AR – Application Development Cloud

Always On

On what we already know about minds that never look away — and what it might mean to wear the screen on your face.

A different kind of question

The first three posts in this series were about things we can measure. Eye strain, with tens of thousands of subjects across decades of optometry research. Inattentional blindness, with controlled studies in driving simulators and flight cockpits. Pedestrian deaths, with police accident reports and peer-reviewed papers. All of that is real. All of that is solid ground.

This last post is going to walk us off the solid ground a little, and I want to be honest about that up front. The question of what happens when augmented reality moves from a thing you sometimes use to a thing you always wear is, as of this writing, an open question. The glasses are not yet ubiquitous. The contact lenses don’t exist yet. The data set we’d need to answer the big version of the question hasn’t been collected, because the experiment hasn’t been run on a big enough population for long enough.

So I’m not going to make predictions. I’m not going to tell you what AR glasses are going to do to society in 2035. I have no idea, and anybody who tells you they do is selling you something. What I’m going to do instead is something a little sneakier and a lot more honest: I’m going to walk you through what we already know about what phones have done to human attention, memory, and presence — because phones are basically AR glasses that haven’t quite made it onto your face yet, and the research on phones is a lot further along than the research on glasses. Then I’ll let you do the math.

The Pokémon Go Body Count

What happens when an augmented reality layer forgets you still have a body in the real world — and what the first big real-world dataset has to teach the next generation of builders.

The summer the world went outside

In July of 2016, something happened that the technology industry had been predicting for about twenty years and had nonetheless completely failed to prepare for. A small company called Niantic released a free mobile game called Pokémon Go, which used your phone’s camera and GPS to overlay little cartoon monsters onto the real world. To catch them, you had to physically walk to where they were. To battle in a “gym,” you had to physically stand near the gym’s real-world location. The game’s slogan was Gotta Catch ‘Em All, and within a few weeks, what felt like half of the developed world was outside trying.

If you were old enough to remember it, you remember the surreal sight of grown adults wandering through public parks at midnight in groups of twenty, their faces lit up by phone screens, occasionally letting out a cheer when somebody caught a rare one. People who had not voluntarily been outside in years were suddenly logging miles on foot. Cardiologists wrote excited articles about it. Public health researchers ran studies on the activity benefits. For a brief shining moment, it looked like augmented reality might single-handedly solve the obesity crisis.

And then the other dataset started coming in.

The Gorilla You Didn’t See

On attention, AR, and the strange truth that more information in your field of view often means less awareness of the world.

A famous experiment, in case you haven’t seen it

Sometime around 1999, two psychologists named Daniel Simons and Christopher Chabris ran an experiment that has since become one of the most famous demonstrations in cognitive science. They filmed a short video of six people in a room passing two basketballs back and forth — three players in white shirts, three in black. They asked viewers a simple question: count how many times the players in white shirts pass the ball.

Most people watch the video carefully, count the passes, and report a number — usually correct. Then the experimenters ask: did you see the gorilla?

The viewers stare at them. What gorilla?

They play the video again. About thirty seconds in, a person in a full gorilla suit walks into the middle of the frame, stops, faces the camera, beats their chest, and walks off the other side. The gorilla is on screen for a full nine seconds. It is not subtle. It is not hidden. It is, by any normal measure, the most interesting thing in the video.

And about half of all viewers, on the first watch, do not see it at all.

This effect has a name. It’s called inattentional blindness, and once you know about it, it changes how you think about pretty much every visual interface you’ve ever used. Including, very specifically, augmented reality.

Your Eyes Were Not Built For This

The physical cost of putting a digital layer in front of your face — and why the most exciting thing about AR is also the part nobody’s solved yet.

A weird little experiment you can try right now

Hold your finger up about six inches from your face and look at it. Really look at it — focus on the fingerprint, the little ridges, the half-moon at the base of your nail. Now, without moving your finger, shift your focus past it to something across the room. A doorway. A window. Watch what happens to your finger. It splits into two ghostly versions of itself, blurred and translucent, while the doorway snaps into clarity.

Now do the reverse. Focus on the doorway. The doorway is sharp; your finger is a fuzzy double-image. Switch back. Doorway blurs, finger sharpens. Switch again.

What you just did is two things at once, and your brain handled them so smoothly you probably never noticed. Your eyeballs physically rotated inward and outward to point at the right depth — that’s called vergence. And the lenses inside your eyes physically squished and stretched to focus at the right distance — that’s called accommodation. Your visual system has been doing these two things together, in perfect lockstep, since you were a few months old. They are so tightly coupled that neuroscientists don’t really treat them as two separate systems anymore. They’re one system with two outputs, and the outputs always agree with each other, because in the natural world they always have to agree.

I’m telling you about this because in the last few years, humanity invented a way to make them disagree. And then we strapped that thing to our faces and called it the future.

You Don’t Need to Know Surgery to Build a Surgery Trainer

How real builders work with experts in fields they’ll never master — and why that’s the whole point.

A confession to start with

Let me tell you about a thing that happens to almost every builder I’ve ever met, including me.

You get an idea. A good one. The kind of idea that makes you sit up a little straighter in your chair, open Claude in a new browser tab and start sketching on the back of a receipt. And then, about ninety seconds in, a little voice shows up and says:

“Who do you think you are? You don’t know anything about that.”

And the idea dies right there, on the back of the receipt, because the voice sounds reasonable. You don’t know anything about medicine. You don’t know anything about aerospace. You don’t know anything about education theory or marine biology or whatever the idea was touching. So clearly you have no business building anything near it. Right?

Wrong. And I want to spend the next few minutes showing you exactly why — using one of the coolest examples I know.

Where Two Worlds Become One: Building a Browser-Based Mixed Reality Tool for Education

By Donald Denney | Realm Forge Academy

There’s a moment in every emerging technology’s lifecycle when it crosses over from novelty to necessity. For mixed reality, that moment is now — and it’s happening inside web browsers.

As part of the Realm Forge Academy course Defining a New Reality: Enter the Metaverse, I’ve built a browser-based mixed reality tool that lets students use their device camera to detect a flat surface in their physical environment and place a 3D avatar into a fixed position in the real world. No app store. No headset. Just a browser and curiosity.

[Image: Screenshot of the MR tool detecting a surface and placing an avatar]

AR Art Overlay Prototype

Using AR to bring an art piece to life has rapidly turned into an active and exciting art genre. I was contacted by an artist looking for new ways to share their art with the world. They asked if I would be able to prototype an AR app that will take their finished art pieces and make it deeper by adding audio, animations, and information to the existing piece. I decided to create a prototype to help determine the feasibility of this idea.

I chose my cyber warrior ai art piece

I then enhance it with special FX, artist information, and audio.

Here is a video of the final presentation. The artist is very interested in moving forward with some exciting new projects.

Skills demonstrated in this prototype:

AR Development
Develop AR experiences that use trackable targets in the environment
Build AR projects to mobile devices
Create basic application interactions with Visual Scripting
AR Design
Research future AR use cases to prepare for developing new kinds of applications
Evaluate platform capabilities and limitations in order to determine whether or not a given feature will work.

Augmented Reality AI Character

One of the most exciting uses for augmented reality is to display AI characters into the real world. I was challenged to use the knowledge I have gained in the field of AR to produce a prototype that would place a life-size 3D character into the real world.

AR Face Filter

New recruits use facial tracking filters to take memorable “selfies”

These days most folks are familiar with facial tracking and recognition technology. It has become quite commonplace to have the camera track your face while in a zoom meeting. Who hasn’t been able to resist adding cat ears or a king’s crown to your video feed while the boss drones away during a meeting?

Markerless AR

Markerless Augmented Reality Development Environment

This type of augmented reality technology is called markerless AR. Using this technique you do not need to program an image Target to project your metaverse scene into the real world. Instead the AR engine is analyzing the environment and detecting a ground plane. When the ground plane is detected the reticule will align and you may then project a 3D object or environment that will stay locked in at those exact coordinates.