Jay Goldman

I’ve long been a fan of Virtual Reality — or, perhaps I should say, or the promise of VR. My earliest in-depth exposure to VR came largely from Howard Rheingold’s 1991 book, Virtual Reality. I still remember the stomach flutterby feeling of reading about how we were going to be dropped into immersive worlds at any moment, surrounded by incredibly realistic graphics and lost in rich data visualizations and holodeck-like games. I can clearly remember a rainy day at The Canadian National Exhibition (The Ex to locals), accompanied by my as-always supportive and patient Dad, waiting for hours to try a really rudimentary VR arcade game. This must have been in the early 90s, just as the crazy hype of VR was breaking, and they had setup a network of four ‘pods’ under a tent in a parking lot. For your (probably staggeringly expensive) entry fee, you got to suit up with the standard VR complement of a big, bulky head-mounted display (HMD) and futuristic gloves, all connected by thick cables to an array of impressive gear with blinking lights. I couldn’t find a single online reference to this particular setup, but if memory serves, the pods were garishly outfitted in neon and NOT A STEP style military warnings, giving the whole thing a very CYBER! feel. You played inside a world that was made up of chunky 3D shapes and had to do things like carry balls through hoops or maybe capture flags. Certainly a far cry from the holodeck, but certainly enough to get my young mind buzzing. Oh the possibilities!

A lifetime later, I feel a little Avery Brooks-like in my lament:

It’s the year 2009! Where are the immersive virtual reality worlds? Where is the epic 1994-era Disclosure Digicon Corridor? You all remember that, right?

I’m sure he’s a really nice man and everything, but I honestly think Michael Douglas has exactly one facial expression: aggressively confused. Also, do you think Digicon and Skynet are related at all? Sure sounds like it. At any rate, eighteen years after he so prophetically donned the goggles and gloves, and the best we have to offer is Second Life.

I’ve kept a bit of an ear to the ground when it comes to this topic — largely based on the warm childhood fuzzies it gives me — so I can say that it was with some amateur-expert enthusiasm that I read about researchers from Japan’s ATR Computational Neuroscience Laboratories having developed new brain analysis technology that can reconstruct the images inside a person’s mind and display them on a monitor. I remember reading about a 1999 study in which Stanley, Li, and Dan jacked into a cat’s brain and were able to recreate the images shown to the cat. This was ground-breaking stuff at the time, but messy and somewhat inhumane in its approach. ATR’s research is truly amazing in at least two ways: the readings are taken with an fMRI machine and are completely external to the subject (i.e.: no trepanning required), and the ability to recreate the image is based not on the reading of electrical stimulation of the visual cortex, but rather on an adaptive learning system understanding the actual functioning of the brain. The participant lies inside the fMRI and is shown images of white shapes on black backgrounds while the machine reads the flow of blood in their brain. Some software developed by the researchers analyzes that flow and turns it back into images. Take a look at this video of the results:

Although admittedly fuzzy, what you’re seeing on the right is the beginning of true VR. Mechanical simulations are awesome, but until they reach the sophistication of an entirely immersive (and seamless) world they will always suffer from a meta-awareness problem: you cannot truly become engaged in an environment when you’re peripherally (or proprioceptively) aware of the hardware and systems that create it. As clearly as I remember that rainy day at the Ex, I can distinctly recall the feeling of the weight of the HMD, and the pull of the cables emerging from the back. Sure, things like the CAVE and VirtuSphere can help with the suspension of belief, but it’s hard to forget that you’re playing the role of protagonist in a giant game of Hamster Ball.

(If you’ll permit a brief but humorous aside, do yourself a favour and go watch some of the 5 Minute Romp through the IP video posted by the Electronic Visualization Lab, or EVL. Aside from the fact that their organization has a fantastic acronym, and that the graphics for the Death Star attack strategy presentation in Star Wars were created there by Larry Cuba in 1977, and that the CAVE was invented there, the dude in the video, Dan Sandin, is wearing the most awesome hat ever.)

This is very much early days. The ability to read a highly distorted white square on a black background does not the Metaverse make. It is a very significant first step down that road though. Dr. Kang Cheng, a researcher from the RIKEN Brain Science Institute, commented on the ATR research:

“These results are a breakthrough in terms of understanding brain activity,” says Dr. Cheng. “In as little as 10 years, advances in this field of research may make it possible to read a person’s thoughts with some degree of accuracy.”

Being able to read and write visuals (and sensations!) straight into the brain is the holy grail. One day — though I would say not in the next ten years — we’ll be able to lie down, close our eyes, and be launched into an experience indistinguishable from reality. Even if it’s only to carry ultra-realistic balls through tangible hoops, or to caputre flags.