Damian Hart
Stuart Molthrop
COSC 320.101
12 December 2005

“Just Input That Anywhere”

It’s Christmastime 2005 and gamers are yammering for their new gaming console.  This year’s heavy hitters are Microsoft’s Xbox 360, the PlayStation 3, and the Nintendo Revolution.  In truth, only the Xbox 360 is available for holiday gift-giving, but the fan sites are ablaze with speculation on how the three systems will duke it out when all are finally available for purchase.  The topic that seems to capture the most attention—other than graphics capabilities—is the introduction of the Revolution’s new input device.  Abandoning the “boomerang” style controller, the Revolution’s austere appearance is that of a household television remote controller.  Its function, however, is what has the gaming world divided into love-it / hate-it camps.  Employing an internal gyroscope and wireless technology the player moves the controller in 3D space as if it were a magic wand, a paper airplane, a fishing pole, a drum stick, or a ninja sword (Totilo).   Some claim that it is the next “revolution” in player input, while others compare it unfavorably to other failed innovations.  Studying these comparisons leads us to consider even more alternate input devices and wonder if and how these devices might be used in modern video game play.

[Nov 14, 2007 update: Of course, the Nintendo

Virtual Space

The most notable objection to the Revolution controller recalls the 1989 introduction of Mattel’s Power Glove.  Worn over the fingers, hand, wrist, and lower arm of one arm, the Power Glove employed technology of the era to register the user’s hand and finger movements; a keyboard of sorts on top of the forearm allowed the other hand to input typed data as necessary.  Data was transmitted to the Nintendo Entertainment System via cables and ultrasonic sounds.  Sadly the aforementioned technology was too immature to provide quick or accurate response to user movements and the Power Glove was abandoned (Barton).  However, the potential for glove-based entertainment input exists and has been seen.  Disney’s EPCOT center once had a display in which a demonstrator, wearing a pair of gloves and standing before a gigantic monitor, was able to use on-screen building blocks to construct a cityscape merely by moving her hands in the space before her body and flexing and releasing her fingers.  Program options were accessed by moving the onscreen hand representation to the menus and “poking” the choices.  The company Essential Reality seems to feel that glove-based input might be due for a renaissance and has developed a new glove-like device called the “P5” (Barton).  Imagine casting a wizardly spell in which the spell is not merely cast as a hotkey keypress but the successful gyrations of the player’s hands?

[Nov 14, 2007 update: The P5 Glove has apparently been discontinued.]

Why only consider the movement of the player’s hand in 3D space?  Why not track the gamer’s entire body?  This question is answered by Sony’s “Eye Toy,” a device that has been on the market as a PlayStation peripheral since late 2003.  Essentially a webcam that sits atop the television, the Eye Toy analyzes the space before it, reacting to the presence and actions of the player in that space (Torres).  The selection of games currently available for the Eye Toy is somewhat limited and juvenile (Dante), but the device shows great potential for the imaginative designer.  Given the rate of change in computer processing power since its release, one can easily envision the Eye Toy in use for fighting games and dancing games.  But what about other possibilities?  Imagine Black & White in which the player “grabs” the ground and rotates it to change the viewpoint.  Point to a destination on the map and then execute a hand gesture to fly there.  Actually “pick up” villagers to train them, pick up sheep and trees and boulders to move them.  A “stroke” the air in front of the player would praise the pet…while a few backhands would slap him around when he’s bad!

Other data capturing technologies exist, however, that are not being used in the gaming field.  What are they?  Why aren’t they being used?  How might they be used?  We will now explore some of those devices.

Voice Recognition

Nearly every newly purchased computer system contains a flimsy microphone of some sort, yet this microphone is very rarely used.  Voice recognition software has been employed for some time to provide modest control over computer functions.  In the early days of Windows 95, there existed a program called “Dashboard,” which used the microphone to record the user’s voice command.  This command was then mapped to a series of keystrokes to be executed when the command was recognized.  Invoking shades of “Star Trek,” merely saying “Power off!” in the vicinity of the microphone might cause the computer to shut down.  Many hands-free cellular phones employ a similar system, allowing drivers to “voice dial” a number by merely speaking the name of the person they wish to call or by reciting the individual digits.

The problems have been that the voice-recognition software has had difficulty in interpreting fluctuations in intonation, cadence, and in filtering out background noise.  These limitations are being conquered, however.  “Help Desk” automated phone menus have long allowed users to make choices by speaking numbers or names into the telephone.   New software can now analyze the speaker’s intonations and if it determines the caller is getting upset, immediately connect the caller to a live assistant.
Bring this capability into the computer gaming world and we may see combat simulators, emergency room simulators, air-traffic controller simulators, and other voice-intensive games take on new life.

Fitness Equipment

Society seems concerned that constant video gaming tends to discourage exercise.  Hours spent in front of the screen, they say, are hours not spent out of doors engaged in sports or other physical activities.  A number of interesting devices are evolving to address this, most notably the CatEye GameBike.  This stationary exercise bike sports a USB cable that is plugged into the PlayStation 2.  The device allows the player—er, rider—to control the game via peddling, steering, and leaning (Reid).  Any speed-related game, such as motocross, speedboat, or similar racing games can get their data from the player’s actions on the bicycle.  If he wants to move faster, he’s going to have to pump those pedals!

[Nov 14, 2007 update: A fellow has converted his home treadmill into a Second Life input device. Now when he wants his avatar to walk somewhere, he actually has to walk on the treadmill to make the avatar move. http://moriash.blogspot.com/2006/08/more-scenic-walk.html ]

Of course the aforementioned games are the obvious implications of the device, but other possibilities exist.  Take part in the Tour de’ France, or bike through Europe after a day at the office.  Actually peddle through the old Atari game “Paperboy.”  Perhaps enjoy a Jules Verne-type adventure in which the onscreen bicycle is not merely a bike, but a bicycle/dirigible combination and the game also takes place above the landscape as well.  What if the bike instead powers H.G. Wells’ onscreen time-machine?  

Global Positioning Systems

The National University of Singapore’s Mixed Reality Lab has employed a GPS system in conjunction with wireless networking and special glasses that can have computer images superimposed on them in a sort of “heads up” display to create a real-life PacMan game which is played on the streets of the city (Knight).  The person taking on the role of PacMan sees the landscape around him, but superimposed on this—via the HUD of the goggles—are the yellow dots that the player must “eat” as PacMan.  As the player moves around the map—literally moving around the city—his position is transmitted by the GPS system to the coordinating computer which also keeps track of how many dots he has eaten.  Of course, he must avoid “Inky,” “Pinky,” “Blinky,” and “Clyde” the four ghosts of the game—other similarly outfitted players who are also out on the streets searching for the PacMan player.

While visions of Star Trek’s holodeck beckon, truly interactive, computer-generated environments are still far in the future.  But GPS-based game possibilities are intriguing.  Mr Mout uses a similar idea with other tricks to create urban warfare simulations for military training (Hughes), but what of the civilian player?  Consider a player, decked out in a PacMan-like manner, but placed in a properly prepared warehouse.  The player’s goal is merely to move—physically move—from one point in the warehouse to another.  Using a “Laser Tag” like pistol that he must recharge with virtual power-ups hidden around the generated environment, he must fight off hoards of attacking enemies.  Might we call this game:  "Truly Doomed" ?

Eye Tracking

Usability testing makes use of eye tracking systems to analyze how a user scans web pages, what areas captures his attention, and how long the focus remains on any particular area.  Once again using what is essentially a webcam aimed at the reader’s face, the software can track the movements of the eye to an accuracy of one-centimeter (Outing).  Knowing where a user is looking and how long he gazes at a particular location is of paramount importance when designing screen layout for web pages and similar interfaces.

But imagine the theatrical possibilities of a game that employed such technology.  Any sort of horror-adventure game that can place a foe in a location that the player isn’t looking—because the game knows the player isn’t looking there—is going to create a whole new level of suspense and immersion.

Telepathy

Developed primarily to aid the handicapped, researchers at Duke University are developing computer programs and interfaces that can read minds (“Monkey Brains”).  Using lab monkeys, the researchers used electrodes to record and interpret brain waves from the monkeys as they used a joystick to manipulate a robotic arm.  Eventually, the scientists removed the joystick and the monkeys continued to move the robot arm merely by thinking about it.  The computer interpreted the brain signals as instructions to the robotic arm, which is amazing enough, but the notion that the monkeys realized they could control the arm merely by thought—then began to do so— is remarkable beyond belief.

The implications for seemingly psychic games is staggering.  A Civilization-type game could assign order to combat units merely by thinking about those commands.  All sorts of “God” games, like the Populous series could experience new life when the player need only think about a volcanic eruption in order to cause one to happen.

Combinations

What sort of experience could be delivered to the “Totally Doomed” player if, in addition to the GPS and HUD device already employed, the computer used eye-tracking to know where the player is looking, and brain-wave recognition to learn that the player is thinking about doubling back for extra ammunition?  How much would a player pay to play?  How often would he pay that amount to continue to play until he “beat” the game?

In another scenario the EyeToy, reacting to the player’s reaching motion, causes the robotic arms of the onscreen mini-submersible to likewise reach for the chest of gold bullion.  Suddenly from the foggy distance of the underwater realm, the hostile Atlantean army begins its charge.  Frantically, the player peddles his GameBike in retreat, hoping to outdistance the angry mob, all the while frantically shouting commands to the Nautilus II resting a few leagues away.

Assuming the role of the X-Men’s fabled Cyclops, one player uses his optic blasts—aimed and controlled via vision tracking— to vanquish foes and overcome other threats.  Meanwhile the EyeToy translates the Superman player’s action into combat moves, while Spiderman—using the P5 gloves—spins the webs that entangle their foes.  With their efforts interpreted and coordinated through a central computer, these three gamer superheroes defend their virtual country from innumerable threats.

Is the new Nintendo controller truly “revolutionary?”  Not really.  Many clever input devices offer the capability for unique game play.  Today’s Bluetooth technology can do away with cumbersome wires, while our faster and more advanced computer processors can more speedily process data from the input devices.  But as these three hypothetical games show, a designer’s imagination can really take flight when we begin to consider using them in conjunction with each other.

Works Cited

Barton, Matt , and David Torre. “The Power Glove Lives!” 2004. Armchair Arcade.  11 December 2005 <http://www.armchairarcade.com/aamain/content.php?article.39>.

Dante, Tolen. “Eyetoy.” 23 November 2003. The Armchair Empire. 11 December 2005 <ttp://www.armchairempire.com/Reviews/ps2/eyetoy.htm>.

Hughes, Charles E., et al. “Mixed Reality in Education, Entertainment, and Training.” November/December. 2005. IEEE Computer Society Press. 11 December 2005 <http://www.computer.org/portal/cms_docs_cga/cga/content/Promo/promo3.pdf>.

Knight, Will. “Human PacMan Hits Real City Streets.” 18 November 2004.  NewScientist.com. 11 December 2005 <http://www.newscientist.com/article.ns?id=dn6689>.

“Monkey Brains Control Robot Arms.” 13 October 2003. PhysOrg.com. 11 December 2005 <http://news.bbc.co.uk/2/hi/health/3186850.stm>.

Outing, Steve, and Laura Ruel. “Welcome to Eyetrack III.” Eyetrack III. 11 December 2005 <http://www.poynterextra.org/eyetrack2004/about.htm>.

Reid, Carlton. “400mph Cyclists Kill Opponents with Handlebar-Mounted Guns.” 26 January 2004. The Register. 11 December 2005 <http://www.theregister.co.uk/2004/01/26/400mph_cyclists_kill_opponents/>.

Torres, Ricardo. “EyeToy Preview.” 6 October 2003. Gamespot. 11 December 2005 <http://www.gamespot.com/ps2/action/eyetoy/preview_6076428.html>.

Totilo, Stephen. “First Look : Nintendo Revolution Controller Feels Smooth as Puppet Strings.” 8 December 2005. MTV News.  <http://www.mtv.com/news/articles/1517678/20051207/index.jhtml?headlines=true>.