This past June at the Electronic Entertainment Expo (E3), Microsoft announced their newest Xbox 360 gaming peripheral called ‘Project Natal’. Project Natal is a sensor bar that connects to your Xbox 360 offering a controller-free gaming and entertainment experience. The foot long sensor bar houses a RGB camera, a depth sensor, a multi-array microphone, and a custom processor running specialized software. All of this state-of-the-art technology allows users to control and interact with the Xbox 360 without the need to touch a game controller.

This device is capable of sophisticated motion tracking, voice recognition, image scanning, facial recognition, and full 3D human skeletal motion capture (i.e. it can recognize each limb’s movement independently and process them in real-time, perhaps even individual fingers depending on the distance from the device). Technically, the device shown at E3 was capable of simultaneously tracking up to four users for motion analysis, with a feature extraction of 48 skeletal points on a human body at a frame rate of 30 hertz. That’s pretty crazy. This technology allows users to move naturally in front of the screen to interact with the game; catching or trapping, swinging, steering, kicking, stomping, throwing, pivoting, jumping, shouting, ducking, punching, leaning, spinning, strafing, etc.

With so many kids these days spending all their free time playing video games instead of outside exercising, perhaps this new technology can help bridge the gap between entertainment and fitness for future gaming generations.

Project Natal Announcement Video:

More information (including video demos) on Project Natal can be found here.

The potential for this device is HUGE. Not only at how it could knock down many of the known barriers to gaming (e.g. if you know how to drive a car, you can immediately play a racing game by grabbing an imaginary steering wheel and steering – no controls to learn or buttons to memorize), but also how it could radically change other aspects of our lives as well. While there are endless opportunities for game designers opened up by this device (and I can already see my niece wanting ‘Dance Hero/Diva’ and all the crazy custom animations my friends will have captured and uploaded to their Xbox Live character avatar, and did I mention the ‘Minority Report’ like user-interface and navigation gestures? Or maybe even running a program that teaches you sign-language which you then use to quickly chat in an MMO chat channel - sans good voice recognition?), the one that I’d like to explore in this post is the potential for a major revolution in the way we work out and stay in shape (at least for us Gen X/Y+ folks).

Now before you say "sheesh, go outside and get a life" - understand that many of us have an unexplainable obsession with technology and gaming. I'm not suggesting a replacement for traditional sport and fitness, simply another alternative activity to help you stay healthy. To each their own, don't be a hater!

The Gamer Physique

With a passion for gaming, I’ll admit that staying in shape is a challenge for me. Although my weight is within normal range, I am usually winded before I really should be in strenuous activity. In fact, it appears I am not alone on this in this country. I do know many people who find running two miles every morning very satisfying with an apparent ease in keeping that routine (and they might get a chuckle at what is explored here), unfortunately I am not one of them. In my attempts at staying in shape, there are two main truths which have prevailed. The first is that I find exercise for the sake of exercise to be painfully dull (running, machines, weights, etc.), whereas I find sport and competition to be very fun (Basketball, Ultimate Frisbee, Soccer, etc.). Basically, if a fun game or competition takes my mind off the exercise, I enjoy it. I imagine this is pretty standard for most people.

The second truth is that if I am committed or highly visible to someone else in my fitness pursuits, I am much more likely to stick with it and can more easily ‘will’ myself to the gym or planned activities (not that I have a perfect record in these cases). Again, these can easily be explained by normal human nature, but they’re relevant here because Project Natal could provide the right combination of motivating social fun to make staying in shape a really enjoyable and rewarding experience for those of us who dread the usual fitness routine.

The Virtual Gym:

Obviously, this device was designed to be used primarily in the home where most people have their Xbox, but what about a brick and mortar chain of ‘virtual gyms’ that offer a fun and social alternative to working out with virtual ‘sport on demand’.  Rather than exercise machines, treadmills, weights, etc. – all that space is dedicated to ‘fitness cubes’ (or the ‘sweat box’?) which houses an Xbox 360 with a Natal Sensor, an HD digital wall projector, wireless sport headphones (or perhaps surround speakers), thick rubber type exercise flooring, and a small bench for towels & water, and perhaps a fan of some sort. The screen is the entire wall in front of you such that the virtual characters you are portraying or interacting with are all life size (or bigger), and you have plenty of space for the multitude of motions and movements you will make during the gaming workout session. Perhaps these spaces are retrofitted racquetball or handball courts, or they don’t include side walls (just large screens) so you can still see and interact with other people in the gym during your session.

When entering the gym the attendant assigns you an open cube and all you need to do is login on the Xbox (bonus if this is all handled automatically by the gym upon membership verification and cube assignment). As you approach the screen, your virtual personal trainer greets you and after either chiding you for how long it’s been since they’ve seen you, or commending your continued commitment - they suggest a gaming exercise routine for the session based on data available (historical data and preferences, personal fitness goals, age, height, weight, measured vitals, etc.).

Each cube’s Xbox are all networked together locally and also to the outside internet, with all the fitness games and programs preloaded on every machine’s hard drive (no discs needed), and all saved fitness scores and/or program progress is persistently saved on the centralized back-end servers (as currently exists with Xbox Live). This allows me to occupy any fitness program anywhere in the world (other cube or hotel room) and pick up my routine or session right where I left it last time I worked out. My virtual personal trainer comes with me where ever I am and never hesitates to offer motivating antidotes (e.g. “Nice job! That was 20 seconds faster than last session’s time!” or “At your current calorie burn rate, you can lose 10 pounds by the end of the month repeating this session 3 times a week”)

OK, so you’re in your sweat box and ready to work out, what does this Natal fitted gym offer me that an ordinary gym workout wouldn’t?

‘Exergaming’:

As I mentioned, exercise for the sake of exercise is not that fun, sport IS fun. What if you could get into multiplayer pick-up games in a variety of virtual sports on demand – on your schedule? That’s the real challenge of organized sport – the organizing part. We are all at the whim of our hectic and often unpredictable schedules. Well, with virtual gyms all connected to the internet all over the world, surely someone else in the world also wants to compete and is working out when you are. Perhaps you’re virtually sparring in Judo against the guy in the next cube, or playing a pick-up game of 2 on 2 Basketball with 3 of your college buddies who are now all over the country, or maybe you’re playing Tennis against a girl from Japan, or in a bike race with 11 other stationary bikers from around the world.

With a smart real-time matchmaking system based on my fitness history and program skill level (which Xbox Live already uses), I could not only get friendly competition but also talk to my friends or similarly skilled competitors during the session using the already existing voice chat system. And in this virtual setting, there are no cancellations due to bad weather or repercussions if something comes up and you can’t get out of the office until late. With real-time matchmaking, someone is always ready to compete with the worst case being a match against the computer (AI) configured to challenge me at my current skill level. The virtual gym offers fun fitness on your schedule and with Project Natal’s alleged ability to recognize 4 unique individuals at one time, you could also be ‘exergaming’ with real world fitness partners in the same cube (or perhaps a double sized one).

Real-World Achievements (Integrated into Social Media):

Another innovation introduced by Microsoft with the Xbox 360 is persistent gaming achievements. Gamers know all too well that in-game achievements go a long way toward making you feel accomplished in a game, your time and money well spent. It’s gotten to the point now where some gamers simply can’t bring themselves to spend time playing a game if there isn’t some kind of global persistence, stats and/or achievements (something to justify the time spent or recognize the skills developed). And as I mentioned earlier, I’ve found that if someone else is involved, I’m much more likely to follow through on my fitness program than if it were just between me and my will power alone.

What if you could earn real-life fitness achievements that all of your friends on Facebook could see, or automatic tweets of your fitness progress or goals achieved? Pose verifiable fitness challenges to your friends? Gyms could also keep local high scores and host competitions or tournaments for prizes. (I could probably write a whole post on the potential for real world national tax-credit achievements - academic, career, parenting, philanthropy, military service, green/low-impact living, volunteering, etc., but for now I’ll focus just on fitness achievements within the context of Xbox and Project Natal.) Sure, achievements alone won’t get you in the door, but they would certainly help you stay at it, and feel that much more rewarded for doing so (potential peer praise is a pretty good carrot).

Exercise for an hour, three times each week in a month – DING Achievement! Get your resting pulse down to 70 beats a minute (verified by a wireless heartbeat monitor and facial recognition) – DING! Lose 5 pounds since last month, 10 pounds in two months (verified by weight pad and facial recognition) – DING DING! Go undefeated in a virtual boxing season or beat a high score – DING! You get the point, but with these posted to Facebook and auto-tweeted on Twitter for all your friends and followers to see, a little social pressure and competition may just be what the doctor ordered to get your blood pressure down.

Persistence, Monitored Progress, and Friendly Competition:

With all your fitness sessions recorded and saved persistently, you could easily monitor your fitness progress. Perhaps this includes opportunities to compete against a ‘ghost’ of your best performance from a year ago – do you still ‘got it’? Can you still get a score of 40 on the sit-ups challenge? Maybe I can challenge myself against celebrity best ‘ghost’ performances or my Xbox Live friends. Can I do more push-ups in a minute than Chuck Norris? Perhaps gamers could wager some of their own gamer achievement points in competitions or for fitness routine commitment challenges or in heart rate improvement and weight loss challenges.

The software would produce (and email) monthly fitness reports, charts, and suggest activities or games for different muscle groups or better general flexibility. Local community ‘fitness improvement’ challenges, gyms against other gyms, states against other states, etc. And of course there would be long-term achievements such as finishing the entire Tour de France under a set time, or a new best unbroken streak of continued regular work-out sessions. Add in some medical peripherals as well (blood pressure, heart rate monitor, weight scale) and the opportunities for personalized goals really expand. Once you have the historical and health vitals data, the ways to process it in informative practical ways are numerous here.

Forget that you’re Exercising:

Assuming Project Natal works as well as is claimed, game designers can take full advantage to get your heart rate up in creative ways. For most Americans these days, just standing up moving around at all is a good start! Gamers can still have a great time while also getting good exercise (without even realizing it). Snowboarding down black diamond slopes where I’m doing the jumping and switching stances and balancing on rails – awesome! Virtual boxing ‘whack-a-mole’ training program – yes please! Playing virtual Street-Fighter with real movements and shouting special move names (rather than memorizing button combos) - yeppers! Playing goalie in the virtual World Cup finals shootout or pitching and batting in the World Series – oh yeah! Perhaps on my ‘off’ days I load the Yoga program, or learn more Taekwondo techniques and run the kicking challenges, get some pro tips on my golf swing from Tiger Woods, or virtual hacky-sack with my Berkeley buddies. Put a stationary fitness bike in my cube (with Natal recognizing my foot cycle rate, posture, and leans) and perhaps I’ll continue my ride along the virtual Appalachian Trail, or continue the next leg in the virtual Tour de France (and perhaps compete with other virtual bikers along the way).

Maybe the gym provides some exercise props that you can check out, such as a kayak paddle or rowing machine to virtually navigate the Colorado river in the Grand Canyon, small hand weights, a body armor harness with full heavy ceramic front and back plates for that extra challenge, step platforms, or a properly weighted dummy bat or golf club to swing. Integrated with state-of-the-art render techniques and real world recorded media and it could even look and sound exactly like you are there. Ever wonder what downtown Paris looks like, select it as the course for your next cycling session. Throw in the potential for 3D rendering and watch out! Heck, hang a climbing rope from the ceiling above my cube and I’ll play virtual Pitfall!  Perhaps I can scan in my boss’s headshot to be used as the face of my next boxing competitor for that extra motivation and stress relief. The sky’s the limit for virtual sports and fitness games.

Sure, this could all be done in your home, but if you live in a place where square-footage is ridiculously priced (I’m looking at you California) – I’d much rather break a sweat without moving the furniture around before and after each session, or worrying about breaking something. Plus, I’ve got all my other fitness amenities there, such as sauna/whirlpool, fresh towels and water, locker rooms, showers, and any other props/equipment or monitoring peripherals that I don’t want to purchase myself.

Natal Leap-frogs the Wii Fit

Nintendo has certainly innovated with the Wii and it's unique controller. They were also first to release a virtual fitness peripheral - the Wii Fit, which is a weight and pressure sensitive balance board type of controller that you stand on. The Wii Fit combined with the motion sensing Wiimote controller may have been just the innovation push Microsoft needed to focus development on Project Natal.

When it comes to virtual fitness - the Wii Fit was an excellent start. However, Project Natal appears to alleviate many of the limitations found in Wii Fit while offering virtual opportunities only limited by our own imaginations. As a designer of one of the first ‘Serious Games’ used for more than entertainment (America’s Army), I know that as an industry we have only scratched the surface when it comes to the potential of the gaming medium outside of pure entertainment. I didn’t even touch on physical therapy and the potential uses this technology has there as well.

Virtual sport will never replace real organized sports, but for those with hectic schedules and/or limited space, equipment, and opportunities – Project Natal may provide a welcome fitness alternative. 

Thanks for reading! If you have any other ideas or thoughts on this topic, let me know with comments!