Bringing Air Guitar To Music Games
Posted: Thu Mar 03, 2011 9:14 pm
Bringing Air Guitar To Music Games
By Edge Staff
The man behind a Kinect hack on how experimenting with motion control could make music games hum again.
British artist and designer Chris O’Shea wields cutting-edge technology as if it were a brightly coloured paintbrush. His various projects and installations – commissioned by the likes of the BBC, FACT and Design Museum London – inspire playfulness and wonder, jostling unexpectant audiences out of their hurry-along autopilot. We talk to the artist himself about his experimentation with Xbox 360’s Kinect technology for a recent air guitar prototype, the capabilities and limitations of the tools, and what they portend for the future of music games.
You first attempted an air guitar prototype in 2007 – what are your thoughts after revisiting the project using the Kinect sensor?
The air guitar project I tried in 2007, for the charity Shelter at Glastonbury, used a stereo camera – which created a depth image from its two cameras. And the problem I found was that it was very noisy, but also these depth cameras are very expensive. I think that particular camera was about £1,500, and typically you’re looking at anywhere up to about £4,000 for a decent depth camera, to give you the depth of the scene. So I think the biggest impact has been the cost. The Kinect coming out for £120 means it’s really accessible to a much wider range of people. Because the Kinect sensor creates a depth image using an infrared light pattern, rather than using two cameras and working out the stereo image between them, you get a much nicer-quality depth image. It’s a cleaner image, it’s got a lot more detail in it, and it’s a lot more accurate in terms of what it’s seeing in the scene. The technology of the depth camera has been out a while but it’s just been too expensive to buy.
The Guitar Hero and Rock Band games require a certain level of technical mastery. Do you think there’s a commercial niche for less precise music games, such as an air guitar simulator like the one you’ve worked on?
People say, “Well, maybe it could detect fingers, and maybe you could play the guitar more accurately if you could detect fingers.” I don’t think it was ever about that. It was just the feeling of playing air guitar in the same way that when you’re playing tennis in Wii Sports, it’s not looking at the rotation of your hand or accurate position of the racquet. It’s just giving you a fun experience, like what you’d imagine it to be; it doesn’t have to be an accurate simulation of real life. I don’t think it’s dumbing the experience down – it’s just making it accessible and fun without having to worry about people having to be able to play the real sport or instrument.
What limitations did you encounter while working on the prototype?
When the Kinect device driver was written so it ran on Windows or a Mac, basically it would give you back a depth image, but there was no tracking. There’s no skeletal estimation – that’s all done in the Xbox. So what you saw in the air guitar prototype was my attempt at trying to estimate and track where someone’s hands were.
While I was doing this demo, PrimeSense, the technology company behind what’s inside Kinect, released a driver that would enable people to use the camera – and also the new cameras coming from Asus that PrimeSense are working with, which is basically going to be Kinect for PC. And they’ve also released their skeletal tracking system, so you can stand there and it’ll estimate and work out all the points of your body, your legs and things. So, very similar to what it’s doing inside the Xbox, you can now do running off the PC.
Do you think that Kinect will inject new life into the music-game genre?
Dance Central is really good. It’s one of the best launch titles for Kinect. I think that Harmonix, for example, have probably got a huge amount of ideas that are a lot more experimental. Rock Band and Guitar Hero pioneered things that became far more successful in the mainstream than their earlier stuff. Hopefully they’ll use this opportunity to go back and do things that are a lot more experimental and more interesting, that are less about simulating real instruments than just using your body in different ways.
By Edge Staff
The man behind a Kinect hack on how experimenting with motion control could make music games hum again.
British artist and designer Chris O’Shea wields cutting-edge technology as if it were a brightly coloured paintbrush. His various projects and installations – commissioned by the likes of the BBC, FACT and Design Museum London – inspire playfulness and wonder, jostling unexpectant audiences out of their hurry-along autopilot. We talk to the artist himself about his experimentation with Xbox 360’s Kinect technology for a recent air guitar prototype, the capabilities and limitations of the tools, and what they portend for the future of music games.
You first attempted an air guitar prototype in 2007 – what are your thoughts after revisiting the project using the Kinect sensor?
The air guitar project I tried in 2007, for the charity Shelter at Glastonbury, used a stereo camera – which created a depth image from its two cameras. And the problem I found was that it was very noisy, but also these depth cameras are very expensive. I think that particular camera was about £1,500, and typically you’re looking at anywhere up to about £4,000 for a decent depth camera, to give you the depth of the scene. So I think the biggest impact has been the cost. The Kinect coming out for £120 means it’s really accessible to a much wider range of people. Because the Kinect sensor creates a depth image using an infrared light pattern, rather than using two cameras and working out the stereo image between them, you get a much nicer-quality depth image. It’s a cleaner image, it’s got a lot more detail in it, and it’s a lot more accurate in terms of what it’s seeing in the scene. The technology of the depth camera has been out a while but it’s just been too expensive to buy.
The Guitar Hero and Rock Band games require a certain level of technical mastery. Do you think there’s a commercial niche for less precise music games, such as an air guitar simulator like the one you’ve worked on?
People say, “Well, maybe it could detect fingers, and maybe you could play the guitar more accurately if you could detect fingers.” I don’t think it was ever about that. It was just the feeling of playing air guitar in the same way that when you’re playing tennis in Wii Sports, it’s not looking at the rotation of your hand or accurate position of the racquet. It’s just giving you a fun experience, like what you’d imagine it to be; it doesn’t have to be an accurate simulation of real life. I don’t think it’s dumbing the experience down – it’s just making it accessible and fun without having to worry about people having to be able to play the real sport or instrument.
What limitations did you encounter while working on the prototype?
When the Kinect device driver was written so it ran on Windows or a Mac, basically it would give you back a depth image, but there was no tracking. There’s no skeletal estimation – that’s all done in the Xbox. So what you saw in the air guitar prototype was my attempt at trying to estimate and track where someone’s hands were.
While I was doing this demo, PrimeSense, the technology company behind what’s inside Kinect, released a driver that would enable people to use the camera – and also the new cameras coming from Asus that PrimeSense are working with, which is basically going to be Kinect for PC. And they’ve also released their skeletal tracking system, so you can stand there and it’ll estimate and work out all the points of your body, your legs and things. So, very similar to what it’s doing inside the Xbox, you can now do running off the PC.
Do you think that Kinect will inject new life into the music-game genre?
Dance Central is really good. It’s one of the best launch titles for Kinect. I think that Harmonix, for example, have probably got a huge amount of ideas that are a lot more experimental. Rock Band and Guitar Hero pioneered things that became far more successful in the mainstream than their earlier stuff. Hopefully they’ll use this opportunity to go back and do things that are a lot more experimental and more interesting, that are less about simulating real instruments than just using your body in different ways.