Creativity Conducted, or Holographic Flash Projections with WiiMotes Deconstructed was lead by team BLITZ. A company that is world-renowned for developing interactive advertising campaigns, rich internet applications and user interfaces. So world-renowned, in fact, that they don't even have a wikipedia article....
This talk was lead by co-founders Ivan Todorov and Ken Martin as they discussed the trials and tribulations of developing the Creativity Conducted interactive WiiMote display at the Adobe Max party on Tuesday Night.
Going into this session, I really had no idea what they were going to talk about. The title had the word WiiMote in it, so that shot my interest all the way up, but the description didn't say what they were using them for, or how they were using them, or what the hell was going on. I thought that they may have been using these things with Flash or something..
What it was: BLITZ had built this massive 16 foot by 9 foot widescreen display that you can paint the timeline of Adobe onto using a WiiMote, and this session was discussing how they went about doing that.
Adobe approaches BLITZ and asks them to develop a big ass display for their Adobe Max Event. The theme of the event is 3 decades of Adobe, so incorporate that in your product.
Ken Martin took the stage and discussed how they came up with the concept. Adobe reaches out to everyone.. Nearly everyone on the planet has used an Adobe product in their life, and maybe not even realizing it.. I mean, if you've seen a YouTube video, you've used an Adobe product as YouTube videos use the Flash player. Big to small, everyone has been touched by Adobe.
But how do you incorporate the creativity used with Adobe into a Chronology sequence. Typically, the word Chronological implies "bored off your ass". Time-lines are very static and full of information.. How can we make this fun?
Interaction
Well, letting the User interact with the time-line is a good start. Adding dynamic components to a very static concept is always a step in the right direction.
Paintbrushes
Let the User paint their own time-line of events. As the time-line becomes full of life, events in Adobe's lifetime will begin to tell their own story and begin to grow with the User's composition.
Save and Share
Part of making a beautiful art piece is showing it off to the world. Being able to save this image was a key feature to this project that needed to be included from the start.
So we want interaction.. Interaction how? Everyone can use a mouse, but that's boring to do for a massive display. Trackballs are great for a display as well, but how can someone paint with a track ball? Being able to gesture the paint strokes would be great but we'd need a tool..
Why not the WiiMote? With a swoosh of your hand, you'll be able to paint onto the screen, like you were conducting a symphony. WiiMote are easy to find as well, we could add in a multiplayer component to this composition to make it very interactive and physical.
Now that we've got our input device, how are we going to make the thing? Martin explained that they wanted to build this thing with 100% Adobe driven development. The application will be made in Flash 9, using some open source Flash products.
Wii Flash: To interpret the bluetooth signals from the WiiMote.
Papervision 3D For developing 3D graphics with flash.
Export to JPG: To export the final composition to JPG.
Now: How are we going to display this thing? They wanted something big! Something and in widescreen format. Hell, why not 16 feet by 9 feet? That's pretty easy to measure. They ended up making the display with a DIY projector, a mesh fabric background, and a dimly lit room.
This display is all about the User. What are our goals when developing this User's experience and how do we want them to interact with the display?
Intuitive and Simple
They want something very simple the User could pick up and play right away. This meant simple instructions, and a very easy interact that didn't take much for the User to feel at home and ready to create.
Animated Brushes
Let their design bloom. When a User makes a stroke, the painting should bloom and come alive.. kind of like when you use very course paper, the strokes bleed into the page and it feels alive.
Illustrators
To create these animated brushes, BLITZ hired 5 illustrators to design 5 different styles independently from each other from the 3 decades of adobe (the 80s, 90s, and 00s).
Since BLITZ would have 5 independent illustrations, they needed to design a plug-and-play importing system for their application to add and swap these brushes at will.
With all the concepts covered for the project, Martin handed the mic over to Todorov to talk about Implementation.
The primary challenge with using the WiiMote is that the values it emits are mostly relative to it's position. It knows the pitch, acceleration, and points relative to the IR sensor bar.. BUT the WiiMote does not know it's exact position on the screen. Your application must interpret that data and figure out where's it's pointing.
Besides screen location, they also wanted to utilize distance to and from the screen to draw depth into the image. On screen, that's where using Papervision was key. Papervision is wonderful for 3D developers because their models can be directly imported from 3ds max or Maya.
Another challenge that cropped up with how to actually paint? The WiiMote knew what buttons were being pressed, but how would they tell Flash about the depresses? They used a Virtual Mouse program to reprogram their computer to use the WiiMote buttons as mouse clicks.
Pleasant Expectations
Some things did go precisely as planned. Adding a second WiiMote into the mix was dirt simple. Using open source technologies through bytearray.org were great. Papervision is fricken awesome.
Pleasant Surprises
Throughout the course of development, they were surprised a few things. Flash Player 9 is damn fast.. They were able to push and push the new flash player with very minimal lag. Also, the WiiFlash community is very healthy. Any problems that came up with WiiFlash were easily resolved with the help of the community. And they even gave back by finding some bugs too.
Unpleasant Surprises
With any project, you'll come across things that are a pain in the ass. Limitations with the WiiMote were very annoying, as they can't interpret depth as well as they'd like. You have a limited range you can use a WiiMote, and when you break that distance, it'll think you're pointing off screen. Also, all their vector imagery had to be converted into rasterized images when outputting on screen.
Once Todorov was done talking about Flash, Martin came back on stage to talk about creating the display they were going to use for this thing. They needed a really fricken bright lens in their projector for a 16 foot by 9 foot display, so they went through several different bulbs before finding something bright enough. The lens had 10,000 Lumens, for those that know whatever that means.
Google Sketch-up was incredibly helpful for placing everything into space. It was good to see of sketch of how far the user will be from the screen relative to the size of the screen, and the placement of the IR sensor bar, and all that.
And he went through other boring construction type stuff that I didn't take notes on.
This session was pretty fun. It was neat the see the creation of a very non-conventional piece of interactivity using state-of-the-art and very current technologies. Always think outside the box ^_^.
Was it practical for me? Not really.. I suck at flash.