As a collaboration between the SAT and Mobile Muse, we have created a component that allows visual artists to collect media content from their audiences during live events. The project is called RawMaterials, and is targetted at VJs using Modul8 for mixing, yet any software capable of receiving OSC messages can be used. Audience members can use their cell phones or other mobile devices to send SMS, MMS, microblob feeds such as Twitter, and even live video using Livecast software. The public can also upload content to social media sites such as Flickr, YouTube, Picasa, etc., and the system can be configured to aggregate content from those sources as well. This can even happen far before the event starts, so that the public can choose the raw materials that will be used during the performance.

In the summer of 2008, we developed a prototype of a multi-player mobile audio game, called SoundPark. The objective of the game was to collect audio clips that were physically scattered throught the real world, and deposit them in a staging area for playback. Central to the design are the themes of highly coupled interaction and communication between players with different roles and an engaging blend of interaction with both the physical and virtual worlds. To this end, numerous technologies including locative sensing, miniature computing, and portable displays had to be integrated with a game middleware and audio scene rendering engine.

I’ve written a much more detailed description on audioscape.org if you are interested, but here is a little documentation video that we put together to illustrate the concepts:

My main activity is the development and programming for the Audioscape Project, which involves anything to do with spatial organization of audio content and sound processing. Generally speaking, we provide the ability to create virtual worlds that function as musical interfaces or installation.

Most recently, however, we have started to add the ability to include mobile and location-based environments, in the outdoors. By using tiny mobile computing devices, GPS, and ad-hoc wireless audio streaming, we have managed to create shared mixed-reality audio environments that multiple users can experience while roaming around outdoors.

The bulk of this development occurred during the Almost Perfect locative media residency at the Banff Centre New Media Institute in November of 2007.

More information about this project can be found on the Mobile Audioscape page, and below is a video that was filmed during the residency in Banff:

On March 11th, 2008, I presented our mobile audio work at Interface[s] Montreal, which is a conference series held each year at the SAT. The theme for this night’s presentations was “Mobility”, and so I talked about the future use of audio in augmented- and mixed-reality applications.

You can watch my presentation below, or watch it on the Interface[s] Montreal website for the full experience.