I built a fun Max/MSP patch that utilises a webcam to interact with a computer-generated fly. The “fly” is created using optical flow-based tracking, which was produced through Jean-Marc Pelletier’s cv.jit externals. They’re great for doing anything with Computer Vision in Max/MSP/Jitter.

As a sound designer, I am obviously more interested in using Computer Vision to affect sound. Inspired by Andy Farnell’s fantastic book on procedural audio Designing Sound, I built a fly sound generator to give life to the fly.

Computer Vision Fly using Max/MSP/Jitter from Andrew Spitz on Vimeo.

If you want to play with the Computer Vision Fly application yourself and you’re on a Mac:
Download Computer Vision Fly Application (9.4MB)

the process

I mapped the X-axis to pitch (left being low, and right being high). To make it feel a bit more natural, I added a ‘drunk’ function so that the pitch always changes slightly, even if the fly is stagnant. I also mapped the X-axis to the left and right pan to give it movement and create the slight illusion of a Doppler effect when the fly whizzes past.

The Y-axis is mapped to volume to give the illusion of distance. When the fly is farther up the screen the volume is softer and when the fly is lower, the volume is louder.

2 Responses to “Computer Vision Fly { sound + video }”  

  1. 1 jozua

    What a great Sci-fi little prgrm. Awesome friend! I think this one is great…!

  2. 2 web tv

    This design is wicked! You most certainly know how to keep a reader entertained.
    Between your wit and your videos, I was almost moved to start my own blog (well, almost.
    ..HaHa!) Great job. I really enjoyed what you had to say, and
    more than that, how you presented it. Too cool!

Leave a Reply