Pen Pixel Percussion

Pen Pixel Percussion is an audio experiment that converts real-time webcam input to a wavetable, inspired by the optical sound work of Norman McLaren.

Before you run it, I recommend watching the short film Pen Point Percussion to understand the context that inspired how it works, then at least a little of this demonstration video to see what it looks and sounds like when running.

The interface has four parts. The start buttons and setting controls are on top.

Below the setting controls, there is a preview video showing what the camera shows, with a green box showing the current read area. The four "Read area" settings move the box. The read area is clipped to the right and bottom of the video, and making it wider or taller than that doesn't grow it further.

Below the preview video, there is a second preview of just the contents of the read area. In this preview, pixels are only dark or light with no mid-tones. The "White threshold" setting determines the cutoff brightness between light and dark pixels. It helps to point the camera at an evenly lit surface with distinct blacks and whites, as shown in the demonstration video.

The bottom of the interface shows the waveform, as interpreted from the read area. The read area is processed from left to right in a series of one-pixel vertical strips. The ratio of light and dark pixels in the strip determines the waveform intensity at one moment in time. (If the read area is solid dark or solid light, then the waveform will be a flat line and produce no audible sound.) Three copies of the waveform are drawn, so you can see the transition where it loops into itself.

The "Time scale" setting determines the playback rate that converts the wavetable into audible sound. At the default of 60Hz, the sound is produced by sweeping from left to right across the read area sixty times per second. The "Volume" setting lets you play it louder or softer.