Sound Squares is an abstract interactive experience where festival goers can create visuals and sounds by walking in front of the installation. Sound Squares has two modes and switches between them every minute.

This project was shown at the Somerville Ignite Festival on October 2nd 2021 and funded by the Somerville Arts Council.

The first mode is a colorful grid of squares representing sheet music (a drum machine). Each column represents an instrument, each row represents a point in time. As time progresses, rows are played, for each of the 3 instruments if a square is active a sound will play, the sheet music is read from bottom to top and then repeats. Viewers can active and deactivate cells by standing in front of the projection and using their hands to activate and deactivate various cells.

The second mode of Sound Square acts an abstract visual mirror. As people walk in front of the display their image is show represented by oversized rectangular “pixels” of red, green, blue, cyan, magenta and yellow. As they move a subtle soundscape forms changing in pitch and duration in time with their movements. Even in this mode the sounds programmed in the colorful grid mode continue to play. As well, even in the colorful grid mode sounds from a person’s movement continue to form as well.

Technical

Sound squares is written in the programming language Processing using the Maxim sound library for all sound generation. It uses an X-Box Kinect to capture people’s hand positions for programming the sheet music, as well as skeleton tracking to trigger the ambient soundscapes, as well as the depth map to show people’s silhouettes in the abstract visual mirror. Images are rear-projected onto rear projection plexiglass.