Tonepainter is an interactive installation that is part of the opening exhibition of CODE (Center of Digital Experiences) Veszprém, Hungary. Visitors can play a MIDI keyboard, creating unique patterns for different sounds on the screen as an immediate visual feedback. The combination of tunes are represented by a digital brushstroke being painted on an ever-turning sphere acting as a canvas.
The guests can select from pre-defined tune presets which are made specifically to illustrate how to compose a pattern – for example a flower or a heart shape. By doing so, they can get inspiration to try to create their own custom patterns by playing the instrument. The visual output is not only defined by the pressed keys: users can also adjust audio settings with the knobs on the MIDI device, which all affect the live animation – for example the curvature of the brushstrokes, or the speed of the turning globe.
Development
Tonepainter consists of three major parts: the sound control, the visualization app, and the GUI. MIDI messages are first sent to a Pd patch, which filters out the necessary MIDI button messages and the needed interval of the keys. It then communicates with both the GUI and the visualization app, and also alters the sounds by the corresponding effect. The GUI makes it easier to understand what buttons are being pressed, and what values are the knobs currently turned to.
My task was to develop the visualization app, which included a real-time particle system, processing and sending out MIDI messages. The brushstroke is made out of lots of tiny particles. You can switch between a 2D and 3D mode – in 3D mode, the radius of the globe changes based on which key is being pressed, making the pattern more distinguishable and creating interesting structures on the surface.
You can see what patterns the audio presets create – testing with the help of a MIDI player script and a MIDI message simulator tool:
