Skip to main content
SearchLoginLogin or Signup

Polyfōnía: an Interactive Vocal Processing System using Finger Tracking

Published onJun 01, 2021
Polyfōnía: an Interactive Vocal Processing System using Finger Tracking


Polyfōnía is a real-time portable vocal processing system which users can control by means of their fingers, without the use of gloves or wiring. Its vocal processors include a live looping system and a harmonizer. Motion detection is performed by a motion sensor. Through specific gestures, the user can develop and control real-time voice harmonies and live looping. The system's input is the microphone's audio signal and its output is the processed audio signal. The traditional foot-controlled vocal processors are not oriented towards the real-time vocal performer's needs, as their handling interferes with the posture and expressiveness of the performer's body, acting more as a communication barrier and a source of distraction. The goal of Polyfōnía is to offer, through new technologies and interactive performance, a more natural handling of vocal processors, without added components, highlighting them as a valuable tool for vocal artists in real-time conditions. It is intended to be used in conjunction with vocalists' standard performing techniques and aesthetics, as a means for expanding the possibilities of human voice and musical creation.


Polyfōnía consists of a Max/MSP standalone patch with two main Interaction Modules, two Vocal Processors and a Motion Skeletal Tracking component. It requires a motion controller, a microphone and a computer. The microphone’s input audio signal is routed to the Vocal Processors. The data collected from the motion controller pass through the Motion Skeletal Tracking component to the main Interaction Modules. The audio signal is modified in the Processors depending on the Interaction Modules output.

The live-looping control gestures are designed to emulate a more natural movement of “grabbing” and “releasing” the sound. For the harmonizer processor, we set the performer’s options to activate and deactivate voices via toggles (rather than keeping voices active throughout the duration of each movement) so that multiple voices could be manipulated simultaneously with a few movements. In addition, we consider the importance of the gestures’ easy memorization and thus, the numbers are chosen so that there can be a direct correspondence with the musical intervals.  It is important to mention that the performer can easily assign any gesture to any pitch shifting, simply by modifying the number of semitones for the chosen toggle in the Max/MSP patch.

The Live-looping Interaction Module controls the live-looping processor (i.e., starts and stops playback of the recorded signal) with the predefined hand gestures. Through the Harmonizer Interaction Module, the performer can enable or disable each voice switch with a predefined gesture. This module recognises seven movements, each of which activates a separate voice if formed to the left or right of the vertical axis starting from the center of the motion controller. Therefore the vocal performer can activate up to fourteen parallel voices, creating the desired harmony effect.


Natalia Kotsani

Natalia Kotsani holds a diploma in Applied Mathematics and Physical Sciences and is a PhD candidate in the School of Electrical and Computer Engineering of National Technical University of Athens (NTUA), under the supervision of Prof. D. Fotakis. She also holds a Master’s Degree in «Jazz Music Performance and New Technologies» of National and Kapodistrian University of Athens (UOA) and has since been in the team of Scientific Associates of the Music Acoustic and Technology Laboratory of UOA. She is also a member of the Computation and Reasoning Lab and the Artificial Intelligence Lab of NTUA. She studied singing technique and vocal performance, classical and jazz piano, jazz singing and improvisation as well as traditional techniques of folk singing. Since 2009 she is touring Greece and abroad and she has numerous collaborations, and three personal albums. Since 2011 she is a member of the renowned band encardia. She was a research associate at the Herakleidon Museum in Athens, in the "Art & Mathematics" educational program, designing topics and giving lectures to students and undergraduates. Her research interests pertains to Algorithmic Game Theory, Machine Learning, Computational Ethnomusicology and Musical Information Retrieval.

Areti Andreopoulou

Dr. Areti Andreopoulou is an Assistant Professor in Music Technology at the Department of Music Studies, National and Kapodistrian University of Athens and a leading researcher at the Laboratory of Music Acoustic and Technology (LabMAT). She has a bachelor's degree in musicology from the university of Athens, and a Master's and PhD degrees in Music Technology from New York University in the United States. Her research evolves around audio signal processing, virtual and augmented reality auditory perception, acoustics and psychoacoustics.

Audio-Visual Documentation


No comments here

Why not start the discussion?