Abstract

In a search for a symbiotic relationship between the digital and physical worlds, I am developing a hybrid, digital-acoustic wind instrument - the Post-Digital Sax. As the name implies, the instrument combines the advantages and flexibility of digital control with a hands-on physical interface and a non-orthodox means of sound production, in which the airflow, supplied by the player’s lungs, is the actual sound source. The pitch, however, is controlled digitally, allowing a wide range of musical material manipulation, bringing the possibilities of a digitally augmented performance into the realm of acoustic sound.

Author Keywords

NIME, hybrid instrument, post-digital, saxophone

CCS Concepts

•Human-centered computing → Sound based input / output;
•Interaction devices → User interface design;

1. Introduction

Acoustic instruments are still in the center of musical activity in many genres for a number of reasons. From the player’s perspective, they offer immediate response to actions and gestures, as well as direct auditory and tactile feedback. Listeners in turn may appreciate their sonic qualities, such as subtleties of dynamics and unique directivity patterns. [1] Yet, digital technology provides a number of features which are impossible to achieve with traditional acoustic instruments, thus it might be implemented to enhance their capabilities. A typical scenario would involve digital processing of acoustic sound, which is then projected through a speaker system; however, this approach detaches the sound source from the actual instrument, creating an experience which might be perceived as unnatural or confusing. Thus, various approaches have been developed to deal with this issue.

The Post-Digital Sax (abbreviated further as P-D Sax) aims to preserve the key features of acoustic instruments, while taking advantage of digital technology. It uses a vibrating reed and the player’s breath as an actual sound source, thus it might be considered an acoustic instrument. Nevertheless, as the pitch is controlled digitally by mechanical vibrations induced in the reed by an electromagnetic actuator, a wide array of digital interventions can be implemented.

1.1 Prior Work

Digitally augmented acoustic instruments span a broad spectrum, which could be divided into following categories:

1. fully digital instruments, in which only the interface resembles an acoustic instrument;

2. augmented acoustic instruments, in which the acoustic sound is digitally transformed;

3. actuated instruments, where the acoustic generators are actuated with digitally controlled mechanisms, while still enabling the hands-on control;

4. fully robotic acoustic instruments;

Regarding the P-D Sax, the actuated instruments category is the most relevant. It is distinguished by preserving both the ability to play the instrument in an usual way and the acoustic character of the sound, while benefiting from the digital augmentations. Notable examples of this approach include magnetically actuated pianos by Per Bloland [2] and Andrew McPherson [3] , bowed string instruments such as the Overtone Fiddle [4] , Svampolin [5] and Halldorophone [6] , plucked string instruments such as the Cyther [7] and percussion instruments, such as the Haptic Drum [8] .

While the woodwind-inspired electronic instruments also tend to fall into similar categories, it’s difficult to find examples fulfilling the criteria for an actuated instrument:

1. fully electronic instruments/controllers, with user interface resembling that of a saxophone. Their main purpose is to provide a familiar user interface and to allow additional expression by utilizing breath/lip pressure sensors, as is the case with EWI [9] and EMEO [10] . They usually require external speakers, although some have a built-in speaker, like Yamaha YDS-150 [11] , making it an embedded acoustic instrument [12] . Some examples also provide haptic feedback, such as the Hybrid Clarinet Project [13] .

2. acoustic instruments with digitally augmented sound, usually with a microphone and additional controllers attached to the instrument; the digitally processed sound is reproduced by external speaker, as is the case with Gluisax [14] and Metasaxophone [15] ; some instruments, such as the HypeSax [16] aim to blend the original acoustic sound with its processed version by placing the speaker on or inside the instrument

3. ?

4. fully acoustic robotic wind instruments, such as the NICTA Robot Clarinet [17] and Waseda Saxophonist Robot [18] ; they focus on the ability to control the acoustic sound digitally, but usually fall short of providing a comfortable interface or a satisfying user experience; although at least one example - the Infinitone [19] - preserves the ability to control the sound by blowing an actual mouthpiece with a reed; the predominant way to digitally control the pitch is a set of actuators, operating on the keywork of the instrument; their common trait is that the user interface is a separate entity, with the instrument itself being overwhelmed with mechanical actuators, which impose additional latency and noise;

1.2 Advantages of the P-D Sax

The P-D Sax proposes an alternative approach of controlling the frequency of reed vibration by the electromagnetic force. To the best of my knowledge, the only devices producing sound by similar means, i.e. by mechanical interruption of the air stream, are the obsolete compressed-air speakers/gramophones [20] and musical sirens [21] , which would make the P-D Sax the first practical implementation of this sound making principle in the form of an embedded acoustic instrument, as well as the first wind instrument falling into the actuated instruments category.

Further advantages of this approach are as follows:

• in the purely electronic saxophone-like instruments, the parameters of a breath/pressure sensor are arbitrarily mapped to various features of the sound; in the P-D Sax the reed and breath are actual sound generators; changes in the air pressure and embouchure influence the volume and timbre of the sound directly;

• in the magnetically actuated string instruments, while the frequency of vibration of electromagnetic field can be varied, it doesn't change the physical properties of the string, i.e. its natural frequency; in the case of the P-D Sax, the magnetic field directly controls the frequency of the sound generator;

• the P-D Sax can produce a much lower pitch than could be provided by an acoustic instrument of the same length;

• the mapping of the interface may be customized, either to fit a specific performance scenario, ot to suit the needs of a specific group, e.g. children or musicians with disabilities;

• nonstandard musical scales can be freely assignable and switchable;

• a number of interventions in the musical material is possible;

1.3 Research Overview

Initial motivation for the research was to test a new approach to acoustic sound production. First experiments resulted in the prototype 1 - a proof of concept prototype of the mouthpiece. After successful initial tests, prototype 2 was built to test the approach in the form of an embedded acoustic instrument. As the instrument showed promise with regards to a planned personal music project, the prototype 3 was built - a refined version of the embedded instrument with a larger sound horn, continuous keys, lip pressure sensor and additional joystick controller. Following sections will focus on the prototype 3.

2. Design

2.1 Basic Principle

While in a traditional single-reed instrument the frequency of reed oscillation is determined by the resonant frequency of the air column, in the P-D sax the reed is forced to vibrate at an arbitrary frequency by an electromagnetic force. Nevertheless, the air from the player’s lungs is still a component necessary to produce sound and its pressure directly affects the volume, while the vibrating reed serves the function of an air valve. As the reed is in contact with one of the player’s lips, the timbre can be controlled by the change of embouchure. In order to vibrate the reed with magnetic field, a permanent magnet has to be attached to the reed [22] .

2.2 Reed and Mouthpiece

Although a standard (slightly modified) tenor saxophone reed is used, the custom, 3D-printed mouthpiece differs from that of a saxophone. The reed is attached in a way which allows a hinged movement; consequently, a permanent magnet can be attached to the far end of the reed, allowing the other end to be placed inside the player’s mouth. The electromagnet, which attracts the movement of the reed, is attached with a bolt; the reed is fixed with a bolt-adjustable rubber o-ring and a neoprene strip.

As the electromagnetic actuation of the reed shows a significant nonlinearity, experiments with different waveforms controlling the power of the electromagnet yielded similar results, with a sudden jump of the reed after the electromagnetic field value exceeded a certain threshold. At 1 Hz frequency, the motion pattern of the reed always resembles the pulse wave, only the duty cycle changes depending on the shape of the wave driving the electromagnet. Therefore, for further experiments, a pulse wave was chosen as a default waveform modulating the magnetic field.

The reed has a certain inertia, which causes a slightly sloped movement at the onset of the cycle; it also bounces off the mouthpiece at extreme positions, creating a small ripple. At 1 Hz these artifacts are barely noticeable; however, with rising frequency, they consume a gradually larger portion of the cycle, resulting in the pattern of reed movement changing from a pulse wave to a nearly sawtooth wave, which is reflected in the instrument’s timbre.

2.3 Body

While in a traditional woodwind the body acts both as a container of the resonating air column and as an acoustic horn, in the P-D Sax only the latter function is preserved - although the instrument’s body doesn’t influence the frequency of reed oscillation, it still influences the volume and timbre of the sound.

Apart from acoustic considerations, both the ergonomics and aesthetics of the instrument were taken into account. It was inspired by the Paetzold recorder [23] and Bart Hopkins’ wooden saxophones [21] . It was designed in Autodesk Fusion 360 software and built from plywood. The body forms a closed pipe approximately 94 cm long. The circular hole in the front wall can be equipped with a 3d-printed bells.

2.4 Electronics and user interface

The electronic part of the instrument is based on a Bela Mini board, running a custom Pure Data patch; analog and digital GPIO are used to provide control signals from the user interface. As the Bela provides only 8 analog inputs which are used for the keys, a Teensy 3.2 board in a midi controller mode is used for additional 2 analog inputs, necessary for the thumb joystick.

A set of 3D-printed keys forms the basis of the user interface. The eight keys on the front are based on a typical recorder layout; the lower six are responsible for the diatonic scale; the top two transpose the notes up and down a semitone to provide a chromatic range. The keys are continuous, enabling smooth note transitions. Each key mechanism has three embedded magnets, two of which act as a spring; a third one is placed directly above a SS49E proportional hall sensor - when the key is depressed, the readout of the sensor changes proportionally to the key’s position (a similar approach have been used by Conforti & Güsewell [23] ). A custom rocker switch enables up/down octave transposition, giving the instrument a 3-octave range. As the prototype 3 was primarily meant to be tested and played by myself due to Covid 19 restrictions, it was adjusted to my personal needs by arranging the keys in a left-handed layout.

As the line output of Bela Mini provides the voltage range of 0-2.3V, it was chosen over the headphone output, as its 0V-based range is more suitable for driving the electromagnet. The signal is amplified by a BD241C NPN bipolar transistor in a simple circuit.

All the electronic components are connected by a custom PCB. Additional PCBs are used for the hall sensors and tact switches, used as the key sensors and octave switches. The instrument is powered by a 5V power bank.

As the vibration of the reed is slightly audible even without action on the player’s part, an endstop switch with an additional lever, pressed by the upper lip of the player, is attached to the mouthpiece. Apart from muting the reed vibration when unnecessary, the lip sensor also provides the gate signal at the beginning of each phrase, which is currently used to control the phrase looping feature.

2.5 Software

The P-D Sax uses a simple Pure Data patch; its main function is to interpret the data from the sensors and to generate a sound wave of appropriate frequency, which is then amplified and sent to the electromagnet. 

The signals from the keys’ hall sensors are mapped to the 0-1 range and translated into midi notes. Currently, the instrument is tuned in twelve-tone equal temperament, but the software may be adjusted to accommodate any other tuning system.

Depending on the position of the thumb joystick, the oscillator can be controlled either directly by the signals from the keys, or by data stored in a loop. While the joystick is in the default position, each gate signal from the lip sensor engages a new loop capture; when the joystick is moved to the right, a gate signal restarts the playback of the recently captured loop. The loop playback speed can be manipulated with the joystick; the looped notes can be transposed using the keys.

3. Results

Prototype 3 of the P-D Sax is a fully functioning, playable acoustic embedded instrument. To test the influence of different sound horns and bells, as well as different playing techniques on the timbre and loudness of P-D Sax, samples under various conditions have been recorded and evaluated. The experiments showed that the instrument produces a complex waveform, with both even and odd harmonics present; the timbre and loudness of the instrument are affected by different sized sound horns and bells, and by changes in the embouchure. Waveforms and spectra of the samples, loudness measurements, as well as sound samples are included in the appendices A, B, C and D.

According to the formula f = v/λ the resonant frequency of the P-D Sax body is 91.22 Hz. Nevertheless, as the instrument’s pitch isn’t determined by the length of the air column, a lowest note was set at 23.12 Hz (F#0), which is just above a typical lower limit of the human hearing range. A pipe of a traditional woodwind instrument would have to be almost 4 times longer to achieve such a low note. The side effect persists in the timbre of the instrument, which has a slightly “nasal” quality, as the lowest frequencies aren’t properly amplified (as is the case with the viola [24]). As shown in the Appendix D, formant frequencies of the instrument are centered around 140, 300 and 500 Hz. Nevertheless, the fundamental frequency is always present.

The highest octave of the instrument’s range (F#2 - F#3) gets gradually more difficult to play, with high air pressure needed to voice a note even in the piano dynamics. This is most likely caused by aforementioned inertia of the reed.

The P-D Sax has been used for a single, unamplified live performance in a small venue and the sound volume was sufficient. It was also tested by playing along alto saxophone and bass clarinet. While it’s noticeably quieter than the former, its loudness is sufficient to play in duet with the latter; the timbres of both instruments seem to complement each other.

Apart from the aforementioned evaluation, three expert woodwind players have been invited to express their opinions and observations. Each of them spent approximately 1 hour with the instrument; they were free to ask any questions and were provided with all the necessary information. The left-handed arrangement of the keys was an initial obstacle for all three experts, but they managed to get over it and focus on other aspects of the instrument. Their general observations were that the simplified keywork and effortless embouchure make it a good entry-level instrument; however, for a skilled player these aspects form obstacles, which would take time to get used to. The obstacles, as well as the unusual, acoustic timbre might be seen as inspiring challenges, though. The continuous key interface was seen as an advantage to both the acoustic saxophone and EWI, to which the P-D Sax was compared favorably. Detailed expert opinions are presented in the Appendix E.

4. Future work

According to the evaluation, certain technical aspects of the instrument need further refinement. The “nasal” timbre of the instrument might be treated as a part of its character, but the underemphasis of the low frequency range might be improved by a larger sound horn. The high register issues can be addressed by adjusting the pulse wave duty cycle to overcome the reed’s inertia. Regarding the experts' comments, the mouthpiece needs to be modified by providing a wider adjustment range to fit various embouchures. The interface could benefit from providing additional keys, which would make it more familiar to saxophonists.

The instrument will be used for a musical project as a part of a small ensemble, offering opportunity to develop the software by tailoring the digital augmentations to specific musical needs of the project.

Potential usefulness of the instrument for children, beginners and manually disabled musicians requires further study.

Ethics Statement

The research outlined in this paper involved three expert woodwind players, male and female, from different musical backgrounds. They expressed full voluntary consent to take part in the research and were assured that they will remain anonymous and that no sesnitive data will be collected. They were informed on a way in which their participation will be included in the paper. They were free to ask any questions an received all the necessary information about the method and merit of the study. The instrument has been disinfected between the consultations to ensure the participants' safety.

Materials used in the prototypes are mostly eco-friendly (wood, plywood, aluminium, PLA filament). The prototypes utilize Pure Data, which is an open source software.

Appendix A

Effect of different sound horns and bells on the timbre of the P-D Sax

Waveforms and spectra of normalized P-D Sax samples, recorded with different sound horns and bells: A) mouthpiece only, B) mouthpiece with the prototype 2 sound horn, C) mouthpiece with the prototype 3 sound horn, D) mouthpiece with the prototype 3 sound horn + hyperbolic bell, E) mouthpiece with the prototype 3 sound horn + pear-shaped bell, F) bass clarinet, added for reference. All samples were recorded with Microtech Gefell MK 250 capsule and MV 220 P48 preamplifier from approximately 23 cm distance with similar dynamics.

Appendix B

Effect of lip pressure on the timbre of the P-D Sax

Waveforms and spectra of normalized P-D Sax samples, recorded with different lip pressure, from A) softest, to E) hardest. All samples were recorded with Microtech Gefell MK 250 capsule and MV 220 P48 preamplifier from approximately 23 cm distance with similar dynamics.

Appendix C

Formant frequencies of the P-D Sax

Sonogram of the full chromatic scale of the instrument, recorded with Microtech Gefell MK 250 capsule and MV 220 P48 preamplifier. The resonant frequency of the body alone has been determined at ~90 Hz, but according to the sonogram, the formant frequencies of a complete instrument are centered around 140, 300 and 500 Hz. Even though a square wave is used to drive the reed, the resulting soundwave has a more complex shape; both even and odd harmonics are present in the spectrum of the sound, whereas in a pure square wave even harmonics would be missing.

Appendix D

Effect of different sound horns and bells on the loudness of the P-D Sax

Relative loudness in Db RMS of P-D Sax samples recorded with different sound horns and bells: A) mouthpiece only, B) mouthpiece with the prototype 2 sound horn, C) mouthpiece with the prototype 3 sound horn, D) mouthpiece with the prototype 3 sound horn + pear-shaped, E) mouthpiece with the prototype 3 sound horn + hyperbolic bell. All samples were recorded with Microtech Gefell MK 250 capsule and MV 220 P48 preamplifier from approximately 23 cm distance with similar dynamics.

Appendix E

Results of the consultations with expert woodwind players

Observations of expert A (classically trained alto saxophone player):

• simplified keywork seen as an obstacle, he would prefer it to resemble the Boehm system more closely, which would require additional keys for semitones, although existing p-d sax keywork could be remapped to better reflect the Boehm system;

• the loose embouchure might be more familiar for double-reed players, but is difficult for a single reed player at the first glance; it might get some time to get used to; it also might be responsible for diminished dynamic range, as compared to the saxophone;

• on a saxophone, holding the mouthpiece firmly with mouth/teeth also stabilizes the instrument and leaves more freedom for finger movements, which is impossible on the P-D Sax;

• on the other hand, the embouchure doesn’t require much effort so it might be useful for amateurs and beginners;

• the neck profile might be to narrow; widening it might positively affect the dynamic range;

• a harder reed might improve P-D Sax’s performance in the high register;

• the fact that the body in the instrument acts as a sound horn and affects the volume and timbre is a beneficial feature;

• the continuous keys, allowing smooth note transitions, are an advantage compared to both the saxophone and the EWI;

Observations of expert B (experienced jazz and pop alto and soprano saxophone player with classical training; also a music teacher):

• at a first glance, a seasoned woodwind player finds it difficult to produce sound, regarding the habit of tight embouchure and “biting” the mouthpiece at the tip; inserting the mouthpiece deeper into mouth, which is required to properly produce the sound on the P-D Sax, feels unnatural, as well as more prominent reed vibration does;

• playing with articulation other than legato is difficult, it would need getting used to

• she compared P-D Sax favorably to EWI which she considers uncomfortable both in terms of the touch key interface and the highly resistant breath sensor

• the octave switch and the key interface feels unintuitive, although it can be gotten used to 

• when asked if she would consider using the p-d sax herself, she answered that she could see a limited use as an instrument for experimental and world music as it has an unusual timbre;

• she sees its strength mostly as an educational instrument - small weight, simplified diatonic keywork and the ease of embouchure makes it easy to start playing, as compared to traditional woodwinds’ steep learning curve; she commented that “children would love it”;

Observations of expert C (Bb, bass and contrabass clarinet player, with experience in contemporary music performance and free improvisation):

• the P-d sax has an exceptionally low pitch for its size;

• while testing the p-d sax himself, he almost immediately found the right embouchure;

• the interface is immature as compared to the clarinet, on the other hand its simplicity might be inspiring;

• he observed that for a creative musician, working with such an instrument, which poses various challenges in terms of unusual playing technique, might be more inspiring than trying to find new features in a well known instrument;

• when asked if he was interested in spending more time with the instrument and how it would fit in his toolset, he answered that he would immediately start thinking of compositions that would use specific abilities of the P-D Sax, such as unusual sounds and effects;

• he observed that the P-D Sax and bass clarinet sound good in duet and that the P-D Sax sounds indeed like an acoustic instrument;

• he considers the timbre of the P-D Sax similar to some folk instruments, especially bagpipes;

• he considered the P-D Sax more interesting than EWI, which he feels doesn’t have much advantage over keyboard instruments;

• he repeatedly observed that the P-D sax is a perfect transition in between the EWI and acoustic woodwind instruments, the fact that the horn and bells change the sound of the P-D Sax is an advantage over fully electronic EWI; 

• the continuous keys are an advantage both compared to EWI and bass clarinet;