Disability studies, virtual reality, parameter mapping, accessibility, input devices
I am interested in creating an immersive digital musical instrument (also known as a DMI) in collaboration with bedridden and bed-bound people that is accessible and for pleasure and play. In addition to building an instrument that is accessible to more bedridden and bed-bound people, this research will offer a framework for other DMI creators to consider accessibility through a disability justice perspective.
The DMI prototype will use popular creative tools Unity and Max/MSP—incorporating Ilsar and Kenning’s framework of ‘remapping’ (Ilsar and Kenning 2020) by bringing the DMI into an immersive environment and using a webcam (Oliveros et al. 2011; Robidoux 2019; Dvorak and Boresow 2019). Building a DMI in an immersive environment, opposed to using a traditional instrument or even a physical DMI, can offer elements of control and expression that otherwise may not be achieved from bed. This is in part due to how popular computers are in daily interactions, and also because the control parameters can truly be altered to incorporate individual access needs, while being time and cost efficient. A headset with a microphone will be connected to the software, which will allow for Max to listen to the sound of movement and process the sound signal in a noninvasive way. The serial communication between Max and Unity will then allow for visual feedback with the DMI.
I am particularly interested in disabled and chronically ill people who are bedridden or bed-bound because 1) this encompasses many groups of disabled people without also focusing on diagnostics. Due to the inadequacies of healthcare, many disabled and chronically ill people remain undiagnosed or misdiagnosed, which would automatically exclude them if this project was to focus on a particular diagnosis such as fibromyalgia or lupus. Removing diagnostic criteria will give this project the opportunity to be more accessible to a wider audience as well in that it will incorporate the needs of people who may not have much in common with each other, other than being bedridden, 2) disabled and chronically ill people who are bedridden and bed-bound are very much an underrepresented group of people in both music technology and disability studies, 3) the access needs that many in this group have are contradictory to what DMI designers/practitioners assume one always has the capacity for. In other words, someone experiencing chronic fatigue may not have the energy or desire to engage in ‘pulling’ as it could exacerbate their fatigue. And 4) this project relates to some of the struggles I’ve had with the inaccessibility of sound art and music tech, as someone who is often bedridden due to disability and illness. Interviewees will be members of various disabled and chronically ill communities that I am a part of, and will be asked to help with a snowball sample for further recruitment. Of course, disabled and chronically ill people who do not have access to DMIs due to inaccessibility are my priority, but it is important to note that such work will benefit non-disabled people’s needs too, through “resonant design” (Pullin 2009 93).
This work also has implications for bringing music technology beyond flow. The use of metaphor has been found to be helpful in the creation of new digital musical instruments for increased transparency and control intimacy (Wessel and Wright 2002; Fels et al. 2002; Wanderley and Depalle 2004 633; Caramiaux et al. 2014) as metaphor can give the designer and user the faculty to make use of what is (thought of as being) familiar to them. The metaphors do too, however, have the capacity to make music technology even less accessible by assuming that all body-minds ‘glide’ or ‘pull’ (and so on) similarly, or at all. Metaphors used in DMI design must be rethought, especially those that relate to “flows.” In addition to creating a DMI that is accessible to more bedridden and bed-bound people, my goal is to provide a framework for other DMI creators to make their controllers an object that is for pleasure and play, rather than the sole purpose of ‘rehabilitation.’
As part of this research, I will also conduct ethnographic interviews with disabled and chronically ill people who want to use music technologies. Following the prototype stages and the ethnographic interviews—which will give me the opportunity to delve into individual access needs—I will engage in user-testing with bedridden or bed-bound people to ensure that the DMI is actually accessible, engaging, immersive, and may be incorporated in real-time performance settings (Hunt et al. 2003 429).
My eventual goal is to prototype an immersive digital musical instrument that is created for, by, and with the interviewees, which will contribute to the betterment of control intimacy for digital musical instruments as a whole. In the Doctoral Consortium, I hope to get feedback from other artists, designers, or engineers who may have particular forms of expertise that I do not. I also hope to meet other graduate students who may be interested in music research as it pertains to disability justice.
Caramiaux, Baptiste, Jules Françoise, Norbert Schnell, and Frédéric Bevilacqua. "Mapping through listening." Computer Music Journal 38, no. 3 (2014): 34-48.
Dvorak, Abbey L., and Elizabeth Boresow. "Using the Adaptive Use Musical Instrument (AUMI) in music therapy clinical practice." Music Therapy Perspectives 37, no. 1 (2019): 1-13.
Fels, Sidney, Ashley Gadd, and Axel Mulder. "Mapping transparency through metaphor: towards more expressive musical instruments." Organised Sound 7, no. 2 (2002): 109-126. Gelber, Elizabeth. "Black Oil Business." In Subterranean Estates, pp. 274-290. Cornell University Press, 2015.
Hunt, Andy, Marcelo M. Wanderley, and Matthew Paradis. "The importance of parameter mapping in electronic instrument design." Journal of New Music Research 32, no. 4 (2003): 429-440.
Ilsar, Alon, and Gail Kenning. "Inclusive improvisation through sound and movement mapping: from DMI to ADMI." In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility, pp. 1-8. 2020.
Malloch, Joseph, Stephen Sinclair, and Marcelo M. Wanderley. "Distributed tools for interactive design of heterogeneous signal networks." Multimedia Tools and Applications 74, no. 15 (2015): 5683-5707.
Oliveros, Pauline, Leaf Miller, Jacklyn Heyen, Gillian Siddall, and Sergio Hazard. "A musical improvisation interface for people with severe physical disabilities." Music and Medicine 3, no. 3 (2011): 172-181.
Pullin, Graham. Design meets disability. MIT press, 2009.
Robidoux, Matthew. "Aumi in Practice: The Adaptive Instrument Ensemble." PhD diss., Mills College, 2019.
Sedgewick, Augustine. "Against flows." History of the Present 4, no. 2 (2014): 143-170.
Wanderley, Marcelo M., and Philippe Depalle. "Gestural control of sound synthesis." Proceedings of the IEEE 92, no. 4 (2004): 632-644.
Wessel, David, and Matthew Wright. "Problems and prospects for intimate musical control of computers." Computer music journal 26, no. 3 (2002): 11-22.