Skip to main content
SearchLoginLogin or Signup

Dialogic Design of Accessible Digital Musical Instruments: Investigating Performer Experience

An introduction and commentary on dialogic design approaches to building ADMIs by critiquing commonly used participatory design methods

Published onJun 16, 2022
Dialogic Design of Accessible Digital Musical Instruments: Investigating Performer Experience
·

Abstract

While it is accepted that accessible digital musical instruments (ADMIs) should be created with the involvement of targeted communities, participatory design (PD) is an unsettled practice that gets defined variously, loosely or not at all. In this paper, we explore the concept of dialogic design and provide a case study of how it can be used in the design of an ADMI. While a future publication will give detail of the design of this instrument and provide an analysis of the data from this study, in this paper we set out how the conversations between researcher and participant have prepared us to build an instrument that responds to the lived experience of the participant.

Author Keywords

ADMI, DMI, accessibility, dialogic design, experience

CCS Concepts

Applied computing Sound and music computing; •Hardware Sensor devices and platforms; •Human-centered computing Accessibility design and evaluation methods;

Introduction

Existing literature [1][2][3][4][5] suggests that participatory design (PD) [6] is essential for the design of accessible digital musical instruments (ADMIs). We set out to query what researchers mean when they say they use PD, since it is an unsettled practice [7] that gets defined variously, loosely or not at all [8]. In this paper, we subscribe to the concept of dialogic design [9], exploring how it is a departure from commonly-used PD methods while getting back to PD’s radical roots, and provide a case study of how it can be used in the design of an ADMI.

In this paper, we also ask ourselves why so much of design and engineering practice is preoccupied with specifications. We set out that a fixed problem statement in a conservative engineering approach can lead to medicalising the disabled performer and misses the actual experience of music making [10], by being too preoccupied with information transfer and control.

The study described in this paper is concerned with the design of a guitar-like instrument off the back of Strummi [11][12][13]. Below, we give a glimpse of the dialogic design conversations between the first author and the guitarist participant/collaborator which have prepared us to build the artefact, and provide a discussion of the usefulness of dialogic design for ADMI design.

Background

Participation is an important topic for the HCI and NIME communities, with designers moving progressively closer to the users/performers of their designs [6]. However, there is a need for more literature on (a) one-to-one design scenarios where researcher and participant attempt to share the designer role equally, and (b) co-design processes in the design of ADMIs1 or disabled-artist-led DMIs.

Below, we critique what PD has become in DMI design – an umbrella term for any sort of user involvement, which often gets mentioned by researchers without drawing specific lines of influence between user input and design decisions. This critique is made challenging by the lack of detail about the specifics of participatory methods in research publications. Furthermore, we look at why accessibility is an area in which participation on both sides is of crucial importance, and point to dialogic design, as a subset of PD which highlights the significance of multidirectional flow of ideas, getting back to PD’s radical origins of workplace user involvement in Scandinavia [14].

ADMIs and Participatory Design

The case study we provide fits in the growing field of ADMI design. Examples of work in this area include a review of publications focusing on ADMIs [1], a thesis on the ability of ADMIs to address technical and social barriers that prevent access to music making [15], and a NIME paper that advocates for the formation of communities of practice for fostering the longevity of ADMIs [2]. Frid’s review of ADMIs states that most of the included instruments were created through some form of PD. To better understand the state of affairs, we have looked at how some of those and other examples employed PD:

  • The WamBam [16] is “a self-contained electronic hand-drum designed for music therapy sessions with severely intellectually disabled clients”. The paper explains they used a co-design process involving interviews, brainstorming sessions and user testing a prototype.

  • MEDIATE [17] is “a multi-sensory environment design for an interface between autistic and typical expression”. The paper explains they involved a 10-year-old boy and his mother in trials of designs and testing.

  • A collaborative music system [18] based on a touchscreen controller, designed for persons with dementia. The paper mentions the collaborative design of the music controller.

  • LoopBlocks [19] is “a tangible wooden step sequencer that uses photoresistors and wooden blocks to trigger musical events”. The paper explains they based their design on a qualitative interview study with teachers of German Special Educational Needs (SEN) schools and they outline the criteria mentioned in those interviews.

  • Kellycaster2 [15] is a bespoke accessible guitar-based DMI which John Kelly uses in his practice as a performing musician. The video is a thorough exploration of collaboration in the context of DMI design. They explain how the dialogue between John Kelly and Charles Matthews, with the support of Drake Music3, led to the instrument design.

  • Instrument One and Instrument Three [20] are a bespoke, motion-controlled digital guitar and a touch-sensing accessible violin. They were each designed in conjunction with a disabled musician, who “used the instruments over two months to help identify factors contributing to sustained use”.

While some of the above publications provide better links between user involvement and design decisions, we find that the details of participation are underspecified in many papers. While this might reflect constraints of space or prioritising other aspects of the design, if the mode of participation has not been seen fit to specify, this should be considered and reported.

Disability and Dialogic Design

The focus on dialogic design is prioritised by the need to take into account the 'Nothing About Us Without Us' [21] adage from disability rights activism, which encapsulates the idea that disabled people know what is best for them and should be involved in any decisions that concern them [22]. This adage responds to colonialist approaches to technology and design that put responsibility on the disabled person to the benefit of the non-disabled [23]. By paying attention to disability culture and politics, the project outlined in this paper seeks to propose an equitable approach that comes from the community instead of 'fixing' a problem perceived by the non-disabled community. This approach lies in harmony with the social model of disability, which shows us that it is the environment, society and culture that surrounds someone that creates barriers and obstacles which disable and exclude them [24].

Furthermore, this paper explores what it means to do PD and what an appropriate equilibrium of participation is. I propose to use the existing term 'dialogic design' [25][9][26] to refer to a middle ground between participationism [25] on the one hand and tokenistic forms of participation on the other. Participation-ism can be understood as the tendency of the designer to become an administrator, a mute facilitator, a post-it note [25]. This is perhaps the naive idea that the designer can conduct design without exerting any influence over the design product. Tokenistic forms of participation can be understood as an abuse of empathy as a design tool. These are seen in examples of human-centred-design based on the designer's idea of what it is like to be like a particular human, but without being with [27] the humans who are stakeholders in the design. Dialogic design, in the middle, comes from the words ‘dialog’ and ‘dialogics’. We embrace this term because it highlights that there are ideas that flow in different directions. In dialog, the substance of the contribution comes from two people bringing different perspectives to a conversation or musical engagement, reading into each other “meanings and possibilities that each alone could not see” [9](p. 574).

From Specification to Experience-Focused Design

Finally, this study delves into how design for disability can and should be based on experience rather than solely on specification. Through conversations with our participant/collaborator, we seek to articulate his experience of being a guitarist and music making [10], which has informed and guided our design. This responds to a conservative engineering approach which is concerned with "going from a fixed problem statement (or requirements specification), to an abstract specification of the solution" [28](p. 3). Instead of trying to make music a problem to be solved, this study aims to account for the diversity of music and providing for fun, curious and taskless interaction [29][30][31].

Methods

Our London-based lab was approached online by an Australian musician looking for a guitar-like instrument that would be suitable for his access needs relating to nerve damage, and after the Strummi [11][12][13] being suggested as an option (with some modifications), I (the first author) was put in touch with him due to the relevance of this request to my research project. After accepting a proposal to take part in this study, we agreed the participant, Victor Chiruta, would receive the instruments resulting from this study and he requested to be named rather than pseudonymised.

The study so far has been comprised of seven design conversations over the course of 13 months. The seven sessions have been held remotely between the participant and researcher (the first author). These one-to-one sessions were recorded and agreement to take part implied consent. The conversations were based on a dialogic design approach, aiming to involve the participant at every stage of the design process. The conversations were similar to semi-structured interviews and 30 minutes to one hour in duration each. Informed by the conversations, prototypes according to what we agreed were made and shipped to the participant. After each conversation between researcher and participant, we recorded a conversation between both authors, which served as an opportunity for reflection and ideation. The sessions were divided into the following two phases:

  1. Scoping & exploration: This phase took place between October and November 2020 and was composed of four design conversations. It involved an evaluation of the participant's access needs, abilities and interests, and an exploration of the cultural roles of the instrument to be preserved. This was followed by a discussion of a ready-made instrument – Strummi (the replication of which was documented in [32]), which was sent the participant, for him to play for the coming months. Cardboard mock-ups, which served as a generative toolkit to prototype possible instrument layouts, were also sent to the participant.

  2. Ideation & making: This phase happened between January 2021 and November 2021. So far, we have held three design conversations, with one outstanding conversation currently delayed for reasons personal to the participant. The researcher is currently making the finished instrument based on everything agreed so far and will send it to the participant, who has been actively involved in the decisions and will report back about his experience.

The current point in this project is opportune for the publication of our findings and discussion relating to dialogic design, while a future publication will focus on the design specification, an evaluation phase, and analysis of all the data collected from our sessions.

Glimpse of the Conversations

Below is a list of the seven sessions so far, which are all design conversations. After each session, I (first author) have gone through the recording and made a summary of what was discussed. These are some of the condensed outcomes from those summaries, with key takeaways in bold:

  1. 26th October 2020. An introduction to the project and each other. Victor described how trying to play guitar again had “felt nice and really good” just holding it despite it being hard to pluck and make chords. His focus was “to play around people and have a jam”. He stressed how it was important to be able to play melodically and rhythmically.

  2. 4th November 2020. A discussion on what it means to be a guitarist. I tried some micro-phenomenology-inspired [33][34] interview questions, which worked well in putting Victor in an evocative state and reminisce having fun playing guitar. The result was a very detailed account of an iconic moment in his past as a guitarist. Victor expressed his interest in the instrument having batteries, so he could play out and about like in the anecdote he had recollected, because his happiest moments playing guitar had been out with mates out camping or on a rooftop. He explained that the accident that had led to his impairment had happened four years ago. As a result, he had reduced mobility and strength resulting from these injuries.

  3. 9th November 2020. An opportunity to set objectives. I asked Victor to mimic how he would play the instrument we are developing. After gesticulating exactly like playing guitar, he said “I still want it to kind of resemble a guitar even if the functionality might be completely different but I would like that shape ’cause when I play with other people I want it to be like I’m playing guitar, not like I’m playing a different instrument”. We also discussed how the instrument should sound like, with Victor saying that it should resemble the sound of an acoustic guitar.

  4. 24th November 2020. A discussion of digital guitars and iPad apps. We weighed the ups and downs of having keys, a touch sensor or strings for the right-hand interaction. Victor commented that Garageband was quite slow to use with his finger movements and did not realise he could strum the virtual strings. Victor said he had realised that playing chords would be more important when in social situations like playing in the background to their friend’s singing and plucking notes would be more of an activity to do on his own.

  5. 27th January 2021. Initial impressions with Strummi and mock-up pack. Victor had been quite confused with the button chord mapping. He said the string sensitivity was very appropriate and excited him a lot about having a similar guitar he could use quite comfortably. He said plucking is actually how he had been using it, since he could not hold his fingers together to make strumming possible. The first thing he would change about it is making it into a guitar shape. Regarding the mock-up pack, it helped making a decision over having a full-length guitar neck. He said it would make sense for him to have the sensors sit diagonally on the neck.

  6. 9th March 2021. Discussion on videos of Victor playing Strummi and an electric guitar. Victor commented he found it hard to strum on both guitar and Strummi. On Strummi, fingers got caught when trying to strum and could not pluck from string to string because of his shakiness. We discussed having a resting bar for the arm over the strings as that would improve the stability of his hand. I said it might be worth trying the Strummi for a longer period to see if strumming got better. I then asked what he thought about the idea of not having to toggle between states (melodic/rhythmic) and he said he liked the idea of having a more organic instrument.

  7. 22nd November 2021. We talked about the two proof-of-concept models that I sent with Thundergut strings (thicker and softer than regular strings) and tines (larger surface area compared to a string), put together as possible to possibly make plucking and strumming easier. We discussed the right-hand interface and how a capacitive touch sensor could overcome the difficulty of strumming and plucking at the cost of losing dynamics.

Discussion

The main aspect we want to demonstrate in this paper is the usefulness of dialogic design for new ADMIs. The insights provided are only available through having followed that process. The major element to consider is that it has been a way of articulating the performer’s experience and channelling it into specific design decisions. Performers’ objectives can be quite abstract when it comes to new instruments. In our case, a running theme of the conversations was how good it felt to just hold a guitar again after acquiring an impairment and the motivation to take it up again and play with others. Through our conversations, it became obvious that Victor wants to play out and about, and he has communicated this through specific imagery, for example, of him playing with friends on the beach.

Nonetheless, dialogic design in the context of our project is about both sides of ideas. While the conversations were a way for Victor to articulate their experience and transform that into a design, the expertise of the first author also played an important part. Therefore, as well as being a careful listener, to be and become [26] with the participant, the first author brought their own ideas to the conversation and built several instruments and prototypes to be discussed, ultimately shaping the final instrument. In this sense, dialogic design led to a richer outcome than just following the performer’s specification.

Performer Experience

We have also investigated the role of performer experience in the design of ADMIs. Influenced by experience-driven innovation, rather than specification-based design, we have tried to convey how research dialogue can lead to answers to real-world situations. In this approach, knowledge-making is the result of a process led by the performer and their lived experience. As we have seen, specification-based design, or a conservative engineering approach, can be medicalised because it is a closed circle that attempts to solve someone’s impairment through a technology brief. Therefore, we believe that, in accordance with the social model, we should consider the complexity of disability when designing new instruments, and that one way of doing this is to focus on performer experience.

The conversations so far have featured exercises that we recommend to members of the NIME community who want to use dialogic design in their practice. These exercise have helped elicit performer experience. For example, asking the performer to gesticulate what their envisioned interaction would look like gives a reliable idea of the instrument’s size, shape and ergonomics. Also, asking about a specific moment in which the performer enjoyed playing music and trying to evoke that moment can lead to very rich descriptions which can then be channelled into the design. Another exercise is to ask the performer to try out and give feedback on various technology that has overlapping purpose with the instrument being designed.

From Participatory Design to Dialogic Design

We have explored the reasons why PD is often used in the development of ADMIs, with the caveat of this discipline being variously defined. As a way forward, we adopt dialogic design [25][9][26] as a specific subset of PD which accepts the designer/researcher’s influence, expressed both as domain-specific expertise and pre-existing artefacts, in the process and outcome of the design, while rejecting tokenistic forms of participation. We argue that more attention should be given in publications to the concrete ways in which we practice dialogic design.

Acknowledgments

We thank Victor Chiruta for his involvement as design partner throughout this project. This work is supported by the EPSRC and AHRC Centre for Doctoral Training in Media and Arts Technology (EP/L01632X/1) and by the Royal Academy of Engineering under the Research Chairs and Senior Research Fellowships scheme.

Ethics Statement

The participant has consented for our conversations to contribute to a research publication such as this one. They have consented for the recordings to be kept according to the authors’ university’s data protection policy. Their real name appears throughout the article, as this is their preferred way of referring to them, rather than in pseudonymised or anonymised form. The participant was selected due to their initial interest in having an instrument made by our lab and their agreement to take part in this study. They are being compensated with keeping the output artefacts from our study, including the final instrument. This research was reviewed and approved by the first author’s university’s ethics board. We observe no potential conflicts of interest.

Comments
0
comment
No comments here
Why not start the discussion?