What makes a mapping work well? Results from two mapping design studies.
We present new results combining data from a previously published study of the mapping design process and a new replication of the same method with a group of participants having different background expertise. Our thematic analysis of participants' interview responses reveal some design criteria common to both groups of participants: mappings must manage the balance of control between the instrument and the player, and they should be easy to understand for the player and audience. We also consider several criteria that distinguish the two groups' evaluation strategies. We conclude with important discussion of the mapping designer's perspective, performance with gestural controllers, and the difficulties of evaluating mapping designs and musical instruments in general.
mappings, evaluation, digital musical instruments
•Human-centered computing → User studies;
•Applied computing → Media arts; Sound and music computing;
A digital musical instrument (DMI) might be viewed as composed of a control interface, a sound synthesizer, and a mapping that connects the two [1]. This model of the instrument suggests that the careful design of the mapping is at least as important as the design of the control interface and the sound synthesizer [2]. Numerous studies provide evidence that this is true, showing that the mapping design affects how an instrument feels [3], how it engages the player [4], and even how the audience appreciates a performance [5].
Although the literature is clear that mappings are vitally important, it remains unclear exactly how to design a mapping that allows the designer to achieve their design goals. Indeed, it is not clear what design goals are most common or important, let alone how mappings can be designed to achieve them. Much of the literature reports on the insights gained by digital instrument designers through their practice of making instruments, e.g. [6][7], where design criteria for mappings are implied by broader recommendations for DMI design in general. In works focused specifically on the design of mappings, the literature often focuses on the structural, topological, and technical aspects of the mapping design, such as recommending the use of intermediary layer [8], dimensionality reduction [9], machine learning [10], and other novel implementation techniques. In all of this work, the specific design goals of the instrument and/or mapping are often only implied by the context of the presentation.
In this paper, we explore mapping design criteria using a novel approach. Rather than reporting on the insights of a single or small group of designers, as is usually done in the literature, our study considers the design goals of a relatively large number of skilled NIME users when making a mapping for live performance, and the criteria that they use to evaluate whether their mapping design allows them to achieve their goals. Two groups of participants were asked to design a mapping effective for a live performance, and then to reflect on their design process and decisions. We examine what our participants considered to be important properties for their mappings to have, and consider how these criteria may change in relation to the designer's experience with the gestural controller used. Finally, we situate the role of mapping design and its evaluation criteria in the overall DMI design process.
This paper reports on novel analyses of the dataset produced in a previously reported study, combined with a new dataset produced by repeating the same methodology with a new group of participants who have additional experience using the T-Stick. The main method of the study has been explained in detail in prior publications [11][12]. In addition, details and software needed to replicate the study are available in an online appendix.
Two groups of participants followed the same study methodology. Participants were invited to make a mapping that they would consider effective for a live performance, using a T-Stick [13] as interface, a generic subtractive synthesizer for sound production, and Webmapper+libmapper [14] as an environment for making the mapping. These tools were chosen for their availability, ease of data collection, and as reasonable representatives of a gestural interface, sound synthesizer, and mapping design environment respectively. We recorded activity data with time stamps, including what signal associations participants created and removed, as well as changes to transfer functions in the associations (e.g. applying an absolute value function or scaling from the input to the output).
Both groups of participants used the same version of Webmapper, and the same T-Stick, however the closed-source commercial synthesizer plugin used in the first study (Native Instruments Massive) was replaced with an open-source Max/MSP-based re-implementation. Although the synthesizer was not exactly the same, we feel it was similar enough (practically the same) such that the difference had no influence on the results. We were motivated to make this change in order to allow the method of the study to be more readily replicated and more fully examined.
Participants were asked to take at least 20 minutes but not more than 60 minutes to make their mapping design, after which their design was exported from Webmapper using its save functionality. Participants were then interviewed about their design process and the decisions they made.
Participants in the first group (N=12) were musicians with 4 to 25 (mean 9.5) years of experience making music using DMIs. Following the first group’s completion of the study described in [11][12], a second group of participants was recruited (no member of the first group was also a member of the second group). Participants in the second group (N=8) were musicians having slightly more experience on average (5 to 21 years, mean 11 years) and were distinguished from the first group by at least 3 months of experience working closely with the T-Stick. Participants in the first group had no prior experience with the T-Stick before the study.
The results presented here stem from a thematic analysis of the interview transcriptions; the activity data are not referenced in these results. Transcripts from the first group of participants were initially analysed separately from the second group, according to the method described in [11][12]. The interviews were re-read in multiple passes, coded and re-coded several times, and finally grouped into themes [15].
In the follow up study with the 8 skilled T-Stick users, a similar approach was used to analyse the new interview transcriptions. After reading the interviews several times, several coding passes were taken, applying codes identified from the first study and introducing new codes. New codes were sorted into themes. The interviews from the first study were then re-read with the new codes in mind in case they might apply to some of the remarks made by the first group of participants.
Participants in our study were asked to design an “effective” mapping for a live performance. This term was chosen very carefully. We didn’t want to bias participants to make a certain kind of mapping or adopt a certain aesthetic or design approach, other than limiting them to the context of live performance. We imagined that participants would come with diverse ideals for performance and thus with rich and varied design goals. Our own ideas about what makes a mapping desirable might be different from our participants’ ideals. Even the term “good” could be controversial; what if a participant wanted to make a mapping that is gritty, chaotic, and unruly? That could be a valid design goal, leading to an interesting performance scenario, but it might not be reasonably described as “good”. We also wanted to avoid terms like “expressive” that come loaded with a lot of cultural and historical connotation that might preclude certain kinds of design approach.
We settled on the word “effective” reasoning that something is “effective” when it meets the relevant criteria for evaluation. The term “effective” thus only implies that the design should meet the criteria for evaluation, and is reasonably neutral with respect to what those criteria should be. We recognize that even this choice might be controversial. The term could have been (mis-)interpreted by the participants in the study to imply a certain kind of mapping design. It may be that no prompt is acceptably neutral to all concerned parties. Let it be noted that our terminology was carefully considered, and that the choice of the term “effective” was intended to imply as little as possible about how the mapping design should be approached and evaluated.
In hindsight, perhaps we should have simply asked participants to “design a mapping for a live performance,” without qualifying what kind of mapping to design. Future research may take this approach. However, the intentionally neutral use of the word “effective” remains useful for discussing the results of such research.
Some direct quotes from the interview transcripts are presented below along with the participant’s anonymized numeric ID. Participants in the first group are numbered 3 through 15 (participants 1 and 2 were pilot study participants, and participant 5 was disqualified for lack of experience). Participants in the second group are numbered 20 through 27, allowing them to be easily identified below. The full interview transcripts are available in the online appendix for reference.
Based on the thematic analysis, we identified three themes that were common to both participant groups: control, legibility, and sound. These themes were first identified in the interviews with the first group of participants and were found to apply strongly to the second group as well, suggesting their generality regardless of prior experience with the control interface used in the mapping design task.
The first theme identified relates to the way the mapping design gives the player a sense of control. 15 out of 20 participants (9/12 from the first group and 6/8 from the second) mentioned the way in which the mapping allows the player to control the synthesizer as an important aspect of a mapping’s effectiveness. Some of these remarks had the quality of establishing the precise meaning of "effectiveness" as the participants understood the word:
I guess a good mapping is a system in which the output meets your intentions. (Laughs), that seems obvious, but the details of that are kind of... so varied. (Participant 10)
These remarks establish that, in the context of live performance, participants wished to deliberately perform specific actions with the intention of producing specific results as a consequence. The instrument and the mapping serve to empower the player with musical agency and allow them to make the specific sound they intend to create.
Although none of the participants in the study completely rejected the desire to control the sound, some also offered another perspective:
It has certain things that are controllable. [...] But it also is a little bit chaotic, and a little bit unpredictable, and that's actually a nice thing for me, for the type of music I would make. (Participant 7)
It has to be simple enough. So I still have control and kind of know what I'm doing. But at the same time, it's fun to have a little bit of chaos and have some sounds that I... I wouldn't know how to redo but somehow I made them. That's kind of nice, too. (Participant 24)
In this case, these participants (6 in total, 2/12 from the first group and 4/8 from the second) wished for the instrument, as well as the player, to have some measure of musical agency. Interestingly, the participants in the second group, with more experience using the T-Stick, were more likely to express appreciation for the instrument’s capacity to surprise and act unexpectedly.
This theme reveals the importance of control in a live performance. Although the participants in our study emphasized that control should lie primarily with the player, they also highlighted that the instrument itself may be given some control, depending on the aesthetic goals of the designer and/or player. The balance of agency between the player and the instrument is therefore identified as a crucial aspect of the mapping design and of instrument design in general.
This theme relates to how easy the mapping is to understand, both for the player and the audience. Others have described this feature as "transparency" [16], but we prefer the term "legibility". "Legibility" implies that the mapping is a kind of text that the player and the audience read and interpret; we feel that this accurately reflects the complex social and embodied relationship of these stakeholders with the instrument and mapping, whereas "transparency" seems to imply some kind of physical or innate property of the mapping that is separate from its interpretation by the player and audience.
Every participant in our study identified legibility as an important property for an effective mapping.
It’s fairly simple. The public has to quickly understand the link between the gesture and the sound. That’s it. (Participant 15)
There were three main expressions of this theme: intuition, metaphor, and correlation. 15 out of 20 participants (9/12 from the first group and 6/8 from the second) expressed that an effective mapping makes sense intuitively, without having to think about it.
Something that I think is important for mappings is clarity. I want to be able to do something and it's clear to somebody who isn't necessarily trained in these things. They can understand that something has happened in terms of what they can hear, and it's not obscure; there is a gesture happening and you know there is something changing in the sound and it’s clear for the audience. (Participant 27)
11 out of 20 participants (8 in the first group and 3 in the second) employed some kind of metaphor or analogy to explain how they made sense of the mapping.
It’s nice actually to be able to squeeze the instrument and for it to be silent, it’s kind of like a resonating body, if you squeeze it it dampens it. (Participant 7)
8 out of 20 participants (6/12 in the first group and 2/8 in the second) found it effective when the perceptual effect of the gesture had a clear correlation with the resulting sound.
What I found worked well was correlating the high intensity motions to high intensity sonic output in my opinion, or in my perception of it. (Participant 10)
However they explained it, 19 out of 20 participants highly valued a mapping that made sense to them, that was easy to understand and seemed like it would be easy for the audience to understand as well. This also relates to the theme of control. With the theme of legibility, participants wished to be able to apprehend the cause-and-effect relationship between a gesture and the sound it produced. With the theme of control, they wished for the cause-and-effect relationship to exist in the first place.
However, beyond cause-and-effect, this theme also implies something deeper about what makes a mapping effective. If music is meant to communicate something, it seems clear that the mapping (as part of the overall musical system) plays a role in this communication. A legible mapping is one which facilitates communicating both how an instrument works, and how the performer's gestures relate to the sound the instrument makes.
Remarks about legibility dominated participants remarks about the effectiveness of their mapping designs, accounting for 41% of all the remarks from both groups that were coded under the broad category of effectiveness of mappings.
The last shared theme between the two groups is considering the sounds that the mapping facilitates making. It comes as no surprise that when designing a mapping for a live musical performance that participants would consider the sounds produced by the instrument. Although there was little consensus among participants about exactly what kind of sounds to make (indeed, no two participants made the same exact mapping), 13 out of 20 participants (9/12 from the first group and 4/8 from the second) specifically remarked about the sound, demonstrating that this was an important consideration in the mapping design process.
Two themes were expressed repeatedly by participants in the second group but were found only infrequently in the remarks of the first group.
It is clear from the second group of participants’ remarks that their previous experience using the T-Stick had an important effect on their design process and evaluation criteria. 7 out of 8 of these participants used their prior experience to guide their decision making process. In most cases, drawing on their prior experience allowed participants to make mapping decisions that they knew would work well for their purposes.
Because of my previous attempts, working with the T-Stick […] I had a good idea of why I want to use this, and maybe you could check this with the data captured, but that might be one of the first ones I might have used also. (Participant 26)
In terms of like doubling it and knowing that works, this is totally cheating because this is something I did a lot with the sopranino T-Stick when I was mapping for that piece. (Participant 27)
However, 3 of these participants expressed wanting to avoid repeating their previous mapping design choices, and instead chose to do something different during the study.
In my project, I use the pressure to control amplitude. I just wanted to try something else, I guess. But this is yeah, this is quite effective. Shake. (Participant 24)
In either case, participants demonstrated that prior experience plays an important role in how designers evaluate their design choices. This comes as no surprise. Indeed, participants’ use of metaphor (described above) to explain how the mapping they made is legible demonstrates the important role of prior experience in a broader sense. However, none of the participants in the first group explicitly remarked about the role of prior experience in their design process, demonstrating that specific background in using the interface at hand in the mapping design is especially important.
5 out of 8 participants in the second group described ways in which the T-Stick’s gestural signals played an important role in their design evaluation process. This is in contrast with only 2 out of 12 participants in the first group who mentioned the importance of the gestural signals themselves.
What would be useful is, in terms of data flow, I need something that is quite stable. Like, this is stable to me (squeezes). Because it's a clear gesture, and it does, if I see the data, it's very stable. (Participant 21)
A handful of other themes were identified related to the way participants evaluated their mappings. These themes were less prominent, being remarked on by fewer participants overall, but often enough that they should be mentioned.
9 out of 20 participants (5/12 from the first group and 4/8 from the second group) described using specific implementation techniques, such as scaling, convergent mappings, and non-linear transfer functions, as being an important part of what made their mapping effective.
I think the biggest evolution was just tweaking the scaling, both of the input, and the output, and some of the expressions, to be able to get something that for me sounded a little bit less random and a little bit more controlled and repeatable. (Participant 12)
If I had all the time in the world? I would from the patch that I have right now, I would go in further. And I would start tweaking the range of each variable to pinpoint areas that would be very effective, because right now I think that the patch, there's a lot of range in it. (Participant 23)
8 out of 20 participants (4/12 from the first group and 4/8 from the second) stressed the importance of controlling amplitude. Many of these remarks emphasize the importance of being able to silence the instrument.
I want to be able to start and stop the sound easily without having to think too much about it. (Participant 4)
I wanted to--when you stop, when you weren't touching it, for it to be silent. I wanted there to need to be some energy injected into the system for it to make sound. (Participant 7)
I need a clear parameter to stop the sound. (Participant 21)
Finally, 5 out of 20 participants (4/12 in the first group and 1/8 in the second) emphasized that what makes a mapping effective depends on the subjective preferences of the mapping designer, and the context for which the design is intended to be used.
Depends on the context, obviously. Yeah, so there isn’t a single answer to this question1, even for me, let alone everybody. So it’s a hard question to generalize. [...] I get that it’s a crucial question. And again, I want to reassert, there’s not a single solution to this because lots of different instruments exist, and it’s nice to have the diversity. (Participant 7)
Based on the thematic analysis of participants’ interview transcription, we are able to identify three important qualities that participants valued in mappings meant for live performance, regardless of the amount of prior experience they had with the gestural controller used in the study.
An effective mapping for a live performance should consider the balance of musical agency between the player and the instrument, primarily empowering the player to perform specific sounds as they intend to, but perhaps sometimes allowing the instrument to behave unexpectedly. Furthermore, an effective mapping for a live performance should be easy to understand, for the player as well as the audience. This legibility may come from a clear metaphor or analogy, from a distinct correlation between the percept of the gesture and that of the sound, or perhaps from simple ineffable intuition. Finally, an effective mapping for live performance must allow the performer to produce effective sounds, although determining exactly what sounds are effective is outside the scope of this work. Participants in both groups also emphasized, though to a lesser degree, the usefulness of implementation techniques such as scaling and convergent mappings for designing effective mappings, as well as stressing the importance of being able to control the output amplitude of the instrument.
The strongest theme in our participants remarks was that of legibility: an effective mapping is easy to understand for both the player and the audience, whether by intuition, metaphor, or apparent correspondence. That this was the main quality by which participants evaluating their mappings implies something interesting about their perspective on the mapping.
Doug Van Nort and colleagues [17][18] identified that there are three main views of mappings useful for designers of mappings and mapping tools: a systems-oriented perspective, a functional perspective, and a perceptual perspective. If our participants had adopted a functional view of their mapping designs, we would have expected them to remark on the topology and dimensionality of the mapping. Such remarks were completely absent, perhaps unsurprisingly given the predominantly non-technical background of our participants. On the other hand, given a systems-oriented view of their mappings, we would anticipate a large number of remarks about the structure of the mapping (e.g. one-to-one, divergent, convergent), and the connections chosen. Although participants did often mention specific connections, and the theme of technical implementation also suggests a systems-oriented perspective, these remarks were often made while otherwise justifying the effectiveness of the connection e.g. by its legibility or controllability. Participants would also have been biased towards such remarks, since Webmapper presents a strongly systems-oriented view of the mapping.
Instead, it appears that participants were largely concerned with the perceptual impact of their mapping design choices. What mattered most to them was not the structure or topology of the mapping, but the resulting look and feel that the mapping imparted on their instrument.
The theme of control suggests that the design of the mapping can empower the performer with musical agency, but it can also allow the instrument itself to act unexpectedly, lending the instrument a certain amount of musical agency as well. That designers would value circumstances in which the instrument adds “a little bit of chaos” (Participants 7 and 21) is especially interesting compared to traditional acoustic musical performance. Imagine practicing a sonata on a piano that sometimes adds a little bit of chaos. These remarks suggest something unique about performance with gestural controllers that is not necessarily shared with other kinds of musical practice. A digital musical instrument has the potential to act in the role of a collaborator in a way that acoustic instruments do not, or at least do differently.
The theme of legibility also suggests something interesting about this performance context. It’s not unusual for instance that a piano player’s hands should be invisible to much of the audience without seriously detracting from this part of the audience’s enjoyment of the music; the player’s gestures, while certainly adding something, are not an essential part of the performance. Yet participants in this study strongly emphasize that the gestures made when performing with the T-Stick should have a clear and meaningful relationship with the sound, suggesting that the gestures of the T-Stick player are somehow a more essential consideration than e.g. those of a piano player.
The themes of control and legibility may be considered as criteria for live performance itself as much as they are criteria for designing mappings meant for that context. This relates to the theme of subjectivity identified in the interview transcripts. Design criteria are inherently context and goal dependent. It may well be that designer's evaluation criteria are relatively independent of the interface, sound synthesizer, and design tools they use; instead, design intent and context of use may have a greater influence.
If this is the case, we might expect practitioners to describe similar evaluation criteria based solely on a stated design goal and context, even without the rigmarole of asking them to actually perform a mapping design. This could be verified by simply interviewing DMI designers about their mapping design goals and evaluation criteria outside the context of a user study.
It's important to insist on the limitations of the results presented here. Most significantly, participants had relatively little time to develop their mapping designs (20 - 60 minutes), and they were asked to make a mapping for a live performance context. It seems very likely that given more time or a different design context, participants would have approached the design differently.
The results presented here may reasonably be expected to reflect the evaluation process that a mapping designer would use in their first sessions working on a new mapping. It seems reasonable that many of the considerations in such a first encounter, those criteria reflected in the themes above, would continue to play a role even in a longer design process. However, the relative importance of these criteria over time is not certain and cannot be indicated by the data from our study.
The results of our study are not known to generalize to mapping design over a longer period, using a different controller, synthesizer, or mapping environment, or when the design goals and context are different from live performance. Future work must consider how mapping design criteria differ depending on these important factors. It seems likely that design goals and context should be the most important factor in determining evaluation criteria; as such, future work should work to shed light on the diverse goals of artists and mapping designers so that criteria for mapping design can be situated in relation to these goals.
We are also curious to consider the influence that the mapping design environment may have had on participants evaluation criteria. We wonder if participants would have designed different mappings and evaluated them according to different criteria had they been using a tool where the design process is based on demonstrating input-output relationships, e.g. [19] or [20].
As music technology continues to proliferate, the design of mappings is likely to become increasingly important. The inherent nature of music technology as well as its historical development both result in an abundance of modular devices that are most useful when combined together: controllers, synthesizers, sound processors, recorders, sequencers, and so on. Mappings are the connective tissue that bind these otherwise separate devices together. They are at the core of digital musical instruments, and we would argue that they are at the core of most forms of electronic artistic practice. As such, it is crucial to develop a better understanding of how mappings are designed, and what makes certain mappings effective. This information will help researchers and developers to imagine more useful tools for making mappings between modular devices, as well as helping designers to make more effective mappings for fully integrated musical instruments.
This research was funded in part by a grant from the Social Sciences and Humanities Research Council of Canada (SSHRC). The authors also extend an enormous thanks to all the participants in the study. Special thanks also to Benjamin Muzart for his work on an earlier pilot study on which the methodology used was based.
The Research Ethics Board II of McGill University reviewed and approved this project in accordance with the requirements of the McGill University Policy on the Ethical Conduct of Research Involving Human Participants and the Canadian Tri-Council Policy Statement: Ethical Conduct.