Author keywords

live coding, versioning, exploratory programming, creative support tools

Problem and Research Questions

The term live coding refers to a style of programming where parts of an evolving program are continuously modified and rerun [1]. Live coding has been used extensively in making and performing music with computers [2][3]. The aim of this research is to examine how live coding musicians keep track of the evolving state of their programs and to understand how new tools could be built to aid them in the process. In other words, we want to explore ways for live coders to keep track of what they have tried so far and to easily switch between different versions of their code. We wish to consider what that might look like both in a live performance setting and during practice or ideation. Throughout this paper, we will use the term “version” to refer to a program’s code at any particular point in time, and not just at specific checkpoints (e.g., like the version number in a software release).  

To make things more concrete, we provide the following motivating example. Suppose a live coder goes through several iterations to create a looping bassline. Then, they move on to develop a melodic line, and now they want to see how the new melody sounds with some previous version of their bassline. The question is: how do they get back to the previous bassline code? Maybe they remember just enough about the previous bassline version to recreate it from scratch. This is error-prone, and of course might not be possible. Maybe they were lucky and commented out the older bassline version, saving it for easy access later. This can work, but only if they knew ahead of time which versions they would want later. Or maybe they resort to scanning through their code’s history, either by looking at everything that has been sent to the interpreter, or through the text-editor’s undo command. Scanning through all the changes may be prohibitively slow in a performance setting. In a practice setting, scanning through the history may still be tedious, and furthermore, the history is likely to be lost across different coding sessions. All of these pitfalls suggest room for improvement in tracking versions in live coders’ programs. Thus, we set out to understand live coders’ current versioning practices and needs, and what types of tools they might be willing to use.

This project can be divided into two tracks. The first is gathering information to better understand the current state of affairs for live coding musicians.  The main research question is: what do live coders do now to keep track of evolving versions of their code? We also want to understand how versioning strategies differ based on context, e.g., during a performance versus preparing for one. In addition, we wish to understand possible opportunities for a versioning tool. What features would the tool need? And what types of interfaces would live coders be amenable to using? Or would live coders be reluctant to use such a tool in the first place?  

The second part of this project involves building and testing a versioning system. The research questions here revolve around what is actually useful to live coders in practice. Does the tool we build increase the amount that live coders switch between versions? Does it decrease the time it takes to switch? There is also the question of whether such a tool fits neatly into existing workflows or suggests new ones. And how might access to a versioning tool affect the final artistic product? 

The anticipated research contributions of this work are:

• An understanding of how live coders currently use versioning techniques in their code and how that varies with context (e.g., practice, rehearsal, live performance, etc.)

• A versioning tool to assist live coders 

• An understanding of how such a versioning tool might change musicians’ practices and enable novel musicality in live coding performances

• Advancing the discussion on versioning in creative practices

Context and Theory

Exploratory Programming

Creating music with live coding is an iterative process where the programmer adjusts their code to explore different musical ideas based on auditory feedback. Furthermore, the programmer’s overall goals might change as they discover new sounds through experimentation. This type of programming, involving rapid exploration and amorphous, evolving goals, is considered “exploratory programming”[4]. A common concern in exploratory programming is the need to navigate between many different versions of code. For example, you might want to try out several values for a variable or you may want to backtrack to a previous state after some exploration proves unfruitful. 

One common way to keep track of code versions is through software versioning tools like Git or Subversion. In an exploratory coding environment, this type of versioning is unideal because the overhead to track changes is too high. The high cost of versioning means either many versions are not tracked at all or the programmer must frequently interrupt their work to switch contexts and use the versioning tool. Instead, exploratory programmers often develop their own ad-hoc versioning systems, for example saving multiple copies of a file, having multiple versions of a function commented out, etc.[5] These ad-hoc strategies attempt to fill a gap, but still, exploratory programmers often bring up the need for better versioning tools[5].

One other domain in which exploratory programming is applicable is data science. Within the data science domain, researchers have explored different tools to aid in program versioning. One approach is to use light-weight versioning of code snippets within a file, as is done with the Variolite system[6]. A similar approach may be useful to live coders, for example, to track different blocks of code corresponding to separate tracks or instruments. Another approach tested in the data science domain is a “post-hoc” versioning system, where the user selects a certain output, like a graph, and then the versioning tool gathers the relevant code necessary to create the graph (and possibly doing the same for all previous versions of the graph)[7]. This post-hoc approach could be appealing in a live-coding setting, as the programmer does not have to distract themselves with the versioning tool at all until they have a need to use it. 

Considerations for Live Coding

One major factor that sets apart live coding from many other exploratory programming practices is that it is often done as a performative act in front of a live audience. Part of what can make live coding performance compelling is the element of risk that comes with improvisation[8]. A robust versioning system might encourage improvisation by providing the performer with a “safety net”, where they know they can easily backtrack to a previous state if things go awry. Being able to quickly change program state can also be used as a musical effect. The ability to quickly swap a program to a new audio scene has been noted as a difficulty in live coding, requiring some amount of skill and planning ahead[9]. A versioning tool might allow the performer to quickly go back to a previous program state without prior planning, making it easier to introduce repetition on larger timescales in live coding performances. 

In live coding music performances, a common norm is that the performer’s screen is shared with the audience[8]. If this is the case, a version control tool used by a performer would also be visible to the audience. A version-control system which visualizes the evolution of blocks of code during a performance could act as a type of “secondary syntax” that may help both the performer and the audience better understand the code[10]. On the other hand, sometimes sharing the performer’s screen can be more of an aesthetic consideration, for example, if the audience is expected to dance rather than spectate [2][11]. In this case, a performer might not want to use a visible version control tool if it does not match their desired aesthetic.

A version-control system could also be used to create a kind of recording of a live performance. Indeed, existing tools, like SuperCollider’s History class, are already used to record and analyze code from performances[9]. The History class contains a simple timeline of the code that was sent to the interpreter. This type of data could be enriched with a code-blocks-level versioning system, which may make the history easier to understand and explore. Such a tool is not limited to performances, and could also be used to record and understand practice sessions or even a piece’s development over multi-day timespans.

Outside of performance, a version control system for live coders can help with understanding code in general. As previously mentioned, a version control system could act as a secondary syntax to aid in understanding how the code changes unfolded[10]. A version control system could also indicate when code on the screen does not match the current version of the code that is running, thus resolving a tension between the “state of the world” and the “state of the code” that is particularly problematic in live coding environments[12]. 

There are certainly reasons live coders may not want a more powerful versioning tool. For example, a tool might not fit flexible workflows involving multiple different live coding languages. Artists may also purposefully opt to work in constrained spaces. For example, some digital artists choose not to use the “undo” button provided by their software tools[13]. If artists might eschew such a basic and wide-spread method for version navigation, it seems plausible that not all live coders would want to use a powerful versioning tool for aesthetic or philosophical reasons.

Methods

Through interviews and user studies, we wish to answer the following research questions:

• RQ1: How do live coders keep track of different versions of their code, and how does that vary in performances vs. practice?

• RQ2: What are live coders’ perceptions towards a potential versioning tool  that can automate their efforts in restoring certain musical states?

• RQ3: How might the use of a versioning tool shape the musical aesthetics of a live coding performance?

Interview

The first part of this study, which aims to understand the current practices and opinions of live coders, will be done via semi-structured interviews. We plan to interview around 15 live coders who have some experience performing. The interview will be divided into three parts. In the first part, we will primarily collect information about their current practice: what languages do they use, what settings do they perform in, do they use external controllers, etc. In the second part, we will more explicitly probe for how they currently handle versioning in their code and how that differs based on the setting and context. To do so, we will need to understand many aspects of their performance practice, like how improvisatory it is, how they incorporate musical transitions, etc. In the third part, we will probe for what they might want out of a versioning tool, what types of interfaces they might be amenable to, and whether they would want to use a versioning tool at all.

Building a Versioning System

The second part of this study is to build a versioning tool for live coding. Rather than waiting for our interviews to be complete, we are developing this system in parallel with initial designs based on our own experiences with live coding. It is an evolving process, and of course there is still room for the interviews to influence the system. After building a working prototype, we plan to run studies with live coders to assess its usability, usefulness, and appeal. 

We have designs for an initial version of a version-control system which is built for use with the live coding language Tidal. One of the main abstractions in Tidal is a “pattern”, which is a sequence of musical events that are looped over a subdivision of time called a cycle[14]. The typical code block sent to the interpreter will define a pattern with a specified name, and will contain a series of composed functions which create the sequence of events that make up the pattern. Typical usage of Tidal is stateless: a block of code defining a pattern does not depend on any other blocks of code. This has a useful property for version control: whenever we send a block of code to the interpreter, we can consider the whole block as a new version of that pattern.

In our design, we propose two different levels of versioning: pattern versioning and global versioning. As the name suggests, pattern versioning keeps track of the different versions for a particular pattern. A new version of a pattern is created whenever a code block which is sent to the interpreter redefines that pattern. Thus, there is a version history for each pattern name. The version history is displayed in an in-line, collapsible GUI which represents the history as a graph with nodes (Figure 1 B). The user can hover over a node to display the code for that version and click it to change the text editor’s code back to the previous version. If the user makes changes starting from a previous version, the new version is shown as a branch on the version graph.

Our designs also include a global versions timeline which keeps track of which versions of each pattern were active over time. This will be displayed as a timeline at the top of the text editor (Figure 1 A), with tick marks representing times when new code was sent to the interpreter. The user can hover over the timeline to get more detailed information about which code versions were active at a given time, and they can click on the timeline to revert the blocks of code in the text editor back to their previous states.

Expected Outcomes

I hope this work contributes to a broader conversation in the NIME community about versioning in creative practices. Replicability has been noted as an important issue at NIME[15], and it could be an interesting conversation to compare replicability at the piece or performance level (like this project) with replicability at the tool-building level. I think this work could also spark good discussion about the role of creative support tools at NIME. For example, what similar tools might be built to address difficulties in other types of workflows, like developing digital musical instruments?

From participation in the Doctoral Consortium, I hope to gain a better understanding of how this work is situated in the broader context of digital music-making and creative coding practices. I think this will give me ideas on where this type of research can go next. I hope to meet other folks with live coding experience who can provide perspective and feedback. Perhaps I will learn about some interesting tools which already exist. I also hope to engage in discussions about research methodology, and wish to learn what we may do to improve ours, and what we might consider doing differently.