What’s going on in the video:

The Sophtar is a feedback instrument, there is a transducer inside the head sound box. Somax2 – a multiagent system for human-machine co-improvisation – runs on the embedded computer responds to the human player by evoking structures of the musical data it is trained on. It tunes the bandpass filters in the electroacoustic feedback loop in response to how the bottom three strings are played. The model responds with pitches and onsets following a local stylistic knowledge space. These output pitches are mapped to 12 harmonics, and onsets are used to activate the solenoids and trigger amplitude envelopes that shape the electroacoustic feedback of the top five strings. The result is an encounter between the human player and the ghostly traces of the musics that formed the training dataset. The strings effectively become a place of contact between human, machine players, and the musical knowledge they encode. 

I have chosen to collaborate with the REACH group at IRCAM because their work is currently at the forefront of the research on using algorithms for human-machine co-creativity. REACH is a research project funded by the European Research Council and led by Gérard Assayag, and IRCAM is one of the leading music and technology research centres in the world. Their research work aims at understanding, modeling, and developing musical co-creativity between humans and machines through improvised interactions. Researchers at REACH study shared musicianship occurring at the intersection of the physical and digital spheres and have produced models and tools as vehicles to better understand human creativity in a context where it becomes more and more intertwined with computation. 

The residency period was very productive. I had access to IRCAM’s Studio 6 for several days, I successfully embedded Somax2 in the Sophtar and refined the algorithm that filters strings harmonics in the feedback loop in order to make Somax2 interact with it more expressively. At the end of the two weeks, I had the opportunity to perform and present my work to other researchers and composers at IRCAM. The session lead to very interesting exchanges with other musicians and researchers, including a hands-on session with Alois Cerbu shortly after to experiment with a frequency-domain saturator in the Sophtar feedback loop. 

I already made use of the results of the residency in a recent performance with Ensemble KNM in Berlin (more info here: The Nine Lives of Zosimos  – Ensemble KNM Berlin) and will use the system also in an upcoming performance at the International Conference on New Interfaces for Musical Expression (NIME) in London. I will play on Thursday 25 June at 19:00 at Rich Mix in Shoreditch. I think the line up of the evening is really cool, and the programme of the whole conference also looks very promising.

Grateful to Gérard Assayag, Marco Fiorini (who also kindly took some of the photos in this post), and Vasiliki Zachari for hosting me at IRCAM. Big thanks to Musikfonds and Culture Moves Europe for supporting my residency. 

Read more:

• Project: Co-Improvising with the Non-human
• Previously: Residency at the Centre for the Advanced Study of Collective Behaviour

I am particularly interested in how groups of non-human beings find ways to organise and dynamically adapt to unforeseen circumstances. What emerges from such group behaviours is more than the sum of its parts: seemingly simple individual strategies become beautifully complex, as swarm dynamics come into play. I find this inspiring in the context of musical co-improvisation, where the interplay between musicians may also result in something difficult to predict that is more than the sum of its parts. 

During my stay in Konstanz I learnt more about how fireflies synchronise, how bats use ultrasound for echolocation, the vocal repertoire of meerkats and the meaning of their many calls, how bees use sound to inform others in the hive of potential dangers, how locusts change their behavioural patterns when they join a swarm, and more. 

The research environment at CASCB is very interdisciplinary by design. Biologists, ethologists, neuroscientists, roboticists, and computer scientists often sit in the same room contributing to the same project. Coffee breaks become opportunities for very interesting casual conversations, where one can learn more about things they didn’t even know they were interested in. 

I presented my work in a seminar, part of a series organised by Andreagiovanni Reina, the Principal Investigator of the Group Intelligence and self-Organisation Lab (GIO Lab). The turn out and interest surprised me, particularly given that I come from other research domains. The questions and conversations that followed were also very stimulating. A few days later I performed with the Sophtar at a Friday evening social event at the Max Planck Institute of Animal Behavior and that was a lot of fun. 

I visited many laboratories during my stay, and was impressed by the variety and scale of the experiments going on at CASCB. I saw virtual reality rigs for insects, groups of robots using echolocation inspired by bats, tools capable of capturing the physiological signals of bees and microscopic fish, among other things. I even had the chance to go on a mountain hike as well as on a “bat walk.” That is an evening walk in the forest with bat detectors, devices that capture the ultrasonic calls of bats and transpose them to a frequency range that humans can hear in real time. 

I am very grateful to Andreagiovanni Reina for hosting me in his group and introducing me to many of the people working at CASCB. I am also very grateful to all the researchers I met during my stay who very generously took the time to show me their work and answer my questions. 

The next residency of the project will take place at IRCAM, where I’ll work with the REACH team at the STMS Lab to embed Somax2—an AI-based multi-agent system for human–machine co-improvisation—into the Sophtar.

Central to the project is the Sophtar, a musical instrument I designed that has been at the centre of my research and practice for the past two years. The Sophtar is an electroacoustic feedback string instrument. The transducer inside its headstock can be used to make its strings vibrate continuously at specific frequencies. The intensity and frequencies at which the strings vibrate can be influenced through custom controls, as well as through algorithms running on the computer embedded in the instrument. In addition to feedback, algorithms can access electromechanical actuators to hit the strings.

The Sophtar will act as a boundary object, a place of encounter between co-improvising algorithmic agents and human performers.

Listening to technology itself

In broader terms, I see studying the dynamics of group behaviour and improvisation as a way to resist, escape, and find alternatives to some of the current polarising mainstream narratives on artificial intelligence. I resonate with the sentiment expressed by the artist and writer James Bridle, who sees artificial intelligence

“not as something which overrides and supplants our own agency, but as something which might bring us to an accommodation with the intelligence and agency of other beings, non-human and more-than-human. Perhaps this is AI’s political role too: to warn us of the dangers of trying to segregate and oppress other beings and to point us towards a better path. […] Placing solidarity at the heart of our relationship with technology is part of this shift in attitudes. It means listening to technology itself, when it tries to tell us something about its uses, affordances and actual outcomes.”¹

Resisting corporate narratives

As part of the Wilding AI collective, I promote a view of generative AI in the arts that resists dystopian-utopian binaries and corporate imperatives to instead make space for collectively developing more diverse AI imaginaries². AI systems with a potential “wildness”—an agentic alterity that functions as a collaborator in the production of novel creative work. Systems “that can be free—if not to imagine then to generate—speeding through possibilities, junctures that are idiotic until they are not”³.
I subscribe to the view that improvisation is a mode of knowledge, critical dialogue, and collaboration across diverse domains⁴. Improvising can be a way of getting to know that alterity—that “wildness”—that may lie within artificial agents and machine learning models.

Soon at the Centre for the Advanced Study of Collective Behaviour

I will develop the algorithms and models in collaboration with several international laboratories currently carrying out research on artificial intelligence from different perspectives. My first research residency will begin next week at the Centre for the Advanced Study of Collective Behaviour (CASCB) in Konstanz, Germany. There, I will be hosted by the Group Intelligence and self-Organisation lab. I am grateful to the research group leader Andreagiovanni Reina for the invitation, and I look forward to meeting him and the other researchers at the centre. On Wednesday I’ll give a short seminar about my work, part of the Short Cyber-Physical Seminars series.

Since December last year, most of the time I dedicated to the project involved work on establishing collaborations, planning trips for the residencies, budgeting, sourcing materials, and developing the artistic vision further.

In Berlin, I am very excited to be working with multimedia artist and technologist Jonas Margraf and designer and mechanical engineer Aaron Verlooy.

The project will continue throughout 2026 with more artistic research residencies, development work, performances, and a public workshop/symposium.

[I wrote this while listening to Eliane Radigue‘s beautiful Feedback Works 1969-1970.]

Next: Residency at the Centre for the Advanced Study of Collective Behaviour