Federico Visi

Category: Research

  • Residency at IRCAM, Paris

    Residency at IRCAM, Paris

    Back in April I did the second artistic research residency of the Co-Improvising with the Non-human project, this time at IRCAM. During the residency period I worked closely with Marco Fiorini of the REACH team to integrate Somax2 — an AI-based multi-agent system for human–machine co-improvisation — into my custom instrument, the Sophtar. Here is a Sophtar + Somax2 improvisation I did the night before leaving Paris.

    What’s going on in the video:

    The Sophtar is a feedback instrument, there is a transducer inside the head sound box. Somax2 – a multiagent system for human-machine co-improvisation – runs on the embedded computer responds to the human player by evoking structures of the musical data it is trained on. It tunes the bandpass filters in the electroacoustic feedback loop in response to how the bottom three strings are played. The model responds with pitches and onsets following a local stylistic knowledge space. These output pitches are mapped to 12 harmonics, and onsets are used to activate the solenoids and trigger amplitude envelopes that shape the electroacoustic feedback of the top five strings. The result is an encounter between the human player and the ghostly traces of the musics that formed the training dataset. The strings effectively become a place of contact between human, machine players, and the musical knowledge they encode. 

    I have chosen to collaborate with the REACH group at IRCAM because their work is currently at the forefront of the research on using algorithms for human-machine co-creativity. REACH is a research project funded by the European Research Council and led by Gérard Assayag, and IRCAM is one of the leading music and technology research centres in the world. Their research work aims at understanding, modeling, and developing musical co-creativity between humans and machines through improvised interactions. Researchers at REACH study shared musicianship occurring at the intersection of the physical and digital spheres and have produced models and tools as vehicles to better understand human creativity in a context where it becomes more and more intertwined with computation. 

    The residency period was very productive. I had access to IRCAM’s Studio 6 for several days, I successfully embedded Somax2 in the Sophtar and refined the algorithm that filters strings harmonics in the feedback loop in order to make Somax2 interact with it more expressively. At the end of the two weeks, I had the opportunity to perform and present my work to other researchers and composers at IRCAM. The session lead to very interesting exchanges with other musicians and researchers, including a hands-on session with Alois Cerbu shortly after to experiment with a frequency-domain saturator in the Sophtar feedback loop. 

    I already made use of the results of the residency in a recent performance with Ensemble KNM in Berlin (more info here: The Nine Lives of Zosimos  – Ensemble KNM Berlin) and will use the system also in an upcoming performance at the International Conference on New Interfaces for Musical Expression (NIME) in London. I will play on Thursday 25 June at 19:00 at Rich Mix in Shoreditch. I think the line up of the evening is really cool, and the programme of the whole conference also looks very promising.

    Grateful to Gérard Assayag, Marco Fiorini (who also kindly took some of the photos in this post), and Vasiliki Zachari for hosting me at IRCAM. Big thanks to Musikfonds and Culture Moves Europe for supporting my residency. 

    Read more:

  • Residency at the Centre for the Advanced Study of Collective Behaviour

    Residency at the Centre for the Advanced Study of Collective Behaviour

    I recently got back from two very inspiring weeks at the Centre for the Advanced Study of Collective Behaviour (CASCB) in Konstanz. There, I did the first artistic residency of my Musikfonds AI Scholarship project Co-improvising with the Non-human. I chose to visit this interdisciplinary research cluster in order to broaden my perspective on what intelligent behaviour is, and find new sources of inspiration for creating algorithmic agents for musical improvisation. 

    I am particularly interested in how groups of non-human beings find ways to organise and dynamically adapt to unforeseen circumstances. What emerges from such group behaviours is more than the sum of its parts: seemingly simple individual strategies become beautifully complex, as swarm dynamics come into play. I find this inspiring in the context of musical co-improvisation, where the interplay between musicians may also result in something difficult to predict that is more than the sum of its parts. 

    During my stay in Konstanz I learnt more about how fireflies synchronise, how bats use ultrasound for echolocation, the vocal repertoire of meerkats and the meaning of their many calls, how bees use sound to inform others in the hive of potential dangers, how locusts change their behavioural patterns when they join a swarm, and more. 

    The research environment at CASCB is very interdisciplinary by design. Biologists, ethologists, neuroscientists, roboticists, and computer scientists often sit in the same room contributing to the same project. Coffee breaks become opportunities for very interesting casual conversations, where one can learn more about things they didn’t even know they were interested in. 

    I presented my work in a seminar, part of a series organised by Andreagiovanni Reina, the Principal Investigator of the Group Intelligence and self-Organisation Lab (GIO Lab). The turn out and interest surprised me, particularly given that I come from other research domains. The questions and conversations that followed were also very stimulating. A few days later I performed with the Sophtar at a Friday evening social event at the Max Planck Institute of Animal Behavior and that was a lot of fun. 

    I visited many laboratories during my stay, and was impressed by the variety and scale of the experiments going on at CASCB. I saw virtual reality rigs for insects, groups of robots using echolocation inspired by bats, tools capable of capturing the physiological signals of bees and microscopic fish, among other things. I even had the chance to go on a mountain hike as well as on a “bat walk.” That is an evening walk in the forest with bat detectors, devices that capture the ultrasonic calls of bats and transpose them to a frequency range that humans can hear in real time. 

    I am very grateful to Andreagiovanni Reina for hosting me in his group and introducing me to many of the people working at CASCB. I am also very grateful to all the researchers I met during my stay who very generously took the time to show me their work and answer my questions. 

    The next residency of the project will take place at IRCAM, where I’ll work with the REACH team at the STMS Lab to embed Somax2—an AI-based multi-agent system for human–machine co-improvisation—into the Sophtar.

  • Co-improvising with the non-human

    Co-improvising with the non-human

    Towards the end of 2025 I was very happy to find out that I am among the 15 artists selected by Musikfonds for their amazing Artificial Intelligence Scholarship Programme. My project explores how non-human group intelligence can inform the design of algorithmic agents for musical co-improvisation. I’ll look at several non-human group behaviours, including how bees use sound to communicate, how large groups of bats use ultrasonic calls to detect their surroundings, and how fireflies synchronise.

    Hundreds of fruit bats flying at sunset. © Christian Ziegler

    Central to the project is the Sophtar, a musical instrument I designed that has been at the centre of my research and practice for the past two years. The Sophtar is an electroacoustic feedback string instrument. The transducer inside its headstock can be used to make its strings vibrate continuously at specific frequencies. The intensity and frequencies at which the strings vibrate can be influenced through custom controls, as well as through algorithms running on the computer embedded in the instrument. In addition to feedback, algorithms can access electromechanical actuators to hit the strings.

    The Sophtar will act as a boundary object, a place of encounter between co-improvising algorithmic agents and human performers.

    Different views of the Sophtar.

    Listening to technology itself

    In broader terms, I see studying the dynamics of group behaviour and improvisation as a way to resist, escape, and find alternatives to some of the current polarising mainstream narratives on artificial intelligence. I resonate with the sentiment expressed by the artist and writer James Bridle, who sees artificial intelligence

    “not as something which overrides and supplants our own agency, but as something which might bring us to an accommodation with the intelligence and agency of other beings, non-human and more-than-human. Perhaps this is AI’s political role too: to warn us of the dangers of trying to segregate and oppress other beings and to point us towards a better path. […] Placing solidarity at the heart of our relationship with technology is part of this shift in attitudes. It means listening to technology itself, when it tries to tell us something about its uses, affordances and actual outcomes.”1

    Resisting corporate narratives

    As part of the Wilding AI collective, I promote a view of generative AI in the arts that resists dystopian-utopian binaries and corporate imperatives to instead make space for collectively developing more diverse AI imaginaries2. AI systems with a potential “wildness”—an agentic alterity that functions as a collaborator in the production of novel creative work. Systems “that can be free—if not to imagine then to generate—speeding through possibilities, junctures that are idiotic until they are not”3.
    I subscribe to the view that improvisation is a mode of knowledge, critical dialogue, and collaboration across diverse domains4. Improvising can be a way of getting to know that alterity—that “wildness”—that may lie within artificial agents and machine learning models.

    Soon at the Centre for the Advanced Study of Collective Behaviour

    I will develop the algorithms and models in collaboration with several international laboratories currently carrying out research on artificial intelligence from different perspectives. My first research residency will begin next week at the Centre for the Advanced Study of Collective Behaviour (CASCB) in Konstanz, Germany. There, I will be hosted by the Group Intelligence and self-Organisation lab. I am grateful to the research group leader Andreagiovanni Reina for the invitation, and I look forward to meeting him and the other researchers at the centre. On Wednesday I’ll give a short seminar about my work, part of the Short Cyber-Physical Seminars series.

    A swarm of bees ©Elisabeth Böker.

    Since December last year, most of the time I dedicated to the project involved work on establishing collaborations, planning trips for the residencies, budgeting, sourcing materials, and developing the artistic vision further.

    In Berlin, I am very excited to be working with multimedia artist and technologist Jonas Margraf and designer and mechanical engineer Aaron Verlooy.

    The project will continue throughout 2026 with more artistic research residencies, development work, performances, and a public workshop/symposium.

    [I wrote this while listening to Eliane Radigue‘s beautiful Feedback Works 1969-1970.]

    Next: Residency at the Centre for the Advanced Study of Collective Behaviour

  • Presenting at AIMC, Conference on AI Music Creativity

    Presenting at AIMC, Conference on AI Music Creativity

    Alexandre Saunier and I are presenting a paper co-authored with Maurice Jones about the behaviour generator based on large language models (LLMs) that we used during the Wilding AI workshop at MONOM, part of CTM Festival 2025. Some highlights of what we’re presenting:

    • LLM Behavior Generator that transforms textual prompts into expressive sound modulations and motions in space.
    • Chain of thoughts involving a “recipe” LLM cascading into a “code” LLM.
    • Max for Live implementation for artistic use in Ableton Live with immersive audio systems such as 4D SOUND.
    • Artistic experimentations and user feedbacks suggests such LLM-based system should be contextualized in the history of sound practices employing aleatoric, emergent, and open-ended strategies.

    Here are the title and abstract:

    Large Language Models to generate sonic behaviors: the case of Wilding AI in exploring creative co-agency

    Large Language Models (LLMs) and foundational models play a central role in multimodal text-to-sound systems, such as text-to-speech and text-to-music, as well as in recent musical agent systems designed to automate music production tasks. We explore an alternative approach that employs LLM-based sonic agents as spatial composition techniques. This method, emerging from the Wilding AI research-creation project, integrates LLMs into Max/MSP, Ableton Live, and spatial sound environments. LLMs are used to generate step-by-step sequences controlling spatial audio parameters, including sound motion in 3D space and other assignable sound properties.
    This paper details the artistic framework of Wilding AI, a LLM behavior generator, a live performance at the CTM Festival 2025, and discussions on composition, improvisation, time, and agency. Our work repositions LLMs as tools for shaping sonic experiences rather than merely generating finished audio material.

    Here is the full paper: https://zenodo.org/records/16946605

    Screenshots of the Max for Live interface of the Wilding AI LLM Behaviour Generator.

    This is one outcome of the collective research-creation lab Wilding AI.

    Members of the Wilding AI collective at Funkhaus Berlin during CTM 2025

  • Sophtar presentation at NIME 2024

    Sophtar presentation at NIME 2024

    Next week is NIME week! This year I am not going – Canberra is way too far from Berlin – but I finally got around uploading my presentation from last year’s conference though, which took place in Utrecht, the Netherlands. Features the Sophtar and a somewhat pretentious digression about magic, ergodynamics, and a quote from a book by Benjamin Labatut I was reading on my way there. Here is the paper!

  • Videos from the Embedding Algorithms Workshop

    Videos from the Embedding Algorithms Workshop

    I have made a YouTube playlist with videos from the Embedding Algorithms Workshop.

    The event took place in July 2024 at Universität der Künste Berlin, we brought together a community of artists, researchers, and practitioners and had a great time discussing the practice of embedding algorithms in the arts with presentations, exhibitions, and performances. Enjoy!

    Full playlist:

  • Computer Music Journal article: “An End-to-End Musical Instrument System That Translates Electromyogram Biosignals to Synthesized Sound”

    Computer Music Journal article: “An End-to-End Musical Instrument System That Translates Electromyogram Biosignals to Synthesized Sound”

    I finally put my hands on the physical copy of the Computer Music Journal issue on Human-AI Cocreativity that includes our open-access article An End-to-End Musical Instrument System That Translates Electromyogram Biosignals to Synthesized Sound.

    There we report on our work on an instrument that translates the electrical activity produced by muscles into synthesised sound. The article also describes our collaboration with the Chicks on Speed on the performance piece Noise Bodies (2019), which was part of the Up to and Including Limits: After Carolee Schneemann exhibition at Muzeum Susch.

    I co-authored the article with Atau Tanaka, Balandino Di Donato, Martin Klang, and Michael Zbyszyński, here is the abstract:

    This article presents a custom system combining hardware and software that senses physiological signals of the performer’s body resulting from muscle contraction and translates them to computer-synthesized sound. Our goal was to build upon the history of research in the field to develop a complete, integrated system that could be used by nonspecialist musicians. We describe the Embodied AudioVisual Interaction Electromyogram, an end-to-end system spanning wearable sensing on the musician’s body, custom microcontroller-based biosignal acquisition hardware, machine learning–based gesture-to-sound mapping middleware, and software-based granular synthesis sound output. A novel hardware design digitizes the electromyogram signals from the muscle with minimal analog preprocessing and treats it in an audio signal-processing chain as a class-compliant audio and wireless MIDI interface. The mapping layer implements an interactive machine learning workflow in a reinforcement learning configuration and can map gesture features to auditory metadata in a multidimensional information space. The system adapts existing machine learning and synthesis modules to work with the hardware, resulting in an integrated, end-to-end system. We explore its potential as a digital musical instrument through a series of public presentations and concert performances by a range of musical practitioners.

  • Organised Sound – Call for papers: Embedding Algorithms in Music and Sound Art

    Organised Sound – Call for papers: Embedding Algorithms in Music and Sound Art

    Excited to announce that I am co-editing with Thor Magnusson a thematic issue of Organised Sound on the topic of Embedding Algorithms in Music and Sound Art.
    Please find the full call below and on the journal’s website.

    The idea of a journal edited issue came after the Embedding Algorithms Workshop that took place at the Berlin Open Lab in July 2024.


    Feel free to share this call with others who might be interested in contributing.

    We look forward to receiving your submissions!

    Call for Submissions – Volume 31, Number 2
    Issue thematic title: Embedding Algorithms in Music and Sound Art
    Date of Publication: August 2026
    Publishers: Cambridge University Press
    Issue co-ordinators: Federico Visi (mail@federicovisi.com), Thor Magnusson (thormagnusson@hi.is)
    Deadline for submission: 15 September 2025

    Embedding Algorithms in Music and Sound Art
    Embedding algorithms into physical objects has long been part of sound art and electroacoustic music practice, with sound artists and researchers creating tools and instruments that incorporate some algorithmic process that is central to their function or behaviour. Such practice has evolved profoundly over the years, touching many aspects of sound generation, electroacoustic composition, and music performance.

    Whilst closely linked to technology, the practice of embedding algorithms into tools for sound making also transcends it. Different forms and concepts emerge in domains that can be analogue, digital, pertaining to recent or ancient technologies, tailored around the practice of a single individual or encompassing the behaviours of multiple players, even non-human ones.

    The processes and rules that can be inscribed into instruments may include assumptions about sound and music, influencing how these are understood, conceptualised, and created. Algorithms can encode complex behaviours that may make the instrument adapt to the way it is being played or give it a degree of autonomy and agency. These aspects can have profound effects on the aesthetics and creative processes of sound art and electroacoustic music, as artists are given the possibility of delegating some of the decisions to their tools.

    In the practice of embedding algorithms, knowledge and technology from other disciplines are appropriated and repurposed by sound artists in many ways. Sensors and algorithms developed in other research fields are used for making instruments. Similarly, digital data describing various phenomena – from environmental processes to the global economy – have been harnessed by practitioners for defining the behaviour of their sound tools, resulting in data-driven sound practices such as sonification.

    More recently, advances in artificial intelligence have made it possible to embed machine learning models into instruments, introducing new aesthetics and practices in which curating a dataset and training a model are part of the artistic process. These trends in music and sound art have sparked a broad discourse addressing notions of agency, autonomy, authenticity, authorship, creativity and originality. There are aesthetic, epistemological, and ethical implications arising from the practice of building sound-making instruments that incorporate algorithmic processes. How do intelligent instruments affect our musical practice?

    This demands an interdisciplinary critical enquiry. For this special issue of Organised Sound we seek articles that address the aesthetic and cultural implications that designing and embedding algorithms has on electroacoustic music practice and sound art. We are interested in the role of this new technological context on musical practice.

    Please note that we do not seek submissions that describe a project or a composition without a broad contextualisation and an underlying central question. We instead welcome submissions that aim at addressing one or more specific issues of relevance to the call and to the journal’s readership which may include works or projects as examples.

    Topics of interest include:

    • Aesthetics in performance of instruments with embedded algorithms
    • Phenomenology of electroacoustic performance with algorithmic instruments
    • Agency, autonomy, intentionality, and “otherness” of algorithmic instruments
    • Co-creation and authorship in creative processes involving humans and algorithmic instruments
    • Dynamics of feedback, adaptivity, resistance, and entanglement in player-instrument-algorithm assemblages
    • Human-machine electroacoustic improvisation
    • Critical reflections on the role of algorithms in the creation of sound tools and instruments and their impact on electroacoustic music practice
    • Meaning making, symbolic representations, and assumptions on music and sound embedded within instruments and sound tools
    • Knowledge through instruments: epistemic and hermeneutic relations in algorithmic instruments
    • Appropriation, repurposing, and “de-scripting” of hardware, algorithms, and data by sound artists
    • Exploration of individual and collective memory through algorithmic instrumental sound practice
    • Roles and understandings of the machine learning model as a tool in instrumental music practice
    • Use of algorithmic instruments in contexts informed by historical and/or non-Western traditions
    • Ethics, sociopolitics, and ideology in the design of algorithmic tools for electroacoustic music
    • Legibility and negotiability of algorithmic processes in electroacoustic music practice
    • Non-digital approaches: embedding algorithms in non-digital instruments
    • Postdigital and speculative approaches in electroacoustic music practice with algorithmic instruments
    • Data as instrument in electroacoustic music practice: curating and embedding data into instruments
    • Aesthetics and idiosyncrasies of networked, site-specific, and distributed instruments and performance environments
    • Entanglement, inter-subjectivity, and relational posthumanist paradigms in designing and composing with algorithmic instruments
    • More-than-human, inter-species, and ecological approaches in algorithmic instrumental practice

    Furthermore, as always, submissions unrelated to the theme but relevant to the journal’s areas of focus are always welcome.

    SUBMISSION DEADLINE: 15 September 2025

    SUBMISSION FORMAT:
    Notes for Contributors including how to submit on Scholar One and further details can be obtained from the inside back cover of published issues of Organised Sound or at the following url: https://www.cambridge.org/core/journals/organised-sound/information/author-instructions/preparing-your-materials.

    General queries should be sent to: os@dmu.ac.uk, not to the guest editors.

    Accepted articles will be published online via FirstView after copy editing prior to the full issue’s publication.

    Editor: Leigh Landy; Associate Editor: James Andean

    Founding Editors: Ross Kirk, Tony Myatt and Richard Orton†

    Regional Editors: Liu Yen-Ling (Annie), Dugal McKinnon, Raúl Minsburg, Jøran Rudi, Margaret Schedel, Barry Truax

    International Editorial Board: Miriam Akkermann, Marc Battier, Manuella Blackburn, Brian Bridges, Alessandro Cipriani, Ricardo Dal Farra, Simon Emmerson, Kenneth Fields, Rajmil Fischman, Kerry Hagan, Eduardo Miranda, Garth Paine, Mary Simoni, Martin Supper, Daniel Teruggi, Ian Whalley, David Worrall, Lonce Wyse

  • The Sophtar

    The Sophtar

    The Sophtar is a tabletop string instrument with an embedded system for digital signal processing, networking, and machine learning. It features a pressure-sensitive fretted neck, two sound boxes, and controlled feedback capabilities by means of bespoke interface elements. The design of the instrument is informed by my practice with hyperorgan interaction in networked music performance.

    I built the Sophtar in collaboration with Sukandar Kartadinata. I presented and performed with it at NIME 2024, here is the paper from the conference proceedings.

    At IIL I am working on extending the Sophtar with actuators and machine learning models to make it respond to my playing in ways that are not easy to predict yet meaningful and inspiring. In particular, I am working on:

    • an extension that allows the instrument to self-play by means of solenoids,
    • embedding notochord models,
    • per-string filtering for harmonic feedback.

    I see the research and development work on the Sophtar as a way to probe and engage with broader research questions on musical improvisation and co-creativity with machines and algorithms.

    I will present the Sophtar at an Open Lab on Friday, 27th September, 15:00-17:00.

    Playing the Sophtar at LNDW 2024 Photo: Christian Kielmann.

    EDIT: here is a video of the performance at the 2025 Guthman Musical Instrument Competition, Georgia Tech Ferst Center For The Arts, 8 March 2025.

    Jeff Albert: trombone;
    Sukandar Kartadinata & Federico Visi: Sophtar

    The Sophtar has been developed with the support of the GEMM))) research cluster at Luleå University of Technology, the Intelligent Instruments Lab at University of Iceland, the European Research Council, the Swedish Research Council, the Boström Fund, and the Helge Ax:son Johnsons foundation.

  • Intelligent Instruments Lab, Iceland

    Intelligent Instruments Lab, Iceland

    My family and I temporarily left Berlin and relocated to Reykjavik, where I am going to join the Intelligent Instrument Lab (IIL) at the University of Iceland. There, I am going to work on extending the capabilities of the Sophtar, the musical instrument I designed to address the needs stemming from my artistic practice involving hyperorgans, networked music performance, and interactive machine learning. I built the Sophtar in collaboration with Sukandar Kartadinata and I describe it more in detail in this NIME 2024 paper. I very much look forward to working with Thor Magnusson and the other super talented researchers at IIL, starting in just a few days!