Tag: music

  • Embedding Algorithms 2

    Embedding Algorithms 2

    algorithmic practices in music and sound

    Embedding Algorithms 2 brings together artists, musicians, and researchers whose work places algorithms at the core of sonic practice. Over two afternoons at UdK Medienhaus, talks and performances will look at how machine learning models, live-coded patterns, biological data, analogue artificial neurons, and feedback circuits become part of how we build instruments, compose, perform, and listen. Topics range from neural audio synthesis and embodied machine learning to neuromorphic computing, art made with living organisms and genetic data, and the politics of training data, alongside spatial sound on the NTSS, a 30-loudspeaker system housed at the Medienhaus.

    The event follows the first Embedding Algorithms Workshop, held at Berlin Open Lab in 2024, and is organised by Federico Visi as part of the project Co-improvising with the Non-human, supported by the Musikfonds AI scholarship (STIP-5). It is kindly supported by the Medienhaus and the Generative Art / Computational Art class at Universität der Künste Berlin.

    Admission is free and open to all.
    The event will be live-streamed.

    If you would like to attend, please register here to receive further information and receive the live streaming links for remote attendance: https://docs.google.com/forms/d/e/1FAIpQLSdge50bUWQwy3mhDmKh3HbjvV6bJ_gSGpMLh7xjrGT2lRqtWw/viewform?usp=publish-editor

    Please register even if you plan to attend in person: it helps us estimate how many people to expect and organise the space accordingly.

    When/Where

    23—24 October 2026
    15:00—19:30
    UdK Medienhaus
    Aula (1st floor)
    Grunewaldstraße 2-5,
    10823 Berlin

    U-Bahn Kleistpark
    Info+registration:
    www.federicovisi.com/emb-algo-2/

    Participants

    Adam Pultz
    Alberto de Campo
    Alex McLean
    Andrea Parkins
    Bruno Gola
    Chris Kiefer
    Eman Safavi
    Federico Visi
    Guilherme Coelho
    Marcello Lussana
    Marco Donnarumma
    Margherita Pevere
    Moisés Horta Valenzuela / Hexorcismos
    Muyang Lyu
    Nicola Leonard Hein
    Society for Nontrivial Pursuits
    Viola Yip

    Short summaries

    Adam Pultz
    Machine Temporalities: The aesthetics and politics of decelerated machine learning
    Since the deep learning revolution in 2012, model size and compute power have followed an exponential growth curve, a feature echoed in the socio-political domain through the fascinating and disturbing project of accelerationism. In this talk I discuss how the artistic use of decelerated machine learning may trouble accelerationist ideology and present a way of speculatively imagining futures beyond technocapitalism.

    Alex McLean
    Non-scribal algorithmic music
    Algorithmic music can feel like abstract, ‘paper rhythms’, which unlike instrumental music, can feel unrelateable to movements of the body. In this talk/performance I’ll share how learning some of the algorithmic patterns underlying traditional South Indian rhythms have grounded my live coding practice, and informed my development of technology around it. I will also reflect on the liveliness of ‘non-scribal’ music culture, and what live coding culture could learn from it.

    Andrea Parkins
    “The Stray”: Diffusing Dis-location with the NTSS
    Andrea Parkins will present an excerpt from “The Stray (Variation 4)”, her electroacoustic composition through which she highlights and implicates correspondences (or ruptures) that take place between gesture, sound and space; and embodied tensions between attempt and fluency. Parkins’ composition draws on Julia Kristeva’s psychoanalytic writing on abjection and the sublime, evoking the subjectivity of a migratory, dislocated (and sounding) body: a “stray”. For “Embedding Algorithms” Parkins will work together with Alberto de Campo and Bruno Gola to diffuse her composition on the NTSS (a non-trivial sound system).

    Bruno Gola
    Neuromorphic Synthesis
    Inspired by the work of David Tudor and Forrest Warthman and the field of Neuromorphic Computing, “Neuromorphic Synthesis” investigates the history and speculates on the future of computation and Artificial Intelligence by means of experimental music, looking at modular systems as a hybrid analog/digital computing system. A central piece of this ongoing artistic research project is the Memristor, an electronic component currently used in research for analog neural networks, overcoming the limitations of the von Neumann architecture. By exploring feedback matrices using Memristors for the weights, signal flow isn’t only affected by the circuits but it also alters the material and the behavior of the circuits itself.

    Bruno Gola, Alberto de Campo, S4NTP
    NTSS – A Nontrivial Sound System
    NTSS is a NonTrivial Sound System that understands spatialisiation as a rich resource for improvised performances: it offers a diversity of loudspeaker sound colors; multiple input sources can be sent on spatial orbits defined by a variety of algorithms; the parameters of these orbits can be informed by machine listening; and it is polyphonic: multiple performers can play their layers of fully spatialised sound into the NTSS simultaneously.
    It consists of 30 loudspeakers living at UdK Berlin Medienhaus. Speaker maps and panning algorithms can be adapted flexibly, which we did for a concert at Silent Green Kuppelhalle, and for a residency at Hosek Gallery Berlin. It is available as an open source SuperCollider library.

    Chris Kiefer
    MEMLNaut: a musical instrument for exploration of musically embodied machine learning
    MEMLNaut is a musical instrument and research platform for exploration of musically embodied machine learning. It can process and generate sound and optimise and run machine learning models on it’s dual core microcontroller chip. MEMLNaut is part of an investigation into the musically expressive potential of machine learning (ML) when embodied within physical musical instruments. It proposes ‘tuneable ML’, a novel approach to exploring the musicality of ML models, when they can be adjusted, personalised and remade, using the instrument as the interface.

    Eman Safavi
    Watermarking the training data of generative audio models: forensic detecability or a matter of intellectual property
    When artists’ audio recordings enter generative AI training datasets, questions of consent, attribution, traceability and control become entangled with model architecture as well as datasets used for the training and fine tuning process. Previous studies have demonstrated successful watermarking, including model-integrated approaches such as DualMark. My thesis investigates a different possibility: can artists mark their recordings once and retain provenance without depending on AI developers? Experiments with AudioSeal and Stable Audio Open in my experiments show that source-level watermarks disappear during representation learning. These findings reframe artist protection as a sociotechnical challenge involving architectural accountability, transparency, and enforceable data governance. The presentation will be a walk through of my experiments, prototypes, hypothesis and discussions.

    Federico Visi
    Embedding algorithms in the electroacoustic feedback loop and co-improvising with the non-human
    After a brief account of my project Co-improvising with the Non-human, I perform with the Sophtar, a feedback string instrument with an embedded computer. Somax2 agents from IRCAM and musically embodied machine learning (MEML) algorithms from the Emute Lab at the University of Sussex run inside the instrument, acting on its electroacoustic feedback loop and electromechanical actuators. Machine and human listen to the strings and act on them, sharing the same vibrating material, and improvise through a single physical system. The session closes with a first glimpse of the NHÉs, a swarm of non-human entities that emit and sense sound and light.

    Guilherme Coelho
    Towards Latent Synthesis: Prompting, Remixing and Blending with Text-to-Audio AI
    This session explores particular approaches to working with text-to-audio AI systems such as Suno and Udio, with a focus on the creative potential of latent synthesis. Through incongruent intersemiotic prompt combinations, cross-materialism, and the remixing and blending of input audio, I show how these systems can be pushed beyond their conventional outputs towards idiosyncratic, emergent sound objects. We will listen to these strange hybrids and consider what sonic materialities, modes of synthesis and musical imaginaries they make possible.

    Marcello Lussana
    Towards Automatic Assessment of the Restorative Potential of Soundscapes
    Evaluating the mental restoration potential of soundscapes is an active field of research, with solid findings but also open questions. While most research on restoration has focused on the visual aspects of environments, soundscape studies have also addressed this issue. However, there is no clear consensus on which auditory features play an important role.
    This presentation outlines ongoing work on fine-tuning Stable Audio Open using pleasantness ratings to teach a machine to recognize pleasant soundscapes.

    Marco Donnarumma
    AI Prosthetics and Sonic Organs
    In a world of social bubbles, immediate gratification and geopolitical upheaval, sound in art can help establish new ways of acknowledging the ‘other’, a first step towards a renewed mutuality. In this talk, Marco will focus on his latest series of works, I Am Your Body, an ongoing collection of performances and installation that leverage participative research to explore Deaf perception of sound as a means for new artistic aesthetics. Computational and prosthetic technology plays a crucial role in these works, but rather than being passive media, they are subverted and transformed into machines that act as new organs for a shared perception.

    Margherita Pevere
    Ode
    Ode imagines urban ponds as spaceships undertaking a mission that, concealed by their apparent stillness, remains enigmatic to humans. Interconnected elements, inspired by the anatomy of pond organisms, shape a contemplative microcosmos. By combining bioinformatics with living samples from Berlin urban ponds, music, sculptural glass, garments, and sound objects, the artwork summons the liveliness of the spaceships. Pevere envisioned the spaceship concept during fieldwork with POUNDER researchers, who sample pond waters to extract genetic data of bacteria inhabiting them (also known as environmental DNA metabarcoding). Ode manifests actual genetic material and datasets from Berlin ponds as a multidimensional repository of ecological relationships and (im)possible adaptations to contaminants.

    Moisés Horta Valenzuela / Hexorcismos
    Building Unheard Music Instruments: Design Methodologies for Independent AI Music R&D
    This performance lecture examines how SEMILLA.AI Studio bridges machine learning research and electronic music practice. Unpacking the end-to-end engineering of real-time neural audio instruments, we detail dataset curation, custom model training, and mapping latent spaces for live performance environments. The session demonstrates how independent computer musicians construct sovereign sonic architectures, concluding with a performance using SEMILLA.AI v2, NEBULA and in-development instruments.

    Muyang Lyu
    A Media Archaeology of Primitive Analog Neurons
    A synthesizer prototype on breadboards, whose circuit is inspired by early artificial neurons in analog perceptrons and by LIF neuromorphic computing based on OTAs and Op Amps. Rather than conforming to conventional categories of musical instruments, the aim is to reflect on the historical epistemology embedded in technology, to question the choice of artistic apparatus between professional ready-made and amateur DIY approach, and to explore a messy but playful reuse and misuse of obsolete techniques from a super-low level.

    Viola Yip and Nicola Leonard Hein
    FO2: A Wearable Optoelectronic Laser-Feedback Instrument – Performing body-space-technology entanglements
    In “Fluid Ontologies”, Transsonic (Nicola Leonard Hein and Viola Yip) continues to expand their intermedial artistic practice in performances. For this work, they further develop their DMI “FO2”. It is an optoelectronic laser-feedback instrument that uses laser modules as sound sources and a solar panel as a microphone, creating audiovisual feedback loops that generate images and sound using the same optoelectronic mechanism. Fluid Ontologies explores and defines new concepts of the instrumentality of light in audio circuits, putting space, body, technology, and instruments into one dynamic feedback system, exploring forms of cybernetic and transmedial modes of listening, as well as the emerging dances of agency.

  • Videos from the Embedding Algorithms Workshop

    Videos from the Embedding Algorithms Workshop

    I have made a YouTube playlist with videos from the Embedding Algorithms Workshop.

    The event took place in July 2024 at Universität der Künste Berlin, we brought together a community of artists, researchers, and practitioners and had a great time discussing the practice of embedding algorithms in the arts with presentations, exhibitions, and performances. Enjoy!

    Full playlist:

  • Computer Music Journal article: “An End-to-End Musical Instrument System That Translates Electromyogram Biosignals to Synthesized Sound”

    Computer Music Journal article: “An End-to-End Musical Instrument System That Translates Electromyogram Biosignals to Synthesized Sound”

    I finally put my hands on the physical copy of the Computer Music Journal issue on Human-AI Cocreativity that includes our open-access article An End-to-End Musical Instrument System That Translates Electromyogram Biosignals to Synthesized Sound.

    There we report on our work on an instrument that translates the electrical activity produced by muscles into synthesised sound. The article also describes our collaboration with the Chicks on Speed on the performance piece Noise Bodies (2019), which was part of the Up to and Including Limits: After Carolee Schneemann exhibition at Muzeum Susch.

    I co-authored the article with Atau Tanaka, Balandino Di Donato, Martin Klang, and Michael Zbyszyński, here is the abstract:

    This article presents a custom system combining hardware and software that senses physiological signals of the performer’s body resulting from muscle contraction and translates them to computer-synthesized sound. Our goal was to build upon the history of research in the field to develop a complete, integrated system that could be used by nonspecialist musicians. We describe the Embodied AudioVisual Interaction Electromyogram, an end-to-end system spanning wearable sensing on the musician’s body, custom microcontroller-based biosignal acquisition hardware, machine learning–based gesture-to-sound mapping middleware, and software-based granular synthesis sound output. A novel hardware design digitizes the electromyogram signals from the muscle with minimal analog preprocessing and treats it in an audio signal-processing chain as a class-compliant audio and wireless MIDI interface. The mapping layer implements an interactive machine learning workflow in a reinforcement learning configuration and can map gesture features to auditory metadata in a multidimensional information space. The system adapts existing machine learning and synthesis modules to work with the hardware, resulting in an integrated, end-to-end system. We explore its potential as a digital musical instrument through a series of public presentations and concert performances by a range of musical practitioners.

  • Talk at the “Mapping Social Interaction through Sound” symposium, Humboldt University, Berlin

    Talk at the “Mapping Social Interaction through Sound” symposium, Humboldt University, Berlin

    I was invited to participate in the Mapping Social Interaction through Sound symposium on 27-28 November 2020. The symposium is organised by Humboldt University, Berlin and – as it is customary these days – will take place on Zoom.

    This is the abstract of my talk.

    Building and exploring multimodal musical corpora:
    from data collection to interaction design using machine learning

    Musical performance is a multimodal experience, for performers and listeners alike. A multimodal representation of a piece of music can contain several synchronized layers, such as audio, symbolic representations (e.g. a score), videos of the performance, physiological and motion data describing the performers movements, as well as semantic labelling and annotations describing expressivity and other high-level qualities of the music. This delineates a scenario where computational music analysis can harness cross-modal processing and multimodal fusion methods to shift the focus toward the relationships that tie together different modalities, thereby revealing the links between low-level features and high-level expressive qualities.

    I will present two concurrent projects focussed on harnessing musical corpora for analysing expressive instrumental music performance and design musical interactions. The first project is centered on a data collection method – currently being developed by the GEMM research cluster at the School of Music in Piteå – aimed at bridging the gap between qualitative and quantitative approaches. The purpose of this method is to build a data corpus containing multimodal measurements linked to high-level subjective observations. By applying stimulated recall (a common qualitative research method in education, medicine, and psychotherapy) the embodied knowledge of music professionals is systematically included in the analytic framework. Qualitative analysis through stimulated recall is an efficient method for generating higher-level understandings of musical performance. Initial results suggest that this process is pivotal in building our multimodal corpus, providing insights that would be unattainable using quantitative data alone.

    The second project – a joint effort with the Computing Department at Goldsmiths, University of London – consists in a sonic interaction design approach that makes use of deep reinforcement learning to explore many mapping possibilities between large sound corpora and motion sensor data. The design approach adopted is inspired by the ideas established by the interactive machine learning paradigm, as well as by the use of artificial agents in computer music for exploring complex parameter spaces. We refer to this interaction design approach as Assisted Interactive Machine Learning (AIML). While playing with a large corpus of sounds through gestural interaction by means of a motion sensor, the user can give feedback to an artificial agent about the gesture-sound mappings proposed by the latter. This iterative process results in an interactive exploration of the corpus, as well as in a way of creating and refining gesture-sound mappings.

    These projects are representative of how the development of methods for combining qualitative and quantitative data, in conjunction with the use of computational techniques such as machine learning, can be instrumental in the design of complex mappings between body movement and musical sound, and contribute to the study of the multiple facets of embodied music performance.

    Further reading

    Visi, F. G., Östersjö, S., Ek, R., & Röijezon, U. (2020). Method development for multimodal data corpus analysis of expressive instrumental music performance. Frontiers in Psychology, 11(576751), doi: 10.3389/fpsyg.2020.576751
    Download PDF (pre-print)

    Visi, F. G., & Tanaka, A. (2021). Interactive Machine Learning of Musical Gesture. In E. R. Miranda (Ed.), Handbook of Artificial Intelligence for Music: Foundations, Advanced Approaches, and Developments for Creativity. Springer Nature, forthcoming.
    View on arXiv.org
    Download PDF (pre-print)

    Visi, F. G., & Tanaka, A. (2020). Towards Assisted Interactive Machine Learning: Exploring Gesture-Sound Mappings Using Reinforcement Learning. In ICLI 2020 – the Fifth International Conference on Live Interfaces.
    Download PDF

    Presentation slides
    Download PDF

  • Physically Distant #3: the network, the pandemic, and telematic performance

    Physically Distant #3: the network, the pandemic, and telematic performance

    PD#3 will be part of Ecology, Site And Place – Piteå Performing Arts Biennial 2020. Participation in the conference is free, but registration is compulsory. Register by sending an email to piteabiennial@gmail.com

    After the two previous editions in June and July, the third Physically Distant Talks will take place on 26 and 27 October 2020. The talks are going to be part of the online event of Ecology Site and Place – Piteå Performing Arts Biennial.

    The format will be different this time, as there are going to be more telematic performances and the talks will be structured in three panels. Each panel member is invited to prepare a 3-minute provocation/reflection related to the topic. This collection of provocations from the panelists will set the tone for an open discussion in the style of the previous Physically Distant talks. As in the previous editions of the talks, Stefan Östersjö and myself, Federico Visi, will be moderating the discussion.

    Programme (all times are CET)

    Monday, 26 October 2020

    17:30 Introduction. Stefan Östersjö and Federico Visi
    17:40 Simon Waters and Paul Stapleton: Musicking online: your technical problem is actually a social problem. A performative conversation.

    18:00-19:00 Panel I. Instrumentality in Networked Performance
    Panelists: Nela Brown, Nicholas Brown, Juan Parra Cancino, Franziska Schroeder, Henrik Von Coler.

    19:00-19:45 Telematic Performance: A concert hall organ in the network.
    Live-streaming from Studio Acusticum. Telematic performances with the University Organ remotely controlled from several locations.
    Robert Ek, clarinet, performing in Piteå (SE)
    Mattias Petersson, live-coding, performing in Piteå (SE)
    Federico Visi, electronics, electric guitar, performing in Berlin (DE)
    Scott Wilson, live coding, performing in Birmingham (UK)
    Stefan Östersjö, electric guitar, performing in Stockholm (SE)

    19:45-20:00 Break

    20:00-21:00 Panel II. Network ecology: Communities of practice for the digital arts
    Panelists: Shelly Knotts, Thor Magnusson, Mattias Petersson, Rebekah Wilson, Scott Wilson.

    Tuesday, 27 October 2020

    17:45-18:00 Marcin Pączkowski: rehearsing music online: possibilities and limitations

    18:00-19:00 Panel III. The network as place
    Panelists: Ximena Alarcon Diaz, David Brynjar/Angela Rawlings/Halla Stefansdottir, Chicks on Speed (Melissa Logan, Alex Murray-Leslie), Maja Jantar, Marcin Paczkowski, Roger Mills, Luca Turchet.

    19:00-19:30 Telematic Performance: iða
    David Brynjar Franzson, technical concept and streaming (US)
    Maja Jantar, performer and composer of visual score (BE)
    Angela Rawlings, performer and composer of visual score (IS/CA)
    Halla Steinunn Stefánsdóttir, performer and composer of visual score (SE)

    19:30-20:00 Break

    20:00-21:00 Where do we go from here? (plenary discussion)

    For more details on the Ecology, Site And Place – Piteå Performing Arts Biennial 2020 online event, download the book of abstracts.