What does it feel like to improvise with AI?

Pianist Zubin Kanga returns to the Southbank Centre on 17 October 2026 with a host of premieres in a  double bill of  Ghost in the Machine + After Dark: Club Cyborg. Zubin Kanga is currently based at Royal Holloway University where he is leading a £1.4m grant-funded research project ‘Cyborg Soloists,’ which explores AI and technology in music. [see Robert’s 2024 interview with Zubin Kanga]

In advance of Zubin Kanga’s premiere of George Lewis‘s Blombos Playground on 17 October at the Purcell Room, Southbank Centre, London, he asks the question ‘What does it feel like to improvise with AI?‘

I’ve often experienced a deep connection with other musicians onstage when playing
chamber music or improvising together, like we’re having a musical conversation. But how
does it feel to have a musical conversation with AI? Does it feel anything like playing with a
human musician on stage? And what can it tell us about what makes human creativity
unique? These are questions I’ve been grappling with while working with American
composer George Lewis on his new work composed for human and AI keyboardists,
Blombos Playground, which I’m about to premiere at the Southbank Centre in London.

While many people today think of AI art as being a recent development and associate it
with the lazy production of bland results, trained via the internet through unethical means, it
actually has a long and much more interesting history in experimental music, going back to
the 1950s. Early developers of AI music systems never sought to replace human
musicians, but instead to explore new ways for human performers to interact with musical
technologies – and with each other.

George Lewis [Photo: Eileen Barroso]

George Lewis is one of these pioneers, a legendary figure in the field of human-computer
interaction in music. Lewis first started exploring the use of musical machines that could
independently interact with human musicians in 1979, developing into his first AI
experiments in 1984 with Rainbow Family. His iconic breakthrough system, Voyager, was
initially developed in 1985-87 and he continues to develop it to this day – it’s one of the two
AI systems in his new piece for me, Blombos Playground. Voyager is completely different
from the type of AI we’ve all become familiar with in recent years. Rather than machine
learning, the system uses a set of musical rules for listening and playing that Lewis has
programmed, based on his own years of experience as an improvising musician (he’s
performed on trombone with many of the world’s leading improvisers since the 1970s,
including Anthony Braxton, John Zorn, Derek Bailey and Evan Parker). The result of this
approach is a computer program that seems to have its own personality as an improvisor.
Voyager listens to the human co-performers, and sometimes follows and accompanies, but
at other times pushes to the fore, like a human improvisor who senses it might be time for
their ‘solo’. Over the four decades since Lewis first programmed it, he’s improved it with
every new development in computing, and many of the world’s leading performers and
ensembles have performed with it – from saxophonist Roscoe Mitchell, to the International
Contemporary Ensemble (USA), to the SWR Symphony Orchestra (Germany). It’s with
Voyager’s latest developments that I enter the picture.

It’s been one of the privileges of having a career as a contemporary music performer that
I’ve been able to work with the legends I learnt about as a student. As an undergraduate 20
years ago, I read articles about Lewis’s AI experiments, and I’ve listened to many
recordings of musicians playing with Voyager over the years. So it was a career highlight
when Lewis said ‘yes’ to a new commission as part of my Cyborg Soloists music technology
research project. Lewis’s central idea was about pushing Voyager to the next level by
combining it with the Somax improvising software created by IRCAM, a renowned music
research institute in Paris.

Though Voyager and Somax are both types of ‘AI’, they work completely differently.
Whereas Voyager uses rule-based approaches to improvisation, Somax uses machine-
learning on a set of user-provided recordings. In my case, it was trained on a couple of

hours of improvisations that I sent to the IRCAM team, who then created an avatar twin, a
musical doppelgänger. This new combination of the two AIs has been coined ‘SoVo’ by
Lewis and his research team (Damon Holzborn at Columbia University, and Marco Fiorini
and Gérard Assayag at IRCAM).

Blombos Playground puts me at the centre of a complex weaving of these two AIs. Each
section provides me with scored material, drawing on musical gestures from the
improvisations I had sent the team, ranging from meditative chorales to jazzy clusters to
quicksilver passagework. At the end of each section, Lewis provides guidance on the many
different ways of using this material to improvise with the AI co-performers. Using a MIDI
keyboard, the sounds I make are as malleable as the AI’s. Sometimes we’re all using piano
sounds, sometimes we’re duetting on harpsichord and celeste, and every combination of
these three sounds. The spatialization of the sounds adds further layers to this musical
conversation, using multiple speakers to place me and the different AIs in different corners
of the room, have us chase each other about the room, or spin an AI’s sounds around me.

Although this might seem complicated and technical, in performance the duetting between
the human and AI keyboardists feels visceral and immediate. There’s a logic to both the
combinations of sounds and their movement around the space. The audience can
sometimes hear the clear differences between human and AI, and sometimes they blend
together, challenging the listener to distinguish between the real and virtual musicians.

But what does it feel like to play? For me, it feels like being inside the minds of these
artificial improvisors as they interact with each other, and with me. I know that these
software systems can’t think, but they seem to have personalities: Somax’s improvising
persona is closer to mine (since it’s trained on my recordings), while Voyager is like a
maverick free jazz improviser who’s comfortable being both an accompanist and a soloist.
But they also play in a way that’s subtly different from human co-performers, and their
‘intelligence’ is unmistakably artificial and non-human. They don’t understand my intentions,
and I know that these systems can’t intend or understand anything, despite seeming to
have musical empathy.

The systems aren’t made to replace human performers, but to reveal to us new ways
of thinking about improvisation, musical personality and musical communication. Real
musicians are less virtuosic, and more fallible, but there’s a natural and deep understanding
and connection when improvising with a real musician that can never be replicated. And
what makes these systems interesting is not the AI technology itself, but the musical
insights and choices that George Lewis and the Columbia/IRCAM team have programmed
into them, using their own musical experience and instincts to create these virtual co-
performers. What inspires me about Blombos Playground is that it’s pushed me to new
kinds of virtuosity and made me think afresh about my own approaches to both
performance and improvisation. It’s challenged my creative habits – and this artificial
musical intelligence has made me a better musician.

Zubin Kanga 

Zubin Kanga will perform the world premiere of Blombos Playground by George Lewis on
17 October at the Purcell Room, Southbank Centre, London.