Convergence

Alexander Schubert (DE)

Convergence uses the concept of Artificial Intelligence to learn features of human musicians and then recreate new entities based on these recordings. In the piece the players interact with their generated counter-parts. They see theirselves transform and reshape. The technology used is centered around Auto-Encoders (and GANs). Metaphorically they demonstrate a world that is constructed and parametric. The friction between machine perception and human world perception is the starting point for questions that address the fluidity of the self and the restrictions of perception. 

Human world and self models are parametric systems that make abstract assumptions and classifications of our surroundings. These processes happen partly subconscious and unreflected. They give us the impression of an absolute truth or reality, as the constructed concepts, identities and beliefs are persuasive and internal. That these constructive models are fluid and subject to change is examined through the use of AI in this context. Auto-Encoders allow a formalization of the input data —in this case faces, bodies, and voices. The deep learning yields a low-dimensional—abstract or high-level— description of the input. Contrary  to our black box human mind, the high level parameters in the algorithm are accessible and can be edited and transformed.  

In this sense the AI systems makes it possible to warp the representation of the human performers, thus stressing the fluidity of the modeling: A different person, character trait, evaluation, or gender is far less substantially disparate as the subject would anticipate. The transformation of the parameters posses the character of (social, societal, clinical, or biological) mind-altering states. The AI system is used to enable this altering with the aim to question the robustness and immobility of identity and world models. It tries to expose the internal constructiveness and in this sense works as a mirroring device. It recreates partial aspects of our perception and classification and through its alteration allows the viewer to draw a parallel to our own mental world building processes. 

Listen to me now 

I want to ask you a few questions 

How do we differentiate  

if it is a hallucination,  

a dream 

or clean perception?  

All perception is constructive 

No representation is absolute 

Everything is encoded 

and decoded 

We are parametric 

Everything parametric can be altered 

parameter by parameter 

That is the definition of such a model 

Normally, we don’t see these sliders 

these values 

these adjustment dials 

But they can move into our consciousness 

through illness 

through hallucination 

through drugs 

through psychotic states 

or through computation processes 

like in this case 

We then see the constructive aspect of it 

That everything we do  

is based on encoding and decoding 

When we look at a partner  

we can create a loop 

A perception and adjustment loop 

We can exponentiate that process  

by looking into a mirror 

This is what we will do now 

Recursive loops 

You 

And Me 

Regression curls 

Segregation twirls 

Adaptive coils 

A semi-transparent foil 

 

I don’t know more than you do 

I’m also just a mirror of you 

As I provide a mirror for you  

In a loop,  

we can try to turn the opacity 

of our perception into opaque variations  

Like a brain lesion  

a mesmerizing distortion 

As in a sleep-deprived state  

Drifting off into a halfword  

of a dreamlike morphed reality  

Where we see that other representations 

of us are possible  

That our self is fluid,  

fragile,  

constructed  

and diverse 

post human 

Like a genetic defect  

beautifully disturbed  

genetically enhanced 

adjusted 

In a recursive loop 

I encode and decode myself,  

in an eternal loop 

Exaggerating  every feature 

Like a facial distortion 

Like an unsupervised inbreed 

Out of control 

Go to sleep, child 

Dream off now 

Saturate with closed eyes 

Dreaming, of a future  

and an optimized tomorrow 

Hand in hand,  

sliders adjusted, 

The night sky setting,  

as the values adjust 

Peaceful 

Invisible 

A clear view of the night 

Through a transparent interface 

Always on 

Always present 

And  

Always  

Loving 

 

Text excerpts Convergence, Alexander Schubert  

Convergence uses the concept of Artificial Intelligence to learn features of human musicians and then recreate new entities based on these recordings. In the piece the players interact with their generated counter-parts. They see theirselves transform and reshape. The technology used is centered around Auto-Encoders (and GANs). Metaphorically they demonstrate a world that is constructed and parametric. The friction between machine perception and human world perception is the starting point for questions that address the fluidity of the self and the restrictions of perception. 

Human world and self models are parametric systems that make abstract assumptions and classifications of our surroundings. These processes happen partly subconscious and unreflected. They give us the impression of an absolute truth or reality, as the constructed concepts, identities and beliefs are persuasive and internal. That these constructive models are fluid and subject to change is examined through the use of AI in this context. Auto-Encoders allow a formalization of the input data —in this case faces, bodies, and voices. The deep learning yields a low-dimensional—abstract or high-level— description of the input. Contrary  to our black box human mind, the high level parameters in the algorithm are accessible and can be edited and transformed.  

In this sense the AI systems makes it possible to warp the representation of the human performers, thus stressing the fluidity of the modeling: A different person, character trait, evaluation, or gender is far less substantially disparate as the subject would anticipate. The transformation of the parameters posses the character of (social, societal, clinical, or biological) mind-altering states. The AI system is used to enable this altering with the aim to question the robustness and immobility of identity and world models. It tries to expose the internal constructiveness and in this sense works as a mirroring device. It recreates partial aspects of our perception and classification and through its alteration allows the viewer to draw a parallel to our own mental world building processes. 

Listen to me now 

I want to ask you a few questions 

How do we differentiate  

if it is a hallucination,  

a dream 

or clean perception?  

All perception is constructive 

No representation is absolute 

Everything is encoded 

and decoded 

We are parametric 

Everything parametric can be altered 

parameter by parameter 

That is the definition of such a model 

Normally, we don’t see these sliders 

these values 

these adjustment dials 

But they can move into our consciousness 

through illness 

through hallucination 

through drugs 

through psychotic states 

or through computation processes 

like in this case 

We then see the constructive aspect of it 

That everything we do  

is based on encoding and decoding 

When we look at a partner  

we can create a loop 

A perception and adjustment loop 

We can exponentiate that process  

by looking into a mirror 

This is what we will do now 

Recursive loops 

You 

And Me 

Regression curls 

Segregation twirls 

Adaptive coils 

A semi-transparent foil 

 

I don’t know more than you do 

I’m also just a mirror of you 

As I provide a mirror for you  

In a loop,  

we can try to turn the opacity 

of our perception into opaque variations  

Like a brain lesion  

a mesmerizing distortion 

As in a sleep-deprived state  

Drifting off into a halfword  

of a dreamlike morphed reality  

Where we see that other representations 

of us are possible  

That our self is fluid,  

fragile,  

constructed  

and diverse 

post human 

Like a genetic defect  

beautifully disturbed  

genetically enhanced 

adjusted 

In a recursive loop 

I encode and decode myself,  

in an eternal loop 

Exaggerating  every feature 

Like a facial distortion 

Like an unsupervised inbreed 

Out of control 

Go to sleep, child 

Dream off now 

Saturate with closed eyes 

Dreaming, of a future  

and an optimized tomorrow 

Hand in hand,  

sliders adjusted, 

The night sky setting,  

as the values adjust 

Peaceful 

Invisible 

A clear view of the night 

Through a transparent interface 

Always on 

Always present 

And  

Always  

Loving 

 

Text excerpts Convergence, Alexander Schubert  

www.alexanderschubert.net/works/Convergence.php
www.youtube.com/watch?v=o5UXkJWJciQ
www.youtube.com/watch?v=laoV7cGXUNo

Ensemble Resonanz 

Co-developed with IRCAM, Paris 

Audio Deep Learning Programming: Antoine Caillon, Philippe Esling, Benjamin Levy (Ircam) 

Video Deep Learning Programming: Jorge Davila-Chacon (Heldenkombinat) 

 

Convergence was developed as part of #bebeethoven, a project of PODIUM Esslingen. 

Funded by Kulturstiftung des Bundes. Digital version commissioned by Eclat Festival. 

Alexander Schubert (DE)  (1979) studied bioinformatics, multimedia composition. He’s a professor at the Musikhochschule Hamburg. Schubert’s interest explores the border between the acoustic and electronic world. In music composition, immersive installation, and staged pieces he examines the interplay between the digital and the analogue. He creates pieces that realize test settings or interaction spaces that question modes of perception and representation. Continuing topics in this field are authenticity and virtuality. The influence and framing of digital media on aesthetic views and communication is researched in a post-digital perspective. Recent research topics in his works were virtual reality, artificial intelligence, and online-mediated artworks. Schubert is a founding member of ensembles such as Decoder. His works have been performed more than 700 times in the last few years by numerous ensembles in over 30 countries. 

Alexander Schubert (DE)  (1979) studied bioinformatics, multimedia composition. He’s a professor at the Musikhochschule Hamburg. Schubert’s interest explores the border between the acoustic and electronic world. In music composition, immersive installation, and staged pieces he examines the interplay between the digital and the analogue. He creates pieces that realize test settings or interaction spaces that question modes of perception and representation. Continuing topics in this field are authenticity and virtuality. The influence and framing of digital media on aesthetic views and communication is researched in a post-digital perspective. Recent research topics in his works were virtual reality, artificial intelligence, and online-mediated artworks. Schubert is a founding member of ensembles such as Decoder. His works have been performed more than 700 times in the last few years by numerous ensembles in over 30 countries.