AIELSON

PAOLA TORRES NÚÑEZ DEL PRADO (PE)

Machine Learning is an innovation that is already reshaping art and society in general, nevertheless, the algorithms developed tend to reproduce the ethnic and cultural biases that prevail in contemporary society. By using a pre-Trained Transformer-based Language Model (GPT-2) retrained with poetry in Spanish, fine-tuned to examples of South American poetry (including my own), and then using this output to feed a text-to-speech synthesis system (Tacotron 2 + Waveglow, trained on an Argentinean voice data set and then on voice examples of Peruvian poet Jorge Eduardo Eielson), I came up with a selection of spoken-word poems in a distinctly Latin American voice that ended up as a vinyl album called El Tiempo del Hombre (The Time Of Man) that is the first of its kind.

In the 60s, Peruvian poet Jorge Eduardo Eielson imagined a talking doll made up of, among other things, a magnetic tape that stored poetic texts that would allow it to “continually recite the most beautiful poems conceived by man.” With the support of Google’s Artists + Machine Intelligence program, I came up with what I call the AIELSON (neologism that mixes “A.I.” with “Eielson”) Spoken-Word Poetry Generation System. It is relevant to point out that this album is not a deepfake (fictitious audiovisual content made with Machine Learning tools), as it is not presented as if it were the late poet’s work, but as an artificial entity that emulates the poet’s voice when reciting.

 

Intrigued by the possibility that new TTS systems offer regarding the recollection (picking-up) of stylistic qualities that could be considered typical of the region (South America), I intended to draw attention to speech features such as accent and intonation, characteristic of the inhabitants of the Central Coast region, Peru (where the poet was from), as well as stress and rhythm, in this case, the particular way the poet had of speaking beyond his origin while also referencing the presence of the Quipu, that ancient Andean textile notation system, not only in the late poet’s work, but within the history of Art, theme that connects the poetry selected for fine-tuning the model.

Machine Learning is an innovation that is already reshaping art and society in general, nevertheless, the algorithms developed tend to reproduce the ethnic and cultural biases that prevail in contemporary society. By using a pre-Trained Transformer-based Language Model (GPT-2) retrained with poetry in Spanish, fine-tuned to examples of South American poetry (including my own), and then using this output to feed a text-to-speech synthesis system (Tacotron 2 + Waveglow, trained on an Argentinean voice data set and then on voice examples of Peruvian poet Jorge Eduardo Eielson), I came up with a selection of spoken-word poems in a distinctly Latin American voice that ended up as a vinyl album called El Tiempo del Hombre (The Time Of Man) that is the first of its kind.

In the 60s, Peruvian poet Jorge Eduardo Eielson imagined a talking doll made up of, among other things, a magnetic tape that stored poetic texts that would allow it to “continually recite the most beautiful poems conceived by man.” With the support of Google’s Artists + Machine Intelligence program, I came up with what I call the AIELSON (neologism that mixes “A.I.” with “Eielson”) Spoken-Word Poetry Generation System. It is relevant to point out that this album is not a deepfake (fictitious audiovisual content made with Machine Learning tools), as it is not presented as if it were the late poet’s work, but as an artificial entity that emulates the poet’s voice when reciting.

 

Intrigued by the possibility that new TTS systems offer regarding the recollection (picking-up) of stylistic qualities that could be considered typical of the region (South America), I intended to draw attention to speech features such as accent and intonation, characteristic of the inhabitants of the Central Coast region, Peru (where the poet was from), as well as stress and rhythm, in this case, the particular way the poet had of speaking beyond his origin while also referencing the presence of the Quipu, that ancient Andean textile notation system, not only in the late poet’s work, but within the history of Art, theme that connects the poetry selected for fine-tuning the model.

khipucamayoc.github.io

Initially funded and developed as part of Google’s Artists + Machine Intelligence 2020 Residency Program. Creative Technologist Holly Grimm supported the implementation of the Machine Learning models used in the generation of both text and speech required for the creation of this album and its connected artworks.

Paola Torres Núñez del Prado (PE) is an artist and a researcher of transdisciplinarity. Her work is essentially complex: she explores the limits of the senses, examining the concepts of interpretation, translation, and misrepresentation in order to analyze the construction of a cultural hegemony in relation to the history of technology and the arts. She was awarded the Google Artists + Machine Intelligence Residence Grant in 2019 and Vivo Arte.mov's Mobile Media Award, Brasil, in 2013. She has presented her work in various countries of the Americas, Central Europe, and Scandinavia, where she is based. Her work is part of the collections of Malmo Art Museum and the Public Art Agency of Sweden.

Paola Torres Núñez del Prado (PE) is an artist and a researcher of transdisciplinarity. Her work is essentially complex: she explores the limits of the senses, examining the concepts of interpretation, translation, and misrepresentation in order to analyze the construction of a cultural hegemony in relation to the history of technology and the arts. She was awarded the Google Artists + Machine Intelligence Residence Grant in 2019 and Vivo Arte.mov's Mobile Media Award, Brasil, in 2013. She has presented her work in various countries of the Americas, Central Europe, and Scandinavia, where she is based. Her work is part of the collections of Malmo Art Museum and the Public Art Agency of Sweden.