AI music could revolutionize the industry — and this artist is leading the way

This story was originally published on September 29, 2022.

As artificial intelligence has gotten better, fears about our AI-enabled future have grown. Even small steps toward more sophisticated AI have triggered concerns that robots will take away human jobs or that AI-generated deepfakes will warp our sense of reality.

In the world of art, misgivings about AI have been particularly acute. An AI-generated piece of art recently won first prize in a competition at the Colorado State Fair, leading to an outcry from artists and critics alike. Other experiments with replacing illustrators and writers have caused similar handwringing.

Despite this widespread apprehension, Holly Herndon — an experimental musician and artist — doesn't think AI is nefarious or will ruin art. These technologies are here to stay, she told me recently, and we might as well learn how to live with them.

"I think the best way forward is for artists to lean into developments with machine learning," she said, suggesting that they "think of ways to conditionally invite others to experiment with them."

Last year, Herndon launched a project called Holly+, a tool that enables other artists to make music using an AI-generated likeness of her voice. The works created using Holly+ are essentially vocal deepfakes sanctioned and encouraged by Herndon.

Herndon's usage of AI shows a way forward for not just artists but also the rest of us. Instead of rejecting new technology, everyone from workers to businesses to amateur creators can learn to use these tools to make their lives easier or open up new creative avenues. By embracing AI, would-be skeptics can help to ensure the burgeoning technology is used for good, rather than leaving it to be shaped by ill-intentioned parties. This doesn't ensure a wholly positive outcome, but with projects like Holly+, Herndon highlights the advantages of a technology like AI, while proving it isn't a mere substitute for human creativity.

From an AI 'baby' to deepfakes and crypto

In Berlin, Herndon has made AI a recurring theme of her work over the past decade, experimenting with human-computer collaboration in her recordings and live performances. An early instance of this was her 2015 album, "Platform," which combined the organic qualities of the human voice with machine sounds and digital-processing techniques. For her 2019 album, "Proto," Herndon took this approach further by creating Spawn, an AI "baby" that merged her voice and the voice of her partner, Mat Dryhurst, into a synthetic female entity that provided additional vocals.

The Holly+ project synthesizes the insights that Herndon has gathered over a decade of engaging with AI, a process of inquiry that has included her earning a doctorate of musical arts at Stanford University's Center for Computer Research in Music and Acoustics. The core of Holly+ is straightforward: Anyone can upload an audio file, and the tool will create a new version with Herndon's processed voice, ready for download. In her blog post announcing Holly+, Herndon said she envisioned a future where digitally created voices "soon become standard practice for artists and other creatives." 

A wide variety of music has been created using Holly+, from glitchy electronic tracks to ambient compositions to more conventional pop songs. Many of the tracks push the human voice to its limits, chopping Herndon's voice into nearly unrecognizable fragments or repurposing them as pieces of a sonic collage. Herndon has also used Holly+ in live performances, appearing at the Helsinki Festival with a local choir.

While the project has yielded an ever-growing array of fascinating art, Holly+ also raises questions about the future of that art. If AI can replicate a unique voice like Herndon's, or mine the vast archive of existing human-produced content, then will it eventually replace humans' role in making that art going forward? And who owns the art that an AI model produces?

Who owns what? 

AI's prospective influence on the art world is not without precedent. The ease of copying, editing, and distributing digital files today means that content can always change, frequently without any acknowledgment that it has been altered. Analog media, in contrast, is more difficult to modify, and doing so typically leaves a mark. Now, the official versions of songs are endlessly remixed, films and television episodes are altered after their initial release, and text and images go viral as unattributed, ever-evolving memes. Sampling — reusing a portion of a song in another recorded song — has become a staple of pop music over the past few decades. 

"In the same way that trying to shut down sampling of media was a dead end, we have entered into an AI era where it will become effortless to generate media using someone's voice, face, or style." Herndon told me. As with sampling, she argued, AI presents a great opportunity for artistic play and innovation "so long as we can mitigate the economic shock of these powerful new tools."

Holly+ highlights a few shocks that might come about, particularly around the ability for artists to control their intellectual property — the legal basis for copyright laws. It's hard enough to lay claim to the content you have created now, but emerging AI tools open up a new set of complications. Is an AI-generated image made in the style of a particular artist plagiarism, or is it an original product of the model that generated it? As AI models improve and become more deeply embedded in the digital world, artists will need to understand the nature of the IP they produce and devise strategies for retaining control of that IP.

"I started the Holly+ project because I think this is a generational opportunity to rewrite how we think about IP," Herndon told me. In the Holly+ announcement post, Herndon said, "There will be demand for official, high fidelity, vocal models of public figures."

This is where crypto comes in: the blockchain technology that undergirds crypto allows users to precisely track ownership of digital files like songs and images, while establishing an indelible record of their origins. This enables artists to sell digital works and ensure they will always receive credit for creating them (even if the work itself is copied freely), while giving the works' buyers confidence that they can prove they have the original version — an ability that, in turn, makes them willing to pay more.

Crypto has thus allowed Herndon to help financially reward artists who create songs with Holly+, while receiving compensation herself. To do this, Herndon established a decentralized autonomous organization, or DAO, which is similar to a worker cooperative. Anyone who owns a specific DAO's crypto tokens becomes a member with voting rights. When other artists create work using Holly+, they can submit it to the Holly+ DAO, whose members vote on which submissions are good enough to be turned into nonfungible tokens and sold via an auction process — think of an NFT as the definitive "original" pressing of a song. (Anyone can use the Holly+ voice tool itself without submitting the output to the DAO, however.) Like a company's shareholders, the members of the DAO benefit from the overall quality of the Holly+ brand via the value of their tokens. Herndon says on her website that DAO members "are incentivized to only certify or license new works that contribute to the value of the voice."

This arrangement benefits Herndon by minimizing the incentives to make bad art with Holly+ or use her voice in a negative way, while relieving her of sole responsibility for managing the tool. The proceeds from each NFT sale are split among the work's creator, the members of the Holly+ DAO, and Herndon — an arrangement that reflects each party's contribution to a given piece. As AI continues to evolve and becomes more central to various creative enterprises, the DAO structure offers a way to credit participants more fairly and help artists retain more control over their work.

AI isn't alien

As technology continues to evolve, artists and creators must keep adapting. Herndon's work, especially Holly+, shows how creative people can approach developments like AI with an open mind, while maintaining a pragmatic awareness of the technology's drawbacks. She said: "Most AI processes we are familiar with now are groundbreaking ways of aggregating the products of human intelligence." In other words, AI can be another tool that artists use, instead of just a threat to their livelihoods. 

And working with AI can be fun. "I am in love with the submissions!" Herndon said of the works created with the Holly+ tool. "Artists have taken my voice in all kinds of directions. There are trains whistling at me, tender duets, and insane club tracks." 

Now, Herndon is taking the lessons from Holly+ to her new organization, Spawning, which will equip other artists to reap the benefits of AI by using it to enhance their work, while protecting their IP from the threats it introduces. Herndon told me that the organization aimed "to help others take that leap and make their own decisions about how their voices and styles are used."

While artists are among the first to seriously grapple with the implications of AI's cutting edge, the technology is likely to affect everyone — whether in their work or in another area of their lives. Holly+ and the Spawning organization both offer models for embracing technological change and making the most of it, rather than trying to resist or avoid it.

"I see how many things could go wrong," Herndon told me, "but I'm committed to creating ways to steer things in a better direction."

Drew Austin is a technology critic, writer, and author of the newsletter Kneeling Bus.