Block’s AI research lab has released a speech model it calls Dessa, and the team says it was trained entirely on human conversation. No transcripts, no text and no pretrained models went into it. The lab says Dessa learned to hold a real-time conversation simply by listening, and that as far as it knows, it is the first AI people can talk with realistically that was built only from human speech.

The lab is small, by its own description. It sits inside Block and was founded by former members of Dessa, the Toronto deep learning startup. Block says the team has built deep learning models since 2012 and generative speech models for more than eight years. Its researchers report directly to Jack Dorsey, Block’s cofounder, chairman and Block Head.

The lab’s main pitch is that today’s voice AI throws away too much. Most voice systems convert speech into text first, and the lab argues that process strips out tone, timing, pauses and emotion. Block’s researchers say that most voice assistants feel stilted and slow for that reason. Dessa, by contrast, works speech-to-speech. Audio goes in and audio comes out, with no text step in between.

The model is full-duplex, meaning it can listen and talk at once, much as people do in a conversation. Because it learned from real conversation, the lab says, it knows when to interrupt and when to offer short acknowledgments such as “yeah,” “got it” or “ooh” to show it is paying attention. Block says inference runs faster than real time, so there is no awkward lag before a reply. It also says Dessa runs locally on a MacBook Pro M4 Max or better.

To show how it behaves, the lab posted a series of recordings of people talking with Dessa. In each one, the person speaks freely and Dessa decides what to say, when to say it and how to say it, including its own voice and personality. The lab added subtitles afterward. In one clip, Dessa laughs mid-sentence when something is funny. In another, a person blanks on a movie title and Dessa finishes the sentence. When asked to speak up, it gets louder. The lab says nobody trained it to do that.

Other clips show a wide range in intonation and speed, a moment where Dessa gasps and then responds with what sounds like teary gratitude, and a conversation in which Dessa talks with ChatGPT Live. The lab says more recordings will be shared on X soon.

Block’s researchers also compared Dessa’s training data with that of large language models. They say Dessa learned language from hundreds of times fewer words, though they noted that large language models also have to learn programming languages and other material that Dessa does not need. The lab argues that learning from speech is harder in some ways. Speech never repeats exactly, even when a person says the same sentence twice. A single spoken word takes thousands of times more bytes than the same word written down. Accents, pauses, breathing and stress can all change what a sentence means.

The lab says it trained Dessa on a single NVIDIA GB200 NVL72 system. It gives three reasons for building the model: curiosity, better understanding between humans and AI, and safety. On safety, the researchers argue that AI systems are gaining capability while still being largely unable to detect emotion in a person’s voice. They offer a scenario in which a person is in serious pain and a voice AI keeps going without noticing.

Block says it is now giving a small group of AI safety researchers early access to Dessa. The goal is to find risks the lab has not anticipated, since a model that learns to talk from sound alone is new territory. Ragavan Thurairatnam, the Dessa lead, is named on the announcement. The lab has listed its researchers, interns and former interns on its page, and it is inviting interested researchers to get in touch through a contact link on the same page.

Questo articolo è stato scritto con l'assistenza dell'IA.
News Factory APP - notizie agentiche per potenziare il tuo SEO e AEO.