Monday, Oct 26 | --:--
Back to home

Block's AI Lab Unveils Dessa, a Voice Model It Says Learned to Talk Only From Recorded Speech

Block says Dessa, built by a small lab that reports to Jack Dorsey, is a full-duplex speech-to-speech model trained from scratch on human conversation, with no text and no pretrained models, and that it runs locally on a MacBook Pro M4 Max or better. Block has shared only its own recordings. It has published no benchmarks, model size, weights, API, pricing or release date, and early access is going to a small group of AI safety researchers.

Times of AI Desk 4 min read Oakland, CA View as Markdown
A warm sound wave and a cool one braiding across a quiet room, the cool wave rising from a closed laptop on a wooden table, faint ripples drifting above.

If Block's description holds up, Dessa is a conversational voice model that learned language by listening alone, without the text-trained language model that most voice AI builds on. For now, the evidence is Block's own account and the recordings it chose to publish. Block introduced the model on Friday, October 23, on its website and on X.

What Block says Dessa does

Block says it trained Dessa "from scratch on nothing but human conversation," adding: "No text, no pretrained models." Block calls Dessa "a true speech-to-speech model" and says it is full-duplex, meaning it listens and talks at the same time. Block says the model learned from real conversation when to cut in and when to make short listening sounds such as "yeah!" and "got it!"

Block says inference "runs faster than real time" and that Dessa "runs locally on a MacBook Pro M4 Max or better." Ragavan Thurairatnam, who leads the project, wrote on X that Dessa is a model "that you can run on consumer GPUs." Neither gives memory requirements or speed figures.

Block says Dessa learned language from "hundreds of times fewer words" than large language models, and that the team trained it on "only one NVIDIA GB200 NVL72," a single rack of Nvidia data-centre hardware. TSN Media, which covered the launch, noted that "there are no benchmarks, no model size, no data figures and no number behind 'hundreds of times fewer words'."

Block's page calls Dessa "the most authentic full-duplex speech model ever built" and says: "As far as we know, Dessa is the world's first AI you can have a realistic live conversation with that was trained only from human speech." Thurairatnam called it "a GPT3 moment for speech." Those are Block's and Thurairatnam's own descriptions, and no outside group has tested them.

What the published recordings show

Block says every clip on its page "is a live recording of a person talking to Dessa" and that subtitles were added afterward. The Verge's Jay Peters wrote that the chats Block shared "can occasionally be a little awkward and uncanny." In one, Dessa takes the speaker for someone called "Ronaldo," he wrote.

Another clip pairs Dessa with GPT-Live-1; Block's caption says "here it talks to ChatGPT Live," and its disclaimer calls GPT-Live-1 "a third-party AI model" whose developer "is not affiliated with Block and has not reviewed or endorsed Dessa." The Verge identifies it as OpenAI's GPT-Live-1 voice model, which OpenAI launched in ChatGPT Voice in July, and wrote that Dessa "comes across as more natural" in that exchange.

Who built it, and why

Block describes the team as a "(very) small AI research lab at Block, founded by former members of Dessa, the Toronto deep learning startup," and says it reports directly to Jack Dorsey, Block's co-founder and chairman. Square, as Block was then called, said in February 2020 that it had acquired Dessa, a Toronto company known for work that included deepfake detection. Square changed its name to Block on December 1, 2021, according to a note on that release.

Block gives three reasons for the project: "curiosity (aka fun), better understanding between humans and AI, and safety." It argues that voice AI that turns speech into text misses emotion: "Imagine being in serious pain while a voice AI has no idea and just keeps going. That's the safety problem."

Who can use it

Thurairatnam writes on the page that Block is giving "a small group of AI safety researchers early access to find the risks we haven't anticipated." The page also has a waitlist button and says more recordings will be shared on X.

Limits

  • Every capability claim here comes from Block or Thurairatnam, and the only evidence published is recordings Block selected.
  • Block has not released benchmark results, the model's size, the amount of audio used, weights, an API, pricing or a release date.
  • The two hardware claims differ: Block's page says a MacBook Pro M4 Max or better, while Thurairatnam's post says consumer GPUs.
  • Block's page does not name GPT-Live-1's developer; the identification as OpenAI's model is The Verge's.
  • We read Block's page, both X posts and the Verge and TSN Media reports in full; we did not test the model or watch the X videos.

Sources

Prior Coverage

Earlier Times of AI reporting on this thread.

Scroll to continue reading