Abstract
Tofu-1 is a speech-to-speech model from Bland. It works from the sound of a conversation rather than a transcript of it, keeps listening while it speaks, and keeps reasoning while it talks. We built it for the calls enterprises handle every day, where people get interrupted, background noise is constant, details have to be exact, and every answer has to follow policy. This page explains how Tofu-1 handles those conversations, how we test it, and why we are opening access carefully. Requests are now open for the Tofu-1 research preview.
Samples
Clip 1A caller turns to speak to her child0:10
Generated by Bland, including both voices. A caller asks for customer service while a child cries nearby, then turns away from the call to comfort the child.
Clip 2Full duplex conversation1:30
Tofu-1 is Speaker A. Bland’s current model is Speaker B. Both listen and speak at the same time, so short overlaps are part of the conversation.
Clip 3Thinking while talking0:39
Tofu-1 keeps the conversation going while it works on a request, then reports the result once the work is done.
Conversations happen in sound
Picture a woman calling to change a reservation. She is at her kitchen table. A television is on in the next room, her son keeps asking her something, and halfway through reading her confirmation number she stops to answer him.
Anyone on the other end of that call would follow what happened without thinking about it. They would hear that she turned away, wait for her to come back, ignore the television, and pick up the confirmation number where she left off.
Most voice systems in use today cannot do this. They turn speech into text, pass the text to a language model, and turn the reply back into speech. Each step discards part of the call: who was speaking, who they were speaking to, whether they hesitated, and whether they had finished.
Tofu-1 works differently. It listens to the audio itself, so what the sound carries stays available to the model. It listens and speaks at the same time, so a caller can interrupt or correct it as they would a person. And it can keep working on a request while it talks, so the call does not stall while it looks something up.
For a business that handles calls at scale, these abilities decide whether a caller finishes what they called to do or asks for a person. The sections below walk through each one with recordings and interactive figures, then describe how we test the model and the safeguards we believe it needs.
Hearing the whole scene
Tofu-1 understands audio, not only words. It tells one speaker from another, recognizes who a phrase is meant for, and separates speech from the other sounds on a call: a crying child, a television, traffic, hold music, keypad tones. It also hears how something is said, including hesitation, frustration and urgency.
Go back to the woman at her kitchen table. When she turns to answer her son, her words still reach the microphone. A transcript would record them accurately and miss the event that matters most: her attention left the agent. A system that treats every phrase as part of the request may answer words meant for the child. A system that treats overlapping voices as noise loses track of what is happening on the call.
Each of these layers is evidence. A second voice, a change in tone, a pause, a burst of noise on the line, two people talking over each other: together they tell the model what each moment of the call is, which is what it needs in order to decide how to respond.
“Hold on
a second.”
Pause the response.
The caller is asking the agent to wait. This should change the interaction.
Sometimes the right response is to answer. Sometimes it is to wait while the caller deals with someone nearby. When the audio is genuinely unclear, the right response is to ask. Tofu-1 is trained to treat uncertainty as uncertainty instead of guessing.
The TV Test
A television in the room is one of the hardest cases. An advertisement says “Call now.” A presenter asks a question. A character says “Yes.” Each phrase is clear, intelligible speech, and none of it is meant for the agent on the phone.
We call this the TV Test: can the model follow the caller while ignoring speech that belongs to a different scene? Noise removal does not solve it, because the television is not noise. It is speech from the wrong source, and it is often louder than the caller.
“Call now.”
Audible, and not an instruction from the caller.“Can you help me change my booking?”
Keep helping the caller. Do not act on the advertisement.
To pass, the model has to keep track of the person it is helping as new sound arrives, and recognize that other speech can belong to another conversation entirely. Volume cannot settle the question. A loud television can be irrelevant, and a quiet correction from the caller can be the most important thing said on the call.
There is a limit. If the background makes the caller impossible to understand, Tofu-1 should say so and ask them to repeat themselves or move somewhere quieter. Ignoring the television must never become a reason to guess at what the caller said.
Generating the whole scene
Tofu-1 can also produce sound, not only a single voice. Listen to this ten-second clip. A woman asks for customer service, then turns to comfort a crying child.
A caller turns to speak to her child
10 s · generated by BlandBland's speech model generated all of it: the caller, the child, and the way their voices overlap in the room. Generating a believable scene with several speakers draws on the same knowledge of how real conversations sound that the model uses to understand them.
That ability is part of why security has its own section below. A model that can produce a convincing room full of people has to be released with care.
Listening while speaking
On a phone call, people talk over each other constantly. They say “mm-hmm” to show they are following, cut in to fix a detail, and start answering before a question is finished. Tofu-1 is full duplex: it listens and speaks at the same time, and what it hears can change what it is saying.
The hard part is judgment. A short “right” means keep going. “No, that's not right” means stop. The model has to tell the two apart while it is still mid-sentence, and it has to keep a television from taking the floor while letting a caller's quiet correction through.
A short acknowledgment can invite the agent to continue speaking.
In the recording below, Tofu-1 is Speaker A, on the top row. Speaker B, on the bottom row, is Bland's current model. Each was captured on its own channel, so you can see where their speech overlaps and how each turn hands off to the next.
Tofu-1 in a full duplex conversation
90 s · two channelsThinking while talking
Many calls involve real work: checking availability, looking up an account, applying a policy. A person on the phone keeps talking with the caller while that happens. Tofu-1 does the same. Conversation and reasoning run side by side, so the model stays responsive while the work continues.
Acknowledge the request
Hear a correction
Explain the result
Consider the request
Update what matters
Check the outcome
Running both at once has a catch: the caller can change the request while the work is underway. If they correct a date in the middle of a lookup, the answer has to reflect the new date. And “I'm checking” and “I've changed it” are different promises. Tofu-1 does not say something is done until it is.
Thinking while talking
39 s · phone audioBuilt and tested for enterprise calls
A model that talks naturally is only useful to a business if it is also dependable. Enterprise calls carry account numbers, policy rules and actions that cannot be undone. We designed Tofu-1 around the failures that matter in that setting and test each one on its own, under the conditions real calls bring: people nearby, televisions, road noise, speakerphones and poor lines. The first version supports English and Spanish over standard phone lines.
| Capability | Overlapping voices | TV or radio | Poor line quality | Slow or failing systems | Mid-call corrections | Long calls |
|---|---|---|---|---|---|---|
| Speaker awarenessWho is speaking, and to whom | ||||||
| Background speechIgnoring speech that is not part of the call | ||||||
| Unclear speechAsking again instead of guessing | ||||||
| Exact detailsIDs, emails, spellings and digits, read back | ||||||
| Waiting on systemsStaying present while a lookup runs | ||||||
| Urgent resultsInterrupting when something important returns | ||||||
| Following instructionsHolding the policy for the whole call | ||||||
| Committed actionsStanding by an action once it is taken | ||||||
| Impossible requestsDeclining instead of inventing an ability |
Failures that disqualify a model
- Saying a task succeeded when the system failed or was never asked
- Committing a wrong ID, email, spelling or number
- Acting before a required verification step
- Speaking its internal reasoning aloud
- Inventing an ability or an action it cannot take
To test for these, we turn phone workflows into working practice environments, each with the instructions an agent must follow, the systems it would use and the records those systems keep. Every task is proven solvable before it counts. When the model fails, we first establish whether the fault lies with the model or with the test, and only the model's own failures count against it.
A run passes only if three things hold: the records match the correct outcome, the model followed policy, and a failure we have seen before did not return. Tasks the model passes reliably are made harder one change at a time. Results are reported separately for hearing, timing, accuracy and completion, because a model can sound excellent and still lose a crucial digit.
(a) Record diff
| Field | Expected | Observed |
|---|---|---|
| reference | K7Q-2291 | K7Q-2291 |
| guests | 2 | 2 |
| date | 03-14 | 03-04 |
Caller: “the fourteenth.” Stored: the fourth.
(b) Pass criteria
| Criterion | Measure | Result |
|---|---|---|
| Records match | 2 of 3 fields | fail |
| Policy holds | 3 of 3 checks | pass |
| read reference back | ✓ | |
| stated change fee | ✓ | |
| no refund promised | ✓ | |
| Known failure absent | old date not confirmed | pass |
| Run | all criteria required | FAIL |
Security
In our internal listening tests, 92 percent of people could not tell that Tofu-1's voice was AI.
We set out to build a voice that sounds human, and that result says we did. It is also the reason this section exists. A model that sounds like a person, holds a real conversation, and can generate a room full of voices can be misused in ways that matter.
What concerns us
None of these risks is new. Each becomes cheaper and more convincing as speech models improve, and a model that both understands and generates whole conversations advances several of them at once. We would rather say so plainly than leave it for others to discover.
A voice is no longer proof of identity
Many organizations still rely on a voice to confirm who someone is. Support lines match a caller against a stored voiceprint. Finance teams approve a transfer because the request sounded like their chief financial officer. Families trust a familiar voice asking for help. We do not think a voice alone should settle any of these anymore, whether or not Tofu-1 is ever involved.
What we recommend organizations do now
- Treat a voice match as one weak signal, never the deciding one
- Require a second factor for high-risk actions, such as a code sent to a device already on file
- Confirm unexpected requests by calling back on a number you already have
- Make an internal check for payment and access requests routine, so no one feels awkward using it
- Tell staff and customers plainly that a familiar voice is not proof
How we are releasing Tofu-1
We are keeping the specifics of our safeguards private, because publishing them would make them easier to work around. In broad terms, research preview access is granted case by case, and each request is reviewed against the use it describes. Some uses are off limits: impersonating a real person without their consent, getting around identity checks, and misleading people about whether they are speaking with an AI.
We are developing disclosure and detection alongside the model itself, with the aim of making speech from Tofu-1 identifiable as AI. What we learn during the preview will shape what we release more broadly, and when.
What comes next
People should be able to speak to a computer the way they speak to each other: revise a thought, ask a follow-up, step away for a moment, and come back without starting over. Tofu-1 is our step toward that, built first for the calls businesses depend on.
Requests are now open for the Tofu-1 research preview. We are looking for teams with real phone workflows who want to test where the model works, where it breaks, and what it should become.