Haru-4B, Built on Qwen3.5, Listens and Talks at the Same Time Like a Phone Call

Share

Japanese company higurashi open-sourced Haru-4B, a voice dialogue model built on Qwen3.5-4B that listens and talks at once. It backchannels, handles interruptions, and keeps up when two people talk over each other. Licensed Apache-2.0 for commercial use, it runs locally on an NVIDIA GPU with about 12 GB of VRAM or a Mac with an Apple chip and 16 GB of memory. The team measured about 65 ms of processing latency on an M1 Max. It is a pretrained research base model, not a ready-made assistant.

Machine translation. Read the original

SourcehigurashiReported Nov 13, 2026Not yet tested

On November 12, the Japanese AI company higurashi open-sourced Haru-4B, a voice dialogue model built on Alibaba's Qwen3.5. Its biggest difference from everyday voice assistants is that it listens and talks at the same time. While you are still speaking, it can say "mm-hm" or "right" to show it is following along. If you interrupt it, it stops and listens. The code and model weights are both under the Apache-2.0 license, so commercial use is allowed. You can run it locally on one NVIDIA graphics card with about 12 GB of VRAM, or on a Mac with an Apple chip.

Two audio tracks from an English conversation: the model keeps talking while the user says mm-hm, the two voices overlap, and the tone turns serious as the topic gets heavier

Image from the demo video on higurashi's official technical blog. The top and bottom tracks are the two speakers in the conversation.

What Happened

Common voice assistants work like walkie-talkies. They wait until you finish and pause, then start thinking and speaking. You can't cut in. Haru-4B takes the phone-call approach, which the industry calls full duplex:

Turn-based voice assistant Haru-4B
When it starts talking After you finish and it detects a pause Any time. It listens and decides as it goes
When you interrupt Usually you have to wait for it to finish, or press a button to cut it off Just keep talking. It gives way to you
Backchannels and overlap Almost none Says "mm-hm" and "right," and can handle two people talking at once
Tone Writes a text answer, then reads it aloud Changes with the mood of the conversation

Every 133 ms, it processes a small slice of audio. It takes in a short piece of what you say, and at the same time it outputs a short piece of its own speech along with the matching text. So listening and speaking never wait on each other. Training used more than 40 million hours of real speech. The team says it deliberately avoided AI-generated question-and-answer dialogue, so the model would learn the pauses, interruptions, and backchannels of real people talking.

Haru-4B structure diagram: every 133 ms, the user's voice, the model's previous frame of text and audio are combined and fed into the language model, which predicts the next piece of text and audio at the same time

Image from the project's GitHub repository.

What Hardware It Runs On

  • NVIDIA graphics cards: About 12 GB of VRAM. The team provides a web demo, so you can open it in a browser and talk into your microphone.
  • Macs with Apple chips: There is a native macOS app. The model is about 10 GB, and 16 GB of memory is the practical minimum. On a MacBook with an M1 Max, the team measured about 65 ms of processing latency, from microphone input to headphone output, using int8 quantization. Splitting the audio into chunks can add up to another 133 ms.
  • Servers: It can be deployed with vLLM-Omni to serve many conversations at once.

It supports nine languages: Chinese, English, Japanese, and Korean, plus Spanish, Portuguese, German, French, and Russian.

Want to try it? Give this prompt to your AI agent:

Help me try the open-source voice model Haru-4B on this computer. Repo: https://github.com/higurashi-ai/Haru
1. First read the repo README and tell me whether my computer can run it (an NVIDIA GPU with about 12 GB of VRAM, or an Apple chip Mac with 16 GB of memory), and how big the model download is.
2. Ask me before installing any software or downloading the model.
3. After it is installed, start the demo and tell me which address to open. It needs a clean recording of just one person speaking, 3 seconds to 3 minutes long, as the voice. I will use my own voice.
4. Do not add the --share flag, and do not expose the demo to the public internet.

Notes

  • It is not a ready-made assistant: The team is clear that Haru-4B is a pretrained base model for research. It talks naturally, but its facts are not reliable, and it won't follow instructions the way an assistant does. To build a product on it, you would need to train it further.
  • It can imitate voices: It speaks in the voice of the reference recording you give it. Only use a voice you have the right to use, and don't use someone else's recording.
  • Bluetooth headsets add lag: The team recommends wired headphones and a wired microphone. Bluetooth adds more than 100 ms.
  • Share links go through third-party servers: Running the demo with --share creates a public link that anyone can use. Your audio also passes through Gradio's servers.
  • The model has only 4 billion parameters. The team itself says answer quality is limited by that size, and it is training a larger version with more than 100 billion parameters.

References

Done with this one? Mark it to track your path.Sign in
Save it for laterSign in
Questions about this piece?Ask AI about this

Comments (0)

No comments yet. Start the conversation.

Sign in to join the discussion.Sign in