ultravox-v0_6-llama-3_1-8b

Model overview

Ultravox is a multimodal Speech LLM built around a pretrained LLM (Llama, Gemma, Qwen, etc) and a speech encoder (whisper-large-v3-turbo) backbone. Ultravox is a multimodal model that can consume both speech and text as input (e.g., a text system prompt and voice user message). The input to the model is given as a text prompt with a special ` ` pseudo-token, and the model processor will replace this magic token with embeddings derived from the input audio. Using the merged embeddings as input, the model will then generate output text as usual. In v0.6 series, ultravox models are trained on expanded Hindi speech data, resulting in significantly improved speech understanding performance on Hindi and modest degradation on other languages. Additionally, the v0.6 models are also trained on noise datasets for improved noise robustness and the ability to output a special string ``((noise))`` if the input audio is too noisy or doesn't contain clear speech.

Key features

特点摘要待依据来源补齐，不凭模型名称推断能力。

Limitations & considerations

Sources are insufficient to establish specific limitations. Verify task performance, latency, data policies, licensing and version lifecycle.

Source prices & cost analysis

Prices vary by channel, region and configuration. Channel quotes are listed independently and are not treated as a single official provider price.

Public benchmarks & run configuration

暂无已采集的可核验公开评分。模型未上榜不代表能力较低。

Frequently asked questions

Sources & traceability

许可证：mit。模型介绍与第三方评测均以原来源的具体版本和方法为准。