AI is happening. The future is updating.Qwen3-Embedding-4B-FP8-Dynamic
Embeddings are the pooled last hidden state; the head is unused and left unquantised. Cosine similarity is scale-sensitive, so compare rankings rather than raw distances when validating this checkpoint. `WikipediaRetrievalMultilingual` from MTEB — retrieval over Wikipedia in 16 languages, scored by `ndcg_at_10`. Serving Qwen/Qwen3-Embedding-4B in bf16 leaves little room for KV cache on a small GPU: the weights take what the cache needs, and the context length has to be cut until it fits. Halving the weights gives that memory back — the same card serves a longer context without any other change.
Embeddings are the pooled last hidden state; the head is unused and left unquantised. Cosine similarity is scale-sensitive, so compare rankings rather than raw distances when validating this checkpoint. `WikipediaRetrievalMultilingual` from MTEB — retrieval over Wikipedia in 16 languages, scored by `ndcg_at_10`. Serving Qwen/Qwen3-Embedding-4B in bf16 leaves little room for KV cache on a small GPU: the weights take what the cache needs, and the context length has to be cut until it fits. Halving the weights gives that memory back — the same card serves a longer context without any other change.