Skip to main content

Qwen3-Embedding-8B

‍

Description​

The Qwen3 Embedding model by Alibaba is designed for creating embeddings, which are essential for a retrieval-augmented generation (RAG) system. This model does not support chat interaction. It is used to convert documents and texts into individual vectors, which are stored in a vector database by the application, enabling semantic search. The strength of this model lies in its support of over 100 languages with high performance.

This model has the lifecycle status LTS‍. If we shut it down or replace it, we announce that at least 6 months in advance.

The following limits and defaults apply:

  • Maximum context length: 32,768 tokens
  • Default embedding dimension: 4,096
  • Maximum number of entries in input per request: 200
  • Supported values for dimensions: 256, 512, 768, 1,024, 1,536, 2,048, 3,072, and 4,096

It is recommended to format embeddings for search queries using the following template:

Instruct: {task_description}
Query: {query}

Here, {query} should express the individual search query in one sentence, and {task_description} should describe the task, for example:

task = 'Given a web search query, retrieve relevant passages that answer the query'
query = 'Explain gravity'

Batch inputs​

You can pass an array of texts to /v1/embeddings to generate multiple embeddings in one request. The index on each response item identifies the position of its input text, starting at 0:

from openai import OpenAI

client = OpenAI(
    base_url="https://llm.aihosting.mittwald.de/v1",
    api_key="YOUR_API_KEY",
)

texts = ["First document", "Second document"]
response = client.embeddings.create(
    model="Qwen3-Embedding-8B",
    input=texts,
)

for item in response.data:
    print(texts[item.index], len(item.embedding))

A request with more than 200 entries in input is rejected with HTTP 400 and the error message Batch too large. Split larger lists into several requests.

Reducing embedding dimensions​

Qwen3-Embedding-8B supports server-side dimension reduction through the OpenAI-compatible dimensions parameter. This is a supported model capability based on Matryoshka Representation Learning‍.

Set dimensions to 256, 512, 768, 1,024, 1,536, 2,048, 3,072, or 4,096. The endpoint returns a vector with exactly the requested number of values and L2-normalizes it after reducing its dimensions. Values below 256 are not supported because retrieval quality degrades at smaller dimensions.

For example, request an embedding with 256 dimensions:

from openai import OpenAI

client = OpenAI(
    base_url="https://llm.aihosting.mittwald.de/v1"
)

response = client.embeddings.create(
    model="Qwen3-Embedding-8B",
    input="Testing 123",
    encoding_format="float",
    dimensions=256,
)

print(len(response.data[0].embedding))  # 256

Smaller embeddings reduce transfer and storage requirements but can also affect retrieval quality. Evaluate the chosen dimension with data representative of your use case.

Terms of use and licensing​

The model is provided by Alibaba under the Apache 2.0 License‍, and reuse of the generated content is not subject to any additional restrictions.

On this page