- Supports synchronous usage. No dependency on Tokio.
- Uses @pykeio/ort for performant ONNX inference.
- Uses @huggingface/tokenizers for fast encodings.
- Python: fastembed
- Go: fastembed-go
- JavaScript: fastembed-js
Click to list models
- BAAI/bge-small-en-v1.5 - Default
- BAAI/bge-base-en-v1.5
- BAAI/bge-large-en-v1.5
- BAAI/bge-small-zh-v1.5
- BAAI/bge-large-zh-v1.5
- BAAI/bge-m3
- sentence-transformers/all-MiniLM-L6-v2
- sentence-transformers/all-MiniLM-L12-v2
- sentence-transformers/all-mpnet-base-v2
- sentence-transformers/paraphrase-MiniLM-L12-v2
- sentence-transformers/paraphrase-multilingual-mpnet-base-v2
- nomic-ai/nomic-embed-text-v1
- nomic-ai/nomic-embed-text-v1.5 - pairs with nomic-embed-vision-v1.5for image-to-text search
- intfloat/multilingual-e5-small
- intfloat/multilingual-e5-base
- intfloat/multilingual-e5-large
- mixedbread-ai/mxbai-embed-large-v1
- Alibaba-NLP/gte-base-en-v1.5
- Alibaba-NLP/gte-large-en-v1.5
- lightonai/ModernBERT-embed-large
- Qdrant/clip-ViT-B-32-text - pairs with clip-ViT-B-32-visionfor image-to-text search
- jinaai/jina-embeddings-v2-base-code
- jinaai/jina-embeddings-v2-base-en
- google/embeddinggemma-300m
- nomic-ai/nomic-embed-text-v2-moe - requires nomic-v2-moefeature (candle backend)
- Qwen/Qwen3-Embedding-0.6B - requires qwen3feature (candle backend)
- Qwen/Qwen3-Embedding-4B - requires qwen3feature (candle backend)
- Qwen/Qwen3-Embedding-8B - requires qwen3feature (candle backend)
- Qwen/Qwen3-VL-Embedding-2B - requires qwen3feature (candle backend, multimodal viaQwen3VLEmbedding)
- snowflake/snowflake-arctic-embed-xs
- snowflake/snowflake-arctic-embed-s
- snowflake/snowflake-arctic-embed-m
- snowflake/snowflake-arctic-embed-m-long
- snowflake/snowflake-arctic-embed-l
Quantized versions are also available for several models above (append Q to the model enum variant, e.g., EmbeddingModel::BGESmallENV15Q). EmbeddingGemma additionally ships a 4-bit build as EmbeddingModel::EmbeddingGemma300MQ4.
Click to list models
Click to list models
Click to list models
To support the library, please donate to our primary upstream dependency, ort - The Rust wrapper for the ONNX runtime.
Run the following in your project directory:
cargo add fastembedOr add the following line to your Cargo.toml:
[dependencies]
fastembed = "7"use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel};
// With default options
let mut model = TextEmbedding::try_new(Default::default())?;
// With custom options
let mut model = TextEmbedding::try_new(
TextInitOptions::new(EmbeddingModel::AllMiniLML6V2).with_show_download_progress(true).with_intra_threads(4),
)?;
let documents = vec![
"passage: Hello, World!",
"query: Hello, World!",
"passage: This is an example passage.",
// You can leave out the prefix but it's recommended
"fastembed-rs is licensed under Apache 2.0"
];
// Generate embeddings with the default batch size, 256
let embeddings = model.embed(documents, None)?;
println!("Embeddings length: {}", embeddings.len()); // -> Embeddings length: 4
println!("Embedding dimension: {}", embeddings[0].len()); // -> Embedding dimension: 384use fastembed::{SparseEmbedding, SparseInitOptions, SparseModel, SparseTextEmbedding};
// With default options
let mut model = SparseTextEmbedding::try_new(Default::default())?;
// With custom options
let mut model = SparseTextEmbedding::try_new(
SparseInitOptions::new(SparseModel::SPLADEPPV1).with_show_download_progress(true),
)?;
let documents = vec![
"passage: Hello, World!",
"query: Hello, World!",
"passage: This is an example passage.",
"fastembed-rs is licensed under Apache 2.0"
];
// Generate embeddings with the default batch size, 256
let embeddings: Vec<SparseEmbedding> = model.embed(documents, None)?;use fastembed::{ImageEmbedding, ImageInitOptions, ImageEmbeddingModel};
// With default options
let mut model = ImageEmbedding::try_new(Default::default())?;
// With custom options
let mut model = ImageEmbedding::try_new(
ImageInitOptions::new(ImageEmbeddingModel::ClipVitB32).with_show_download_progress(true),
)?;
let images = vec!["assets/image_0.png", "assets/image_1.png"];
// Generate embeddings with the default batch size, 256
let embeddings = model.embed(images, None)?;
println!("Embeddings length: {}", embeddings.len()); // -> Embeddings length: 2
println!("Embedding dimension: {}", embeddings[0].len()); // -> Embedding dimension: 512use fastembed::{TextRerank, RerankInitOptions, RerankerModel};
// With default options
let mut model = TextRerank::try_new(Default::default())?;
// With custom options
let mut model = TextRerank::try_new(
RerankInitOptions::new(RerankerModel::BGERerankerBase).with_show_download_progress(true),
)?;
let documents = vec![
"hi",
"The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.",
"panda is animal",
"i dont know",
"kind of mammal",
];
// Rerank with the default batch size, 256 and return document contents
let results = model.rerank("what is panda?", documents, true, None)?;
println!("Rerank result: {:?}", results);Alternatively, local model files can be used for inference via the try_new_from_user_defined(...) methods of the respective structs.
Helpers in the similarity module score and rank the vectors embed returns, so a quick in-memory search needs no extra crate:
use fastembed::similarity::{cosine_similarity, top_k};
// `embeddings` is the Vec<Embedding> from model.embed(...)
let query = &embeddings[0];
// Score two vectors directly ([-1.0, 1.0], higher = closer)
let score = cosine_similarity(query, &embeddings[1]);
// Or rank the corpus: (index, score) pairs, best first
let hits = top_k(query, &embeddings, 5);
println!("Closest: {:?}", hits);For larger corpora or persistence, push the vectors to a vector search engine (e.g. Qdrant) and query there.
use fastembed::{SparseInitOptions, SparseModel, SparseTextEmbedding};
let mut model = SparseTextEmbedding::try_new(
SparseInitOptions::new(SparseModel::OpenSearchNeuralSparseDocV3Gte).with_max_length(8192),
)?;
// This model emits one score per vocabulary entry per token.
// So keep the batch size small.
let documents = model.embed(vec!["Hello World"], Some(4))?;
// Queries are from the tokenizer and a precomputed IDF table alone, without any inference.
let queries = model.query_embed(vec!["Hello World"])?;Qwen3 embedding models are available behind the qwen3 feature flag (candle backend).
[dependencies]
fastembed = { version = "7", features = ["qwen3"] }use candle_core::{DType, Device};
use fastembed::Qwen3TextEmbedding;
let device = Device::Cpu;
let model = Qwen3TextEmbedding::from_hf(
"Qwen/Qwen3-Embedding-0.6B",
&device,
DType::F32,
512,
)?;
// Text-only usage with the Qwen3-VL embedding checkpoint is also supported:
// let model = Qwen3TextEmbedding::from_hf("Qwen/Qwen3-VL-Embedding-2B", &device, DType::F32, 512)?;
let embeddings = model.embed(&["query: ...", "passage: ..."])?;
println!("Embeddings length: {}", embeddings.len());For multimodal text/image usage with Qwen/Qwen3-VL-Embedding-2B:
use candle_core::{DType, Device};
use fastembed::Qwen3VLEmbedding;
let device = Device::Cpu;
let model = Qwen3VLEmbedding::from_hf(
"Qwen/Qwen3-VL-Embedding-2B",
&device,
DType::F32,
2048,
)?;
let image_embeddings = model.embed_images(&["tests/assets/image_0.png", "tests/assets/image_1.png"])?;
let text_embeddings = model.embed_texts(&["query: blue cat", "query: red cat"])?;
println!("Image embeddings: {}", image_embeddings.len());
println!("Text embeddings: {}", text_embeddings.len());The nomic-embed-text-v2-moe model is available behind the nomic-v2-moe feature flag (candle backend). First general-purpose MoE embedding model with 100+ language support.
[dependencies]
fastembed = { version = "7", features = ["nomic-v2-moe"] }use candle_core::{DType, Device};
use fastembed::NomicV2MoeTextEmbedding;
let device = Device::Cpu;
let model = NomicV2MoeTextEmbedding::from_hf(
"nomic-ai/nomic-embed-text-v2-moe",
&device,
DType::F32,
512,
)?;
let embeddings = model.embed(&["search_query: ...", "search_document: ..."])?;
println!("Embeddings length: {}", embeddings.len());The BGE-M3 model produces dense, sparse, and ColBERT embeddings simultaneously in a single forward pass.
use fastembed::{Bgem3Embedding, Bgem3InitOptions, Bgem3Model};
// With default options
let mut model = Bgem3Embedding::try_new(Default::default())?;
// With custom options (supporting custom max length up to 8192 tokens)
let mut model = Bgem3Embedding::try_new(
Bgem3InitOptions::new(Bgem3Model::BGEM3Q)
.with_max_length(1024)
.with_show_download_progress(true),
)?;
let documents = vec![
"Hello, World!",
"This is an example passage.",
"fastembed-rs is licensed under Apache 2.0",
"i dont know"
];
// Generate all three representations in a single forward pass
let output = model.embed(documents, None)?;
println!("Dense dimension: {}", output.dense[0].len()); // -> Dense dimension: 1024
let sparse_emb = &output.sparse[0];
println!("Sparse non-zero tokens: {}", sparse_emb.indices.len());
println!("ColBERT token count: {}", output.colbert[0].len());Models download on first use and load from cache afterwards (no network needed at runtime once cached).
- FASTEMBED_CACHE_DIR— cache location (default:- .fastembed_cache). Equivalent to- TextInitOptions::with_cache_dir.
- HF_HOME— if set, takes precedence over the above.
- HF_ENDPOINT— Hugging Face mirror base URL, for restricted networks.
- HF_TOKEN— Hugging Face access token, for gated repositories and higher rate limits. Without it, the token saved by- huggingface-cli loginis used when present.
To run models on a GPU via DirectML on Windows, enable the directml feature:
[dependencies]
fastembed = { version = "7", features = ["directml"] }Then pass a DirectML execution provider when initializing a model:
use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel};
use ort::ep::DirectML;
let model = TextEmbedding::try_new(
TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)
.with_execution_providers(vec![DirectML::default().into()]),
)?;When DirectML is detected, fastembed automatically disables memory pattern optimization and parallel execution on the ONNX Runtime session, as required by the DirectML execution provider.
Fastembed returns a typed fastembed::Error. The type is re-exported from the crate root. The enum is #[non_exhaustive]. New variants can be added in minor releases without breaking match arms.
use fastembed::{Error, Result, TextEmbedding};
fn load() -> Result<TextEmbedding> {
let model = TextEmbedding::try_new(Default::default())?;
Ok(model)
// ...
}To handle an error, match on the variant that applies:
use fastembed::{Error, TextEmbedding, TextInitOptions, EmbeddingModel};
match TextEmbedding::try_new(TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)) {
Ok(model) => { /* ... */ }
Err(Error::ModelRetrieval { file, source }) => {
eprintln!("could not fetch {file}: {source}");
}
/*
...
*/
Err(Error::Ort(err)) => {
eprintln!("ONNX runtime error: {err}");
}
Err(e) => eprintln!("{e}"),
}