# Embedding endpoint returning \[None\] embeddings **URL:** \ **Category:** Inference Endpoints on the Hub\ **Created:** [October 4, 2024, 7:58pm UTC](https://discuss.huggingface.co/t/embedding-endpoint-returning-none-embeddings/110159 "2024-10-04T19:58:03Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![lgbird](https://avatars.discourse-cdn.com/v4/letter/l/848f3c/32.png) [@lgbird](https://discuss.huggingface.co/u/lgbird)\ **Post date:** [October 4, 2024, 7:58pm UTC](https://discuss.huggingface.co/t/embedding-endpoint-returning-none-embeddings/110159/1 "2024-10-04T19:58:03Z")
I’m trying to deploy [dunzhang/stella\_en\_1.5B\_v5 · Hugging Face](https://huggingface.co/dunzhang/stella_en_1.5B_v5) as an embedding model. The endpoint launches successfully but the returned embeddings are all [None, None, None, …]. It’s also worth noting that the returned embeddings have dimension 1536 but the model default is actually 1024… I tried simply deploying the model from the official repo above with “Sentence Embeddings” as the task and also creating a custom endpoint handler using sentence\_transformers as described in the inference endpoints guide (cloned repo here: [lgbird/stella\_1.5B\_custom · Hugging Face](https://huggingface.co/lgbird/stella_1.5B_custom)) and the task set as “Custom”, both with the exact same result! I’m not sure what’s going wrong/what to try next… I’ll add the logs of the “standard” endpoint(the one from the official model repo) down below, the ones from the custom endpoint handler are the same(and look fine to me). Thanks in advance for any help! ```auto - 2024-10-04T17:33:52.482+00:00 {"timestamp":"2024-10-04T17:33:52.482653Z","level":"INFO","message":"Args { model_id: \"/rep **** ory\", revision: None, tokenization_workers: None, dtype: None, pooling: None, max_concurrent_requests: 512, max_batch_tokens: 16384, max_batch_requests: None, max_client_batch_size: 32, auto_truncate: false, default_prompt_name: None, default_prompt: None, hf_api_token: None, hostname: \"r-lgbird-stella-embeddings-exqg6mkq-96fea-57q2f\", port: 80, uds_path: \"/tmp/text-embeddings-inference-server\", huggingface_hub_cache: Some(\"/repository/cache\"), payload_limit: 2000000, api_key: None, json_output: true, otlp_endpoint: None, otlp_service_name: \"text-embeddings-inference.server\", cors_allow_origin: None }","target":"text_embeddings_router","filename":"router/src/main.rs","line_number":175} - 2024-10-04T17:33:52.740+00:00 {"timestamp":"2024-10-04T17:33:52.740581Z","level":"WARN","message":"Warning: Token '<|endoftext|>' was expected to have ID '151643' but was given ID 'None'","log.target":"tokenizers::tokenizer::serialization","log.module_path":"tokenizers::tokenizer::serialization","log.file":"/root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokenizers-0.19.1/src/tokenizer/serialization.rs","log.line":159,"target":"tokenizers::tokenizer::serialization","filename":"/root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokenizers-0.19.1/src/tokenizer/serialization.rs","line_number":159} - 2024-10-04T17:33:52.740+00:00 {"timestamp":"2024-10-04T17:33:52.740612Z","level":"WARN","message":"Warning: Token '<|im_start|>' was expected to have ID '151644' but was given ID 'None'","log.target":"tokenizers::tokenizer::serialization","log.module_path":"tokenizers::tokenizer::serialization","log.file":"/root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokenizers-0.19.1/src/tokenizer/serialization.rs","log.line":159,"target":"tokenizers::tokenizer::serialization","filename":"/root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokenizers-0.19.1/src/tokenizer/serialization.rs","line_number":159} - 2024-10-04T17:33:52.740+00:00 {"timestamp":"2024-10-04T17:33:52.740618Z","level":"WARN","message":"Warning: Token '<|im_end|>' was expected to have ID '151645' but was given ID 'None'","log.target":"tokenizers::tokenizer::serialization","log.module_path":"tokenizers::tokenizer::serialization","log.file":"/root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokenizers-0.19.1/src/tokenizer/serialization.rs","log.line":159,"target":"tokenizers::tokenizer::serialization","filename":"/root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokenizers-0.19.1/src/tokenizer/serialization.rs","line_number":159} - 2024-10-04T17:33:52.742+00:00 {"timestamp":"2024-10-04T17:33:52.742207Z","level":"INFO","message":"Maximum number of tokens per request: 512","target":"text_embeddings_router","filename":"router/src/lib.rs","line_number":199} - 2024-10-04T17:33:52.742+00:00 {"timestamp":"2024-10-04T17:33:52.742496Z","level":"INFO","message":"Starting 2 tokenization workers","target":"text_embeddings_core::tokenization","filename":"core/src/tokenization.rs","line_number":28} - 2024-10-04T17:33:52.872+00:00 {"timestamp":"2024-10-04T17:33:52.871997Z","level":"INFO","message":"Starting model backend","target":"text_embeddings_router","filename":"router/src/lib.rs","line_number":241} - 2024-10-04T17:33:53.094+00:00 {"timestamp":"2024-10-04T17:33:53.094540Z","level":"INFO","message":"Starting FlashQwen2 model on Cuda(CudaDevice(DeviceId(1)))","target":"text_embeddings_backend_candle","filename":"backends/candle/src/lib.rs","line_number":364} - 2024-10-04T17:34:09.333+00:00 {"timestamp":"2024-10-04T17:34:09.333134Z","level":"INFO","message":"Warming up model","target":"text_embeddings_router","filename":"router/src/lib.rs","line_number":257} - 2024-10-04T17:34:11.624+00:00 {"timestamp":"2024-10-04T17:34:11.624385Z","level":"WARN","message":"Invalid hostname, defaulting to 0.0.0.0","target":"text_embeddings_router","filename":"router/src/lib.rs","line_number":319} - 2024-10-04T17:34:11.626+00:00 {"timestamp":"2024-10-04T17:34:11.626272Z","level":"INFO","message":"Starting HTTP server: 0.0.0.0:80","target":"text_embeddings_router::http::server","filename":"router/src/http/server.rs","line_number":1778} - 2024-10-04T17:34:11.626+00:00 {"timestamp":"2024-10-04T17:34:11.626296Z","level":"INFO","message":"Ready","target":"text_embeddings_router::http::server","filename":"router/src/http/server.rs","line_number":1779} - 2024-10-04T17:34:39.911+00:00 {"timestamp":"2024-10-04T17:34:39.911560Z","level":"INFO","message":"Success","target":"text_embeddings_router::http::server","filename":"router/src/http/server.rs","line_number":706,"span":{"inference_time":"45.087126ms","queue_time":"322.203µs","tokenization_time":"1.567907ms","total_time":"47.05461ms","name":"embed"},"spans":[{"inference_time":"45.087126ms","queue_time":"322.203µs","tokenization_time":"1.567907ms","total_time":"47.05461ms","name":"embed"}]} - 2024-10-04T17:39:13.793+00:00 {"timestamp":"2024-10-04T17:39:13.793842Z","level":"INFO","message":"Success","target":"text_embeddings_router::http::server","filename":"router/src/http/server.rs","line_number":706,"span":{"inference_time":"20.27212ms","queue_time":"204.423µs","tokenization_time":"157.999µs","total_time":"20.693654ms","name":"embed"},"spans":[{"inference_time":"20.27212ms","queue_time":"204.423µs","tokenization_time":"157.999µs","total_time":"20.693654ms","name":"embed"}]} - 2024-10-04T17:40:43.781+00:00 {"timestamp":"2024-10-04T17:40:43.781391Z","level":"INFO","message":"Success","target":"text_embeddings_router::http::server","filename":"router/src/http/server.rs","line_number":706,"span":{"inference_time":"17.160806ms","queue_time":"205.497µs","tokenization_time":"299.634µs","total_time":"17.715855ms","name":"embed"},"spans":[{"inference_time":"17.160806ms","queue_time":"205.497µs","tokenization_time":"299.634µs","total_time":"17.715855ms","name":"embed"}]} - 2024-10-04T17:41:09.750+00:00 {"timestamp":"2024-10-04T17:41:09.750369Z","level":"INFO","message":"Success","target":"text_embeddings_router::http::server","filename":"router/src/http/server.rs","line_number":706,"span":{"inference_time":"16.989841ms","queue_time":"201.616µs","tokenization_time":"354.533µs","total_time":"17.603286ms","name":"embed"},"spans":[{"inference_time":"16.989841ms","queue_time":"201.616µs","tokenization_time":"354.533µs","total_time":"17.603286ms","name":"embed"}]} - 2024-10-04T17:43:24.712+00:00 {"timestamp":"2024-10-04T17:43:24.712499Z","level":"INFO","message":"Success","target":"text_embeddings_router::http::server","filename":"router/src/http/server.rs","line_number":706,"span":{"inference_time":"72.323516ms","queue_time":"54.927283ms","tokenization_time":"2.630849ms","total_time":"174.609825ms","name":"embed"},"spans":[{"inference_time":"72.323516ms","queue_time":"54.927283ms","tokenization_time":"2.630849ms","total_time":"174.609825ms","name":"embed"}]} - 2024-10-04T17:44:03.242+00:00 {"timestamp":"2024-10-04T17:44:03.242162Z","level":"INFO","message":"signal received, starting graceful shutdown","target":"text_embeddings_router::shutdown","filename":"router/src/shutdown.rs","line_number":27} ``` ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 8, 2024, 10:13am UTC](https://discuss.huggingface.co/t/embedding-endpoint-returning-none-embeddings/110159/2 "2024-10-08T10:13:53Z")
It may be quicker to send mentions (@+username) to the authors of these information. > **[Alibaba-NLP/gte-multilingual-base · HERE IS HOW YOU USE THIS WITH TEI OR...](https://huggingface.co/Alibaba-NLP/gte-multilingual-base/discussions/7)** > > testing to see if this works for TEI > **[Deploy Embedding Models with Hugging Face Inference Endpoints](https://huggingface.co/blog/inference-endpoints-embeddings)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. > **[GitHub - huggingface/text-embeddings-inference: A blazing fast inference solution for text...](https://github.com/huggingface/text-embeddings-inference)** > > A blazing fast inference solution for text embeddings models ---
**Author:** ![dweb](https://avatars.discourse-cdn.com/v4/letter/d/ee59a6/32.png) [@dweb](https://discuss.huggingface.co/u/dweb)\ **Post date:** [March 12, 2025, 2:16pm UTC](https://discuss.huggingface.co/t/embedding-endpoint-returning-none-embeddings/110159/3 "2025-03-12T14:16:14Z")
Same issue here using Alibaba-NLP/gte-modernbert-base ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [March 12, 2025, 4:48pm UTC](https://discuss.huggingface.co/t/embedding-endpoint-returning-none-embeddings/110159/4 "2025-03-12T16:48:49Z")
There is a possibility that it may be a bug in Sentence Transformer or TGI or server config. > [@Unable to access public model - status 401](https://discuss.huggingface.co/t/unable-to-access-public-model-status-401/144806): > > i build .net code to access sentence-transformers/all-MiniLM-L6-v2. while running, i get the following error: {StatusCode: 401, Unauthorized] although token is right , it has read permission using (var client = new HttpClient()) { client.DefaultRequestHeaders.Add(“Authorization”, “hf\_DimrceBJNUZfMjpICMAQfXzNkrioZErQVI”); var requestContent = new { inputs = new string[] { text1, text2 } }; var response = await client.PostAsync("https://api-inference.huggingface.co/models/sentence-tran…