# Git clone often gives 403 or 429 errors **URL:** \ **Category:** Beginners\ **Created:** [March 6, 2025, 8:24pm UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510 "2025-03-06T20:24:26Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![certik](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/certik/32/42682_2.png) [@certik](https://discuss.huggingface.co/u/certik)\ **Post date:** [March 6, 2025, 8:24pm UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510/1 "2025-03-06T20:24:27Z")
We download data from Hugging Face at our CI (GitHub Actions) like this: ```auto + git lfs install Updated Git hooks. Git LFS initialized. + git clone https://huggingface.co/datasets/certik/fastGPT Cloning into 'fastGPT'... fatal: unable to access 'https://huggingface.co/datasets/certik/fastGPT/': The requested URL returned error: 403 ``` Quite often in the last month we started encountering “403 Forbidden” and “429 Too Many Requests” errors. We were getting similar errors when using `curl` over https. We do not want to overload your servers. I have a few questions to figure out how to move forward: 1. Is it ok to use Hugging Face at our CI to download data? 2. If so, what is the rate limit? 3. How should we handle the 403/429 errors — should we retry with exponential delay iterations (1, 2, 4, 8, … seconds)? 4. Should we use some authentication? Thanks for maintaining this service, it has been very useful to us. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [March 6, 2025, 10:20pm UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510/2 "2025-03-06T22:20:36Z")
SSL configuration errors and token configuration errors are common causes. Also, in Windows environments, it is easier to stabilize git-lfs as well as git by reinstalling them using the installer. The default ones are kind of old. > [@Connection Error when Accessing Dataset URL on Hugging Face](https://discuss.huggingface.co/t/connection-error-when-accessing-dataset-url-on-hugging-face/67418): > > I am currently experiencing an issue while trying to access my private dataset in my repo for example: ([https://huggingface.co/dataset/Mustafa21/dataset](https://huggingface.co/dataset/Mustafa21/dataset)) The error message I encountered is as follows: ionError(f"Couldn't reach {url} ({repr(head\_error)})") ConnectionError: Couldn't reach https://huggingface.co/datasets/Mustafa21/dataset/resolve/main/dataset\_infos.json (ConnectionError('Unauthorized for URL https://huggingface.co/datasets/Mustafa21/dataset/resolve/main/dataset\_infos.json. Pleas… > [@Error 403! What to do about it?](https://discuss.huggingface.co/t/error-403-what-to-do-about-it/12983): > > I can not save the model, it gives HTTPError: 403 Client Error: Forbidden for url: [https://huggingface.co/api/repos/create](https://huggingface.co/api/repos/create) - You don’t have the rights to create a model under this namespace how to fix? ---
**Author:** ![certik](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/certik/32/42682_2.png) [@certik](https://discuss.huggingface.co/u/certik)\ **Post date:** [March 16, 2025, 7:35pm UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510/3 "2025-03-16T19:35:45Z")
Thanks @John6666. Note that the repository I am trying to download is public. I nevertheless created a Huggingface token and I can download it using curl and the token like this: ```auto curl -f -L -H "Authorization: Bearer ${HUGGINGFACE_TOKEN}" -o model.dat https://huggingface.co/datasets/certik/fastGPT/resolve/main/model_fastgpt_124M_v1.dat ``` I tested that if I use invalid token that this download will fail. So I think it is working when I do it by hand in a terminal. However in order to use this at the CI, I first tried creating a secure variable at GitHub, but it doesn’t get exposed to PRs from forks. I could expose the token I created online, but this doesn’t seem like a good practice even if the token has no permissions, since it is still tied to my account and there might be some other consequences. Also, from Huggingface’s perspective – if there were some special privileges, like not getting download errors for people that use the token, then if I exposed the token publicly, anyone could use the same token, so it would defeat the measure. So that doesn’t seem like a solution. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [March 17, 2025, 4:10am UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510/4 "2025-03-17T04:10:26Z")
Hmm… huggingface-cli login? > **[Command Line Interface (CLI)](https://huggingface.co/docs/huggingface_hub/main/en/guides/cli#huggingface-cli-login)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. ---
**Author:** ![certik](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/certik/32/42682_2.png) [@certik](https://discuss.huggingface.co/u/certik)\ **Post date:** [March 17, 2025, 4:58am UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510/5 "2025-03-17T04:58:07Z")
The `huggingface-cli` still requires a token, so at our CI the token would have to be public, which I think is not a good idea. However, I found this page that shows how to download files: [Download files from the Hub](https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-from-the-cli), maybe it doesn’t fail like a direct `curl` invocation. ---
**Author:** ![certik](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/certik/32/42682_2.png) [@certik](https://discuss.huggingface.co/u/certik)\ **Post date:** [March 18, 2025, 5:41pm UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510/6 "2025-03-18T17:41:26Z")
We also tried the following `curl` options: `--retry-all-errors --retry 10 --retry-delay 20`, but unfortunately that still sometimes fails, here is an example: > > > https://github.com/lfortran/lfortran/actions/runs/13531928535/job/37815936445?pr…=6418 > > \`\`\` > + git lfs install > Updated Git hooks. > Git LFS initialized. > + git clone https://huggingface.co/datasets/certik/fastGPT > Cloning into 'fastGPT'... > fatal: unable to access 'https://huggingface.co/datasets/certik/fastGPT/': The requested URL returned error: 403 > Error: Process completed with exit code 128. > \`\`\` > > Background: this \`git clone https://huggingface.co/datasets/certik/fastGPT\` was introduced in https://github.com/lfortran/lfortran/pull/6406 as a fix for https://github.com/lfortran/lfortran/issues/6318. The error is now much better, it fails at the point where it should fail. > > What remains to figure out is if we should be using huggingface at all, given that it is not very reliable for our CI. We can also ask at https://github.com/huggingface/huggingface\_hub, I found some similar issues there. Maybe we need to authenticate with a token when downloading, we might be getting rate limited. That is over 3 minutes of trying every 20 seconds, and it did not succeed. So I think this is not some transient error, it is likely some rate limit. So I think we should not be downloading from Huggingface using curl at our CI. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [March 19, 2025, 5:17am UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510/7 "2025-03-19T05:17:19Z")
For a moment I thought it might be a Windows git problem, but if it’s happening with curl, it’s a different error that’s unrelated… Hmm, I wonder if something is being blocked in the connection path… > [@Proxy Issues While Accessing Hugging Face](https://discuss.huggingface.co/t/proxy-issues-while-accessing-hugging-face/100169): > > Hello, I’m experiencing issues while trying to access Hugging Face from my institute’s network. Here are the details of my problem: Error Message ProxyError: (MaxRetryError(“HTTPSConnectionPool(host=‘[huggingface.co](http://huggingface.co)’, port=443): Max retries exceeded with url: /api/whoami-v2 (Caused by ProxyError(‘Cannot connect to proxy.’, OSError(‘Tunnel connection failed: 403 Forbidden’)))”)) Steps Taken: Checked and unset the proxy variables. Changed the token key and tried to connect. Attempted to conne… > [@403 Client Error: Forbidden for url](https://discuss.huggingface.co/t/403-client-error-forbidden-for-url/12563/16): > > I’m kicking off the batch job from a sagemaker notebook instance, but the batch job runs in sagemaker. It still fails with 403. I tried it from a different aws account with the same results. More detail in this thread [Error 403 when downloading model for Sagemaker batch inference - #6 by philschmid](https://discuss.huggingface.co/t/error-403-when-downloading-model-for-sagemaker-batch-inference/12571/6) ### For Windows users > **[Git - Install for Windows](https://git-scm.com/downloads/win)** > **[Git Large File Storage](https://git-lfs.com/)** > > Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise. ---
**Author:** ![thordata](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thordata/32/62596_2.png) [@thordata](https://discuss.huggingface.co/u/thordata)\ **Post date:** [September 1, 2026, 1:53am UTC](https://discuss.huggingface.co/t/git-clone-often-gives-403-or-429-errors/144510/8 "2026-09-01T01:53:08Z")
For public datasets in CI you shouldn’t need a token at all, and the 403/429 you’re seeing is the edge throttling the runner’s outbound IP, not your auth. A few things that moved the needle for me in GitHub Actions: - Drop git-lfs for downloads and use the Hub’s file download path instead (hf download / huggingface\_hub snapshot\_download with allow\_patterns). git clone pulls every LFS object; you usually want a subset. - Set HF\_HUB\_ENABLE\_HF\_TRANSFER=1 — the accelerated downloader uses fewer, larger connections, which trips per-connection limits far less than many small curl/LFS requests. - Back off on the actual signal: on 429, read Retry-After and sleep that long (plus jitter) rather than a flat --retry-delay 20. A fixed delay is why --retry 10 still intermittently fails. - Narrow the scope with allow\_patterns so you fetch 3 files, not 3000 — fewer requests, lower chance of tripping the limit. - Cache the downloaded files between runs (actions/cache keyed on revision) so only changed shards re-download. If it must stay a single git clone for your workflow, then the shared-IP angle is the real fix: GitHub Actions runners come from a pool, so a busy egress IP is throttled globally and it looks random from your side.