# Fine-tune code llama on private source code

**URL:** <https://discuss.huggingface.co/t/fine-tune-code-llama-on-private-source-code/54846>\
**Category:** Beginners\
**Created:** [September 13, 2023, 12:23pm UTC](https://discuss.huggingface.co/t/fine-tune-code-llama-on-private-source-code/54846 "2023-09-13T12:23:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![boblevy8](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/boblevy8/32/19428_2.png) [@boblevy8](https://discuss.huggingface.co/u/boblevy8)\
**Post date:** [September 13, 2023, 12:23pm UTC](https://discuss.huggingface.co/t/fine-tune-code-llama-on-private-source-code/54846/1 "2023-09-13T12:23:30Z")

</div>

What’s the best approach to fine-tune code llama to answer questions about source code on my local disk, without sending the code into the cloud?

Assume the local machine has sufficient GPU (via petals) and the source code in question is ~1m LOC of C# but is unlabelled (there are no “questions” in the training set). So I assume this will necessitate an unsupervised learning approach?

Know of any good papers or other work addressing this use case?

Thanks!  
-Bob

---

<div class="post-metadata">

**Author:** ![stormchaser](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/stormchaser/32/20128_2.png) [@stormchaser](https://discuss.huggingface.co/u/stormchaser)\
**Post date:** [October 6, 2023, 5:26pm UTC](https://discuss.huggingface.co/t/fine-tune-code-llama-on-private-source-code/54846/2 "2023-10-06T17:26:33Z")

</div>

hey hello! have you been able to fine tune your code yet? It will be great if you write here on your findings.

---

<div class="post-metadata">

**Author:** ![aakashgoel12](https://avatars.discourse-cdn.com/v4/letter/a/13edae/32.png) [@aakashgoel12](https://discuss.huggingface.co/u/aakashgoel12)\
**Post date:** [February 27, 2024, 8:53am UTC](https://discuss.huggingface.co/t/fine-tune-code-llama-on-private-source-code/54846/3 "2024-02-27T08:53:26Z")

</div>

@stormchaser @boblevy8 Any findings/approach/code, please share ?

---

<div class="post-metadata">

**Author:** ![himapatel](https://avatars.discourse-cdn.com/v4/letter/h/74df32/32.png) [@himapatel](https://discuss.huggingface.co/u/himapatel)\
**Post date:** [May 30, 2024, 5:45am UTC](https://discuss.huggingface.co/t/fine-tune-code-llama-on-private-source-code/54846/4 "2024-05-30T05:45:08Z")

</div>

You could use this for your data preparation work [GitHub - IBM/data-prep-kit: Open source project for data preparation of LLM application builders](https://github.com/IBM/data-prep-kit)