# Code to train a model on my dataset is not working

**URL:** <https://discuss.huggingface.co/t/code-to-train-a-model-on-my-dataset-is-not-working/59736>\
**Category:** Beginners\
**Created:** [October 24, 2023, 6:12pm UTC](https://discuss.huggingface.co/t/code-to-train-a-model-on-my-dataset-is-not-working/59736 "2023-10-24T18:12:16Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Choso](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@Choso](https://discuss.huggingface.co/u/Choso)\
**Post date:** [October 24, 2023, 6:12pm UTC](https://discuss.huggingface.co/t/code-to-train-a-model-on-my-dataset-is-not-working/59736/1 "2023-10-24T18:12:16Z")

</div>

Hey there.  
For a project I want to create sentence embeddings of text data. I managed to produce embeddings with the paraphrase-multilingual-MiniLM-L12-v2 model, but they were not satisfactory. Therefore, I wanted to train the model on a part of my data and then use the finetuned model to create the embeddings.  
Since I have no idea how to code, I mainly used GPT4 to help me. However, it does not succeed in creating a working code, so I would like to ask you to help me produce a code that works.

Here’s my code:  
import sys  
if ‘google.colab’ in sys.modules: # If in Google Colab environment  
# Installing requisite packages  
!pip install datasets transformers evaluate  
!pip install accelerate -U  
!pip install -U sentence-transformers

```
# Mount google drive to enable access to data files
from google.colab import drive
drive.mount('/content/drive')

# Change working directory to desired folder
%cd /content/drive/MyDrive/HS_23_Msc

```

# Import necessary libraries

import pandas as pd

import torch

from torch.utils.data import DataLoader, Dataset

from transformers import AdamW, get\_linear\_schedule\_with\_warmup

from sentence\_transformers import SentenceTransformer, InputExample, losses

from sentence\_transformers import models

# Load your dataset

import pandas as pd

dat = pd.read\_csv(‘Embeddings\_text.csv’)

sentences = dat[‘word’].tolist()

# Create the training samples. Here, we’re using each sentence as its own label for simplicity.

train\_examples = [InputExample(texts=[sentence], label=0) for sentence in sentences]

# Create a DataLoader for the training samples

train\_dataloader = DataLoader(train\_examples, shuffle=True, batch\_size=16)

# Define the model and the training procedure

model\_name = ‘sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2’

word\_embedding\_model = models.Transformer(model\_name, max\_seq\_length=256)

pooling\_model = models.Pooling(word\_embedding\_model.get\_word\_embedding\_dimension())

model = SentenceTransformer(modules=[word\_embedding\_model, pooling\_model])

# Define optimizer and scheduler

optimizer = AdamW(model.parameters(), lr=1e-5)

scheduler = get\_linear\_schedule\_with\_warmup(optimizer, num\_warmup\_steps=1000, num\_training\_steps=10000)  
from torch.utils.data import DataLoader, Dataset  
from sentence\_transformers.losses import CosineSimilarityLoss  
class SentenceDataset(Dataset):  
def **init** (self, sentences):  
self.sentences = sentences

```
def __len__ (self):
    return len(self.sentences)

def __getitem__ (self, idx):
    return self.sentences[idx], self.sentences[idx] # returning sentence, label (they are the same in this case)

```

# Use your dataset

sentence\_dataset = SentenceDataset(dat[‘word+comment’].tolist())  
train\_dataloader = DataLoader(sentence\_dataset, batch\_size=batch\_size, shuffle=True)

# Define loss and optimizer

train\_loss = CosineSimilarityLoss(model)  
optimizer = torch.optim.Adam(model.parameters(), lr=learning\_rate)

# Train the model

model.train()  
for epoch in range(num\_epochs):  
for sentences, labels in train\_dataloader:  
# Directly compute loss using the sentences and labels  
loss\_value = train\_loss(sentences, labels)  
loss\_value.backward()  
optimizer.step()  
optimizer.zero\_grad()

print(“Training complete!”)

# Creating embeddings for the entire dataset

features = model.encode(sentences, convert\_to\_numpy=True)

# Exporting the embeddings to a csv

features\_df = pd.DataFrame(features)

features\_df.to\_csv(‘features\_tuned.csv’)