# Using Huggingface Trainer for custom models

**URL:** <https://discuss.huggingface.co/t/using-huggingface-trainer-for-custom-models/16882>\
**Category:** Beginners\
**Created:** [April 16, 2022, 8:52pm UTC](https://discuss.huggingface.co/t/using-huggingface-trainer-for-custom-models/16882 "2022-04-16T20:52:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ivnle](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@ivnle](https://discuss.huggingface.co/u/ivnle)\
**Post date:** [April 16, 2022, 8:52pm UTC](https://discuss.huggingface.co/t/using-huggingface-trainer-for-custom-models/16882/1 "2022-04-16T20:52:15Z")

</div>

Say I want to train a simple LSTM or MLP with Trainer (Pytroch `nn.Module`s). Do I just need to ensure the model adheres to the following?

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/5/54f7813e40c2d50891055cd9b7713288fc5eec6d.png)

Is there an example of using Trainer to train models that are not HF Transformers models? Best practices?

---

<div class="post-metadata">

**Author:** ![kishore](https://avatars.discourse-cdn.com/v4/letter/k/8edcca/32.png) [@kishore](https://discuss.huggingface.co/u/kishore)\
**Post date:** [April 17, 2022, 2:48am UTC](https://discuss.huggingface.co/t/using-huggingface-trainer-for-custom-models/16882/2 "2022-04-17T02:48:23Z")

</div>

I think HF trainer API is specifically for transformers but not for other models.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [April 18, 2022, 12:25pm UTC](https://discuss.huggingface.co/t/using-huggingface-trainer-for-custom-models/16882/3 "2022-04-18T12:25:21Z")

</div>

We don’t have an example, but as long as you follow the recommendation in that list of the documentation, you should be fine.

---

<div class="post-metadata">

**Author:** ![julien-c](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/julien-c/32/41937_2.png) [@julien-c](https://discuss.huggingface.co/u/julien-c)\
**Post date:** [April 19, 2022, 7:10pm UTC](https://discuss.huggingface.co/t/using-huggingface-trainer-for-custom-models/16882/4 "2022-04-19T19:10:16Z")

</div>

and if you use it successfully and want to do a short writeup, publish it, we’ll make sure to share your writeup!

---

<div class="post-metadata">

**Author:** ![ivnle](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@ivnle](https://discuss.huggingface.co/u/ivnle)\
**Post date:** [April 26, 2022, 5:32pm UTC](https://discuss.huggingface.co/t/using-huggingface-trainer-for-custom-models/16882/5 "2022-04-26T17:32:45Z")

</div>

Confirmed that you can train a simple LSTM or MLP with Trainer. This is nice since I can just stay within the HF ecosystem. I’m not sure I’ll have time to do a write-up but as long as you follow that list in the original post, it will work.

---

<div class="post-metadata">

**Author:** ![Petrina](https://avatars.discourse-cdn.com/v4/letter/p/a3d4f5/32.png) [@Petrina](https://discuss.huggingface.co/u/Petrina)\
**Post date:** [May 29, 2023, 7:50am UTC](https://discuss.huggingface.co/t/using-huggingface-trainer-for-custom-models/16882/6 "2023-05-29T07:50:42Z")

</div>

Hello ivnle!

I am currently trying to train an LSTM model that takes as input the embeddings that are outputted from a pretrained model from the hf hub. I am following the example at [Sharing custom models](https://huggingface.co/docs/transformers/custom_models) and my model class that inherits from PreTrainedModel is the following:

```auto
from transformers import PreTrainedModel

class LSTMModel(PreTrainedModel):
    config_class = LSTMConfig

    def __init__ (self, config, pretrained_model):
        super(). __init__ (config)

        self.model = SentimentLSTM(pretrained_model, 
                                   output_size=config.output_size, 
                                   hidden_dim=config.hidden_dim, 
                                   n_layers=config.n_layers)

    def forward(self, tensor, labels=None):
        logits = self.model(tensor)
        if labels is not None:
            loss = torch.nn.cross_entropy(logits, labels)
            return {"loss": loss, "logits": logits}
        return {"logits": logits}

```

where SentimentLSTM is my custom LSTM model.

Then I initialize the training\_arguments and the trainer object and try to perform trainer.train(). However I get an error, due to the tensor argument, because the function cannot find it.

Did you use the same example to perform your own training?

If so, what was your forward function in the corresponding LSTMModel class and how could you pass an extra argument for this function through Trainer?

Thank you in advance,  
Petrina