# Tabular classification/regression pipeline

**URL:** <https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030>\
**Category:** 🤗Hub\
**Created:** [August 24, 2022, 9:00am UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030 "2022-08-24T09:00:08Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![StyrbjornKall](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/styrbjornkall/32/37485_2.png) [@StyrbjornKall](https://discuss.huggingface.co/u/StyrbjornKall)\
**Post date:** [August 24, 2022, 9:00am UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/1 "2022-08-24T09:00:08Z")

</div>

Hi!

I have a fine-tuned pytorch model which builds upon a pretrained roberta model from huggingface. The pytorch model simply has two fully connected layers after the roberta model where I add some additional parameters to predict a single value (i.e. a regression task).

I would very much like this model to be available for inference on the hub and I believe the Tabular Classification or Tabular Regression tasks are suitable widgets since they allow me to both pass text to the transformer in one column and the other additional parameters in the other columns.

However, I’ve been trying to follow this [tutorial](https://huggingface.co/templates/tabular-classification) which links to the [wine-quality](https://huggingface.co/osanseviero/wine-quality?data=%7B%22fixed_acidity%22%3A%5B%227.3%22%2C%227.8%22%2C%2210.3%22%5D%2C%22volatile_acidity%22%3A%5B%220.7%22%2C%220.88%22%2C%220.32%22%5D%2C%22citric_acid%22%3A%5B%220%22%2C%220%22%2C%220.45%22%5D%2C%22residual_sugar%22%3A%5B%221.9%22%2C%222.6%22%2C%226.4%22%5D%2C%22chlorides%22%3A%5B%220.076%22%2C%220.098%22%2C%220.073%22%5D%2C%22free_sulfur_dioxide%22%3A%5B%2211%22%2C%2225%22%2C%225%22%5D%2C%22total_sulfur_dioxide%22%3A%5B%2234%22%2C%2267%22%2C%2213%22%5D%2C%22density%22%3A%5B%220.9978%22%2C%220.9968%22%2C%220.9976%22%5D%2C%22pH%22%3A%5B%223.51%22%2C%223.2%22%2C%223.23%22%5D%2C%22sulphates%22%3A%5B%220.56%22%2C%220.68%22%2C%220.82%22%5D%2C%22alcohol%22%3A%5B%229.4%22%2C%229.8%22%2C%2212.6%22%5D%7D) example by @osanseviero, but there I can’t find any pipeline.py file and don’t really understand how to implement the functions specified in the tutorial.

There are some users who have successfully implemented the Tabular Regression widget on sklearn models, has anyone figured out how to do so with transformers models?

Thank you in advance! 😊

---

<div class="post-metadata">

**Author:** ![osanseviero](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/osanseviero/32/38738_2.png) [@osanseviero](https://discuss.huggingface.co/u/osanseviero)\
**Post date:** [August 29, 2022, 9:24pm UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/2 "2022-08-29T21:24:31Z")

</div>

Hey there! This is an excellent question!

The Inference API (widgets) don’t work out of the box at the moment for tabular tasks with `transformers`. The reason is that there is no [pipeline](https://huggingface.co/docs/transformers/main_classes/pipelines) in `transformers` for these two tasks, which leads to this issue.

The `generic` solution is a bit hacky and more of a proof of concept. It requires changing the model library to `generic` in the `README.md` by setting `library_name: generic`. In `pipeline.py` you can then load the model in ` __init__ ` and run inference in ` __call__ `. Note that this will ideally be fixed by adding the pipeline to `transformers` and then just using the inference API, as the mentioned before is a proof of concept. Note that you will also need to update the model card with an example table so it knows which columns are expected. Here is an [example](https://huggingface.co/osanseviero/wine-quality/blob/main/README.md#L8)

---

<div class="post-metadata">

**Author:** ![StyrbjornKall](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/styrbjornkall/32/37485_2.png) [@StyrbjornKall](https://discuss.huggingface.co/u/StyrbjornKall)\
**Post date:** [August 30, 2022, 2:16pm UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/3 "2022-08-30T14:16:50Z")

</div>

Thank you for your answer! I’m however still a bit unsure on how to write the pipeline.py file. Currently my file simply looks like this:

```auto
from typing import Dict, List, Union
from transformers import AutoModel, AutoTokenizer
import os

class PreTrainedPipeline():
    def __init__ (self, path=""):
        # IMPLEMENT_THIS
        # Preload all the elements you are going to need at inference.
        # For instance your model, processors, tokenizer that might be needed.
        # This function is only called once, so do all the heavy processing I/O here"""

        self.tokenizer = AutoTokenizer.from_pretrained("StyrbjornKall/new_dummy")
        self.model = AutoModel.from_pretrained("StyrbjornKall/new_dummy")

    def __call__ (
        self, inputs: Dict[str, Dict[str, List[Union[str, float]]]]
    ) -> List[Union[str, float]]:
        """
        Args:
            inputs (:obj:`dict`):
                a dictionary containing a key 'data' mapping to a dict in which
                the values represent each column.
        Return:
            A :obj:`list` of floats or strings: The classification output for each row.
        """

        encodings = self.tokenizer.batch_encode_plus(
            inputs['text'],
            padding = 'max_length',
            max_length = 100,
            pad_to_max_length=True,
            truncation=True)

        tok = encodings['input_ids']

        return tok

```

I thought an easy starting point would just be to return the tokens of the text held in the “text” column in the table but this does not seem to work… Now I get a time-out error as well when trying to load the model which I have not seen before.

Here’s my repo for reference: [new\_dummy](https://huggingface.co/StyrbjornKall/new_dummy?data=%7B%22SMILES%22%3A%5B%22c1ccccc1%22%5D%7D)

---

<div class="post-metadata">

**Author:** ![osanseviero](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/osanseviero/32/38738_2.png) [@osanseviero](https://discuss.huggingface.co/u/osanseviero)\
**Post date:** [August 30, 2022, 7:18pm UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/4 "2022-08-30T19:18:33Z")

</div>

THe `library_name` you’re using right now is `sklearn`. This [PR](https://huggingface.co/StyrbjornKall/new_dummy/discussions/1) should fix that

---

<div class="post-metadata">

**Author:** ![StyrbjornKall](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/styrbjornkall/32/37485_2.png) [@StyrbjornKall](https://discuss.huggingface.co/u/StyrbjornKall)\
**Post date:** [August 31, 2022, 11:01am UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/5 "2022-08-31T11:01:57Z")

</div>

Thank you so much, that seem to have resolved some things. However, now I get the following error from the API:

```auto
The task `tabular-classification` is not recognized by api-inference-community

```

On another note: how are the inputs accessed? Is it `table = inputs['data']` to access the data stored in the API-table and then further `table['text']` to access the contents of column ‘text’?

---

<div class="post-metadata">

**Author:** ![osanseviero](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/osanseviero/32/38738_2.png) [@osanseviero](https://discuss.huggingface.co/u/osanseviero)\
**Post date:** [August 31, 2022, 11:54am UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/6 "2022-08-31T11:54:34Z")

</div>

Sorry about that! [This PR](https://github.com/huggingface/api-inference-community/pull/95) will fix things once deployed. In the meantime you should be able to play with `structured-data-classification` instead, which was the old name.

---

<div class="post-metadata">

**Author:** ![StyrbjornKall](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/styrbjornkall/32/37485_2.png) [@StyrbjornKall](https://discuss.huggingface.co/u/StyrbjornKall)\
**Post date:** [August 31, 2022, 12:27pm UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/7 "2022-08-31T12:27:19Z")

</div>

Thank you! I also suggest the same be done to the tabular-regression task which seems to generate the same error 🙂

Is the `structured-data-classification` not supported as a widget in the API? The table from the README is no longer visible for example

---

<div class="post-metadata">

**Author:** ![osanseviero](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/osanseviero/32/38738_2.png) [@osanseviero](https://discuss.huggingface.co/u/osanseviero)\
**Post date:** [August 31, 2022, 12:42pm UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/8 "2022-08-31T12:42:33Z")

</div>

Yes, the PR will update it for both `tabular-` tasks.  
You’re right, the widget is now disabled for that. Sorry for misleading you! We need to wait until the PR is merged+deployed and this will work afterward.

---

<div class="post-metadata">

**Author:** ![StyrbjornKall](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/styrbjornkall/32/37485_2.png) [@StyrbjornKall](https://discuss.huggingface.co/u/StyrbjornKall)\
**Post date:** [August 31, 2022, 12:44pm UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/9 "2022-08-31T12:44:01Z")

</div>

Thank you!

---

<div class="post-metadata">

**Author:** ![varun500](https://avatars.discourse-cdn.com/v4/letter/v/dec6dc/32.png) [@varun500](https://discuss.huggingface.co/u/varun500)\
**Post date:** [June 12, 2023, 6:11am UTC](https://discuss.huggingface.co/t/tabular-classification-regression-pipeline/22030/10 "2023-06-12T06:11:47Z")

</div>

I am working on a regression problem and I am looking forward to using Transformers for it but before jumping into the implementation and all stuff, I am curious that can you use transformers for a regression problem? I have around 90 features (floating points) and one target. I couldn’t find any paper on transformers for regression problems so please let me know if any of you used transformers for this purpose.

I am working on a problem where I am having tabular data having more than 90 features and one target and all the features are in integers (continuous). I want to use pre-trained BERT, GPT2 but when it comes to the tokenizer the tokenizer is expecting the input in the text format. I can change the integer data in the text format like this:

original\_data = [1,2,3,4,5,…,94]  
transformed\_data = [“1,2,3,4,5,…,94”]

Now if I pass the transformed\_data to the tokenizer then surely it will work but I wanna know if someone tried to use transformers for this purpose and if yes, then what was the outcome, and how did the results look like?

How can I use the transformers library for this purpose all the tokenizers are trained for the text data so I am kinda lost. Any help will be appreciate.