# Use two sentences as inputs for sentence classification **URL:** \ **Category:** 🤗Transformers\ **Created:** [April 12, 2021, 4:29pm UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444 "2021-04-12T16:29:08Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](https://discuss.huggingface.co/u/saireddy)\ **Post date:** [April 12, 2021, 4:29pm UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444/1 "2021-04-12T16:29:08Z")
for an example, we have a datasource with three columns column\_a: text data which describes one feature column\_b: text data which describes another feature column\_c : category/label If i have to approach this kind of text classification problem with BERT, how can we pass column\_a and column\_b as inputs to bert model? is there a way to concatenate two sentences using separator token or is there a way to achieve this using encoder\_plus method? any help is appreciated! ---
**Author:** ![Maimonator](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/maimonator/32/1452_2.png) [@Maimonator](https://discuss.huggingface.co/u/Maimonator)\ **Post date:** [April 12, 2021, 6:21pm UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444/2 "2021-04-12T18:21:55Z")
Not an expert (and far from being one) but I’m interested in what you have tried so far. If I wouldn’t get good results using simple methods (let’s say by just concatenating the two columns) I’d try having an ensemble of two BERT models where one receives `column_a` and the other receives `column_b`. Let’s say one column is an expert opinion about the medical condition of a patient, and the other is the patient’s opinion on his medical condition. Then to me it makes sense to have two models one fine-tuned to the expert opinion and the other fine-tuned to the patient’s opinions. Hopefully I didn’t say too much nonsense 😊 ---
**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](https://discuss.huggingface.co/u/saireddy)\ **Post date:** [April 13, 2021, 12:23am UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444/3 "2021-04-13T00:23:11Z")
thanks @Maimonator, any ideas on passing two text columns into encoder\_plus method? ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [April 13, 2021, 10:06am UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444/4 "2021-04-13T10:06:03Z")
Hi @saireddy, BERT supports sentence pair classification out-of-the-box. There’s no need to ensemble two BERT models. In BERT, 2 sentences are provided as follows to the model: [CLS] sentence1 [SEP] sentence2 [SEP] [PAD] [PAD] [PAD] … You can prepare them using `BertTokenizer`, simply by providing two sentences: ``` from transformers import BertTokenizer tokenizer = BertTokenizer.from_pretrained("bert-base-uncased") sentence_a = "this is a sentence" sentence_b = "this is another sentence" encoding = tokenizer(sentence_a, sentence_b, padding="max_length", truncation=True) ``` This encoding is a Python dictionary, which includes keys including input\_ids. If you decode them back as follows: `tokenizer.decode(encoding["input_ids"])` Then this will print [CLS] this is a sentence [SEP] this is another sentence [PAD] [PAD] [PAD] … ---
**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](https://discuss.huggingface.co/u/saireddy)\ **Post date:** [April 20, 2021, 7:51pm UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444/5 "2021-04-20T19:51:46Z")
hi @nielsr , thanks for the clarification. I have used encode\_plus method to pass values from two columns and its taking two sentences by inserting [SEP] token between . AS BERT had limitation of using 512 tokens, I will try with other variants of bert, which can take longer sentences. Any suggestions on models to start with for long text? ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [April 22, 2021, 7:48am UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444/6 "2021-04-22T07:48:19Z")
Actually, the `encode_plus` method is deprecated, it’s advised to just call the tokenizer, as shown above. BERT indeed has a token limit of 512, so if you provide two sentences they should fit into this budget. If you want to try longer sequences, you can take a look at [LongFormer](https://huggingface.co/transformers/model_doc/longformer.html) or [BigBird](https://huggingface.co/transformers/model_doc/bigbird.html). ---
**Author:** ![sadhaklal](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sadhaklal/32/44699_2.png) [@sadhaklal](https://discuss.huggingface.co/u/sadhaklal)\ **Post date:** [April 20, 2022, 5:05pm UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444/8 "2022-04-20T17:05:58Z")
Do this @xap: ```auto a1 = data["user"].tolist() a2 = data["standard"].tolist() ``` Basically you can’t pass pandas `Series` objects to your tokenizer. You need to convert them to lists first. ---
**Author:** ![sadhaklal](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sadhaklal/32/44699_2.png) [@sadhaklal](https://discuss.huggingface.co/u/sadhaklal)\ **Post date:** [April 21, 2022, 5:16pm UTC](https://discuss.huggingface.co/t/use-two-sentences-as-inputs-for-sentence-classification/5444/10 "2022-04-21T17:16:49Z")
@xap My guess is that each batch in your `DataLoader` is a dictionary (I might be wrong; I need to check your `DataLoader` creation code to confirm). Why don’t you print out one batch & check its structure. Is it a tuple of tensors or is it a dictionary of tensors? You can check by doing this: ```auto for batch in train_dataloader: break print(batch) ``` If it’s a dictionary, then follow the steps outlined here: [A full training - Hugging Face Course](https://huggingface.co/course/chapter3/4?fw=pt) In particular: ```auto outputs = model(**batch) ``` The problem with the following line is that it will pick up the keys of the dictionary rather than the values: ```auto for batch_idx, (pair_token_ids, mask_ids, seg_ids, y) in enumerate(train_dataloader): ``` You can confirm this by adding some print statements inside the loop like so: ```auto print(pair_token_ids) print(mask_ids) etc... ```