# Distilbert Seq2clas **URL:** \ **Category:** 🤗Transformers\ **Created:** [June 25, 2021, 1:46am UTC](https://discuss.huggingface.co/t/distilbert-seq2clas/7323 "2021-06-25T01:46:37Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![natank](https://avatars.discourse-cdn.com/v4/letter/n/46a35a/32.png) [@natank](https://discuss.huggingface.co/u/natank)\ **Post date:** [June 25, 2021, 1:46am UTC](https://discuss.huggingface.co/t/distilbert-seq2clas/7323/1 "2021-06-25T01:46:37Z")
Hello I have two questions: 1 We can view the. pooled layer by using output\_hidden\_states=true and follow the logic, but is there a generic way to do so? 2 When we are doing inference, can we output the hidden\_states from trainer.predict() as we do by model(input/output\_hidden\_states=True)? Thanks ---
**Author:** ![pratikbhatia0011](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@pratikbhatia0011](https://discuss.huggingface.co/u/pratikbhatia0011)\ **Post date:** [July 18, 2021, 10:59am UTC](https://discuss.huggingface.co/t/distilbert-seq2clas/7323/2 "2021-07-18T10:59:29Z")
I have a similar doubt as regards point 2. I am working on Question Answering with Distilbert. The predict function in the Trainer does not work if output\_hidden\_states = True. It works fine if the same argument is set to False. Is this a bug? If not, then how is one to use a model for prediction if one has set the argument output\_hidden\_states = True while initializing the model ? ---
**Author:** ![pratikbhatia0011](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@pratikbhatia0011](https://discuss.huggingface.co/u/pratikbhatia0011)\ **Post date:** [July 18, 2021, 12:01pm UTC](https://discuss.huggingface.co/t/distilbert-seq2clas/7323/3 "2021-07-18T12:01:09Z")
Found the solution. Posting it here just in case someone else too gets stuck with my particular problem. I just had to pass ignore\_keys = [‘attentions’] in the predict function and everything works fine. ---
**Author:** ![pratikbhatia0011](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@pratikbhatia0011](https://discuss.huggingface.co/u/pratikbhatia0011)\ **Post date:** [July 18, 2021, 12:22pm UTC](https://discuss.huggingface.co/t/distilbert-seq2clas/7323/4 "2021-07-18T12:22:45Z")
@natank To answer your second question, I don’t think so because as seen in the predict function source code, it only returns predictions, label\_ids and metrics. ![trainer_predict](https://us1.discourse-cdn.com/hellohellohello/original/2X/9/9548774cb4fa6980305311e4894aa48f00876563.png) ---
**Author:** ![natank](https://avatars.discourse-cdn.com/v4/letter/n/46a35a/32.png) [@natank](https://discuss.huggingface.co/u/natank)\ **Post date:** [July 19, 2021, 11:53am UTC](https://discuss.huggingface.co/t/distilbert-seq2clas/7323/5 "2021-07-19T11:53:32Z")
Thanks for you replies. Regarding one it indeed resolved me a problem. Regarding your answer on q2. It seems that Huggingsface offer two working modes, research : use model , and model taining - use predict. Am I right?