# How to plot models using torchviz or hiddenlayer

**URL:** <https://discuss.huggingface.co/t/how-to-plot-models-using-torchviz-or-hiddenlayer/24392>\
**Category:** 🤗Transformers\
**Created:** [October 13, 2022, 3:04pm UTC](https://discuss.huggingface.co/t/how-to-plot-models-using-torchviz-or-hiddenlayer/24392 "2022-10-13T15:04:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Indramal](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/indramal/32/61744_2.png) [@Indramal](https://discuss.huggingface.co/u/Indramal)\
**Post date:** [October 13, 2022, 3:04pm UTC](https://discuss.huggingface.co/t/how-to-plot-models-using-torchviz-or-hiddenlayer/24392/1 "2022-10-13T15:04:55Z")

</div>

I am trying to plot models using torchviz and hiddenlayer but both gets errors.

torchviz - [GitHub - waleedka/hiddenlayer: Neural network graphs and training metrics for PyTorch, Tensorflow, and Keras.](https://github.com/waleedka/hiddenlayer)  
hiddenlayer - [GitHub - szagoruyko/pytorchviz: A small package to create visualizations of PyTorch execution graphs](https://github.com/szagoruyko/pytorchviz)

Common Code:

```auto
from transformers import AutoModel
model1 = AutoModel.from_pretrained("bert-base-uncased")

```

Code for torchviz :

```auto
from torchviz import make_dot, make_dot_from_trace
make_dot(model1)

```

Error:

> AttributeError: ‘BertModel’ object has no attribute ‘size’

Code for hiddenlayer:

```auto
import hiddenlayer as hl
hl.build_graph(model1)

```

Error:

> AssertionError: Argument args must be provided for Pytorch models.

How to solve these errors or do you have any other model plot method?

---

<div class="post-metadata">

**Author:** ![mert-kurttutan](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mert-kurttutan](https://discuss.huggingface.co/u/mert-kurttutan)\
**Post date:** [December 10, 2022, 9:40am UTC](https://discuss.huggingface.co/t/how-to-plot-models-using-torchviz-or-hiddenlayer/24392/2 "2022-12-10T09:40:52Z")

</div>

> [@Indramal](#):
>
> `AutoModel.from_pretrained("bert-base-uncased")`

Hi,  
There is a new package developed just for this purpose, called `torchview`. I tried your model with torchview with the following code

```auto
from transformers import AutoModel, AutoTokenizer
from torchview import draw_graph
model1 = AutoModel.from_pretrained("bert-base-uncased")

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModel.from_pretrained("bert-base-uncased")

inputs = tokenizer("Hello world!", return_tensors="pt")

model_graph = draw_graph(model1, input_data=inputs)

model_graph.visual_graph

```

The output is the following, (zoom in the image for clearity)

 ![model](https://us1.discourse-cdn.com/hellohellohello/original/2X/1/13f515e0a11fd15dc8c036e02f2f28cea768a025.png)

It also works very well with many other model types  
The link for the project: [torchview project](https://github.com/mert-kurttutan/torchview)  
Disclaimer: I am the author of the package.

---

<div class="post-metadata">

**Author:** ![Indramal](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/indramal/32/61744_2.png) [@Indramal](https://discuss.huggingface.co/u/Indramal)\
**Post date:** [December 11, 2022, 1:30am UTC](https://discuss.huggingface.co/t/how-to-plot-models-using-torchviz-or-hiddenlayer/24392/3 "2022-12-11T01:30:22Z")

</div>

wow, thank you very much. I also try to build a library but I have no time so I skip it.

---

<div class="post-metadata">

**Author:** ![jeffnju](https://avatars.discourse-cdn.com/v4/letter/j/fbc32d/32.png) [@jeffnju](https://discuss.huggingface.co/u/jeffnju)\
**Post date:** [November 18, 2023, 7:05pm UTC](https://discuss.huggingface.co/t/how-to-plot-models-using-torchviz-or-hiddenlayer/24392/4 "2023-11-18T19:05:47Z")

</div>

Hi Mert, thanks for your efforts in torchview, which is very useful to visualize models.

Though it works in some models (e.g. “bert-base-uncased” as you mentioned), there is a problem for some other models like “google/flan-t5-small”. Details can be found below

```auto
tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-small")
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-small") 
input_text = "translate English to German: How old are you?"
inputs = tokenizer(input_text, return_tensors="pt")

model_graph = draw_graph(model, input_data=inputs).visual_graph
# Line below triggers an error: "RuntimeError: Failed to run torchgraph"
model_graph.render(directory='/home/jeff/tmp/graphviz_output').replace('\\', '/')

```