Oh nice, hadn't seen it. It looks good, and they've clearly got some decent stuff I don't (chrome extension, embedding in model cards, family pages).
The main difference appears to be how the graph gets made. From what I can tell they analyze the config on the server (their page says the first request can take a few minutes). I actually build the model on PyTorch's meta device and run a fake forward pass through it, so I get the real execution order and the tensor shapes at every module. That's what the animated replay and the compute/KV cache numbers are based on.
Hi! Not from the config alone, the config just tells transformers which classes to build. I let it actually build the model, but on PyTorch's meta device, so every module and parameter exists with its real shape and dtype but no memory behind it. That gives me the true nn.Module tree (the same one print(model) would show), and I hash repeated subtrees so 36 identical layers show as one stack with a ×36 badge.
Then I run a fake forward pass (dummy inputs, also on the meta device) with a forward hook on every module. Most ops do shape inference fine without real data, so that records the execution order and every module's input/output shapes. Both things you're asking about are already in the UI:
- Tensor shapes: click any module and the inspector shows its traced input/output (e.g. [1 batch × 7 seq × 4096 hidden], the labels come from matching dim values against config) plus its weight shapes ([151936 vocab × 4096 hidden]). The flow-replay HUD shows the shape transformation at each step.
- Per-layer parameter counts: every node shows its param count and share of the model, there's a treemap of children by params, and a "cost" lens that switches the whole map to compute (MACs), activation memory, or KV cache — with sequence length as a slider.
Params/dtypes are cross-checked against the safetensors headers (fetched via HTTP range requests, no weight download).
I'd love to hear more about that. Are you able to fine tune specific layers?
Specifically, one particular otherwise excellent model I use has an alignment problem (sycophancy) that I've isolated to a specific layer. I can nuke the layer with lora and the behaviour stops - but I'm not sure what else I'm nuking in the process. I'm quite new at this so I'd love any advice. Thank you!
I saw a similar page a month back. The design aesthetics are bit different
https://hfviewer.com/
Oh nice, hadn't seen it. It looks good, and they've clearly got some decent stuff I don't (chrome extension, embedding in model cards, family pages).
The main difference appears to be how the graph gets made. From what I can tell they analyze the config on the server (their page says the first request can take a few minutes). I actually build the model on PyTorch's meta device and run a fake forward pass through it, so I get the real execution order and the tensor shapes at every module. That's what the animated replay and the compute/KV cache numbers are based on.
How are you generating the architecture graph from model configs, and do you plan to surface tensor shapes or layer-level parameter counts as well?
Hi! Not from the config alone, the config just tells transformers which classes to build. I let it actually build the model, but on PyTorch's meta device, so every module and parameter exists with its real shape and dtype but no memory behind it. That gives me the true nn.Module tree (the same one print(model) would show), and I hash repeated subtrees so 36 identical layers show as one stack with a ×36 badge.
Then I run a fake forward pass (dummy inputs, also on the meta device) with a forward hook on every module. Most ops do shape inference fine without real data, so that records the execution order and every module's input/output shapes. Both things you're asking about are already in the UI:
- Tensor shapes: click any module and the inspector shows its traced input/output (e.g. [1 batch × 7 seq × 4096 hidden], the labels come from matching dim values against config) plus its weight shapes ([151936 vocab × 4096 hidden]). The flow-replay HUD shows the shape transformation at each step. - Per-layer parameter counts: every node shows its param count and share of the model, there's a treemap of children by params, and a "cost" lens that switches the whole map to compute (MACs), activation memory, or KV cache — with sequence length as a slider.
Params/dtypes are cross-checked against the safetensors headers (fetched via HTTP range requests, no weight download).
Visually inspecting how a particular model is architected is quite interesting. Thank you for coming up with this!
extremely cool. I’ve had Claude walk model architectures to debug Loras and fine tunes, but this is delightful
I'd love to hear more about that. Are you able to fine tune specific layers?
Specifically, one particular otherwise excellent model I use has an alignment problem (sycophancy) that I've isolated to a specific layer. I can nuke the layer with lora and the behaviour stops - but I'm not sure what else I'm nuking in the process. I'm quite new at this so I'd love any advice. Thank you!
Love it, I thought it only shows the "guts" of a model, turns out it also estimates the cost of serving that model.
Thanks for good vibes. This is awesome
[dead]
[dead]
[dead]