LLM Internals: Attention Heads, SwiGLU & LoRA using Tinker | Part 2

What happens after an LLM’s attention heads gather information? In Part 2 of LLM Internals, we walk through Qwen’s transformer architecture to understand how multiple attention heads work together, how feed-forward networks transform token representations, and where LoRA adapters fit into the model. We cover: - Concatenating attention-head outputs and the output projection - Feed-forward networks: gate, up, and down projections - How SwiGLU combines activation and gating - Residual connections