Thursday, July 04, 2024

AI: Spreadsheet Is All You Need

 dabochen/spreadsheet-is-all-you-need: A nanoGPT pipeline packed in a spreadsheet

A nanoGPT pipeline packed in a spreadsheet

This is a project to help ... understand how GPT works.
It is pretty fun to play with, especially when you are trying to figure out what exactly is going on inside a transformer.
This helped to visualize the entire structure and the data flow.
All the mechanisms, calculations, matrices inside are fully interactive and configurable.

It contains all the transformer components including:
  • embedding
  • layer norm
  • self attention
  • projection
  • MLP
  • softmax
  • logits

It is based on Andrej Karpathy's NanoGPT structure which includes roughly 85000 parameters.
It is clearly a very small size, but it is both complex enough to understand how it works, and also not too big to crash computer. In contrast to chatgpt, this project is a character based prediction system, meaning that each token is a character, and to reduce the complexity, only letter A/B/C are being tokenized.

No comments: