Thursday, July 04, 2024

AI: Spreadsheet Is All You Need

 dabochen/spreadsheet-is-all-you-need: A nanoGPT pipeline packed in a spreadsheet

A nanoGPT pipeline packed in a spreadsheet

This is a project to help ... understand how GPT works.
It is pretty fun to play with, especially when you are trying to figure out what exactly is going on inside a transformer.
This helped to visualize the entire structure and the data flow.
All the mechanisms, calculations, matrices inside are fully interactive and configurable.

It contains all the transformer components including:
  • embedding
  • layer norm
  • self attention
  • projection
  • MLP
  • softmax
  • logits

It is based on Andrej Karpathy's NanoGPT structure which includes roughly 85000 parameters.
It is clearly a very small size, but it is both complex enough to understand how it works, and also not too big to crash computer. In contrast to chatgpt, this project is a character based prediction system, meaning that each token is a character, and to reduce the complexity, only letter A/B/C are being tokenized.

Rimac Robotaxi? EV self driving:

 Rimac Just Exposed Tesla's Robotaxi Playbook. - YouTube

Rimac Automobili - Wikipedia

Rimac Automobili (Croatian pronunciation: [rǐːmats automobǐːli]REE-mahts) is a Croatian automotive manufacturer headquartered in Sveta Nedelja, Croatia, that develops and produces electric sports cars. Its sister company, Rimac Technology (part of the Rimac Group) also produces drivetrains and battery systems for automotive businesses.

In July 2021, VW Group's Porsche and Rimac announced a joint venture that would incorporate Volkswagen's high-performance Bugatti brand as well as Rimac Automobili. The new venture is called Bugatti Rimac.[15] Parent company Rimac Group announced a 55% stake in the joint venture, with Porsche holding a 45% stake in the company.[16] in June of that year, Porsche stated it was one of the participants in a new €500 million round of fundraising.[17]