I trained a 1.1B-parameter LLM from scratch on 20B tokens for about $200

A few months ago, I had the idea of making a LLM from scratch as a personal project (for learning and partly for improving my resume). Since I learned a lot from other posts on here over the past year, I wanted to...