nanochat
Nanochat was developed by Andrej Karpathy, who previously served as the Director of AI at Tesla and co-founded OpenAI. This project builds upon his previous work with nanoGPT. Nanochat is an extensive effort aimed at creating comprehensive chatbot-like language models through a complete end-to-end training and inference pipeline. It consists of approximately 8,000 lines of clean code and serves as the culmination of Karpathy’s LLM101n course at Eureka Labs. The goal is to make it easier and more educational for researchers, students, and developers to develop large language models.
Visit Website- Full-Chain Training WorkflowEnd-to-end implementation from tokenization to web serving, with all components integrated into a single codebase that can be executed using simple scripts.
- Affordable LearningProvides basic ChatGPT-like capabilities for around $100 worth of computing power (equivalent to 4 hours of training on 8 X100 GPUs), making it suitable for individual researchers and smaller teams.
- Low DependenciesA clean, hackable codebase with few external dependencies, which makes it straightforward to comprehend and adjust.
- Scale-Out ArchitectureSupports training of variously budgeted large models, ranging from $100 for basic models to $1000 for more advanced versions.