Sebastian Raschka
@rasbt
ML/AI research engineer. Ex stats professor.
Author of "Build a Large Language Model From Scratch" ( & reasoning (
加入 October 2012
1.2K 正在关注 511K 粉丝
Nice showcase that interesting LLM work can be done on single GPU!
I extended the GPT-2-style code from @rasbt's "Build a Large Language Model (from Scratch)" so that it was a 6-expert (2 active) mixture-of-experts, and trained it from scratch over 8 days. It worked well! Full writeup with maths and code at
显示更多
