注册并分享邀请链接,可获得视频播放与邀请奖励。

Avi Chawla
@_avichawla
Daily tutorials and insights on DS, ML, LLMs, and RAGs • Co-founder @dailydoseofds_ • IIT Varanasi • ex-AI Engineer @ MastercardAI
加入 September 2019
152 正在关注    80.5K 粉丝
8 AI model architectures, visually explained: LLMs get so much attention that it's easy to overlook the rest of the field. Different inputs, tasks, and deployment constraints have led to very different model designs. Here are eight specialized AI model architectures: 1. LLM (Large language models) Text is split into tokens, mapped to embeddings, and processed through transformer layers. The model generates its response by predicting successive tokens. ↳ GPT, Claude, Gemini, Llama. 2. LCM (Large concept models) Moves prediction to the sentence level. Sentences become SONAR embeddings, and the model predicts the next sentence representation. Meta explored both autoregressive and diffusion-based versions. ↳ Meta's LCM research introduced this approach. 3. LAM (Large action models) Connects a user's request to actions in an environment. These systems interpret intent, plan steps, and use interfaces or tools, often checking results before continuing. ↳ Rabbit R1, Microsoft UFO, Claude Computer Use. 4. MoE (Mixture of experts) A router assigns token representations to selected expert networks inside the model. Activating a subset of experts adds model capacity without running every expert for every token. ↳ Mixtral, DeepSeek, Switch Transformer. 5. VLM (Vision-language models) Combines visual representations with language processing. A vision encoder extracts image features, which are connected to a language model so it can answer questions about visual content. ↳ GPT-4V, Gemini, LLaVA. 6. SLM (Small language models) Uses fewer parameters to reduce memory and compute requirements. These models can run locally, with quantization often reducing their deployment footprint further. ↳ Phi-3, Gemma, Mistral 7B, Llama 3.2 1B. 7. MLM (Masked language models) Learns by predicting hidden tokens using context on both sides. This bidirectional training produces representations useful for retrieval, classification, and other language-understanding tasks. ↳ BERT, RoBERTa, DeBERTa. 8. SAM (Segment Anything Model) An image encoder processes the image, while a prompt encoder handles points, boxes, or masks. A mask decoder combines them to identify the requested regions. ↳ Meta's SAM supports interactive segmentation and image annotation. These aren't eight mutually exclusive categories. A small model can process images, and a language model can use MoE layers. Some labels describe architecture, while others describe training or intended use. There's another approach I kept outside the image because it alters how a model handles context rather than introducing a separate model family. Recursive language models, from MIT, put the input into a Python environment that a root model can inspect through code. The model can search or slice that input, then make recursive model calls on selected portions instead of loading everything into its own context window. The reported experiments handled inputs beyond 10 million tokens without retraining, with gains over the base model even on some tasks whose inputs already fit its context window. I recently wrote a full breakdown of how this works, including why accessing a large input and reasoning reliably over it are different problems. Read it below.
显示更多
0
17
101
23
转发到社区