Marc Sun
@_marcsun
Machine Learning Engineer @huggingface Open Source team
524 Following 1.7K Followers
Your LLM knows the answer before it writes a single token.
So we stopped making it write.
transformers serve now turns any LLM or VLM into a decision model with `/v1/systemone` endpoint.
Decision model standardization coming soon in transformers by @art_zucker
Demos below👇
Show more
Millions of GGUF downloads later, those same llama.cpp checkpoints can now run in 🤗 transformers.
Same models, more ways to use them, and fast local inference on Mac powered by ggml kernels!
Blog:
ggml kernels:
Show more
