GPT-6.1 Terra beats Astra in our vision benchmarks
wtf OpenAI?
had to double-check the results
- text
- object detection
- counting
also ~47% faster and ~92% cheaper
↓ leaderboards and examples
We are hosting the Visual Intelligence Summit on October 22 in SF.
The physical world is AI's biggest opportunity and the best way to advance humanity in our lifetimes. Few people are building for AI beyond the screen.
The Summit is for the people who see it differently and want to be at the frontier of bringing AI into the physical world.
If you’re building for that future, join us.
Ever wanted to know if you're cooked in an IRL game of chess? 💀
So I built Stockfish IRL. A camera watches a real board, tracks every move, and literally yells at you when you blunder.
Built with @roboflow 👇
Gemini 4 Argon vs GPT-6 Astra YouTube stream
tomorrow, Oct 15th, 8 AM PT / 11 AM ET / 5 PM CET
save the time and see you tomorrow
- detection
- segmentation
- transcription
- data extraction
- efficiency and cost
what else?
link:
Astra dethroned
Gemini 4 Argon is the best "vision" model
#1# for detection:
- tight, precise boxes
- dense scenes
#1# for text extraction:
- small text
- handwritten text
- complex tables
- rotated text
4× cheaper than Astra at high
Muse Wonder is pretty good vision model
suuuuuuuper expensive
vs Muse Spark 1.3
- better text extraction
- better object detection
- 2x faster
- 4x more expensive (almost Astra price)
↓ benchmarks and examples
finally releasing our new OCR benchmark
tons of hard examples like this mechanical dial meter
- single value extraction
- JSON data extraction
- document transcription
- text localization and recognition
- 48 models evaluated
↓ examples and leaderboards
My favourite thing about @roboflow Trackers is how easy we made it to swap things and create the best tracker for your case.
Swap the tracker, the IoU variant, camera motion compensation, dynamic frame rate or masks, one line change, and see what works best!
Any detector works.
At COLM? Join Roboflow Labs tonight to talk vision, language models, and everything in between.
Meet fellow researchers and builders over light bites, 7–10pm PT.
Space is limited. Request a spot:
Haiku 5.5 first look
I tested new model from Athropic on object detection and our new text benchmark
Haiku 5.5 is better and cheaper than GPT-6 Luna
leaderboards and examples ↓
openai just released Decisions API, Jev-like "system 1" multi-modal model based on GPT6 Luna!
Each egg gets cropped and sent to OpenAI (~270ms latency) and returns P(clean, dirty, cracked).
One image crop consumes ~280 input tokens (~$0.03/1K imgs). @roboflow RF-DETR for detecting eggs
I spend a lot of time finding out what VLMs still can't do
pretty excited to help change that
we launched Roboflow Labs at @roboflow
training data, evals and environments for real-world tasks. vision, multimodal reasoning, robotics and coding.
link:
I just labelled over 1000 segmentation masks for utility poles in under 90 s 🤯
-GPT Astra for box detections
-SAM3 for masks on those boxes
How long would it take a human to do the same??
the Driveline team is pushing the frontier of what's possible with AI + sports
if you're interested in how AI is shaping the future of sports, you have to come check out their session
Driveline Baseball x Roboflow
Come see @SamEhrlich and I speak in San Francisco at the @roboflow Visual Intelligence Summit on October 22nd.
@DrivelineBB is pushing the boundaries of applied AI/CV in sports and we're honored to be selected for VIS!
you can label data in roboflow with astra
millions of vision products are now unblocked by labeling time/cost going almost to $0
crazy next 12 months ahead
GPT-6 Astra is now the top model for vision tasks
sure 3-10x more expensive but 50%+ faster
extremely capable model, OpenAI really pushed the frontier in all directions with this one
Gemini dethroned
GPT-6 Astra is the best "vision" model I've seen
- notices tiny details
- draws conclusions
- produces precise and tight boxes
- sees and reads text in different contexts
- knows a lot about a lot
- fast
- expensive
↓ GPT-6 Astra detection deep dive
new Gemini releases aren't moving much on the vision leaderboard in terms of accuracy scores, Gemini 3.5 Flash is still the top for example
but a lot of movement happening when it comes to cost and speed
every big shift in speed+cost is making new production use cases possible, exciting times
Gemini 3.8 Flash might be my new favorite Gemini model
- #1# image reasoning
- #1# data extraction
- #2# object detection
- 30% faster than Gemini 3.7 Flash
↓ Gemini 3.8 Flash deep dive
Gemini 3.8 Flash and Fable 5.1 vision evals are live, new top 10 ranking order
Google still dominating
Fable improving, notably ~50% faster than the Gemini models beating it
Meta has 2 spots, bet you didn't see that coming
Fable 5.1 is the best "vision" model Anthropic ever released
big gains in object detection and counting
still very strong in OCR
still very expensive
↓ deep Fable 5.1 dive
Giving Microduck object-tracking vision capabilities!
🦆👀🤖
It can now play fetch and doesn't get confused by other objects.
Using @roboflow Trackers library I made Microduck keep track of the different objects in its sight and has a control policy for reaching the ball it tracks.
Open-Source code below 👇
it's really hard to understand and estimate costs for vision tasks with frontier models
adding features to make it as transparent as possible
once you have fully visibility into models, decisions on which ones to use can totally shift
Fable 5 is so expensive
we added a feature today that lets you check how many tokens a model generated when responding to your computer vision query and how much it cost
same image, same prompt
- Google 3.7 Flash - $0.005
- Qwen3.8-Max - $0.018 (3.6x)
- Fable 5 - $0.170 (34x)