AJ
@ItsmeAjayKV
Bullish on local AI
OSS FTW
Independent AI R&D.
quants, repe, kernel.
1x 3060, 1x 3090: 64GB RAM
N4 (日本語勉強中) 🎌
862 Following 9.1K Followers
@deepseek_ai Deepseek-v4.1-flash-v2 coming 👀!!
I'm more convinced something has changed behind the deepseek-platform api. The "deepseek-flash" endpoint seems to be routing to a different model now.
It could be:
- Deepseek-v4.2-flash
- An updated checkpoint of v4.1-flash
Since it's also a little slower, i wonder if V4.1-pro is being quietly tested behind the endpoint.
But either way, i'm getting way better result, the difference is noticeable.
See the comparison video, same prompt, same settings.
Show more
It's Monday and get ready best week of Local AI
Labs, let the model releases begin ! 👀
With so many models expected over the coming weeks, this could be an exciting week for the local AI community.
Which one do you think we will get first?
Show more
GLM 5.3 Flash on a single 3090 at 24.3 t/s decode and over 1000t/s prefill 🔥
And yes, prefill has been optimised and it does get faster the bigger the context window, not magic, just hard work on Maya engine 🫡
Show more
More proof that we need specialized inference engine.
GLM-5.3-Flash is now faster on my 3090 + 64GB RAM
After some work on Project Maya (from @PeasantSmith), my own IQ2_XXS quant now runs at
- 22 - 24 tok/s gen
- 910 t/s prefill at 8k
- 1150 t/s prefill at 32k
Full system spec:
- RTX 3090
- 64GB DDR5 RAM
These are number i never thought i'll see in my system.
Get my quant here:
Maya:
Show more
What a time for 3090 !
GLM-5.3-Flash just got way faster on my 3090 + 64GB ram.
Project-Maya as inference engine:
My PR is in work, but i cannot hold back from teasing.
@alphaprimeus @ItsmeAjayKV Dont forget to donate! Always helps
And remember I’m one man team!
You don't want to miss out on Maya!
Running glm-5.3-flash on my 3090 at this speed was not possible before.
Give it a try !
Maya is on 🔥
57k downloads and growing!
Thank you all for trying and sharing the models and engine
I hope you are having fun 🫡
What a time for 3090 !
GLM-5.3-Flash just got way faster on my 3090 + 64GB ram.
Project-Maya as inference engine:
My PR is in work, but i cannot hold back from teasing.
GLM-5.3-Flash running faster on my 3090 thanks to Project Maya from @PeasantSmith !
After some tweaking (PR coming soon) i'm able to run my Aj-IQ2_XXS mixed quant with Maya.
- ~20t/s avg
- low 200t/s prefill.
Without MTP btw.
I'm working on some pr to improve speed, at this rate even GLM-5.3-flash will be usable in my 3090 before end of this month.
Checkout Maya:
Get my quant:
Show more
My prediction about Qwen4's improvement might just come true after all 🤩
"Alibaba has already been running RSI continuously for over a month since Qwen 3.8 Max!"
"Qwen 4.0 full lineup (Max/Plus/Flash/27B) will continue to deeply adopt it."
If this holds true, which i know it will, we can expect major improvements across the Qwen4 lineup.
Let's gooo !!!!
Show more
💥划重点！Qwen 4.0全系模型将深度RSI，其实Alibaba早在Qwen 3.8 Max就已采用RSI连续跑了一个多月！
云栖大会上，Qwen负责人刘大一恒明确披露：内部已实际使用RSI（递归自我改进），并且Qwen 4.0全系（Max/Plus/Flash/27B）将继续深度采用。
模型从真实日志自动找短板→自造训练数据→小模型多轮验证→最终推入旗舰训练，全程近乎零人工干预。
他补充说到：模型连续跑一个多月、33轮有效迭代，Qwen3.8-Max在AA智能指数上30天内从40飙到45，直接冲上第一梯队。
Qwen 4.0 全系正在自己推着自己进化，加速键已经按下多时了。
#Qwen4# #RSI# #递归自我改进# #大模型# #云栖大会# #刘大一恒# #AI# #自进化# #阿里巴巴#
Show more
Now we can schedule meetings in X itself.
This is really good, everything in this platform itself makes things more easier.
My M1 MacBook Air is still going strong.
What a machine.
I'll be upgrading it prob next year.
a single 3090, a single 3090, a single 3090, a single 3090, a single 3090
320B parameters, 320B parameters, 320B parameters, 320B parameters
18B active, 18B active, 18B active, 18B active, 18B active, 18B active, 18B active
Project Maya 💎
Show more
My favorite firearm platform of all time.
Locally generated the models with trellis on a 5090, and had Opus make it a showroom experience.
Which other model gets this much love?
@Alibaba_Qwen, you should be really proud of your work.
Keep open-weight model drops coming !
Confirmation !!!
We are going to get both:
- Qwen4-27
- Qwen4-max (not sure of size)
open weight drop at same time !!!!
GLM-5.3-flash is going to get even faster on 3090 + 64GB RAM.
With other pr and mine coming soon, we will be able to run it even faster.
Local AI is going through a great time now, especially for small consumer hardware.
Special shoutout to @PeasantSmith and @needmorevram.
Show more
GLM-5.3-Flash running faster on my 3090 thanks to Project Maya from @PeasantSmith !
After some tweaking (PR coming soon) i'm able to run my Aj-IQ2_XXS mixed quant with Maya.
- ~20t/s avg
- low 200t/s prefill.
Without MTP btw.
I'm working on some pr to improve speed, at this rate even GLM-5.3-flash will be usable in my 3090 before end of this month.
Checkout Maya:
Get my quant:
Show more
GLM-5.3-Flash running faster on my 3090 thanks to Project Maya from @PeasantSmith !
After some tweaking (PR coming soon) i'm able to run my Aj-IQ2_XXS mixed quant with Maya.
- ~20t/s avg
- low 200t/s prefill.
Without MTP btw.
I'm working on some pr to improve speed, at this rate even GLM-5.3-flash will be usable in my 3090 before end of this month.
Checkout Maya:
Get my quant:
Show more
Confirmation !!!
We are going to get both:
- Qwen4-27
- Qwen4-max (not sure of size)
open weight drop at same time !!!!
Power of Strata 🦾
Little GPU bro runs Strata 50+ tps on dual 3060 12GB and 64GB DDR4 like a champ! E5-2696v4 quad channel ram still can lift 🦾
As someone who's spent years setting up Windows on pc and laptops, let me tell you Linux and macos are soo easy to setup.
Holy shit is setting up a new Windows 11 computer a shitty experience. When did that happen.
Violently spams enter !!
