AJ
@ItsmeAjayKV
Bullish on local AI
OSS FTW
Independent AI R&D.
quants, repe, kernel.
1x 3060, 1x 3090: 64GB RAM
N4 (日本語勉強中) 🎌
加入 September 2016
862 正在关注 9.1K 粉丝
What a time for 3090 !
GLM-5.3-Flash just got way faster on my 3090 + 64GB ram.
Project-Maya as inference engine:
My PR is in work, but i cannot hold back from teasing.
GLM-5.3-Flash running faster on my 3090 thanks to Project Maya from @PeasantSmith !
After some tweaking (PR coming soon) i'm able to run my Aj-IQ2_XXS mixed quant with Maya.
- ~20t/s avg
- low 200t/s prefill.
Without MTP btw.
I'm working on some pr to improve speed, at this rate even GLM-5.3-flash will be usable in my 3090 before end of this month.
Checkout Maya:
Get my quant:
显示更多
