Dmytro Dzhulgakov
@dzhulgakov
Co-founder and CTO @FireworksAI_HQ. PyTorch core maintainer. Previously FB Ads. Ex-Pro Competitive Programmer
가입 April 2013
845 팔로잉 중 7.9K 팬
pretraining implies unique model arch and/or optimization setup - hard to abstract as API while allowing full flexibility
midtraining is different: finetuning on Ts of tokens, same arch
you need full param, not only LoRA - check Fireworks Training, we’ve got it covered
더 보기
if tinker launched a pretraining API it would rip so hard. probably mostly by quote/enterprise but theres really no good OSS pretraining stack, unlike RL
