I re-read the articles about data, and it says that the LLM labs cares more about the speed and quantity rather than quality of the data. They would rather get a lot of data with mixed qualities rather than missing any of the good ones. Might this be the reason why the in-house data teams is still not enough? They can't have this quantity and speed without purchasing from outside. (Rather than just the "domain expertise" reason that I originally thought). Another reason that makes data venders inreplaceable from non-technical perspective is that, companies want to use data vendors as arm’s-length proxies to launder data, such as distilling rival models like OpenAI’s and Anthropic’s or scraping user data from their own apps, shifting legal and reputational risk the companies can’t or won’t take on themselves to those data venders.