One of the thing we realized early is that AI is so good at copy pasting. Whenever it hallucinated things that didn't exist was basically a failure of injecting the right things in the context. So we engineered the Astryx project to harness this behavior with amazing results!
These social experiments around models are so fascinating. As the models are being used in more and more daily situations, the way they react becomes critical.