Just met with one of our engineering leaders. Some takeaways about AI engineering best practices + more:
1) Spec writing and strong reading comprehension are two of the most valuable skills in ai engineers today.
2) Being hyper structured and opinionated in engineering workflows is how you get probabilistic models to behave deterministically when you want them to and also get models to spend tokens efficiently.
3) Creating a standardized schema/metadata on markdown files in your workflows allows you to make non-software tasks verifiable which allows you to close the agent loop more successfully.
4) One of the bigger behavioral changes in knowledge work is learning to thoughtfully structure/organize your files like good engineers have always done to get the least entropy from models.
5) Building a strong immune system around markdown files is important. As a workflow evolves and gets more reps it’s easy for specs to get bloated with conflicting guidance/unnecessary rules. Hermes agents solves this with a thin memory layer. There’s still a lot of optimization to be done with memory/markdown autophagy.
6) The key ingredients of our engineering process are CLI + deeply opinionated folder/file system + markdown metadata + coding agents + linear as source of truth.
7) Our mental model is always how can we make sure the agent has one way to do things and can validate its output.