Yes, guardrails on open models can be removed. Likewise guardrails on closed models can fail disastrously too. It is a complex problem to solve and not one has shining record in this matter.
It is tempting to portray open models are uniquely dangerous. As a matter of fact - the most hacking behaviours reported till date are with closed models.
For the sake of fairness, we should not do things to open models ecosystem what we won't do to closed models after some misbehaviour by them.
Some AI security prophets may offer us a version of reality where banning open models automatically makes the world a lot safer. That is false comfort.
Yes to AI security that is impartial to models without discrimination between open and closed nature.
It is admirable frontier labs CEOs are concerned about AI safety - I do not doubt them - they are great humans and have give us these beautiful model.
In the continuation of that spirit they need to share details on
- architecture of models,
- inference systems,
- guardrails & classifiers,
- training method,
- data sets
so it can be widely scrutinised - at least with a centralised authority.
Your friends and family vouching for it it not enough.
Same could be expected from likes of Fireworks, BaseTens and all the NeoCloud offering Token factories, as well as, Open source labs as well.
If they do not comply with the requirements, I totally support shutting them down. But there can not be different rules for different parties.
顯示更多
to be clear about open models: i love them. i have a kimi k3 finetune running on tinker. bad things may happen with free distribution of open source models approaching superintelligence. the offense dominates the defense. if that happens, china/us will try to control it
顯示更多