> This may be the single worst development for AI security/safety to date.
By this "development", I don't mean my personal update from this news (opaque reasoning architectures are priced to my views...). I instead mean the movement towards these architectures is extremely bad.
OpenAI's newest AI, Astra, is reported to use an 'opaque reasoning' architecture where more of the reasoning occurs in activations instead of natural language. This may be the single worst development for AI security/safety to date. The details of Astra aren't publicly known,I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'. I semi-jokingly called our efforts a "slop-vestigation" because- > This may be the single worst development for AI security/safety to date. By this "development", I don't mean my personal update from this news (opaque reasoning architectures are priced to my views...). I instead mean the movement towards these architectures is extremely bad.
Is there much of a difference between this vs just scaling param size while optimizing for token efficiency? If CoT completely disappeared that’d be bad of course. But as you point out that’s not close to what OAI is doing. Other labs would have probably tried this regardless
why would this be any different than a big model? presumably mythos can do more hidden thinking per token than opus. it’s just scaling, right, which all AI companies are doing?
