To be clear, @GaryMarcus, your 2023 post did say "some kind of hybrid may well work."
But to be precise about it, for years, what you've meant by "hybrid" is neurosymbolic architecture (some form of symbol manipulation built into the system itself). It did not mean an LLM that's
This wins the prize for sleazy misrepresentation.
@mattShumer took my 2023 argument for hybridizing LLMs with symbolic tools – which is *exactly* what everyone does nowadays – and made it sound like I said something completely different, by taking nine words entirely out of
Gary Marcus is right that math ≠ all of science.
But his bar just keeps moving:
2012: deep learning is "at best, only a small step toward the creation of truly intelligent machines"
2022: "deep learning is hitting a wall" months before ChatGPT
2023: fine, chatbots, but they
the last clause in particular (golden age of science) is a total leap of faith, the same overgeneralization from formal problems to difficult-to-formalize problems fallacy i have seen repeatedly all day.
i wouldn’t even be sure that GPT-next will be immune from deleting user
Solving just one of these problems would have been unbelievably impressive.
The next OpenAI model solved ten.
Looks like GPT-next is going to make Fable look like a toy, and usher in a golden age of science.
An internal version of Astra, @OpenAI’s next major model family, solved 10 major open problems in mathematics, quantum complexity, and theoretical computer science.
We believe it will be a major step for scientific reasoning. openai.com/index/ten-adva…
