> you cannot conclude that [a model] has no understanding based solely on what it is trained to do
agreed.
but that's not everything we're basing our conclusions on – we also know that GPT-3 was trained purely on text, and i (and presumably GP) don't think that's a path towards "understanding".
in other words, i think being a language model [trained only using a text corpus] is a valid reason to be skeptical of its potential :)
There's this really weird idea that you need high-bandwidth highly-multimodal sensory data to ‘truly’ understand things, but this makes no sense to me.
1. It's all just data in the end. The signals from your eye are not any more real because they are in response to photons.
2. Deafblind people exist, and can even learn to speak vocally. They might have misconceptions about the visual world, but it's certainly not true that they lack general intelligence.
3. As an objection to GPT-3 as a pathway to AGI, the only thing stopping us training these models on high-bandwidth highly-multimodal data is scale, anyway, so the objection doesn't work.
4. GPT-3 is clearly capable of reasoning in ways that can only be explained with more sophisticated world models than at smaller scales. People asserted reasoning was at a limit with GPT-2, and it just clearly isn't.
agreed.
but that's not everything we're basing our conclusions on – we also know that GPT-3 was trained purely on text, and i (and presumably GP) don't think that's a path towards "understanding".
in other words, i think being a language model [trained only using a text corpus] is a valid reason to be skeptical of its potential :)