Yeah, this concerns me. If someone believes they can upper bound LLM capabilities, I think the onus is on them to explain where and why scaling laws break down. Regardless, it seems like we'll get to AGI relatively soon (say, within a century), whether that's using transformers and LLMs or not.