Trying to put his in an uncontroversial way: the human brain (or a brain plus paper and a pencil) can be turning complete/equivalent. Therefor a human sitting down with a pen and pencil could, in a painstakingly long time, compute the backwards and forward passes of a transformer network.

Therefor a human with no understanding of grammar/language, and using no innate biological circuits, could process grammar and respond with language.

The flaw in this argument would be how to teach a human to do this without grammar ...