Crucially, this was before the transformer paper, "Attention is all you need", or any of the diffusion models had been released.

This just looks like some painful RL or evolutionary algorithms, at best.