Indeed, I just run it (GPT-3.5-turbo) in a loop on increasingly long sequences of "b b b b ....", and it went off the rails somewhere around 200+ 'b's.