Exactly. Point of the poem is how a marginal different in options will later be construed as "made all the difference", and thus the colloquial memory of the poem is construed as "made all the difference" when the point of the poem is that there wasn't a meaningful difference but would be remembered as though it was.

Hard to describe precisely because the "life imitates art" is so recursive/tautological here.