Weight placed on the face makes a qualitative difference. More weight means bigger straps and more pressure against the face to keep it there. If the Apple vision is light enough then it might not be an issue, but the size of the strap they're using hints that it's much more significant than most everyday wearable objects.
It’s not a completely valid comparison - the ski goggles are much closer to your face so have less of a moment on your neck. While the vision pro notionally looks like a set of goggles from the front, the side view shows that they’re just as big as every other VR headset.
I don’t think we’re really going to know what they feel like until someone not from Apple puts them on their head!