Well with an exponential curve, even if the amount of the initial dose changes 10 fold, from 20,000 to 200,000 virons you will get to hundreds of millions in pretty much the same time. I believe adaptive immunity takes about a week or two to kick in, and in that time the virus is going to keep multiplying like crazy, regardless of the dose.

Also, suppose you were living with a partner and both of you got infected. Both of you will continually shed and "re-infect" the other, so would we expect that the infection will necessarily be severe?