Yes, it absolutely has to taper, with an exponential curve, to keep stress constant over the length of the cable. The taper ratio (in terms of cross-sectional area) for the best currently available engineering materials for an Earth space elevator would be in the tens to hundreds of millions, but a carbon nanotube cable might only require a taper ratio of around ten. (A steel cable would need a taper ratio on the order of 10^33.)
It’s a function of area so if we scale equally in both dimensions, a taper ratio of 10 means a thickness ratio of ~3.2. But yeah, that doesn’t exactly buy us much.