v6 actually has longer headers on average (40B instead of 20B) but is generally more efficient to process because it's a fixed header length.
ON the other hand, theoretically IPv4 header length could vary depending on the presence of IP options, so you'd need to compute the offset to continue processing the packet.
(I say theoretically because conventional wisdom is that IP options are unreliable in the face of middleboxes, so they're mostly unused. But compliant IPv4 processors have to calculate the header size anyways...)
ON the other hand, theoretically IPv4 header length could vary depending on the presence of IP options, so you'd need to compute the offset to continue processing the packet.
(I say theoretically because conventional wisdom is that IP options are unreliable in the face of middleboxes, so they're mostly unused. But compliant IPv4 processors have to calculate the header size anyways...)