You are correct. It's likely that the majority of humans have been lactose intolerant for human history. It's primarily people of Northern European descent that are not lactose intolerant, but lactose intolerance is very common outside of those groups