It just means that there's at least one aspect of music that everyone agrees on. Harmony might be favored more by some people than others, but they're all hearing harmony. And not everyone developed scales, but all the ones who did agreed on octave equivalence.

Also, idk who finds consistent rhythm completely wrong. Some music breaks it more than others. Is there some culture that will hear Mozart and think "what, this isn't even music"?