I suspect that whether it's considered more rude will vary by culture, but yes I think "I don't care about x" is a way to specify a lack of want in contrast to a negative want. It's also probably the most common way, but still used rarely I find, maybe because people consider it more rude.