Not listening to the majority, on the other hand, may lead to your project being forked. Take Gnome 3 as an example. This in turn may lose you a significant amount of developers so that you have two projects that each are weaker than the original one. This seems to be an inherent problem with projects like this. The only thing resembling a solution is to have a strong project leader who is able to convince people to trust him or her on radical UX changes.
UX is the most under appreciated area of open source software. It requires change, following/setting design trends and not listening to the majority.