> I just give you two books that you've read and you tell me which is better. Do this loads of times and eventually you have a solid ranking of every book you've ever read. Aggregate everyone's rankings and you have a much more robust system then "please rate this book out of 5 stars".
We've done exactly this with Flickchart - https://www.flickchart.com - presenting the user with two movies at a time and letting them choose what, in their opinion, is the better film. We then aggregate everyone's rankings to generate a combined chart of the best films of all-time that's always in flux and changing.
We take into account each user's total number of rankings (refinement), and the total number of movies they've ranked (breadth of knowledge). We also use time as a factor - given that something that just came out has had less time to be "an all-timer" than something that has existed in the world longer and had the time to be experienced by more people.
We created this way back in 2009 and are currently building the next-generation of it on a modern web stack. This new iteration is also more modularized so we can expand more easily from just movies to other domains (books, games, music, TV, etc.).
We've done exactly this with Flickchart - https://www.flickchart.com - presenting the user with two movies at a time and letting them choose what, in their opinion, is the better film. We then aggregate everyone's rankings to generate a combined chart of the best films of all-time that's always in flux and changing.
We take into account each user's total number of rankings (refinement), and the total number of movies they've ranked (breadth of knowledge). We also use time as a factor - given that something that just came out has had less time to be "an all-timer" than something that has existed in the world longer and had the time to be experienced by more people.
We created this way back in 2009 and are currently building the next-generation of it on a modern web stack. This new iteration is also more modularized so we can expand more easily from just movies to other domains (books, games, music, TV, etc.).