Old timers (‘scuse me while I slam a geritol and bourbon) will remember that when reddit first launched, there was a recommendation engine that purportedly took your votes and turned them into a personal page of stories you would enjoy.
It was scrapped and eventually subreddits were introduced. I think people like the idea of communities.
That being said... The fact that something failed one, two, or a hundred times in the past doesn’t mean it won’t work today. Things may be different today, or perhaps the approach may be slightly different. If Google can make bazillions of dollars using machine learning to optimize the ads displayed on a page, I’m pretty confident machine learning can be used to optimize the likelihood that you’ll upvote articles suggested by a bot.
The question of whether this becomes an echo chamber and you stop finding things that are interesting but outside of your current tastes is deep. There’s an old saying, “The best present is something you didn’t know you wanted until you unwrapped it."
Reading HN will also not expose you to non-hacker-related information. At the same time, it's always possible to make the stories you're more likely to like more likely to be chosen, and introduce some randomness to be exposed to the odd lower-rated story.
More than that, subreddits are why Reddit didn't turn into digg.
My theory is that as communities grow, you have one of two choices to maintain cohesion: aggressively police who is let in or give people with divergent interests space to express those interests.
If you don't, and your community is growing, eventually the most valuable members will no longer find value in the community. This is just reversion to the mean. But the consequence is that those valuable people will then leave, and the cycle repeats until the community evaporates entirely.
Subreddits are a "pressure valve" that allow high-value users to self-select and maintain their own set of norms without sacrificing the growth of the main public square.
It's not the stories and comments that are fundamentally valuable to HN, it's the people.
I remember there being a lot of discussion before subreddits where most people (myself included) wanted tagging instead of subdomains. The subreddits have obviously worked well but I think they (along with self posts and image previews) also changed reddit from a news aggregator into a glorified forum.
The community is very important. But there's something to be said for the ability to filter properly - there was a blog post on HN recently discussing how aggregate sites end up with the lowest common denominator. What ends up on the first page is only a small % of the submissions that are actually interesting. I probably miss half of the stories that I would like just because they fall off before I even see them.
The question is, how do you figure out what's interesting to who? Joel is attempting to answer this for himself. A site that could answer it for everyone while still somehow maintaining social bonds would do very well I think.
I have been working at it with http://hubski.com. In short, you choose people to filter content for you based on what they post, and what they share. You in turn choose what to pass on to those who follow you. It's been working pretty well.
I'm still working on recommendations. There are suggestions once you follow someone. It's a good way to get started, but the best way to get good content is to hand pick folk based on what they have shared.
I remember about 5 years ago when I was looking for something else like reddit only more technical, saw HN and thought, "wow, I don't have a clue what 90% of these people are talking about. Seems interesting."
It was scrapped and eventually subreddits were introduced. I think people like the idea of communities.