Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Twitter is returning it's pages with all of the links marked as rel=nofollow. This means Google can't index the content that is linked by Twitter users.


That's not true. It just means that it won't contribute to pagerank. But the GoogleBot has discretion to crawl or not crawl the links.


So you're saying that the Google spider should break with expected and normal behavior (honoring nofollow) in order to crawl pages that someone else has gone to the trouble of making noncrawlable?

Yeah, that won't get them into any PR hot water or anything! </sarcasm>




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: