Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Even if LinkedIn loses and scrapers can no longer be blocked, they still just switched to putting all profiles behind an authwall, or at least it's very hard to not get an authwall. So could HiQ even carry on if they won anyway?


I'm not very familiar with neither LinkedIn nor HiQ, but what would be the problem with logging in before scraping?


The reason the pages are public to begin with is that Google will only scrape public pages for search indexing. LinkedIn wants to provide the pages ONLY to google, so they tried telling hiQ to stop scraping without any physical blockers (so as to not impede google's scraper).

If LinkedIn loses this case they (and others) might try to get Google to change their policy (either use auth or some whitelisted IP addresses or something).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: