Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
jxramos
on Jan 16, 2018
|
parent
|
context
|
favorite
| on:
Google Memory Loss
you'd think it would at least come up in the internet archive if not anywhere else.
paulcole
on Jan 16, 2018
[–]
https://web.archive.org/robots.txt
emmelaich
on Jan 16, 2018
|
parent
[–]
That's unfortunate. But understandable in a way.
# robots.txt web.archive.org 2013-10-02 User-agent: * Disallow: / User-agent: ia_archiver Allow: /
jxramos
on Jan 16, 2018
|
root
|
parent
[–]
touche, I don't suppose the old non commercial websites mentioned in the article suffer the same problem though right? Maybe an accidental robots.txt file was mistakenly left around?
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
