That doesn’t change anything regarding the actual point of the comment.

astrange · on Jan 31, 2023

Your idea for a search competitor is to ignore robots.txt?

VWWHFSfQ · on Jan 31, 2023

or an advertising competitor that ignores DNT!

oh wait

Nuzzerino · on Jan 31, 2023

Yes, clearly that's the best possible interpretation of what I said. /s

chihuahua · on Jan 31, 2023

This seems perfectly logical to some people:

Google ignores robots.txt: "Google is evil! They're trespassing on my webserver!"

Google follows robots.txt: "Google search results suck! They're not indexing GitHub!"

berkle4455 · on Jan 31, 2023

It's a just a text file.