I think they can, but they won't because robots includes search engines and blacklisting search engines from user profiles will very negatively impact their metrics.
First off, robots.txt is optional. It's neither a technical nor a legal limitation at this point.
Second, OP's argument suggests a UI option that gives or removes user consent from all robots in general. Unless they plan to word it: "allow robots that we like that are good for us. but disallow other robots", I don't think it's okay to discriminate by either allowing/banning a particular robot, as that is not what the user agreed to.
