Theft is a potential problem, but so is shoulder-surfing. At least if your auth token goes missing then you know you've been compromised.