Assuming you don't have an address tied to your account, the risk can be mitigated somewhat. If you KeyMe account is hacked or phone is lost, it's similar to losing the physical key itself-- the thief doesn't really know what the key goes to.
But, if the KeyMe user database is hacked along with personal information (billing address), then it becomes an issue.
Also, if you lose your phone and have emergency contact information (or can lookup the address associated with the phone number), then the thief also knows where to go.
Hopefully everyone has a locked phone, but if they have a stolen phone and have gotten past the lock it is not hard to figure out someones home address on their phone by looking around apps and looking at the contacts.
But, if the KeyMe user database is hacked along with personal information (billing address), then it becomes an issue.
Also, if you lose your phone and have emergency contact information (or can lookup the address associated with the phone number), then the thief also knows where to go.