If AI has as big of an impact as it seems like it will, we will ultimately have to design consumer-grade chips around neural networks to run it at the edge. (kind of like what https://www.etched.ai is doing)

Either that or some massive efficiency breakthrough, probably (hopefully!) both. Otherwise AI will be limited to enormous computing banks at datacenters, and it will be severely limited in terms of privacy (because no one wants to upload their personal data to datacenters, especially not to power some black-box "AI")