Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The authors did not really expect it to be such a huge influence. You could also argue, it is a somewhat natural next step. This paper did not invent self-attention nor attention. Attention was already very popular, specifically for machine translation, and a few other papers already did use self-attention at that point in time. It was just the first paper which solely used attention and self-attention and nothing else.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: