There should also be a distinction for dynamic and static websites. I've built many responsive static websites and it was fairly easy and not very labor intensive.

Then I built two fairly dynamic sites which were responsive and the game completely changed. There are so many elements you take for granted on a static site which need to be accounted for. Yes, it took more time, but it also opened the site up to less hassles from mobile users. It also made us really focus on the UX aspect to ensure the user was getting a great experience.

I'm a huge proponent of responsive design. I think it's made me a better developer and designer.