I use CoAP in my small DIY projects (ESP8266 mostly for now)… but sometimes I wish I used MQTT, because with the broker model you can put your device in deep sleep and only wake up occasionally, saving power. On the other hand the REST server model lets me adjust polling frequency on the client side, without storing any config in the device…
It is a good example of marketing dollars influencing protocol adoption. I've met a remarkable number of people that work in IoT that know nothing of CoAP.
With Mosquitto, I can run a very tiny broker. And things as small as "Arduino with ethernet chip" can serve as a MQTT client. I'd consider something that can run in 16MHz with 2k ram and 32k storage to not be something "consultant-driven over-engineering and under delivery par excellence."
Indeed. If you're going with an Arduino/ethernet you are not going to get SSL layer between client/broker. Then again, you do get the security of being on a wired network, and little to no RF emissions.
If you "upgrade" to an ESP class microprocessor, you do get MQTT, CoAP, and SSL support. And frankly the more protocols, the better. Some do tend to be better than others in different situations. I know I greatly prefer MQTT, for its clean and basic protocol. It gets out of the way for me to do what I want, basic Pub/Sub and storing relevant data into a MongoDB for timeseries.
I know at work, we're using RabbitMQ (AMQP). It has its own positives and negatives. My biggest concern is it serves as a forward-and-store datastore - another place where backups need to be made, let we lose essential data. I'm against complex moving parts, where simpler moving parts would suffice.