How about this one from just last year: I'm writing an application that encrypts packets using datagram TLS with OpenSSL. Take a raw packet, send it through OpenSSL, it comes out the same on the other end and everything looks fine.
Then I try load testing it and start getting mangled packets. Application data where there should be IP headers and vice versa. So get out the debugger and find out that the data going into SSL_write() on one end isn't the same as the data coming out of SSL_read() on the other end. No TLS error, just mangled data.
So I install the OpenSSL symbols and discover that the data going into the cipher is exactly the same as the data that gets decrypted on the other side, hence no HMAC verification failure. But OpenSSL by default compresses data before encrypting it, and decompression output is not the data that was originally compressed.
TLS requires records to be delivered reliably. Datagram TLS, by contrast, is like UDP. Packets can be lost. And if they are, you can't use stateful compression or the lost data creates a hole in the decompressor's stream and corrupts the output. But OpenSSL was doing exactly that. So disable compression and the problem disappears instantly. (After three days in a debugger.)
FYI, using TLS compression makes you susceptible to the CRIME attack. I think I have a ticket with OpenSSL for them to turn that off by default, but I don't think they've done it, yet. Glad you got there accidentally!
How about this one from just last year: I'm writing an application that encrypts packets using datagram TLS with OpenSSL. Take a raw packet, send it through OpenSSL, it comes out the same on the other end and everything looks fine.
Then I try load testing it and start getting mangled packets. Application data where there should be IP headers and vice versa. So get out the debugger and find out that the data going into SSL_write() on one end isn't the same as the data coming out of SSL_read() on the other end. No TLS error, just mangled data.
So I install the OpenSSL symbols and discover that the data going into the cipher is exactly the same as the data that gets decrypted on the other side, hence no HMAC verification failure. But OpenSSL by default compresses data before encrypting it, and decompression output is not the data that was originally compressed.
TLS requires records to be delivered reliably. Datagram TLS, by contrast, is like UDP. Packets can be lost. And if they are, you can't use stateful compression or the lost data creates a hole in the decompressor's stream and corrupts the output. But OpenSSL was doing exactly that. So disable compression and the problem disappears instantly. (After three days in a debugger.)