This appears to be targeted at Lambda function, which can't reuse existing connections between executions.

Also, establishing an HTTP connection is much faster than establishing a typical database connection, in my experience. I don't know why that is.

Unlike a properly configured RDS cluster, this is available outside of your VPC on the open internet.

That’s the main selling point to me, though the connection pooling of MYSQL connections by the HTTPS proxy is also nice too.

Unless I’m misunderstanding the question, many http clients would pool http(s) connections. Most of them do that by default. So connection establishment cost gets amortized over large number of API calls.