Sounds like you are agreeing with me. You tried to work on it years ago and didn't fix it. Even when it was completing last year it's performance was half of what Rails was, which isn't known to be fast [1].
The database pool needs to be at 512 to 1024? Yet Rails seems to get away with far less than that and still have better performance.[2]
Even if, somehow, Elixir is actually faster, which seemingly lacks evidence at this point other than some anecdotes by people who have a vested interest in people adopting the language (primarily Elixir dev shops and framework/language authors), the fact that the community of Elixir devs cannot get decent results on these tests says a lot.
If these people can't get good results in tuning Elixir, who can?
Not a surprise to see people ITT also noting that they ran into the same types of problems that the tests reveal.
> The database pool needs to be at 512 to 1024? Yet Rails seems to get away with far less than that and still have better performance.[2]
Rails concurrency model is per process. So you will have `n_workers` pools and the amount of connections will be `n_workers * 3` based on the code you shared. The production machine has 28 total threads [1], which means Rails is starting 35 workers, according to their auto tuning code [2]. Overall, Rails uses 35x more pools than the Phoenix application and has twice the number of connections (105). It is absurd to imply that Rails is actually running with far less.
Second of all, the database pool needs to be 512 or 1024 if your app server can push traffic quickly enough to the database. If you are increasing the Rails pool size and it is not getting faster, then Rails is your bottleneck. Based on the runs I did last week, it seems the pool is the bottleneck in Elixir's case. We will know for sure in a couple weeks once the PRs land.
> If these people can't get good results in tuning Elixir, who can?
If you want to compare with Rails in particular, we beat Rails in every benchmark except two:
1. Single query: which will be tackled by better sizing our pools
2. Plain text: where we are currently sending the wrong payload (but when we sent the proper payload, as in the last published round, we beat Rails too)
And it does not include any of my proposed changes yet.
My point is exactly that the feedback cycle is long, so it takes time, and I was not able to invest time before. But given how intent some folks are to misinterpret benchmarks (and assign malice to the whole community), it is probably better to continue looking into it.
Nice to see elixir put up decent benchmarks for once. It looks like it's about 80 percent faster. Still a far cry from "10 to 20x reduction in size and cost".
I wrote, while learning, a monitoring system in Elixir. I put everything in one "Phoenix" app. What does it do?
* Query about 60 oracle databases in parallel every five seconds.
* Query a number of KDB databases every five seconds.
* SSH to dozens of machines every ten seconds
* Ingest statistics from icinga.
* Pull ticket updates from Jira every couple of seconds.
* Provide facilities for connecting in to screen on about 100+ remote machines, and interacting with their interfaces.
* API access to internal and external commands/functions.
* more.
It does much more than this, while serving live-view sessions to users, on a dual core machine (CPU: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz x 2), while averaging about 8% CPU for everything it does.
The SSH component replaces a python version that does the same thing. The Python instance, by itself, pushes a 6 core box (same xeon as above but with 6 cores, rather than 2) to 100% CPU, and spaws close to 3,000 OS tasks.
Many of the other checks that are running in that 8% CPU Elixir process replace python, node, and java checks, each of which required more ram, demanded more CPU, and were spread across eight to ten dedicated machines per check.
We have been retiring machines, and reducing CPU load, so I'd say we do see the 0 to 20 reduction.
Now, my solution might have been written better, but that was what I came up with while learning Elixir.
YMMV. In benchmarks like cached queries, it already outputs a solid 5.5x faster than Rails, and it is worth remembering that the Rails app is way too stripped compared to what folks are actually running in prod (which was one of my original criticisms).
The database pool needs to be at 512 to 1024? Yet Rails seems to get away with far less than that and still have better performance.[2]
Even if, somehow, Elixir is actually faster, which seemingly lacks evidence at this point other than some anecdotes by people who have a vested interest in people adopting the language (primarily Elixir dev shops and framework/language authors), the fact that the community of Elixir devs cannot get decent results on these tests says a lot.
If these people can't get good results in tuning Elixir, who can?
Not a surprise to see people ITT also noting that they ran into the same types of problems that the tests reveal.
[1]: https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
[2]: https://www.techempower.com/benchmarks/#section=data-r21&hw=...