A pause is still a pause though. One system I know of had a solution. Throw in a huge amount of RAM on the server and delay garbage collection until end of day.

But a pause that can happen when you want it is acceptable. A 3 second stall might be totally acceptable as long as it happens outside of a trade, and you can guarantee that.

If you can predict 3 seconds in advance if you’re going to need to trade or not, you’d make millions very quickly. In trading terms, that’s basically having a time machine.

You don't need to predict in advance, you just need the capability in the system. If a system takes X time from the beginning of a trade to being ready to process the next trade, with Y of that time being GC, it doesn't matter how long Y is if you can execute the entire trade before all of the GC happens.

I used to support a derivatives trading system at a company called Patsystems, so I’ve seen this. Our legacy system could process trade triggers from 4ms to 7ms, but tracing the behaviour of the new system it could do it in 3ms, great. Except every now and then a GC pause would halt order processing for around 50ms. Our customers hit the roof.

Trading systems trigger almost all their orders based on detecting market changes, which you cannot predict. If a symbol hits a price that triggers one of your trades you want that trade in the market ASAP every millisecond matters. Randomly add 50ms on to that and your trading system is out of business. The customers will go elsewhere.

(Sorry, I missed this over the weekend) - I think the issue in this case is you're selling this as a guaranteed 3ms, going down from 4-7ms. If you have to pause for this 50ms GC once every X trades, then you force the GC to happen _after_ the X-1th trade has completed, but you need to provision an extra 15(?) machines to cover the pause. Otherwise, you're not delivering 3ms trades, you're delivering (3 + GCTime/X) ms. It doesn't matter if GC takes 1ms or 1s, you "simply" need to have the capacity to cover the extra GC time.

Why waste money hire best engineers when more ram does trick?

As I understand it (before my time), this was a practical solution found on original symbolics lisp machines; trigger the GC and go get lunch...

Interesting but doesn't allocation get slower as the heap increases?

Normally yes but that’s mostly a factor of heap fragmentation. It the GC never runs the “old” buffers are always full so you never bother going through them.

There are open source 'pauseless' (really: ultra low pause) GCs for the JVM now. Azul made good money selling such a thing to the finance industry for a long time.

Typical pauses may be far less than a millisecond in these GCs because they do almost everything in parallel.