Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm curious, what role did Common Lisp play in the original version of your trading system?


Common Lisp was doing all the "high level" stuff. No CL code was on the actual critical path but it still was responsible for some critical things that would be difficult otherwise.

For example, I had piece of Common Lisp code optimize and compile decision trees to machine code. These were used, for example, to check whether the trade is allowed to go to market. This had to be very optimized as it was running in parallel to constructing the message and had to deliver verdict before the message was actually created to have a chance at actually stopping anything from reaching exchange. This was my solution to have order verification in essentially zero time (the cost was single branch instruction that would only be mispredicted if the verdict was to stop the order).

Another piece of Common Lisp generated machine code to parse XDP messages coming from exchange. The spec for these was in XML, Common Lisp macros created extremely optimized machine code that used absolute minimum instructions to parse out the information we were interested in without having to write all that code manually. I actually used concept from Practical Common Lisp book for this (http://www.gigamonkeys.com/book/practical-parsing-binary-fil...). Thanks, Peter Seibel.

The entire application was essentially a Common Lisp application that used FFI to call a library with optimized ANSI C code. Some Common Lisp code was running in parallel to constantly update things like decision trees that required regular recompiling based on market situation but most was really just to setup everything and give control to the C part.

The application did not talk to operating system (Linux) after it started (no syscalls at all) and used full kernel bypass to talk to NIC.


> did not talk to operating system (Linux) after it started (no syscalls at all) and used full kernel bypass to talk to NIC

That's really interesting... How does it work? How can a user space program talk to the NIC directly without using system calls? Shared memory?



A lot of NIC manufactures provide either some hooks into something like DPDK or simple LD_PRELOADable libraries that offload the network syscalls (socket, poll, recv, etc.) into a library that talks to the NIC directly using mmap'd IO I presume.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: