Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

UDP Multicast relies on constantly throwing out the entire state of your system in order to be reliable, whereas with TCP you only need to throw out changes. This is fine in finance because the state is changing constantly anyway so you might as well just continually throw out the multicast traffic.

Message queuing is a more complicated system, but it allows you to push the bulk of your work out to the distributed network instead of concentrating it on the producer.



That is definitely not true about state management - you can build very TCP-like systems on top of UDP (see QUIC), and the final adaptation you need is to figure out how to make that work with multicast (a non-trivial adaptation, but possible). Once you do, you have the foundations of a pub/sub system in the form of a one-to-many (or many-to-many if you are brave enough) network protocol, and now you basically only have to figure out how to do state recovery of new clients and clients who are behind, and that can basically just be a database. The system described in the OP is kind of a combination of a database and load balancer in order to do this over unicast connections.

You have to do this state management with a TCP-based system anyway - the use of TCP doesn't magically come with the full state of past messages.


This is not right in finance at least. Messages updates are typically incremental just like in TCP. The reliability problem is dealt with with careful network stack tuning, core affinity and out of band recovery.


Check out 0MQ (ZeroMQ), Informatica Ultra Messaging (formerly 29 West), and aeron.io They create reliable uni/multicast protocols over UDP, including strategies for NAK storms etc.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: