Build Stuff 2018

Consensus protocols enable distributed systems to agree a common view of shared state. This common view allows a cluster to continue service while a majority of its members are available. Raft was designed to be understandable and based on simple proven protocol, however like most consensus protocols there is little guidance on how to implement it efficiently.
Aeron was designed to be an understandable messaging system, it was also designed to be fast, very fast, even in its Java implementation. If the design principles of Aeron were applied to Raft, could we create a high-performance consensus implementation? Come to this talk if you would like to find out how to build a high performance distributed event system in Java and see what happened when Aeron met Raft.

Since its introduction in 1989, Paxos has been a most prevalent protocol for solving consensus in distributed systems. Not long ago, researches at Stanford published Raft protocol - a more understandble and approachable alternative to Paxos.

In this talk I will talk about design of distributed systems and introduce Raft, reveal its inner-workings and show how it is useful in solving reliability issues inherent to all distributed systems.

We usually think of monitoring as something the Operations team will take care of. In a "monolithic" system that might be enough.
But we like to distribute our systems. We deploy microservices with persistent queues and automated retries. We allow them to tolerate failure, at least for short periods of time. Sometimes it may seem everything is running OK. No heavy load, no memory or CPU surges, no immediately observable slow-downs. But in fact the system may not be functioning and effective performance is degrading. Messages are piling up in queues. Business SLA's are starting to get breached. With distributed systems, we need to do more than basic monitoring.
Let's talk about how we can do a better job at keeping the lights on and ensuring our systems are functioning as designed.
I'll introduce various aspects of monitoring and the things we need to think about when writing code. I'll show how you can optimize your distributed systems and find performance bottlenecks. You'll discover how you can win if you give monitoring a bit more love :-)

In this talk we will look at how to leverage the messaging infrastructure itself to find and identify problems in message driven systems. We will look at what categories of problems can occur and what their symptoms are. Then move on to diagnosing the source of the problem and the taking corrective actions. We will also look at how to use these same tools to implement dynamic resource management. Finally we will wrap up with some thought on how to use tsi information in designing robust high performance systems.