Another quick hint: always make things work first then profile and optimize it. This happens when my colleagues were building a stream processing engine that scales up. At first, they start with all those fancy fine-grained locks but it turns out the final result just does not scale at all. They have to retreat to the version with some big locks and then figure out what was happening and how to solve it.
Am I the only one that found that there wasn't really much actual actionable information in here? There's basically no specific information about how they scaled along the axes they established were important
That’s totally a fair point. I could have given a more concrete talk about how I’ve done these things in real systems (and the original version of this talk, presented internally to my company, had more real details and spoke very candidly about mistakes that I’d made along the way). Here I wanted to give more of a “here’s how you might want to structure your thinking, along with some basic design principles” kind of talk - I wish I’d had time to give more detail!
Yeah it was an good talk from the perspective of laying out the landscape and I found myself agreeing with everything, only to find that I was close to the end of the slide deck :(
Is there a blog somewhere where you've written a bit more about it?
I think of high level talks like these emphasizing timelessness. Technologies change over time and with in different contexts. But the theory remains as a way of thinking abstractly about a problem.
Besides, at the level of jobs I apply to now, interviewers care much more about my thought process than specific technologies.
