John Nunemaker Addicted to Stable

Tuesday Randoms: 2016-04-19

The Error Model. I’ve been doing a lot of resiliency related things lately and that means dealing with errors. This post is one of the longest (and also best) summary of error handling on the planet.

Write code that is easy to delete, not easy to extend.. Kind of lacking in examples, but a lot of good thoughts.

Any system that can wake you up at night is one worth being able to control at runtime.

The Wrong Abstraction. I like most things that Sandi has to say, but this one in particular.

prefer duplication over the wrong abstraction

Giving better code reviews. Love this quote:

Your team is only as good as your weakest reviewer.

Building Mobile-First Infrastructure for Messenger. How facebook improved messenger with a new system.

The new sync protocol reduces Messenger’s non-media data usage by about 40%. Additionally, reducing congestion on the network leads to roughly a 20% decrease in the number of people who experience errors when trying to send a message.

Turning the database inside-out with Apache Samza. Four things that databases do that makes them super valuable. This is an absolutely fantastic look at what makes databases great and how we can replicate that outside of traditional databases using streams (aka kafka in this post).

Another very nice feature of an append-only log is that it allows much easier recovery from errors. If you deploy some bad code that writes incorrect data to the database, or if a human enters some incorrect data, you can look at the log to see the exact history of what happened, and undo it. That kind of recovery is much harder if you’ve overwritten old data with new data, or even deleted data incorrectly. Also, any kind of audit is much easier if you only ever append to a log — that’s why accountants don’t use erasers.

Spotify’s Event Delivery – The Road to the Cloud (Part II). Why spotify chose Google’s Cloud Pub/Sub over Kafka for their event system.

Based on these tests, we felt confident that Cloud Pub/Sub was the right choice for us. Latency was low and consistent, and the only capacity limitations we encountered was the one explicitly set by the available quota.

Golden Rules for Making Money by P. T. Barnum in 1880. I am always fascinated when I find really old writing applicable in the modern age. I did not read all of this, but a skim had some interesting bits.

Young men starting in life should avoid running into debt. There is scarcely anything that drags a person down like debt.

How to Ensure a New Manager Succeeds.

The single biggest mistake that managers make when they promote someone is that they reduce the amount of support and attention they provide to that employee. This is a fatal mistake that sets your employee up to crumble like the train above.


You must model the attitude and work ethic that you would like to see in others. - John Maxwell

From the 21 Irrefutable Laws of Leadership. Cheesy book at times, but some really good thoughts as well.

Simplicity does not precede complexity, but follows it.

Alan Perlis. I’ve said this for a long time, but I’m not famous and evidently not the first. :) It reminds me of Mark Twain’s quote “If I had more time, I would have written a shorter letter.”

The unavoidable price of reliability is simplicity.

C. Hoare. Hat tip goes to Joe Duffy in the error model article I linked above.

Whatever you think you can do or believe you can do, begin it. Action has magic, grace and power in it.

Goethe. I have noticed this. When I start something, I immediately feel better about it, no matter how much I was avoiding it.

Show me what you can do; don’t tell me what you can do.

John Wooden. Actions over words.

The best day to plant a tree is 20 years ago. The second best day is today.

Chinese Proverb. Another quote on action.

Excellence is achieved by the mastery of the fundamentals.

Vince Lombardi. Pretty much every legendary coach emphasizes fundamentals (aka the little things).



Think about what your app would look like without UPDATE and DELETE.

There is no such thing as reliable time.

We want to understand our current architecture and troubleshoot latency problems, in production.

Flow: when a skill is ingrained in your nervous system and it is automatic.

calmly typed on a keyboard by @jnunemaker