Being a developer is like driving: you focus on the final destination and, to overcome obstacles, you take shortcuts or draw your own route if necessary. Debugging and solving incidents on production is a key part of our job. It takes a bit of a different... »
In this second in a three part series on Sinkholing, we will dive into the details of implementation. In part one of our series on Sinkholing, we got into general ideas about how Sinkholing works, why it is needed, and how it fits into our infrastructure... »
This is the first in a three part series on sinkholing, targeted attacks, and how to defend your platform. As our user base (and our popularity) grows, so too does the threat of sudden attacks against our platform. We have encountered many attempts... »
This is a tale of several small issues turning into an absolute clusterf^@k. It’s never just the one bug that causes a catastrophe, it’s always a synergy of multiple small issues - small issues that are so stupid, and so easy to overlook, that it’s... »
Weekends are when we all want to sleep in. After a long week of work, we need time to relax, take our minds off things, and recharge. So when I got the call on early Saturday morning telling me we’re sending HTTP 500 Server Error messages to our end... »