The is a design pattern, used extensively in distributed systems to prevent cascading failures. In this post, we’ll go through the problem of cascading failures and go over how the circuit breaker pattern is used. circuit breaker Motivation: The problem of cascading failures Before jumping into the circuit breaker pattern, let’s try and understand what problem it tries to solve. When service A tries to communicate with service B it allocates a thread to make that call. There are 2 kinds of failures that can occur while making the call. We use the example of a user service making a call to friends service. : friends_service.get_friends(user_id) Exception e: InternalServerError ''' user service ''' : def get_user_info (user_id: str) try except as raise : In immediate failure, an exception is raised immediately (like: Connection Refused) and the service A thread is freed. Immediate Failures : If service_b takes a long time to respond, as we get new requests to service A, we’re getting more and more threads waiting for service_b. If several requests are made while waiting for timeouts this can exhaust service A’s thread-pool and can bring down service A. Timeout Failures ” -Michael T. Nygard, Release It! “Your code can’t just wait forever for a response that might never come, sooner or later, it needs to give up. Hope is not a design method. Let’s walk through an example of a social media application to understand this better. Here we have an service which is what the client interacts with, it aggregates results from a bunch of services including the service. User service calls photo service and friends service which in turn calls . aggregator user friends_db Here, friends service tries to make requests to the , however, friends_db is not responding with an immediate failure, instead keeps the threads from the service waiting. The service tries to retry thereby using more threads. As it gets new requests more threads are waiting on the to respond. friends_db friends friends friends_db We can now see how service is now becoming the source of timeouts for user service. User service exhausts it’s thread-pool waiting for requests from friends service, just how friends service was waiting for . We can now see how failure in friends_db caused a in services indirectly dependent on it. friends friends_db cascading failure Eventually, the aggregator service will also come down with the same reason. The client calls the aggregator service and so our system is effectively shut down for the users. We see how one error in one component of our architecture caused a cascading failure bringing all other services down. Circuit Breaker Pattern The circuit breaker is usually implemented as an /chain of responsibility/filter. It consists of 3 states: interceptor pattern Closed: All requests are allowed to pass to the upstream service and the interceptor passes on the response of the upstream service to the caller. Open: No requests are allowed to pass to upstream and interceptor responses with a default response generally an error response. Half-Open: Some of the requests are allowed to pass to upstream others are terminated and responded with a default response. The following shows the circuit breaker interceptor in its 3 states The circuit breaker is implemented as an interceptor intercepting all requests from user service to friends service. In this picture it is in the “closed” state and allows all requests to be passed to the friends service The circuit-breaker switches to the “open” state when the number of failures to the friend service are more than the failure threshold. It doesn’t allow requests from the user service to reach friends service instead it responds immediately with a default response After a set “recovery timeout” period has passed the circuit breaker switches to a “half-open” state where it allows some of the requests to reach the friends service and the others are terminated and responded with the default response. Let’s look at a Python example for a circuit breaker. You can create your own circuit breaker using: circuitbreaker CircuitBreaker FAILURE_THRESHOLD = RECOVERY_TIMEOUT = EXPECTED_EXCEPTION = RequestException : friends_service.get_friends(user_id) Exception e: InternalServerError from import : class MyCircuitBreaker (CircuitBreaker) 20 60 @MyCircuitBreaker() : def get_user_info (user_id) try except as raise We can also leverage the sidecar pattern to this. In this approach, we don’t have to modify our services by wrapping them around circuit-breakers, but instead, we ship our applications with a sidecar like . All outbound traffic from the service is proxies through Envoy. Envoy supports the circuit breaking out of the box. Following is an example configuration of circuit-breaking with Envoy: Envoy circuit_breakers: thresholds: - priority: DEFAULT max_connections: 1000 max_requests: 1000 - priority: HIGH max_connections: 2000 max_requests: 2000 Resources http://martinfowler.com/bliki/CircuitBreaker.html https://pypi.org/project/circuitbreaker/ Release it (Book) https://books.google.com/books/about/Release_It.html?id=Ug9QDwAAQBAJ&source=kp_book_description Circuit breaking in Envoy: https://www.envoyproxy.io/learn/circuit-breaking Also published here .
