Previous incidents

We are observing increased delay and error rate in processing SCM webhook events. Our team is investigating the issue.

Update 1: Issue is now isolated to CI Triage events. All other event types have been processed and error rates have been addressed.

Update 2: All issues have been addressed and the system has finished processing events backlog.

Ask Unblocked feature is experiencing elevated error rates. Our team has been made aware and are investigating the issue.

Update 1: This issue is impacting CI Triage and Code Review features as well

Update 2: Issue has been identified as result of a third-party service outage. We are working on rerouting traffic to a secondary endpoints.

Update 3: A fix has been implemented and the system has recovered. We will continue monitoring closely.

Issue has been resolved.

We have noticed elevated error rates and longer response times. Our team is is currently investigating the issue.

Issue has been resolved.

Our team is investigating reports of elevated error rates at the moment.
Issue has been resolved as of 9:19 PDT

Our team is investigating increased latencies in API calls.

Update 1 12:26PM PDT: We have applied temporary fix to help reduce latencies while our team is investigating the root cause.

Update 2 12:55PM: Issue has been resolved.

We are observing increased error rates in Ask Unblocked and other inference related features due to ongoing Anthropic service outage.

Issue has been resolved

Issue has been resolved

Our team is investigating an issue causing elevated error rates in API calls

Update 9:13PM: Our team has identified the issue. We are working on deploying a fix.

Update 9:20PM: Fix has been deployed and all services have recovered.