Unexpected Errors related to Certificates
Incident Report for Measuremen
Postmortem

Summary

On January 07, 2024, the Engineering team investigated a report that customers experienced core authorization errors with messages relating to timeouts and expired certificates. Impacted users may have been unable to log in for the duration of the incident. We sincerely apologize for the impact this has caused you and your users.

Root Cause Analysis

The support team isolated the impact to a subset of customers. The service contained a domain whose certificate had expired. The certificate was manually renewed to resolve the incident.

Mitigation Actions

To avoid similar incidents from happening in the future, the following actions have been taken:

● Creating additional layers of monitoring to catch failed certificate renewals and prevent expirations.

● Migrating legacy domains used to point to more closely managed and monitored edge domains.

Posted Feb 22, 2024 - 20:33 CET

Resolved
Services has been restored. We sincerely apologize for the impact this had on you and your users.
Posted Jan 07, 2024 - 04:15 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jan 07, 2024 - 04:00 CET
Identified
The issue has been identified and a fix is being implemented.
Posted Jan 07, 2024 - 03:30 CET
Investigating
Impacted users may experience intermittent timeouts and error messages relating to expired certificates when attempting to log in. We will provide updates as soon as know more.
Posted Jan 07, 2024 - 00:00 CET
This incident affected: Measuremen Authentication (User Authentication).