Yesterday, September 28, between 12:51 and 13:42 CET, our API started responding with errors over HTTPS due to expired SSL certificate. As a result, most website integrations didn't work correctly, including registration forms.
The root cause of the problem was an incorrect configuration of SSL certificate updating service, which didn't restart a web server when a new certificate arrived. On top of that, there was a mistake in our monitoring rules. The monitoring service didn't count this error as critical, and we couldn't react fast enough.
What was done:
- We changed the SSL certificate updating service to restart a web server when the certificate updates.
- We changed our monitoring rules to catch these types of errors in the future.
What we plan to do:
- We will monitor expiring SSL certificates to make sure we react in advance if the automation scripts do not work as expected.
We apologize for the issue and promise to learn on this mistake.
