We've deployed more verbose logging in the platform and collecting diagnostic info. Kentik engineers will continue monitoring the situation closely until we find the root cause.
Posted Dec 03, 2020 - 15:02 UTC
Monitoring
We have introduced a more verbose logging / diagnostic solution to the alerting subsystem which required a restart of those services. In doing so, the delays appear to be temporarily resolved. We will continue to investigate the situation with the newly introduced diagnostics.
Posted Dec 03, 2020 - 12:35 UTC
Update
We are continuing to investigate this issue.
Posted Dec 03, 2020 - 12:01 UTC
Investigating
Alerting notifications are presently severely delayed (12+ hours)
Posted Dec 03, 2020 - 11:28 UTC
This incident affected: Alerting and Mitigation Services.