Possible Service Degradation

Incident Report for ThousandEyes

Resolved

This incident has been resolved.
Posted Sep 27, 2023 - 02:00 UTC

Monitoring

The issue has been mitigated. We are monitoring the results.
Posted Sep 26, 2023 - 20:41 UTC

Update

We have identified a possible degradation for the ThousandEyes Web Application and API service.
Enterprise Agents may not save changes.
Alerts and Alert Notifications may be delayed.
Posted Sep 26, 2023 - 19:03 UTC

Investigating

Service degradation is still present.
Enterprise Agent may not correctly take/save the changes applied to them, also for existing Enterprise Agent we might lose visibility to agent metadata changes sent from agents like hostname, ip address, software versions (agent, browserbot, TEVA), OS, Kernel etc.
Controller/ThousandEyes Platform might reject checkins from agents checking in for the first time.

we are working actively in this issue
Posted Sep 26, 2023 - 18:24 UTC

Monitoring

The issue has been identified and mitigated, we are monitoring the results
Timestamp 16:40 UTC - 17:40 UTC
Posted Sep 26, 2023 - 17:50 UTC

Investigating

We are currently investigating a possible service degradation that could prevent Enterprise Agent to correctly take/save the changes applied to them, also for existing Enterprise Agent we might lose visibility to agent metadata changes sent from agents like hostname, ip address, software versions (agent, browserbot, TEVA), OS, Kernel etc.
Controller/ThousandEyes Platform might reject checkins from agents checking in for the first time.

We are working actively in this issue.
Posted Sep 26, 2023 - 17:44 UTC
This incident affected: Agent Configuration and Data Collection (Cloud and Enterprise Agents: Registration controller, Cloud and Enterprise Agents: Test assignment and configuration controller).