Current Status: Impact Mitigated Resolution Update: * The platform-wide impact has been successfully mitigated * Inference services have resumed normal operations and are successfully handling traffic * Network connectivity has been restored across most services
Known Impact: * Some training jobs may have experienced interruptions during the incident * If your training jobs failed during this period, please restart them * Our monitoring shows that new training jobs should now complete successfully
While we consider this incident resolved, our teams continue to monitor the platform closely. If you experience any persistent issues, please contact our support team.
Thank you for your patience throughout this incident.
Posted Jan 27, 2025 - 21:06 UTC
Update
Our team has implemented a permanent solution and is monitoring the system's behavior.
Posted Jan 27, 2025 - 20:46 UTC
Monitoring
Current Status: Partial Recovery in Progress Progress Update: Essential platform services have been successfully recovered Our team is currently implementing a strategic approach to restore the remaining data plane functionality Monitoring metrics indicate that customer training jobs remain unaffected
Posted Jan 27, 2025 - 20:03 UTC
Identified
Current Status: Investigating and Mitigating We are currently experiencing connectivity issues affecting multiple services: * API Services * Management Console * Virtual Machine Public IP Addresses * Load Balancers
Impact: Users may experience connection problems or service interruptions when accessing these services. Update: Our team has identified the root cause and is actively working on resolving the issue with highest priority. We will provide updates as more information becomes available. We apologize for any inconvenience this may cause.
Posted Jan 27, 2025 - 17:54 UTC
Update
At the moment, we are experiencing issues with Virtual Private Network connectivity.
Posted Jan 27, 2025 - 17:26 UTC
Investigating
We are currently experiencing a partial unavailability of our Compute services, which may affect some of your workloads. Our team is actively investigating the issue and working towards a resolution. We will provide further updates as soon as more information becomes available.
Posted Jan 27, 2025 - 16:58 UTC
This incident affected: EU-NORTH1 (Compute Cloud, Virtual Private Cloud (Networking), Managed Service for Kubernetes®, API, Management Console, Monitoring, Token Factory).