All systems are operational

Past Incidents

Tuesday 19th September 2023

Managed DNS Small period of reduced availability on Managed DNS

The Managed DNS cluster failed to respond to a small number of DNS queries for a short period of time. We are currently investigating the exact scale and root cause of this incident.

  • Last night we were able to mitigate an attack with the fine tuning that happaned earlier.

  • Another brief outage happened last night. Further investigation showed that there was a TCP SYN flood happening from various networks and some UDP floods. We are working with the networks from which we received the traffic and we've put additional protections in place to prevent these incidents.

  • Metrics shows signs of a brief DDoS on various nodes within the DNS cluster. We are further investigating.

    • Monday 18th September 2023

    Managed GitLab Runner Less capacity available on Managed GitLab Runners

    Jobs may be picked up slower within GitLab. This could result in:

    • jobs being queued for a longer period;
    • or longer waiting on "Waiting for Runner to become available and request this job.".

    This is due to an API issue at our cloud provider. Our system is unable to scale up the GitLab runners as a result. We are waiting for a fix from our cloud provider.

  • The provider fixed their API and scaling was back operational.