All systems are operational

Past Incidents

30th April 2020

Web: Dallas Opal4 not responding

Opal4 is not responding. We're looking into the cause.

  • We believe this latest issue was caused by a DNS reflection attack.

    We're consulting with our upstream provider to see what they can do to protect the server and network from future attacks.

  • Opal4 is back online, investigation is ongoing.

    • Web: Dallas Apache was not responding on Opal4

    Apache was once again not responding on Opal4 after hitting its configured ServerLimit. The underlying cause appears to have been a high number of PHP-FPM processes being re-spawned for one customer site. We've adjusted the PHP-FPM configuration for that site to avoid the problem in the future.

    29th April 2020

    No incidents reported

    28th April 2020

    No incidents reported

    27th April 2020

    No incidents reported

    26th April 2020

    No incidents reported

    25th April 2020

    No incidents reported

    24th April 2020

    Web: Dallas Apache was not responding on Opal4

    The Apache service on Opal4 was not responding for approximately 1 hour today beginning 18:23 UTC.

    The problem was that the Apache service had hit its configured ServerLimit which meant that no further Apache processes could spawn to handle requests. That has been corrected and the service is online now.

    We're now investigating the root cause and also looking into why our own monitoring systems did not alert us to this condition.

  • The root cause appears to have been a misconfigured customer application. We've corrected the configuration and have updated our monitoring system to alert us when Apache approaches its configured limits.