All systems are operational

Stickied Incidents

9th March 2025

Web: Dallas OPAL4 MariaDB Outage

MariaDB on OPAL4 has crashed, we're working to identify the cause and solution now.

  • We've seen no further problems since our last update.

  • The issue with database permissions is resolved and the MariaDB service is back online and operating normally. We will continue to monitor.

  • We've discovered that there is still a problem with database permissions. We're working to resolve that at this time.

  • The data restoration has completed and we are continuing to monitor the server as usage increases.

  • The restoration is approximately 75% complete.

  • The data restoration is approximately 50% complete.

  • The MariaDB restore process is still running.

  • The database corruption is unrecoverable so we will proceed with restoring from backups. The most recent valid backups were from 1.5 hours before the server crashed so we'll proceed with the restore from there.

    The restoration process varies in completion time. It may take up to 6 hours or more. We'll keep this post updated.

  • Past Incidents

    28th January 2025

    No incidents reported

    27th January 2025

    No incidents reported

    26th January 2025

    No incidents reported

    25th January 2025

    No incidents reported

    24th January 2025

    No incidents reported

    23rd January 2025

    No incidents reported

    22nd January 2025

    No incidents reported

    21st January 2025

    Web: Washington D.C. opal12 not responding

    opal12.opalstack.com (washington DC shared hosting) is currenty not responding. We're investigating at this time.

  • The outage was caused by a very heavy spike in system load, itself caused by a brief (but intense) attack from an abusive IP address. The problem has been mitigated and the server has been stable since our previous update.

  • opal12 is back online and stable at this time. The cause of the outage is still under investigation.