Below is a real-time overview of our servers where you can check if there's any known issues.
Affecting Server - atl1
Minor update: Feb 27th: Quick reboot, see everyone in five minutes.
We believe the issue to be resolved but we're continuing to monitor the situation, thank you.
Update: @ 12:20 PM EST: We have implemented some fixes that we suspect will correct the original issue, we'll follow-up by email once this is confirmed.
Update @ 12:04 PM EST: Services accessible again. We'll provide additional updates as needed.
Update @ 11:59 AM EST: We're bringing the machines down for a quick reboot (~5 minutes), thank you for your patience. Expect a bit more dust before we're 100% completed.
Update @ 11:20 AM EST: Both machines are back online.
We'll continue to investigate & email when we have a final report. Further updates will be posted here as needed.
In future instances of downtime we should be able to return the machines to service within 5 minutes.
Initial issue:
We're investigating another instance of downtime on this & US-ATL1:
Thanks for your patience.
Affecting Server - atl1
We're investigating continued instability on ATL1. Thanks for your patience.
Update @ 11:27 AM EST: We're working with the datacenter to get the machine back online.
Affecting Other - Phoenix data center
12:54pm Datacenter engineers are investigating connectivity issues at the Phoenix location. Please hold on for further updates, we apologize for any inconvenience this may have caused.
1:01pm Issue has been resolved.
We are seeing DNS issues with our servers, please hold on for further updates.
We are moving our backup infrastructure to a new provider and therefore backups will be unavailable until the new servers have made backups using our new Acronis cloud backup service. As always please ensure you have your own backups at all times. This does not have any impact on your website or web hosting services.
We are investigating intermittment connectivity issues with the Amsterdam location, please hold on for further updates as our NOC engineers work on this.
UPDATE: Datacenter technicians have replaced a faulty cable and services are back to normal. We are closlely monitoring all servers to make sure everything is running smoothly!
Affecting Other - Backup Server
Backup storage server had to be upraded to add more storage, this means your previous backups will be wiped clean and only fresh backups will be available starting today. Please always maintain personal copies of your site or wordpress install at all times.
Affecting Server - usphx1
This server is being rebooted and should be back online in a few minutes.
Affecting Other - Amsterdam
We're currently investigating issues at the Amsterdam data center, our teams are working on isolating the issue and will have an update shortly!
Affecting System - Phoenix Node down
Server has been rebooted and is back online.
Our network engineering team is investigating an outage with the atlanta data center.
Affecting Server - atl1
We're rebooting the atlanta node due to an emergency update.
Affecting Other - Network
We're investigating an outage with an ISP routing to the Atlanta Data center, please stay tuned for more information. Our engineers are working on this!
Affecting Server - us-phx1
We are upgrading server's MySQL/MariaDB 5.5 to MariaDB 10.1.
Affecting Server - ams1
We're rebooting the amsterdam node right now.
Affecting Server - us-phx1
Server is being rebooted after required kernel update.
Affecting Server - phxsd1
Rebooting phxsd1 to apply kernel updates for the backup module.
Affecting Server - us-phx1
Server has been rebooted to apply network configuration changes.
Affecting Server - usc1.wirenine.com
We're moving usc1 to atl1 server, there will be no impact on service and no changes are required on your end.
Affecting Server - atl1
We are migrating server2 accounts into our ddos protected facility.
Update: Transfer is now complete, please feel free to contact us if you need assistance.
Affecting Server - us-phx1
Server is being rebooted to apply network configuration changes. Should be back online in a few minutes.
Affecting Server - us-phx1
DC Network engineers are working on a issue with the routers here in our Phoenix location. Servers and websites are still online however some websites are seeing intermittent connectivity problems from certain locations, this is expected to be resolved within a 2-3 hour timeframe. We apologize for any inconvience in advance.
Affecting Server - phxsd1
We're going to reboot this server for network configuration changes.
Affecting Server - us-atl1
usc1 reseller server is being rebooted.
Affecting Server - atl1
Server2 is under SYN flood attack on port 80 causing time out issues for users intermittently hogging open available connections, we are working on mitigating this attack as quickly as possible. We apologize for the issues caused due to this.
UPDATE: This has been resolved.
Affecting Other - Network
Valued Customer,
Our DC Network engineers we will be adding additional capacity to our backend network
infrastructure in our Phoenix data center.
To help make this process run as smooth as possible, we are planning a two-day
maintenance period starting on Monday December 8th and ending on Tuesday
December 9th on the Phoenix Backend Network to add new equipment to our
infrastructure. Our Network Engineering Team will be taking all the necessary
precautions to ensure the impact is minimal. We do not anticipate any downtime
however there is a very small chance that a brief outage of 10-15 minutes is
witnessed between 9:00 a.m. and 2:00 p.m. Mountain Standard Time (MST) on both
days. Please note that if there is any impact this will only affect the backend
services and not Internet traffic.
Thank you for your understanding and patience as we continue to work toward
providing you with the best possible service. If you have any questions, please
contact us.
Affecting Server - atl1
Server2 has been getting large HTTP Flood attacks causing intermittent timeout issues for users from thousands of infected IP addresses, we are working on resolving this as quickly as possible. We have heightened the security settings on our firewall, so we apologize in advance if your access gets blocked. Open a ticket if you're unable to access your site and it's hosted on server2.
UPDATE: The attack has been successfully mitigated.
Affecting Server - atl1
We need to reboot server2 to fix a stuck backup module, it should be back online within a few minutes.
Affecting Server - atl1
Server2 is being rebooted for an update.
Affecting Server - atl1
We'll be rebooting server2 at midnight tonight to update the R1Soft backup agent.
Affecting Server - atl1
Server2 is rebooting, should be back online in a brief moment.
Affecting Server - atl1
We're investigating the potential mysql database connection issues after MariaDB upgrade. Please stay tuned.
UPDATE: We have restarted MySQL service and all connection issues should now be resolved.
Affecting Server - atl1
We are currently in the process of replacing MySQL with MariaDB MySQL, this may cause intermittent database connection issues on your site for a few minutes. We apologize in advance for any inconvenience this may cause you. To learn about MariaDB and how it improves your database performance please visit https://mariadb.org/
UPDATE: Server2 is now using MariaDB.
