Hardware Migration (Resolved)
  • Priority - High
  • Affecting Server - Shakespeare
  • Update: This has now been completed

    As per our email this past Friday we are in the process of migrating the Shakespeare server over to new hardware.

    The initial migration has been completed and websites are now loading from the new hardware.

    There is one outstanding issue which is causing certificate errors when logging into cPanel and this will be resolved in the early hours of the 5th June when the final IP switch takes place.

    Update: The final IP switch has taken place but we are seeing an issue with some mail clients reporting "unsupported SSL protocols" (or similar) error messages. We are aware of this and are working on it. In the meantime you can still access emails via Webmail:

    https://shakespeare.hyliahub.com/webmail

  • Date - 06/04/2018 13:46
  • Last Updated - 06/06/2018 08:46
Server Down (Resolved)
  • Priority - High
  • Affecting Server - Shakespeare
  • The Shakespeare server is currently down and not responding to Ping. The datacenter are currently setting up a KVM session so we can investigate. We will post more details as soon as we have them.

    Update: The server is now back online.

  • Date - 05/30/2018 08:39 - 06/01/2018 10:56
  • Last Updated - 05/30/2018 09:12
bullseye.d9hosting.com (108.61.50.138) down (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • The server is online now, we have checked the websites and the websites are loading as well.

  • Date - 03/11/2018 10:13 - 03/11/2018 16:33
  • Last Updated - 03/11/2018 16:33
Server Down (Resolved)
  • Priority - Critical
  • Affecting Server - Shakespeare
  • Update: The server has come back up after a reboot.

    The Shakespeare server has gone unresponsive. We are waiting on the DC to hook up a KVM unit for us so we can investigate further.

  • Date - 02/09/2018 16:47
  • Last Updated - 02/09/2018 17:17
UK Network Upgrade (Resolved)
  • Priority - Medium
  • Affecting System - UK Datacenter
  • Our UK network provider will be carrying out scheduled network upgrades from 23:00 to 06:00 starting on the 7th Feb.
     
    There is expected to be around 20 minutes of network downtime during this period affecting all UK servers.
     
    This maintenance is the final piece of work required to fully migrate the network into a brand new Cisco platform which has been a large ongoing operation between the datacenter and Cisco.

  • Date - 02/07/2018 23:00 - 02/09/2018 10:49
  • Last Updated - 02/07/2018 09:29
Bullseye will be taken down for maintenance (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • The server will be taken down for maintenance, we will inform once the maintenance work gets completed.

    Update: THis has now been completed and the server is back online.

  • Date - 12/31/2017 12:01 - 12/31/2017 00:00
  • Last Updated - 12/31/2017 12:42
Packet Loss (Resolved)
  • Priority - Critical
  • Affecting System - PNJ Switch
  • We are seeing an issue with a switch at the NJ DC. Everything is up but there is sporadic packet loss. We have reported the issue to the datacenter but you may notice slower than usual performance whilst the issue is resolved.

    Affected servers are:

    bronte
    wilde
    hermes
    kipling
    shaw
    goron

  • Date - 11/06/2017 09:47 - 11/23/2017 16:03
  • Last Updated - 11/06/2017 09:48
Wilde emergency maintenance (Resolved)
  • Priority - Critical
  • Affecting Server - Wilde
  • We are aware of the ongoing  issue with wilde server. We are working on this to resolve the issue as soon as possible.

    Update: We have tested the RAM but the sticks are fine. We are now running long smart tests on all drives in the server but will try keep the server as stable as possible during this time.

    Update: We will be taking the server down in the next 20 minutes to replace a hard drive in the server and then proceeding to rebuild the RAID array.

    Update: The drive has been replaced and the RAID rebuild is currently in progress. We have stopped MySQL on the server to keep the load down so the rebuild will complete quicker.

    Update: We have managed to stabilize the server but are still seeing issues with the RAID array which we continue to work on. We do also have a new server up and ready to migrate data to should we need to but we will continue to work on the server for the next few hours and if any further work is going to occur that will incur excessive downtime we will send out an email to affected customers.

    Update: The issue has been resolved. We finally tracked it down to a rogue mod_security rule that was rolled out by comodo in an automatic update!

  • Date - 10/05/2017 07:12
  • Last Updated - 10/05/2017 14:42
Drive Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Kipling
  • We will be taking the server down to replace a failed hard drive at midnight on the 29th Sept. Downtime is expected to be 60 - 120 mins.

    Update: This was completed. Downtime was around 20 mins.

  • Date - 09/29/2017 00:00 - 09/29/2017 14:17
  • Last Updated - 09/29/2017 14:17
RAM Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Shaw
  • We will be taking the Shaw server down at 1am EST on the 26th Sept to add more RAM to the server. Downtime is expected to be 30 - 60 mins.

  • Date - 09/26/2017 01:00 - 09/26/2017 08:23
  • Last Updated - 09/25/2017 10:03
RAM Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Hermes
  • We will be taking the Hermes server down for a RAM upgrade at 1am EST on the 21st September. Downtime is expected to be 30 - 60 mins.

    Update: This has now been completed

  • Date - 09/21/2017 01:00
  • Last Updated - 09/21/2017 08:09
Multiple IP Blocks Down (Resolved)
  • Priority - Critical
  • Affecting System - US Datacenter
  • There is currently an ongoing issue at the US Datacenter affecting a number of different IP blocks. The blocks are currently going up and down sporadically but the DC is working on the issue.

    Clients affected are those with VPS's on the node "Kraken" and also a handful of clients with Dedicated Servers.

    Shared and Reseller servers are unaffected so far.

    Update: This was caused when the datacenter needed to do an emergency reboot of a switch to solve some BGP issues. This has now been resolved.

  • Date - 08/25/2017 16:57
  • Last Updated - 08/25/2017 18:19
Network Maintenance (Resolved)
  • Priority - Medium
  • Affecting Other - US Datacenter
  • This notification is to inform you of scheduled maintenance that may affect your D9 Hosting service. Please review the details of this notification to determine how it will affect your service. (all times US eastern).

    Event Type: Network Upgrade
    Start Time: 07-12-17 02:00 EST
    End Time: 07-12-17 03:00 EST
    Location(s): New York City Metro
    Device(s): BR1.PNJ1/BR2.PNJ1
    Client Affecting: Possible

    Event Summary: Important maintenance will be performed on one or more devices that may affect network connectivity to your services. Configuration updates will be applied that may affect traffic into the network in NY/NJ causing higher than normal latency or brief periods of packet loss.

  • Date - 07/12/2017 02:00 - 07/17/2017 12:38
  • Last Updated - 07/10/2017 13:25
Network Maintenance (Resolved)
  • Priority - Medium
  • Affecting Other - US Datacenter
  • This notification is to inform you of scheduled maintenance that may affect your D9 Hosting service. Please review the details of this notification to determine how it will affect your service. (all times US eastern).

    Event Type: Network Upgrade
    Start Time: 07-12-17 02:00 EST
    End Time: 07-12-17 03:00 EST
    Location(s): New York City Metro
    Device(s): BR1.PNJ1/BR2.PNJ1
    Client Affecting: Possible

    Event Summary: Important maintenance will be performed on one or more devices that may affect network connectivity to your services. Configuration updates will be applied that may affect traffic into the network in NY/NJ causing higher than normal latency or brief periods of packet loss.

  • Date - 07/12/2017 02:00 - 07/17/2017 12:52
  • Last Updated - 07/10/2017 13:25
Packet Loss (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • We are currently seeing some packet loss on this server which is resulting in websites not resolving for some users. The datacenter are working on the issue for us.

  • Date - 07/10/2017 13:02 - 07/10/2017 13:23
  • Last Updated - 07/10/2017 13:03
Packet Loss (Resolved)
  • Priority - High
  • Affecting Server - Bullseye
  • We are seeing some packet loss on the server bullseye.d9hosting.com

    We are currently investigating the issue.

  • Date - 07/03/2017 09:18 - 07/10/2017 13:03
  • Last Updated - 07/03/2017 09:18
High Server Load (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • The Wilde server is experiencing high load for the past few hours, we are currently aware of the issue and investigating to find the cause of the issue.

  • Date - 06/05/2017 15:12 - 06/06/2017 10:55
  • Last Updated - 06/05/2017 14:13
Server Migration (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • We will be switching the IP addresses over from the old Hyrule hardware to the new at around 9am EST on Tuesday 30th May, around 1 hour of downtime is expected.

    All affected clients have already been emailed about this work.

    Update: IP change has been completed, we are now stopping MySQL on both servers and doing a final sync of MySQL to esnure data consistency

    Update: This has been completed

  • Date - 05/30/2017 12:42
  • Last Updated - 05/30/2017 16:57
Issue with BBU on Hyrule server (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • There is currently an issue on the hyrule server with the BBU-battery backup unit and this is causing high load on the server.

    Our admins are currently aware of the issue and are working on the server. We are attempting to replace the batterie or raid card with the data center and there will be a server down time for the next 45 to 60 minutes. We will let you know when the problem is resolved.

    Update: The datacenter are still working on this and are testing the BBU to see if it's charging at all, if not it will either need to be replaced or worse case scenario we would need to replace the RAID card but this would require a full data restore so we are hoping to avoid that.

    Update: Server has been brought back online whilst we run additional tests on the BBU. We will check the BBU status again at 9pm EST when the BBU should've fully charged and we can see if it is holding the charge or still failing, at which point we can decide how to proceed.

    Update: Server is back online but we are seeing issues with MySQL which we are currently working on

    Update: MySQL issue has been fixed. Although due to the BBU issue there will be sub-optimum website performance until the issue is resolved. We will have more info in around 12 hours time once the tests on the battery have completed.

    Update: The battery unit on the RAID controller is completely dead, so rather than swap out the RAID controller and risk any data loss and having to restore from backups, we are putting up a new box and will be migrating all data over to the new machine. We will be emailed all clients shortly with more info.

  • Date - 05/23/2017 12:51 - 05/30/2017 12:42
  • Last Updated - 05/24/2017 08:44
High Server Load (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • We are seeing abnormally high load on the Hyrule server that is causing very slow website performance. We are monitoring and working on the server and hope to have the issue resolved ASAP.

  • Date - 05/23/2017 09:09 - 05/23/2017 14:00
  • Last Updated - 05/23/2017 09:10
Web Server Upgrades (Resolved)
  • Priority - Medium
  • Affecting Other - All Servers
  • We will be migrating from EasyApache 3 to EasyApache 4 on all Shared and Reseller Servers from the 19th April onwards.

    Each server will be migrated one by one, and there will be around 30 minutes of website downtime per server.

    The updates will all be done during off peak hours (between the hours of 1am and 6am)

    Currently Completed:

    Shaw

    Hermes

    Shakespeare

    Kipling

    Hyrule

    Bullseye

    Goron

    Wilde

    Baracus

    Fleetwood

    Nash

  • Date - 04/19/2017 08:51 - 04/26/2017 15:25
  • Last Updated - 04/26/2017 15:25
UK VPS Network Issue (Resolved)
  • Priority - High
  • Affecting Other - IP Range
  • We are seeing a network issue for VPS's on the IP range 185.3.166.*

    The issue is only affecting 5 VPS's but we have escalted the issue to the datacentre and they are working on it.

    Update: We have just received an update from the datacentre, which is copied below. We have seen a couple of IP's come back up for a short period then go down again, so hopefully it wont be too long before the issue is resolved:

    "We are still working on an issue with the compute layer of a small section of our infrastructure. Though many of you have your services back online, we are still working hard to bring back those clients with offline services."

  • Date - 04/24/2017 13:37 - 04/25/2017 10:55
  • Last Updated - 04/24/2017 15:11
Server Maintenance (Multiple Servers) (Resolved)
  • Priority - Medium
  • Affecting Other - All Shared and Reseller Servers
  • We are upgrading Apache on all Shared and Reseller servers on the 30th and 31st of March during off-peak hours. We don't expect any disruption but in the unlikely event there is any downtime, this notice is to make clients know we are aware of the issue and will be working on it.

  • Date - 03/29/2017 14:32 - 03/30/2017 14:21
  • Last Updated - 03/29/2017 14:34
Server Maintenance (Multiple Servers) (Resolved)
  • Priority - High
  • Affecting System - Multiple Servers
  • We will be carrying out scheduled maintenance on the following servers on the dates and times listed below. You can find details of this maintenance and how it will affect you below.

    Servers: Goron, Hermes & Hyrule [Completed]

    When: Monday 20th March @ 01.00 EST

    Servers: Kipling, Shaw & Wilde [Completed]

    When: Tuesday 21st March @ 01.00 EST

    Expected Downtime: 2 to 3 hours

    What: We will be taking the servers down to add an additional SSD RAID array to it. This will allow us to improve the overall performance of the server by offloading the MySQL service to the new SSD array.

    All clients with accounts that reside on these servers have already been notified via email.

  • Date - 03/20/2017 01:00 - 03/21/2017 04:00
  • Last Updated - 03/21/2017 09:51
Server Migration (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • As per our previous email notifications, we will be migrating the Bullseye server over to new hardware on the 26th January.

    We will be migrating the IP addresses from the old hardware to the new at around 10am EST on the 26th January, around 1 hours downtime is expected.

    Update: This is now in progress.

    Update: IP's migrated but we need to do a final MySQL sync, so at the present time it looks like we'll be running approx 1 hour behind the scheduled window.

    Update: Migration completed

  • Date - 01/26/2017 10:00 - 01/27/2017 10:24
  • Last Updated - 01/27/2017 10:24
UK VPS Network Issues (Resolved)
  • Priority - Critical
  • Affecting System - All UK VPS's
  • There is currently a network issue affecting all UK VPS's. We are aware of it and the DC are working on fixing the issue.

    Update: This is due to a power outage at the Namesco/Everest Datacentre in Reading, UK. They are currently working to get power restored in the building and we will update you as and when we get any updates.

    Update: Power has now been restored and the datacentre team are now working on fixing network issues that have been caused by the power outage.

    Update: As of 14:40 the datacentre are still working on bringing all affected services back online.

    Update: VPS's are now slowly coming back online but there are a lot of machines to bring back up so please hold tight whilst everything continues to be brough back online.

    Update: [17:22] We are still waiting on our provider in the datacentre to get the majority of services back online. We've seen a small number of IP's come back up but the majority are offline. We aren't able to get any concrete information as to why this is but we will continue trying and post any updates as soon as we get them.

    Update: All servers are now back online and we will be emailing a full RFO to all clients as soon as we receive one from the datacenter.

  • Date - 11/15/2016 08:57
  • Last Updated - 11/16/2016 09:01
Kernel Updates (Resolved)
  • Priority - High
  • Affecting System - All Servers
  • We will be updating all servers to patch against the "Dirty Cow" kernel vulnerability over the following days:

    25th October
    =========

    Now - bullseye [completed]
    Now - goron [completed]

    26th October
    ==========

    1am EST - Hermes [completed]
    2am EST - Hyrule [completed]
    3am EST - Kipling [completed]

    2am GMT - Baracus [completed]
    3am GMT - Fleetwood [completed]
    4am GMT - Nash [completed]

    27th October
    =============

    1am EST - Kipling [completed]
    2am EST - Shaw [completed]
    3am EST - Wilde [completed]

    28th October Onwards
    ===============

    All clients Dedicated Servers & VPS. Clients will receive an email advising of upgrade times.

    Downtime will likely be 15 - 30 minutes per server.

  • Date - 10/25/2016 08:42 - 11/07/2016 20:03
  • Last Updated - 10/27/2016 08:44
VPS Node Migrations (Resolved)
  • Priority - High
  • Affecting System - Old VPS Node
  • We are currently migrating all VPS's running on old CentOS 5 nodes over to new nodes. We expect this to be completed by 13:00 EST on the 20th October.

    If you are going to be impacted by this you will have already received an email giving you more details on the 19th October.

  • Date - 10/20/2016 09:00
  • Last Updated - 10/20/2016 16:07
Network Issues Incident Report (Resolved)
  • Priority - Low
  • Event Type: Network Outage
    Location(s): Piscataway, NJ
     
    Event Type: Network Outage
    Location(s): Piscataway, NJ
     
    Event Summary: A large portion of the Data Center's IP space was targeted by an attack exceeding 500 Gbps in size that caused several major upstream links to be completely saturated.
     
    The DC made several immediate changes to the network infrastructure that prevented the attack from causing complete unavailability.
     
    While the current infrastructure can handle the attacks as a whole, the attack traffic did not come in through balanced routes, causing specific uplinks to be completely saturated.
     
    The DC have now started the process of upgrading uplinks to all upstream providers to multiple 100 Gbps links, which will allow them to completely absorb the attack without saturation in the near future. This permanent solution is expected to be in place soon (date TBA as they are still coordinating with upstream providers) and is considered to be the highest priority project to ensure long term availability and quality of service

  • Date - 09/22/2016 08:43
  • Last Updated - 09/30/2016 09:52
Network Issues on a number of Servers (Resolved)
  • Priority - Critical
  • Affecting Other - goron, hermes, hyrule, kipling, shaw, wilde, keats, kraken, bronte
  • Update: From the US datacenter:

    "Our network gear was targeted by a sophisticated attack, causing significant packet loss across a great portion of our NYC Metro network. The attack was manually mitigated and we're currently working on a permanent solution to prevent this sort of attack automatically


    There was a network failure earlier today on the above servers. Our datacenter in the US was contacted and we have been told that this issue has now been fixed. We have not been told as yet what caused the issue, whether it was a network issue or there was some kind of attack. We will keep this updated as and when we get information. Please open a tech support ticket if you are still experiencing any issues.

  • Date - 09/20/2016 09:53
  • Last Updated - 09/21/2016 09:45
MySQL Upgrades (Resolved)
  • Priority - Medium
  • Affecting Other - All Shared and Reseller Servers
  • We will be performing MySQL upgrades on all Shared and Reseller servers over the coming days.

    During the upgrade window there may be slower than usual performance and also brief moments of downtime for MySQL driven websites.

    You can find the upgrade schedule below:

    4th August - 4am EST (US East Coast)
    ========================= 

    Bronte [completed]
    Bullseye [completed]

    5th August - 1am GMT (UK Time)
    ===============================

    Baracus [completed]
    Fleetwood [completed]

    5th August - 1am EST (US East Coast)
    ===================================

    Goron [completed]
    Hermes [completed]

    6th August - 1am GMT (UK Time)
    ================================

    Nash [completed]

    6th August - 1am EST (US East Coast)
    ===================================

    Hyrule [completed]
    Kipling [completed]


    7th August - 1am EST (US East Coast)
    ===================================

    Shaw [completed]
    Wilde [completed]

  • Date - 08/04/2016 08:52
  • Last Updated - 08/08/2016 08:25
Sporadic Packet Loss (Resolved)
  • Priority - High
  • Affecting System - US Network
  • We have been seeing packet loss at present with the Choopa datacentre and all networks contained within it (our servers are on a network within the Choopa building). It's been going on for a couple of days now and we are waiting to hear about a full resolution. I must stress this isn't something we have any direct control over, since it is an issue with the entire facility rather than something specific to the D9 servers.

  • Date - 07/01/2016 12:49 - 07/05/2016 12:30
  • Last Updated - 07/01/2016 12:50
Servers Down (Resolved)
  • Priority - Critical
  • Affecting Other - Multiple Servers
  • We currently have 4 servers down due to an automated system failure on behalf of our upstream providers, the servers should be back up shortly.

    Update: This has now been resolved

  • Date - 06/14/2016 14:43
  • Last Updated - 06/14/2016 14:48
Drive Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Wilde
  • One of the hard drives in the RAID array has failed, so we are taking the server down at 1am EST on the 25th May to replace the drive. Downtime shouldn't be more than 30 - 60 minutes.

    Update: The drive has been replaced and RAID is currently rebuilding

  • Date - 05/25/2016 01:00
  • Last Updated - 05/25/2016 13:59
RAM Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • Maintenance Notice:
     
    The "Bullseye" Server will be taken down at 2 am EST on the 31st March to replace a couple of RAM sticks. Downtime shouldn't be more than 1 - 2 hours.

  • Date - 03/31/2016 02:00 - 04/14/2016 16:07
  • Last Updated - 03/30/2016 13:19
RAM Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Goron
  • We will be taking this server down at 2am EST on the 27th Feb so add some additional RAM to the server. Downtime is likely to be around 15 - 30 minutes.

    Update: This has now been completed

  • Date - 02/27/2016 02:00 - 02/27/2016 03:00
  • Last Updated - 02/27/2016 11:54
Server Maintenance (Resolved)
  • Priority - Medium
  • Affecting Server - Hyrule
  • We are currently stopping the MySQL and Apache services on this server so that we can re-mount the /home parition with quota support to fix an on-going issue with regards the reporting of disk space usage on this server. Services will be down for around 10 minutes.

    Update: This has been completed and services back online

  • Date - 01/28/2016 14:39
  • Last Updated - 01/28/2016 14:54
Network Issue (Resolved)
  • Priority - High
  • Affecting System - Multiple Servers
  • There appears to be a local network issue affecting multiple servers in our US datacenter, we have reported the issue and the DC techs are checking it.

  • Date - 10/07/2015 16:03 - 10/07/2015 17:40
  • Last Updated - 10/07/2015 16:04
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Hermes
  • Server has gone unresponsive, we are checking it.

    Update: Server is back online after a reboot

  • Date - 08/24/2015 13:32
  • Last Updated - 08/24/2015 14:07
Hyrule Server Load (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • We have found a resource heavy user on the server and are currently moving this account over to a Dedicated Server. There are multiple accounts to move and whilst we rsync these over to a new Dedicated Server load will be high and you may experience sub-optimal performance when loading websites. We are working on getting the resource heavy accounts off the server ASAP whilst keeping the load down to a minimum to allow other sites to load.

    Your patience is appreciated and performance will return to normal once these accounts are moved off the server.

  • Date - 06/22/2015 11:54 - 06/24/2015 13:39
  • Last Updated - 06/22/2015 11:55
Kernel Upgrades (Resolved)
  • Priority - High
  • Affecting System - Multiple Servers
  • Maintenance Details: Kernel Security Patch
    Start & End Date: 19th June 23:00 - 20th June 06:00 EST
    Expected Downtime: < 1 hour per server

    We will be patching the following Dedicated Servers against the recently discovered CVE-2015-1805 Linux Kernel bug that if exploited, could allow an unprivliged user root access to the server.

    Once we have applied the patch, the Dedicated Server will be rebooted to complete the maintenence. Downtime of no more than 1 hour is expected for each server:

    • Kipling
    • Shaw
    • Wilde
    We will also be patching the following UK Dedicated Servers at 01:00 GMT on the 20th June:

    • Baracus
    • Fleetwood
    • Nash

    You can view the full details of the bug that we are patching against here:

    https://access.redhat.com/security/cve/CVE-2015-1805

  • Date - 06/19/2015 23:00 - 06/22/2015 11:54
  • Last Updated - 06/19/2015 08:17
Kernel Upgrades (Resolved)
  • Priority - High
  • Affecting System - Multiple Servers
  • Maintenance Details: Kernel Security Patch
    Start & End Date: 18th June 23:00 - 19th June 06:00 EST
    Expected Downtime: < 1 hour per server

    We will be patching the following Dedicated Servers against the recently discovered CVE-2015-1805 Linux Kernel bug that if exploited, could allow an unprivliged user root access to the server.

    Once we have applied the patch, the Dedicated Server will be rebooted to complete the maintenence. Downtime of no more than 1 hour is expected for each server:

    • Bronte
    • Bullseye
    • Goron
    • Hermes
    • Hyrule


    We will also be applying the same patch to all UK VPS's at 1am GMT on the 19th June, again, only minimal downtime is expected.

    You can view the full details of the bug that we are patching against here:

    https://access.redhat.com/security/cve/CVE-2015-1805

  • Date - 06/18/2015 23:00 - 06/19/2015 06:00
  • Last Updated - 06/19/2015 08:15
Dedicated Server Kernel Pathces (Resolved)
  • Priority - High
  • Affecting Other - Clients Dedicated Servers
  • Maintenance Details: Kernel Security Patch
    Start & End Date: 17th June 23:00 - 18th June 06:00 EST
    Expected Downtime: < 1 hour per server

    We will be patching all clients Dedicated Servers against the recently discovered CVE-2015-1805 Linux Kernel bug that if exploited, could allow an unprivliged user root access to the server.

    Once we have applied the patch, the Dedicated Server will be rebooted to complete the maintenence. Downtime of no more than 1 hour is expected for each server.

    All clients with UK Dedicated Servers please note we will start with the Kernel upgrades at 02:00 GMT, not EST.

    You can view the full details of the bug that we are patching against here:

    https://access.redhat.com/security/cve/CVE-2015-1805 

    Update: This has been completed and all servers are back online

  • Date - 06/17/2015 23:00 - 06/18/2015 08:21
  • Last Updated - 06/18/2015 08:20
VPS Kernel Security Updates (Resolved)
  • Priority - High
  • Affecting System - All OpenVZ VPS's
  • Maintenance Details: Kernel Security Patch
    Start & End Date: 16th June 23:00 - 17th June 06:00 EST
    Expected Downtime: 2 Hours

    We will be patching all VPS's against the recently discovered CVE-2015-1805 Linux Kernel bug that if exploited, could allow an unprivliged user root access to the server.

    Once we have applied the patch, the hardware node will be rebooted to complete the maintenence at which point a each VPS will start to come back online. Downtime of no more than 2 hours is expected.

    You can view the full details of the bug that we are patching against here:

    https://access.redhat.com/security/cve/CVE-2015-1805

    Update: This has now been completed and all VPS's are back online

  • Date - 06/16/2015 23:00 - 06/17/2015 06:00
  • Last Updated - 06/17/2015 08:08
Bullseye server Down (Resolved)
  • Priority - Critical
  • Affecting Server - Bullseye
  • The server is up and back online. We are restoring the accounts now and it will be completed soon.

    Restoration of all the accounts Completed'

    Few account failed during restoration process. We are going to check the failed accounts now.

    We will keep updating here.

    We appreciate your patience.


  • Date - 02/23/2015 03:05 - 03/02/2015 14:27
  • Last Updated - 02/27/2015 03:57
Shaw Server Down (Resolved)
  • Priority - Critical
  • Affecting Server - Shaw

  • We are having network issue on the Shaw Server. We informed the issue on DC and the techs at Data Center is working on the issue.


    Network issue with the Shaw server has been resolved.

    Server is up and back online.

  • Date - 02/23/2015 17:19 - 02/23/2015 21:58
  • Last Updated - 02/23/2015 18:57
Server Unresponsive (Resolved)
  • Priority - Critical
  • Affecting Server - Fleetwood
  • The server is currently unresponsive. We are checking it.

    Update: Server has been rebooted and is back online

  • Date - 12/09/2014 13:51
  • Last Updated - 12/09/2014 14:00
Server Down (Resolved)
  • Priority - High
  • Affecting Server - Bullseye
  • The server has gone unresponsive, we are checking it.

    Update: The server is back up after a reboot and we are monitoring it. Total downtime was around 35 mins.

  • Date - 11/14/2014 12:40
  • Last Updated - 11/14/2014 14:17
Network Disruption (Resolved)
  • Priority - Medium
  • Affecting System - Specific IP's
  • There appears to be a network issue affecting a very small number of IP blocks in our US facility. We're checking it out.

  • Date - 11/10/2014 16:56 - 11/11/2014 11:50
  • Last Updated - 11/10/2014 16:56
MySQL Upgrades (Resolved)
  • Priority - Medium
  • Affecting Other - All Shared and Reseller Servers
  • We will be upgrading MySQL on all Shared and Reseller hosting servers in the early hours of Tuesday - Saturday morning, we'll be upgrading a small number of servers each night. There is likely to be around 1 hour of service interupption for MySQL driven websites.

    Update: The MySQL upgrades on the servers Wilde and Shaw have been completed, the next batch of servers will be upgraded on Wednesday morning

    Update: The MySQL upgrades on the servers Baracus, Hermes and Goron have been completed, the next batch of servers will be upgraded on Thursday morning

    Update: The MySQL upgrades on the servers Kipling, Hyrule and Nash have been completed, the next batch of servers will be upgraded on Friday morning

    Update: All servers have now been updated

  • Date - 09/02/2014 01:00 - 09/06/2014 06:00
  • Last Updated - 09/08/2014 08:49
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Shaw
  • The server has gone unresponsive. We're currently checking on it.

    Update: This has now been resolved, we will continue to monitor the server

  • Date - 08/13/2014 13:57
  • Last Updated - 08/13/2014 16:04
Server Outage (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • The Nash server is suffering sporadic outages, we are checking on it.

    Update: Server is back up and we're monitoring it

  • Date - 08/06/2014 11:38 - 08/06/2014 20:01
  • Last Updated - 08/06/2014 11:44
Server Maintenance (Resolved)
  • Priority - Medium
  • Affecting Server - solus.hyliahub.com
  • We've discovered that the server isn't discovering one of the RAM sticks which is leading to performance issues. We have taken the server down to investigate, we hope to have it back online shortly.

    Update: We have replaced the defective RAM and the server is back up and running. VPS's are being restarted one by one, all VPS's should be back up within the hour.

  • Date - 07/29/2014 09:36
  • Last Updated - 07/29/2014 10:02
Intermittent Network Issue (Resolved)
  • Priority - High
  • Affecting System - New Jersey Network
  • We are seeing intermittent packet loss on our New Jersey network. The datacentre are aware of the issue and are currently working on it.

    UPDATE;

    18.44 - the issue is now resolved

  • Date - 06/30/2014 15:30
  • Last Updated - 06/30/2014 18:23
Apache, MySQL and PHP Updates (Resolved)
  • Priority - Low
  • Affecting System - All servers
  • We will shortly be upgrading the Apache Web Server on all Shared, Managed VPS and Managed Dedicated Servers to address a security vulnerability found in older versions of Apache. Any Managed VPS clients running an older version of MySQL that is causing cPanel updates to fail will also be upgraded at the same time.

    We will also be updating the default PHP version on all Shared Servers at the same time as the Apache upgrades. Managed VPS and Managed Dedicated Server customer wont have the PHP version changed unless you're running an old PHP version that is no longer supported by cPanel/EasyApache/

    The majority of Shared servers were still running PHP 5.2.* as the default version, despite it going into "end of life" status back in January 2011 so we are upgrading the default PHP version to PHP 5.4.*

    If you have a script that wont run on the new PHP version, you can switch back to PHP 5.2.* via cPanel > Select PHP Version. (This applies to Shared and Reseller Customers only)

    No downtime is expected during the upgrades.

  • Date - 06/27/2014 10:49
  • Last Updated - 06/28/2014 13:16
Kernel Upgrades (Resolved)
  • Priority - High
  • Affecting Other - Multiple Servers
  • The following servers will be rebooted at 2am EST on Tuesday 10th June to fix the recently discovered CVE-2014-3153 kernel vulnerability:

    bullseye.d9hosting.com
    goron.hyliahub.com
    hermes.hyliahub.com
    hyrule.hyliahub.com
    kipling.hyliahub.com
    shaw.hyliahub.com
    wilde.d9hosting.com

    The following servers will be rebooted at 2am GMT on Tuesday 10th June:

    baracus.hyliahub.com
    fleetwood.hyliahub.com
    nash.hyliahub.com

    Downtime is likely to be 5 - 10 minutes unless a FSCK is needed.

    Update: This has been completed and all servers are back online

  • Date - 06/10/2014 02:00
  • Last Updated - 06/10/2014 13:51
UK VPS Network Maintenance (Resolved)
  • Priority - High
  • Affecting System - UK VPS Network
  • On 2014-02-15 we'll be performing some maintenance that will affect your service. We will be migrating UK VPS's to a new facility, as part of our expansion programme for services capacity and network offering. The migration will happen overnight, from 22:00-02:00 (GMT).

  • Date - 02/15/2014 16:00 - 02/18/2014 09:49
  • Last Updated - 02/14/2014 10:13
US Backup Server Maintinence (Resolved)
  • Priority - Medium
  • Affecting System - Backup Server
  • We are currently re-configuring the RAID level on our US backup server; vault.bkpvault.com. During this time backups will be unavailable for restoration.

  • Date - 02/06/2014 10:04 - 02/12/2014 15:18
  • Last Updated - 02/06/2014 10:05
New Jersey Network Maintenance (Resolved)
  • Priority - High
  • Affecting System - New Jersey Network
  • This notification is to inform you of scheduled maintenance that is taking place in our New Jersey facility. Please review the details of this maintenance below:

     

    Event Type: Network Upgrade
    Start Time: 2013-12-04 02:00 EST
    End Time: 2013-12-04 02:45 EST
    Device(s): br1.pnj1

    Event Summary: We have worked with our vendor to isolate the root cause of a brief outage on Monday 2 December, 2013 which impacted some customers. The root cause is a recently discovered memory leak in the current software version. To resolve this condition, the OS level must be upgraded immediately and will require a router reload. The other core routers are not experiencing the defect but will be scheduled for upgrade shortly in non-emergency maintenance windows.

    The purpose of this maintenance window is to apply an OS version upgrade to one of our core routers in the PNJ location. This upgrade will require a reload which may impact customer traffic for up to 15 minutes.

  • Date - 12/04/2013 02:00 - 12/10/2013 14:09
  • Last Updated - 12/03/2013 22:12
Server down (Resolved)
  • Priority - Critical
  • Affecting Server - solus.hyliahub.com
  • Kraken went unresponsive and we are working on it. So please hold on.

     

     

    Server back online. Restarting each containers one by one.

     

     

    Containers are back online.

  • Date - 10/20/2013 19:10 - 10/20/2013 19:41
  • Last Updated - 10/20/2013 19:41
Network Issue (Resolved)
  • Priority - High
  • Affecting Other - Pennsylvania Network
  • There appears to be a network issue affecting the Pennsylvania datacenter that houses our budget dedicated server clients. We will post more info when we receive it from our upstream provider.

    The issue is now resolved and we received the following info from the data center:

    "We are aware of the network/routing issues recently arising, and have been working hard to resolve such. The is issue is due to a combination of issues/factors occurring all at once, including: Relocation of carrier circuits to the new PA facility from our old facility, maintenance from carriers on multiple circuits, defective RAM/Memory in a gateway router, BGP flaps from a backbone carrier, Defective carrier equipment card on a long haul transport route, a DDOS attack direct to some router interfaces w/carrier uplink attached, amongst a few other smaller issues. All of these issues hitting at one time, made it a little hard/complex to troubleshoot problems, and get a handle on what was actually going on.

    We believe everything has been resolved to the point, and expect no other major service issues moving forward. There is some clean up work still needing to be done, and some circuits to still migrate, but we do not expect to see major network issues arise from them as per the last few days random problems. There may be a few minor hiccups, such as BGP updates, as some circuits are relocated to the new facility over the next week, and as we shift traffic around.

    We than thank you for your patience and understanding with this, and we expect smoother network performance moving forward."

  • Date - 08/27/2013 20:57
  • Last Updated - 08/29/2013 13:45
High Server Load (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • We are just doing the final data sync on the Wilde server before we complete the migration to the new datacenter, during this time load on the old server will be high and sites will be loading slower than usual but it will only be for an hour or two.

     

     

    Update: We are going to disable httpd and mysql on the server just so we can complete the final sync quicker since it was badly overloading the server. We expect the services to be down for around 1 hour and once the sync is complete we will route traffic through the new server

  • Date - 08/09/2013 09:25 - 08/10/2013 08:57
  • Last Updated - 08/09/2013 12:46
MySQL Down (Resolved)
  • Priority - High
  • Affecting Server - Bullseye
  • We are aware of an issue with the MySQL service on Bullseye, we are checking it.

  • Date - 08/07/2013 11:51 - 08/09/2013 09:24
  • Last Updated - 08/07/2013 11:51
Server Unresponsive (Resolved)
  • Priority - Critical
  • Affecting Server - Bullseye
  • Server is currently unresponsive, we have issued a reboot request

     

    Update: Server back up after a reboot

    Update: Server is overloaded, checking it

     

    Update: Server is back up, we have tweaked a few settings and are monitoring it

  • Date - 07/30/2013 18:47
  • Last Updated - 07/30/2013 22:09
Server nash.hyliahub.com is down (Resolved)
  • Priority - Critical
  • Affecting Server - Nash
  • **Please Scroll To The Bottom of the Page for Latest Updates**

    The server has gone unresponsive again and the DC techs are checking on it.


    Update: THere appears to have been a heating issue causing a kernel panic, we have re-thermal pasted the CPU, and checked the health of the heatsync. Will be attempting to bring the server back up momentaerially. 

    Update: The server died again as it was coming back up, onsite techs are still working on it. Looking like we may have to go for a chassis swap

     

    Update: Server was up for a period but died again. We are working on getting replacement hardawre set up.

     

    Update: Datacenter is sourcing replacement hardware whilst still trying to get the server back up. In the meantime if you have a local backup of your sites please open a ticket with technical support and we can set up your account on a different server and you can upload your data.

     

    Update: We are unable to locate an exact hardware match in the UK facility so it looks like we are going to have to restore data to a US based server whilst the DC are able to order in a new box for the UK facility. Will keep posting updates as and when we get them. As we mentioned before, if you have local backups that you can restore than please let technical support know and we can recreate your account on another server.

     

    Update: We have got a new server ready and will shortly start restoring data to that server from our Last good backup which is from the 24th July. There is just short of 400 GB of data to be restored. We will keep this page updated with the progress.

     

    Restore Updates

    ==============

    Update 1: Currently restoring /var (the database partition) and it's just over 15% done....now 23% done...../var is now 42% done....... now 65% done..... now 100%  and completed.
    Update 2: Currently restoring /tmp  it's over 35% done........ now 95% done........now complete. 
    Update  3: Started restoring  the /home partition ...  29% done ... 31%done .... 38% done...45%done
    Update 4: Restoration of /home partition is still in progress. Currently 49% done...52% done...55% done.... 59% done... 63% done.. 67% done...73% done...76% done.....now 81% done.... now 87% done......now 92% done........now 98% completed.... now 100% completed.
    Update 5: Bare-Metal Restore has been completed. Its time to reboot the server through HDD and after that the new Ip's will be assigned.
    Update 6: Onsite techs are currently booting server into single user mode to reset the password, once done new IP's will be assigned
    Update 7: We should be back up very soon, we can see signs of life in the IP's and are just waiting on final confirmation from the datacenter before we can make any needed nameserver IP changes


    Update
    =======

    The server is now back online with new IP's. Nameserver IP's have been updated and your sites will shortly start to resolve from the new IP addresses if they are not already doing so. If you are using a 3rd party DNS service and just pointing your A records to the server, please update them to use the following IP:

    209.140.31.226

    We will be emailed all affected clients with an incident report shortly.

  • Date - 07/27/2013 13:05 - 07/31/2013 08:11
  • Last Updated - 07/30/2013 05:51
Server Unresponsive (Resolved)
  • Priority - Critical
  • Affecting Server - Bullseye
  • The Bullseye server is currently unresponsive, we are checking it.

  • Date - 07/29/2013 08:18 - 07/29/2013 09:28
  • Last Updated - 07/29/2013 08:18
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • The server nash.hyliahub.com is not responding. We are checking it.

     

    Update: Unfortunately we have no further info we can give you other than the datacenter are still working on the server but if/when we get more information we will post it here.

     

    Update: We are able to get access to the server for brief periods but soon after it goes inaccessible, looks like it could be a PSU issue. Still checking it.

     

    Update: We have been able to bring the server back up. We are closely monitoring it.

     

    Update: We are rebooting the server to install a new kernal.

     

    Update: Server booted into new kernal successfully but disk IO is still very high. We are checking the drives for errors. During this time http may be up and down.

  • Date - 07/25/2013 15:46 - 07/26/2013 09:39
  • Last Updated - 07/26/2013 08:36
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • The server is currently unresponsive, we are checking on it.

     

    Update: Datacenter techs are checking on the server

  • Date - 07/13/2013 04:32 - 07/13/2013 08:01
  • Last Updated - 07/13/2013 06:14
Pennsylvania Datacenter Move (Resolved)
  • Priority - Medium
  • Affecting Other - Clients Dedicated Servers
  • As per our recent emails, a number of clients with Dedicated servers located in PA are being moved (the servers are being physically moved) to our upstream providers new facility 3 miles down the road. Downtime for the move is expected to be 2 - 4 hours. Full details of the move are in the email that was sent out to all affected clients last week.

  • Date - 07/10/2013 08:17 - 07/11/2013 10:09
  • Last Updated - 07/10/2013 08:19
UK Servers Unreachable (Resolved)
  • Priority - High
  • Affecting System - UK Cheltenham Network
  • There is currently a network issue with the UK datacenter that houses our Shared and Reseller UK servers. We are seeing lots of packet loss which is causing sites to time out, the DC is currently working on the issue.



    Update: There is a massive DDOS attack which is overloading the main switch leading to the packet loss, DC techs are working to mitigate the attack



    Update: The IP that was being attacked (it wasn't one of ours, but another client on the same switch) has been null routed and servers are accessible again.

  • Date - 07/04/2013 11:58 - 07/08/2013 03:07
  • Last Updated - 07/04/2013 14:42
IP rerouting to new hardware (Resolved)
  • Priority - Medium
  • Affecting Server - solus.hyliahub.com
  • As per our email notice, we will be routing the IP's on the VPS node kraken.hyliahub.com to new a new box. We have opened a maintinance window for this from 9am - 11am EST on Friday 24th May, you may experience sporadic outages until the new routing is in place.


    Update: The IP's have been routed to the new server successfully, we are now working to bring the VPS's back online. Thanks for your patience.


    Update: This has now been completed and all VPS's are back online, apologies that we went slightly over the 2 hour window we gave ourselves, it was due to an unforseen firewall issue. Please contact tech support if you have any issues after the migration, you may see a few duplicate emails come through from your email accounts but this is normal and nothing to worry about.

  • Date - 05/24/2013 09:00 - 05/28/2013 05:52
  • Last Updated - 05/24/2013 11:14
wp-login.php DDOS (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple Servers
  • We are seeing a reoccurance of the wp-login.php DDOS attack on a smale scale against a small number of our servers, as such we have temporarily suspended global access to wp-login.php pgaes until the attack subsides. If you need access to the admin area of a Wordpress based site in the meantime please follow the instructions in this guide:


    http://d9clients.com/knowledgebase/113/How-To-Enable-Wordpress-Admin-Area-Access-For-Your-IP-Address.html

  • Date - 05/20/2013 16:12 - 05/23/2013 08:45
  • Last Updated - 05/20/2013 16:14
DDOS Attack (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • We are currently experiencing a DDOS attack against the server nash.hyliahub.com which is causing sub-optimal performance, we are currently working on brining this under control.


    Update: We have implemented the global wp-login.php block and this has brought the server load back to normal levels. We will leave the block in place whilst the DDOS subsides and then remove it. If you need access to a Wordpress admin area in the mean time, please follow this guide:


    http://d9clients.com/knowledgebase/113/How-To-Enable-Wordpress-Admin-Area-Access-For-Your-IP-Address.html


    Update: This how now subsided, and we have removed the global wp-login.php block.


    Update: The issue has returned again today, we are working on it.

  • Date - 05/14/2013 04:46 - 05/15/2013 08:26
  • Last Updated - 05/15/2013 07:14
Server Reboot (Resolved)
  • Priority - Critical
  • Affecting Server - solus.hyliahub.com
  • Server has gone unresponsive so we are rebooting it, it should be back up shortly. As there have been an abnormal amount of errors on this node recently we are arranging for new hardware to replace the existing box which should fix these ongoing issues.


    Update: Server is back up and all VPS's online

  • Date - 05/14/2013 08:19 - 05/14/2013 08:51
  • Last Updated - 05/14/2013 08:51
Filesystem Error (Resolved)
  • Priority - Critical
  • Affecting Server - solus.hyliahub.com
  • We are facing some file system related issues on hardware node Kraken. We are working on this issue, so please wait for further updates from our end.

    Update:

    Issue resolved and server back online.

  • Date - 05/13/2013 00:26 - 05/13/2013 02:21
  • Last Updated - 05/13/2013 01:15
Server Reboot (Resolved)
  • Priority - High
  • Affecting Server - solus.hyliahub.com
  • Server went inaccessible so we have rebooted, VPS's should be back online shortly.

    Update: Server is back online and VPS's up and running, please open a support ticket if you have any problems.

  • Date - 04/26/2013 06:00
  • Last Updated - 04/26/2013 06:10
Server Unreachable (Resolved)
  • Priority - Critical
  • Affecting Server - Hermes
  • Server is currently unresponsive, we are checking it.

    Update: Server is back up, please open a support ticket if you are still facing issues

  • Date - 04/25/2013 11:02 - 04/25/2013 12:47
  • Last Updated - 04/25/2013 12:46
Network issue (Resolved)
  • Priority - Critical
  • Affecting Server - Hyrule
  • We are facing a network issue with our server. our network engineers are working on this to resolve this issue asap.

  • Date - 04/22/2013 20:35 - 04/24/2013 12:21
  • Last Updated - 04/22/2013 20:37
wp-login.php DDOS Attack (Resolved)
  • Priority - High
  • Affecting System - Multiple Servers
  • For Instructions On How To Access Your Wordpress Admin Area. Please See The Following Knowledgebase Article:
     

     

     

    Friday Evening Update: 

    The blocking of wp-login.php files has had the desired effect, and servers have remained stable throughout the day. Whilst the attacks do seem to have slowed down over the past few hours, we have seen this behaviour over the past couple of days, only for things to peak up again after a few hours of being idle, so with that in mind we will be leaving all wp-login.php files disabled over the weekend and then reassess things on Monday. The last thing we want to do is let our guard down, and find ourselves back to square one over the weekend when we wont have as many staff on shift the handle the situation.

     

    Users can still access their Wordpress admin areas by whitelisting their IP address, as shown in this article.

     

    ======================================================

    Friday Morning Update:


    As the attacks show now sign of slowing down, we have now taken the steps to globally disable access to all wp-login.php files on all shared and reseller servers. This has already been implemented on all servers apart from Bullseye which was brought down by the DDOS attack a short time ago. Once Bullseye comes back up we will implement the temporary fix in an effort to keep the servers online throughout this industry wide attack.

     

    ======================================================

     

    Historic Updates - for information purposes only, please refer to the updates above for the most recent info

     

    Over the past 24 hours there has been a widespread DDOS campaign targetting the popular CMS script Wordpress, inparticular the file used to log into the admin area of the script; wp-login.php. Large numbers of IP addresses from across the world have been attacking any files they can find, resulting in massive issues for web hosts and the stability of web servers.


    Whilst we have done our best to manage the situation, a number of our servers have been hit hard, some have even been taken offline by the attack.

     

    To get everything back online on the servers that have been hit the hardest (at the time of writing these are the servers Hermes, Kipling and Wilde) we have had to take the only option left that was to available to us, and that is to disable access to all wp-login.php files on the server, and the block any IP's that attempt to access a file with that name.

     

    We understand that this will mean a small number of genuine users will be locked out of the server for a short period (60 mins), but to put it into context, this measure has also allowed us to block malicious IP addresses that were causing the issues, at a rate of around 1,000 per minute. So you can see the sort of numbers we are having to deal with across multiple servers.

     

    So What We Are Asking From You Is This
    ==============================

    - If you are on one of the servers where we are temporarily disallowing access to wp-login.php and you DON'T need urgent access then please sit tight and bare with us, normal service will be resumed once the attacks subside

    - If you DO need URGENT (we must stress this part, please only request access if absolutely vital) access to the admin area of your blog, please open a ticket with technical support and provide us with:

     

    1. The URL of the blog you want access to

    2. Your IP address - this can be found at www.whatismyip.com

     

    Thanks for your patience in this matter, we know it's going to be a bit of a pain for the next few days but it was either this, or have the servers going down every few minutes when the DDOS overloaded them! Also, if you want more details on the nature, and the scale of the attack, you can see how the rest of the web hosting community is dealing with the issue in this forum thread.

     

  • Date - 04/11/2013 17:10 - 04/22/2013 04:09
  • Last Updated - 04/12/2013 10:27
wp-login.php brute force/ddos attack (Resolved)
  • Priority - High
  • Affecting Server - Hermes
  • The Hermes server has just been hit by the wp-login.php ddos attack that has been going round the net for the past 12 - 24 hours and is currently inaccessible, we are working on it. Please see the "Wilde" network issue down the page for full details on the issue.

  • Date - 04/11/2013 15:47 - 04/11/2013 17:27
  • Last Updated - 04/11/2013 15:48
httpd service down (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • The httpd service on this server has gone down, causing websites on the server to become inaccessible. It appears to be due to a large scale brute force attempt from lots of IP's focusing on a lot of different wp-login.php pages. We are aware of the issue and are working on it.

    Update: We are now seeing the wp-login.php brute force issues on a number of other servers, and after some research we have discovered this is a wide scale brute force attack which is affecting lots of other web hosts. More details can be found here. We are currently testing out mod_security rules on a server that is under attack, if successful this would block the IP's making the brute force attempts and bring service levels back to normal.


    Update: We have managed to stabalise all servers apart from Wilde, which is being hit particularly hard by this brute force/ddos attack. We are still working towards a solution on Wilde and really appreciate your patience on this matter.

  • Date - 04/11/2013 06:54 - 04/11/2013 17:27
  • Last Updated - 04/11/2013 12:09
httpd down (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • httpd has crashed, which is causing websites to time out. We are aware of this issue and are currently working to get the service back online.

    Update: We have brought httpd back online and the server is stable. We will continue to monitor the server for the next few hours.

  • Date - 04/11/2013 04:27
  • Last Updated - 04/11/2013 05:28
UK Network Maintenance (Resolved)
  • Priority - Medium
  • Affecting Server - Baracus
  • We will be performing the network maintenance on Tuesday, April 9th at 6AM GMT. The purpose of this maintenance is to optimize the network configuration.
    
    The servers which will be involved in the scheduled maintenance are: baracus.hyliahub.com The maintenance window should not be longer than 30 minutes. You should not experience downtime for longer than 5 minutes.

  • Date - 04/09/2013 02:00 - 04/09/2013 04:58
  • Last Updated - 04/08/2013 05:34
Nash server : network issue (Resolved)
  • Priority - Critical
  • Affecting Server - Nash
  • Nash server is down due to a network config error and we are working on getting it back up

    Update : 2 Our network engineers are still working on the network configuration.

    Update 3: We apologize profusely for the unacceptable downtime on Nash that occurred overnight and into this morning, we have now resolved the issue with the internal network issue/IP routing and your site and emails are back online.

  • Date - 03/25/2013 16:03 - 03/26/2013 00:00
  • Last Updated - 03/26/2013 06:49
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - solus.hyliahub.com
  • The server kraken.hyliahub.com has gone unresponsive, we are checking on it.

    Update: Server is back online after a reboot

  • Date - 03/14/2013 09:13
  • Last Updated - 03/14/2013 09:39
Network Issue with server (Resolved)
  • Priority - Critical
  • Affecting Server - Baracus
  • We are now facing a network issue with the server. 

    Our network engineers are working on this. 
    This should be fixed asap. We will keep you updated.

  • Date - 03/10/2013 13:49 - 03/10/2013 13:58
  • Last Updated - 03/10/2013 13:51
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • A hacked account on the server has flooded the mail queue and sent it unresponsive. We've currently issued a reboot request so the server should be back online shortly. Thankyou for your patience.

    Update: The server is back up after a reboot and FSCK, apologies for any inconvenience caused.

  • Date - 03/07/2013 12:54 - 03/08/2013 03:53
  • Last Updated - 03/07/2013 14:57
Atlanta Network Maintenance: Sunday Febuary 2 (Resolved)
  • Priority - Low
  • Affecting Other - Network
  • The Atlanta datacenter that houses our Shared and Reseller servers will be performing Network Maintenance on Sunday February 24th between the times of 5am to 7am EST. We do not anticipate that you will experience any downtime and there should be no interruptions in your service. 

  • Date - 02/24/2013 05:00 - 03/07/2013 14:57
  • Last Updated - 02/22/2013 03:56
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Hermes
  • The server has gone unresponsive, we are checking it and will get it back up ASAP.

    Update: The server is back online, we have temporarily stopped the httpd and mysql services to keep the load down whilst we investigate what caused the server to go down

    Update: We have re-enabled http and mysql on the server, we found an abusive user on the server which we believe caused the server to become overloaded. The user has been suspended and load has gone back to normal.

  • Date - 02/13/2013 03:14
  • Last Updated - 02/13/2013 04:25
network issue with server (Resolved)
  • Priority - Critical
  • Affecting Server - Hermes
  • Unfortunately we are facing a network issue with our shared server hermes.hyliahub.com.

    Our network engineers are working on this. We hope that the issue will be resolved as soon as possible.
    We really do apologize for the inconvenience this might have caused you.

    Further updates will be notified here.




  • Date - 02/12/2013 20:03 - 02/13/2013 04:11
  • Last Updated - 02/13/2013 03:53
Network issue with Server (Resolved)
  • Priority - Critical
  • Affecting Server - Shaw
  • We are currenlt facing a network issue with our shared server shaw.hyliahub.com.
    Our network engineers are working on this and this should be fixed asap.

  • Date - 02/06/2013 09:56 - 02/07/2013 01:29
  • Last Updated - 02/06/2013 09:59
Server Down (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • Server has gone unresponsive, we are rebooting it

  • Date - 01/10/2013 06:12 - 01/10/2013 06:59
  • Last Updated - 01/10/2013 06:13
Goron Server Not Responding (Resolved)
  • Priority - High
  • Affecting Server - Goron
  • Goron server isn't responding and we're unable to establish kvm and apc. We have contacted the data center and will update when we hear more details.

    Currently server Goron is up and running fine. 

  • Date - 01/09/2013 18:13
  • Last Updated - 01/09/2013 18:49
RAM & MOBO Swap (Resolved)
  • Priority - High
  • Affecting Server - solus.hyliahub.com
  • We will be taking down the VPS hardware node "Kraken" this Friday (December 28th) at 6 am EST to perform a RAM and Motherboard swap to resolve ongoing stability issues we have been facing over the past couple of weeks.

    The hardware swap should take no more than 30 minutes, and once the server is back online individual VPS's will be booted up one by one, which usually takes 1 hour to complete. So total downtime should be around 2 hours.

    Thanks for your continued support.

  • Date - 12/28/2012 06:00 - 12/31/2012 06:13
  • Last Updated - 12/27/2012 17:34
RAM Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - solus.hyliahub.com
  • We will be taking down the VPS node Kraken at midnight on Weds 19th December EST to replace the RAM in the server.

    Downtime will be 1 - 2 hours.

  • Date - 12/19/2012 23:59 - 12/31/2012 06:13
  • Last Updated - 12/15/2012 16:16
UK Network Issue (Resolved)
  • Priority - High
  • Affecting System - UK Network
  • There is currently an issue on part of the UK network we use which is causing connections to the following services/services to fail:

    - Nash (UK shared & reseller server)

    - Some UK VPS's

    The datacenter technicians are already working on the issue and should have it resolved soon.

    Update: We are now back online, apologies for any inconvenience caused.

    Update: The network is experiencing issues again, onsite techs are already working on it.

  • Date - 12/13/2012 10:02 - 12/13/2012 14:09
  • Last Updated - 12/13/2012 10:42
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - solus.hyliahub.com
  • The VPS node kraken.hyliahub.com has gone unresponsive, load was high a couple of hours ago but we brought it under control however it has now gone unresponsive so we are working on it.

  • Date - 12/10/2012 05:29 - 12/11/2012 10:06
  • Last Updated - 12/10/2012 05:30
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • The server wilde.d9hosting.com is currently unresponsive, we are checking it.

    Update: Server is back online after a reboot

  • Date - 12/07/2012 03:32 - 12/07/2012 03:49
  • Last Updated - 12/07/2012 03:49
Kernal Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Kipling
  • We will shortly be rebooting the server to complete a Kernal upgrade on kipling.hyliahub.com, downtime will be around 10 - 30 minutes.

  • Date - 11/02/2012 05:37 - 11/02/2012 06:07
  • Last Updated - 11/02/2012 05:38
Server Performance Issues (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • Server is suffering performance issues, we are checking it.

    Update: An abusive account on the server was sending large amounts of spam, overloading the server. The account has been suspended and the server load is back to normal.

  • Date - 10/29/2012 10:24
  • Last Updated - 10/29/2012 10:40
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Hermes
  • The server hermes.hyliahub.com has gone unresponsive. We have issued a reboot request.

    Update: Server is back up after a reboot. We will keep a close eye on it for the next few hours.

  • Date - 10/02/2012 03:16 - 10/02/2012 03:45
  • Last Updated - 10/02/2012 03:20
RAM Replacement (Resolved)
  • Priority - High
  • Affecting Server - solus.hyliahub.com
  • We have discovered a corrupt memory module in the server, so we will be taking it down at 1am EST on October 2nd to replace the corrupt module. When the new module is installed the server will be booted back up, and the VPS's will come back up one by one. Downtime is expected to be between 60 - 180 minutes.

    Update: This has been completed. Changing the RAM took around 10 minutes, and then all individual VPS's booted up one by one. Total downtime was between 15 minutes and 1 hour depending on the individual VPS.

  • Date - 10/02/2012 01:00 - 10/02/2012 03:05
  • Last Updated - 10/02/2012 03:05
Server Down (Resolved)
  • Priority - High
  • Affecting Server - Hermes
  • The server has gone unresponsive, we are checking it now.

    Update: The server is back up after a reboot, we will keep an eye on it for the next few hours.

  • Date - 09/28/2012 15:23
  • Last Updated - 09/28/2012 15:29
Drive Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Kipling
  • One of the drives in the RAID array is showing signs of deteriation, so we are going to replace it to be on the safe side. We don't expect any downtime but load on the server will be higher than normal whilst the RAID rebuild completes.

  • Date - 09/10/2012 16:12 - 09/11/2012 16:22
  • Last Updated - 09/10/2012 16:13
Drive Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Kipling
  • We have found 2 bad drives in the RAID array, we are replacing them 1 by 1. During this time you may experience sub-optimal performance on the server.

    Update: The 1st drive has been replaced and the RAID rebuild completed, we will now proceed with the 2nd drive.

    Update: This has been completed

  • Date - 08/20/2012 08:14
  • Last Updated - 08/23/2012 09:57
Server Inaccessible (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • The server is currently inaccessible, we are checking it.

    Update: Server has been rebooted and is now back online

  • Date - 08/16/2012 09:41
  • Last Updated - 08/16/2012 09:51
Degraded RAID Array (Resolved)
  • Priority - High
  • Affecting Server - Kipling
  • One of the disks in the RAID array is failing, which is causing instability on the server. We have contacted the DC to get the bad drive replaced.

    Update: Apologies for the lack of updates, still going through the RAID rebuild process which by nature is very resource intensive until it's done, and can take a few hours to complete.

    Update: We've brought httpd back online so sites are now loading again, RAID rebuild is 72% complete so the server will be slower the usual until it completes.

    Update: The RAID rebuild on KIPLING server is now complete!

  • Date - 07/23/2012 10:55
  • Last Updated - 07/23/2012 20:59
Kernal Upgrades (Resolved)
  • Priority - Medium
  • Affecting Other - All Shared Servers
  • We will be updating the kernals on all shared servers during the early hours (EST and GMT) or Friday July 20th. To complete the kernal upgrades each server will need to be rebooted, which will cause sites and email to go offline until the server has booted up in the new kernal. Downtime should be no more than a few minutes, unless we run into issues.

    The servers will be upgraded one by one and we will be starting work at 3.30am GMT (12.30am EST), servers to be upgraded are:

    • Baracus
    • Bullseye
    • Goron
    • Hermes
    • Hyrule
    • Kipling
    • Nash
    • Shaw
    • Wilde

  • Date - 07/20/2012 00:30 - 07/23/2012 11:06
  • Last Updated - 07/19/2012 03:15
Server Unreachable (Resolved)
  • Priority - High
  • Affecting Server - Baracus
  • The server is currently unreachable, we are checking it.

    Update: We are not able to access the server via IPMI either, so we are having to escalate the issue to the datacenter, we will update this page when we get more info.

    Update: The server is now back online, apologies for any disruption caused.

  • Date - 07/13/2012 03:10 - 07/13/2012 05:33
  • Last Updated - 07/13/2012 05:33
Server Inaccessible (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • The server is currently inaccessible, we are checking it.

    Update: Server is back online after a reboot

  • Date - 06/30/2012 04:36
  • Last Updated - 06/30/2012 04:43
Sites not resolving (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • There appears to be an issue with HTTP and/or Named on the server, we are checking it.

    Update: BIND appears to have broken beyond repair, so we've switched to NSD for the time being to get sites back online. We'll open an issue with cPanel regarding BIND and switch back from NSD > BIND once we hear back from cPanel.

    Update: We have resolved the issue with BIND and have reverted back to it, please open a support ticket if you have any issues.

  • Date - 06/18/2012 16:00 - 06/18/2012 16:51
  • Last Updated - 06/18/2012 16:51
Server Unreachable (Resolved)
  • Priority - High
  • Affecting Server - Baracus
  • The server baracus.hyliahub.com is currently unreachable due to a network wide issue at the UKServers datacenter, they are aware of the issue and are working to bring the network back online ASAP.

    Update: This has now been resolved

  • Date - 05/29/2012 03:41 - 05/29/2012 04:09
  • Last Updated - 05/29/2012 04:09
Nameservers Not Responding (Resolved)
  • Priority - High
  • Affecting Server - Hermes
  • The nameservers on the server hermes.hyliahub.com are not responding, as such sites are not loading. We are working on it.

    Update: We are unable to get the nameservers to respond on any IP's, so have contacted cPanel regarding the issue. We will keep this page updated when we receive further information.

    Update: We found that an automated named.conf update broke the nameserver configuration on the server and caused it to stop responding to external NS queries. We have reverted to a backup of the named.conf file and the nameservers now resolve again and sites are back online.

  • Date - 05/19/2012 03:34 - 05/19/2012 04:26
  • Last Updated - 05/19/2012 04:25
DDOS Attack (Resolved)
  • Priority - High
  • Affecting System - Atlanta Datacenter
  • Servers in the Atlanta datacenter were unavailable for a few minutes earlier this evening when the data was under attack from a massive DDOS. This has been nullified and the network is accessible again, apologies for any inconvinience caused.

  • Date - 05/16/2012 16:37 - 05/17/2012 08:45
  • Last Updated - 05/16/2012 16:39
UK Network Issue (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • Update: This should now be resolved, please let us know if it persists.

    We have had reports of intermittent connectivity issues along certain transit routes with certain ISP's when accessing certain UK servers. The DC is checking on this now and we will update you when we get further information.

  • Date - 04/26/2012 04:05
  • Last Updated - 04/26/2012 10:01
Drive Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Kipling
  • Update: Drive replacement and RAID rebuild is now completed

    Update: RAID rebuild is 90% complete

    Update: The drive has been replaced and the RAID rebuild is currently in progress, it is currently 54% complete

    Update: This has been pushed back to the early hours of Sunday Morning (15th April)

    One of the drives in the servers RAID array is degraded, so we are replacing the failing drive in the early hours of Saturday morning (EST). The server will be taken down to replace the drive and when back online the RAID array will be rebuilt, which will cause sub-optimum performance until the RAID rebuild has been completed.

  • Date - 04/15/2012 02:00
  • Last Updated - 04/16/2012 16:06
Atlanta Network Maintenance - 14th April (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • One of our upstream providers (cogent) is doing some maintenance on their circuit April 14th.

    Start time: 3:00 am eastern 04/14/2012
    End time: 6:00 am eastern 04/14/2012

    We are taking steps to work around this and clients should not be affected but we want to make sure this is posted so all can be aware if anything changes.

  • Date - 04/13/2012 15:43 - 04/15/2012 03:54
  • Last Updated - 04/13/2012 15:43
UK Network Issue (Resolved)
  • Priority - High
  • Affecting Other - Network
  • Update: The official cause of the outage we received from the DC was:

    "The Datacentre suffered an unplanned network disruption at 11:24, 11/04/2012, they have yet to find the cause but believe it to be unrelated works at Telehouse London. Whilst re-routing the backup feed, their connection was restored"

    They will also be installing secondary routing equipment to handle the instant fall back to the backup feed, as clearly this would have prevented today's outage.

    Update: The network is now back up, it was down for around 30 mins.

    There appears to be an issue with the UK network, effecting the shared/reseller server:

    Nash.hyliahub.com as well as clients with UK based VPS's on the IP ranges 178.239.167.*

    We have contacted the DC about this.

  • Date - 04/11/2012 05:29 - 04/13/2012 11:05
  • Last Updated - 04/13/2012 11:05
Sporadic Outages (Resolved)
  • Priority - High
  • Affecting Other - Atlanta Network
  • 5th May Outage Cause: The datacenter have provided us with the following information regarding the Atlanta network outage experienced on May 5th:

    "There was maintenance performed by Cogent. Thus we had to close the layer 2 link to Cogent. They anticipated the completion of the maintenance by 7:00 EST.

    Cogent's maint completed, the circuit was restored, the circuit was activated and all is functioning normally."

    Update, Thurs May 5th: The network issue reappeared for around 45 mins this morning meaning we were unable to access our client area to answer tickets. Updates were posted on our Facebook and Twitter pages. Apologies for this unacceptable Atlanta network performance of the past few days, unfortunately it is out of our control and we are at the mercy of the datacentre, we will be taking it up with their management to see if we can get any further updates for you.

    Update: The following has just been sent to us by the datacenter:

    -- Begin DC Update --

    "There is a routing loop between some routers which belong to to the service providers through which the packets are routed to us. This is mainly related to not-fully propagated routes across the service providers around the world. Thus if a packet goes through the network which does not have updated records, it loops until the TTL of this packet is expired.

    We are working on this issue along with the DC techs and will try to get it resolved as quick as possible. The updates will follow on our forums.
    The network loop is noticed through nLayer, Tinet.net and perhaps others are involved. That means if packets from your end-devices are routed through these carriers, the packets will be looped until TTL of the packet is expired.

    We still work along with the DC techs to get this resolved but everything is fine inside our network. We just need the routes to get fully propagated to other service providers around the world."

    -- End DC Update --

    We are experiencing sporadic outages due to a network issue at the Atlanta Datacenter. This appear to be effecting a handful of servers and is causing intermettent connections.

    The status updates provided by the DC indicate the issue is caused by a routing loop effecting multiple IP ranges, they are working on it at present. As the servers are all up, and the issue is at the DC end, then unfortunately we will need to wait for their updates regarding this, as it's effecting their entire network, not just our servers. So we will post updates here as and when we get them.

    This is only effecting servers located in Atlanta, US. Servers in the UK, Miami, California, Dallas and Pennsylvania are uneffected.

  • Date - 04/03/2012 05:06
  • Last Updated - 04/05/2012 09:53
Kernal Updates (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple Servers
  • The following servers will have their Kernals updated in the early hours of Wednesday 21st March. These are both UK servers, so the upgrade will take place in the early hours, GMT:

    Nash

    Baracus

    The servers will be rebooted to complete the upgrade, downtime will be 10 - 20 mins.

    Update: These have both been completed

  • Date - 03/21/2012 23:00 - 03/21/2012 06:31
  • Last Updated - 03/21/2012 06:30
Kernal Upgrades (Resolved)
  • Priority - Medium
  • Affecting System - Multiple Servers
  • Update: Hyrule came back online earlier today after a hard reboot, apologies for any inconvinience caused.

    Update: Onsite techs are checking on Hyrule

    Update: This has been completed, although Hyrule hasn't come back online after the upgrade. We expect it's running a FSCK before it can come back online but have contacted the DC to confirm, will update when more info is available.

    We are updating the kernal on the following servers:

    Bullseye
    Goron
    Hermes
    Hyrule
    Kipling
    Shaw
    Wilde

    The servers will be rebooted to complete the update, so will be down for 10 - 20 minutes whilst they boot back up.

  • Date - 03/14/2012 04:46
  • Last Updated - 03/14/2012 14:13
Kernal Upgrades (Resolved)
  • Priority - Medium
  • Affecting System - Multiple Servers
  • We are currently upgrading the Kernals on the following servers, each server will need to be rebooted to complete the upgrade. Downtime will be 10 - 20 minutes per server:

    Bullseye
    Goron
    Hermes
    Hyrule
    Kipling
    Shaw
    Wilde

  • Date - 03/01/2012 03:46 - 03/02/2012 03:34
  • Last Updated - 03/01/2012 03:47
Server Migration (Resolved)
  • Priority - Medium
  • Affecting Server - Nash
  • We will be migrating data on the server nash.hyliahub.com to new hardware on Monday February 20th.

    The migration is expected to take between 3 - 5 days.

    We will notify all clients via email when this has been completed, no downtime is expected.

    Update: We are now running the final rsync and will be pointing IP's to the new server shortly

    Update; The final rsync has overloaded the server, we've temporarily stopped httpd and mysql so that it completes quicker

    Update: We have postponed the final rsync until tonight, we'll complete the changeover in the early hours of tomorrow morning to avoid disruption for you

    Update: The migration has now been completed, please let us know if you have any issues

  • Date - 02/15/2012 05:13 - 02/23/2012 08:16
  • Last Updated - 02/23/2012 08:16
Server Load (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • The server is currently running sluggishly due to a large number of MySQL queries in "locked" status, causing high memory usage. We are working on this now.

    Update: This has now been resolved

  • Date - 02/09/2012 10:15
  • Last Updated - 02/09/2012 10:42
Server Down (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • Server has gone unresponsive, we are checking it.

    Update: Server isn't atcually down, but memory usage is high which is causing the sluggishness, we are working on it.

    Update: Load has been brought under control and sites are loading normally. Please let us know if you have any further issues.

  • Date - 01/23/2012 09:12
  • Last Updated - 01/23/2012 09:30
MySQL 5.1 Upgrade (Resolved)
  • Priority - Low
  • Affecting Server - Goron
  • MySQL will be upgraded to version 5.1 at 3am (EST), on Friday 20th January.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

  • Date - 01/20/2012 03:00
  • Last Updated - 01/20/2012 03:52
MySQL 5.1 Upgrade (Resolved)
  • Priority - Low
  • Affecting Server - Hermes
  • MySQL will be upgraded to version 5.1 at 3am (EST), on Saturday 21st January.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

  • Date - 01/21/2012 03:00 - 01/21/2012 03:13
  • Last Updated - 01/18/2012 11:02
MySQL 5.1 Upgrade (Resolved)
  • Priority - Low
  • Affecting Server - Hyrule
  • MySQL will be upgraded to version 5.1 at 3am (EST), on Monday 23rd January.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

  • Date - 01/23/2012 03:00 - 01/23/2012 03:22
  • Last Updated - 01/18/2012 11:01
MySQL 5.1 Upgrade (Resolved)
  • Priority - Low
  • Affecting Server - Kipling
  • MySQL will be upgraded to version 5.1 at 3am (EST), on Tuesday 24th January.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

  • Date - 01/24/2012 03:00 - 01/24/2012 03:28
  • Last Updated - 01/18/2012 11:00
MySQL 5.1 Upgrade (Resolved)
  • Priority - Low
  • Affecting Server - Nash
  • MySQL will be upgraded to version 5.1 at 3am (GMT), on Wednesday 25th January.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

  • Date - 01/25/2012 22:00 - 01/25/2012 03:55
  • Last Updated - 01/18/2012 10:59
MySQL 5.1 Upgrade (Resolved)
  • Priority - Low
  • Affecting Server - Shaw
  • MySQL will be upgraded to version 5.1 at 3am (EST), on Thursday 26th January.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

  • Date - 01/26/2012 03:00 - 01/26/2012 03:25
  • Last Updated - 01/18/2012 10:58
MySQL 5.1 Upgrade (Resolved)
  • Priority - Low
  • Affecting Server - Wilde
  • MySQL will be upgraded to version 5.1 at 3am (EST), on Friday 27th January.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

  • Date - 01/27/2012 03:00 - 01/27/2012 05:34
  • Last Updated - 01/18/2012 10:57
MySQL 5.1 Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • MySQL will be upgraded to version 5.1 at 3am (EST), on Wednesday 18th January.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

    Update: This has now been completed

  • Date - 01/18/2012 03:00 - 01/18/2012 04:00
  • Last Updated - 01/18/2012 04:28
Server Unresponsive (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • Server has gone unresponsive, we are checking it.

    Update: Server is back online

  • Date - 01/16/2012 03:32
  • Last Updated - 01/16/2012 03:58
MySQL 5.1 Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Baracus
  • MySQL will be upgraded to version 5.1 at midnight (GMT), on Saturday 14th November.

    Once MySQL has been upgraded, PHP will need to be rebuilt, during this time MySQL driven sites will show a 500 internal server error. The PHP rebuild will take no longer than 1 hour.

    Update: This has been completed

  • Date - 01/14/2012 00:00 - 01/15/2012 05:04
  • Last Updated - 01/15/2012 05:04
Server Sluggishness (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • Load on this server is very high, causing it to be very sluggish, we are checking it now.

    Update: We have temporarily stopped httpd on the server to enable us to investigate what is causing the load

    Update: We've found the drives have stopped running as SATA and are running as PATA, which is causing the performance issues. We'll modify the kernal and reboot the server with the drives running as SATA which will should solve the issue. We are keeping httpd offline for the moment as it will enable us to work faster on the server, without experiencing the sluggishness.

    Update: The issue has now been resolved.

  • Date - 01/12/2012 03:25
  • Last Updated - 01/12/2012 16:06
Server Reboot (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • Server is unresponsive, we have rebooted it. Updates to follow.

    Update: Server is now back online, please open a support ticket if you have any issues

  • Date - 01/09/2012 04:17
  • Last Updated - 01/09/2012 04:44
Drive Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Kipling
  • We will shortly be taking the server down to replace a failing drive in the RAID array. Once the drive is replaced, the RAID array will be rebuilt, this is a resource intensive process and may result in a bit of sluggishness until it has been completed.

    Update: This is now complete, there was no downtime

  • Date - 01/05/2012 11:46 - 01/05/2012 17:07
  • Last Updated - 01/05/2012 17:07
Scheduled Network Maintenance (Resolved)
  • Priority - Low
  • Affecting Other - Network
  • We have the network maintenance scheduled between 2011-12-30 00:00:00 EST and 2011-12-30 01:00:00 EST in our DC in Atlanta.

    Maintenance will be performed on hardware carrying some Atlanta upstream connections on Friday, December 30th. This maintenance will include a brief outage on network connections to PCCW/BtN. Downtime is not anticipated, but some sub-optimal routing may be experienced as traffic is shifted between providers.

  • Date - 12/30/2011 00:00 - 01/05/2012 11:49
  • Last Updated - 12/22/2011 09:02
Server Not Responding (Resolved)
  • Priority - High
  • Affecting Server - Kipling
  • Around 10 minutes ago the server became unresponsive, it's still pinging but we are unable to access the box, so have contacted the DC. Updates to follow.

    Update: The server is now back online, please open a ticket if you are still facing issues

  • Date - 12/20/2011 03:23
  • Last Updated - 12/20/2011 04:55
Kernal Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Nash
  • The server will be rebooted at 2.30am GMT to complete a kernal upgrade. This update is needed to resolve the performance issues you have been experiencing lately - one of the drives is being recognized as an IDE drive rather than SATA which is causing a large IO bottleneck, the kernal upgrade will fix this issue.

  • Date - 12/14/2011 00:00 - 12/14/2011 03:23
  • Last Updated - 12/13/2011 05:51
Server Reboot (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • Server has become unresponsive and has been rebooted, updates to follow.

    Update: Server is now back online, it'll take a few more minutes for sites to start loading whilst the services all restart

  • Date - 12/12/2011 03:21 - 12/13/2011 05:51
  • Last Updated - 12/12/2011 03:29
Server Slowdown (Resolved)
  • Priority - Medium
  • Affecting Server - Nash
  • The server nash.hyliahub.com is currently suffering performance issues, and this is causing the slow/non loading of websites. The server is up but running very slowly, we are working on it.

    Update: The cPanel backup script was causing the load issues, we are manually killing off the proccesses

    Update: Load is now coming down, if you give it 10 minutes or so everything should be back to normal.

    As a sidenote, new hardware for a new-Nash server was shipped from the US this past Tuesday, so as soon as we get it set up in our UK DC, we'll be migrating all users away from the old (slow!) hardware, to bring it inline with our other UK and US servers.

  • Date - 12/08/2011 04:04 - 12/08/2011 04:57
  • Last Updated - 12/08/2011 04:31
Scheduled Rebbot (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • We will be rebooting the server in the early hours of Thursday November 3rd to boot it into the CloudLinux kernal. Downtime will be 10 - 30 minutes.

    Update:The server isn't booting into the new kernal so the DC techs are currently working on it

    Update: It is still being worked on and we are awaiting updates from the DC

    Update: Still awaiting more news, apologies we cannot provide you with more information, the DC are not being forthcoming with it which we find totally unacceptable, but that's something we will have to take up with management once the server is back online. We will try and get them to provide us with KVM access so we can work on the server ourselves.

    Update: There was a problem with grub on the server. The reinstallation of grub resolved the part of the problem. Grub could not find the kernel files which was the second part of the problem which had to be fixed as well. As it was being done through KVM remotely, it took a bit longer to rectify the issue. Now the server is up but we will need to change the network settings to bring it back on the network as it was connected to PXE VLAN to get it loaded on rescue image.

    Update: Server is now back online and sites accessible, please let us know if you continue to have any issues.

  • Date - 11/03/2011 00:00 - 11/03/2011 07:35
  • Last Updated - 11/03/2011 07:34
Failed Services (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • Final Update: All sites should now be back online, if you are still having problems please can you open a support ticket and we will check it out for you. Apologies for the extended downtime, we will send an incident report to all affected Clients in the next couple of days.

    Several services on Nash have failed, including httpd, cPanel and MySQL. We are aware of the issue and are working on it. We will update this report with more info as we get it.

    Update: We have found an issue with one of the drives, so we are rebooting the server and forcing a FSCK. During this time the server will be offline, the FSCK can take anything from 30 mins to a number of hours depending how much data there is to check. We will update you with more info when available.

    Update: The RAID card is failing to be detected, we are replacing it with a new one

    Update: The main drive is completely dead and cannot be found by the server. We have replaced it and we are installing the OS on it. Once that is done we will install cPanel and we will restore the data from the backup drive.

    Update: Server is back online, we are now proceeding to install cpanel on the server.Please hold on

    Update: Data is being synced, please hold on.

    Update: As there are around 468 accounts to be copied over and the size of data to be copied is high, it may take a few more hours. As of now 50GB of data in /home directory have been copied. We estimate there is around 100 GB left to be copied.

    Update: 106 GB has now been copied, we estimate there is just under 50 GB left to go

    Update: The sync is almost complete, accounts are being syncd in order and we are currently on the letter "Y". Once the sync is complete we will repair any permission errors and that should then see sites come back online.

    Update: The sync has been completed and some sites back online. Others showing either an Internal Server error, or a 403 Forbidden message will need file permissions fixing. We are currently running a script that will do this, like the sync, it will run through the accounts in alphabetical order.

    Update: All accounts with a cPanel username starting with A - D should now be back online

    Update: Still having issues with the permissions, we are working on this and will update you soon.

  • Date - 10/31/2011 10:28 - 11/02/2011 04:06
  • Last Updated - 11/01/2011 12:02
Scheduled Network Maintenance- 03:00 EST thro (Resolved)
  • Priority - Medium
  • Affecting System - Network - Atlanta
  • We have been notified by the datacenter that there is a scheduled network maintenance that will be carried out 03:00 EST through 07:00am EST Sunday, Oct 30th.

    Further details follow:

    Maintenance Notification:
    Facility: Atlanta
    Time: 03:00 EST through 07:00am EST Sunday, Oct 30th
    Anticipated impact: 30 minutes

    Description:
    Upstream scheduled maintenance

    Impacted services:
    Atlanta Cogent (Silver Network) upstream connections

    Details of Action:
    We have been informed of scheduled maintenance that will briefly impact our Cogent upstream connection on the Silver network in Atlanta on Sunday, October 30th. During this time period customers on the Silver network may experience some sub-optimal routing as traffic shifts to other providers.

  • Date - 10/30/2011 00:00 - 10/31/2011 05:32
  • Last Updated - 10/29/2011 04:52
IP's Not Resolving (Resolved)
  • Priority - High
  • Affecting Server - Hermes
  • Some of the IP's on hermes.hyliahub.com have gone down which is causing some sites to stop loading, we are currently working on the issue.

    Update: All IP's are now back up apart from h209.217.246.141, we are working on it

  • Date - 08/27/2011 04:17 - 08/27/2011 05:59
  • Last Updated - 08/27/2011 04:39
Change switch port (Resolved)
  • Priority - High
  • Affecting Server - Shaw
  • We'll be taking the server down to put it onto a new switch port that can be managed remotely, during this time websites will be unavailable.

    Downtime should be 30 mins - 1 hour.

    The work will be done at off peak hours for minimum disruption.

    Update: We've had to extend the downtime window as we've run into issues with the server after it's been hooked up in the new rack, we are working on it.

    Update: The server is now back online. However, changes need to be made to the Network Switch and the Engineers will be carrying this out shortly

  • Date - 08/03/2011 00:00 - 08/04/2011 08:56
  • Last Updated - 08/03/2011 12:56
Hardware Check (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • We will be taking the server down at 1am EST on Wednesday August 3rd to perform a check on the RAM, and replace it if need be.

    This should resolve the issues users have been facing over the past few days, downtime is expected to be 2 - 4 hours.

    Update: We've had to extend the downtime window as we've run into issues with the server. The memtest failed to complete and the system shut itself down, we believe it's either due to the motherboard or faulty RAM slots. We will get the server back online and then do more checks at off peak hours.

  • Date - 08/03/2011 01:00 - 08/04/2011 08:55
  • Last Updated - 08/03/2011 05:42
Issue on Wilde Server (Resolved)
  • Priority - Critical
  • Affecting Server - Wilde
  • We are aware that there are issues on Wilde server and have reported this to the data center. We will update when we have more information

    The server was not responding to SSH requests. It has been rebooted and is now responding

  • Date - 08/01/2011 14:24 - 08/01/2011 17:04
  • Last Updated - 08/01/2011 17:04
Performance Issues (Resolved)
  • Priority - Medium
  • Affecting Server - Wilde
  • We are aware of performance issues on the Wilde server, causing websites to load very slowy. We are aware of the issue and are checking it now.

  • Date - 07/31/2011 04:59 - 08/01/2011 04:29
  • Last Updated - 07/31/2011 05:00
Issue on Wilde Server (Resolved)
  • Priority - Critical
  • Affecting Server - Wilde
  • We are aware that there is an issue on Wilde server we are in touch with the data center to fix this. We wil update when we have more information

    This issue is now resolved and the server has been rebooted.

  • Date - 07/30/2011 14:37 - 07/30/2011 16:02
  • Last Updated - 07/30/2011 16:02
Issue on Nash Server (Resolved)
  • Priority - Medium
  • Affecting Server - Nash
  • We are aware that there is an issue on Nash server and have contacted the Data Center for more information and will up-date when we have more information.

    Update: Server has been rebooted and is back online

  • Date - 07/25/2011 17:01
  • Last Updated - 07/25/2011 17:13
Server Reboots (Resolved)
  • Priority - Medium
  • Affecting Other - All shared servers
  • We will be rebooted all of our shared servers onto a new Kernal in the coming days, so during the reboot websites will be unavailable. Downtime shouldn't be more than 30 mins - 1 hour.

    We'll be doing the reboots at off peak hours to cause as little disruption as possible.

    Update: This has now been completed

  • Date - 07/20/2011 05:32 - 07/30/2011 00:00
  • Last Updated - 07/21/2011 03:25
Scheduled Network Maintenance (Atlanta) (Resolved)
  • Priority - Medium
  • Affecting Other - Network
  • Maintenance will be performed on some of our core hardware on Thursday, July 21st, requiring a restart of this hardware. This will impact traffic passing through several providers causing traffic to reroute through other upstream connections. It will also impact traffic passing through nLayer network, causing rerouting on this network as well. Other upstream will not be impacted. Some disruption may be experienced during this maintenance as rerouting takes effect.

    Work on this issue is scheduled between 2011-07-21 23:00:00 EST and 2011-07-22 01:00:00 EST.

  • Date - 07/21/2011 23:00 - 07/22/2011 13:52
  • Last Updated - 07/19/2011 03:52
Sites loading cgi-sys/defaultwebpage.cgi (Resolved)
  • Priority - High
  • Affecting Server - Goron
  • We are aware that some sites are not loading correctly on Goron, and are instead loading cgi-sys/defaultwebpage.cgi

    We are checking this issue now.

    Update: This has been resolved, apologies for any inconvinience caused.

  • Date - 07/04/2011 10:16 - 07/04/2011 12:03
  • Last Updated - 07/04/2011 12:02
UK Server Nash (Resolved)
  • Priority - Critical
  • Affecting Server - Nash
  • We are aware that there are currently problems on the UK Server, Nash. We have contacted the data center about this for more information and will update as soon as we receive the info. Thank you for your patience

    This issue is now resolved and all sites on Nash should be back up

  • Date - 06/26/2011 10:17 - 06/26/2011 12:35
  • Last Updated - 06/26/2011 12:35
cPanel Not Loading (Resolved)
  • Priority - Medium
  • Affecting System - Multiple Servers
  • We are aware of cPanel loading issues on the following servers:

    • Bullseye
    • Goron
    • Hyrule
    • Hermes
    • Shaw
    • Wilde

    This appears to be due to a NFS being down and as such cPanel is hanging when trying to connect to it. We have opened a ticket with the datacenter about this and should have it resolved soon.

  • Date - 05/25/2011 14:28
  • Last Updated - 05/25/2011 16:19
IP Resolution Issue (Resolved)
  • Priority - Critical
  • Affecting Server - Nash
  • The nameserver IP's for Nash are currently not responding and as such sites are not resolving, we are working on this.

    Update: The datacenter will take a bit of time to figure out the resolution issue for the nameserver IP's, so to speed things up we've changed the IP's to new ones and your sites should now be back up. If you are still unable to access please clear your cache, or try accessing from a different browser. If you continue to have issues please open a support ticket.

  • Date - 05/11/2011 04:53
  • Last Updated - 05/11/2011 05:33
Upcoming Network Downtime (Resolved)
  • Priority - Medium
  • Affecting System - Network
  • There will be around 5 minutes of downtime on the network later on today whilst we make some routing/subnetting adjustments, this will effect shared and reseller customers hosted on our US based servers.

  • Date - 03/14/2011 06:14 - 03/16/2011 04:37
  • Last Updated - 03/14/2011 06:15
Shaw (Resolved)
  • Priority - High
  • Affecting Server - Shaw
  • The "Shaw" server hasn't come back up after last nights quota recalculation, we have found that it's currently in a "locked" state - we have contacted the DC about this.

    Resolved at 17.30 GMT

  • Date - 02/26/2011 05:49
  • Last Updated - 02/26/2011 13:25
Quota Recalculations (Resolved)
  • Priority - Medium
  • Affecting Other - Servers: Bullseye, Hyrule, Shaw
  • We need to recalculate the quota on the servers Bullseye, Goron, Hyrule and Shaw as they are showing incorrect disk usage data.

    We are going to be doing this during off peak hours; 1am EST on Saturday 26th Feb, and downtime for each server should be no more than 30 minutes.

  • Date - 02/25/2011 00:00 - 02/26/2011 13:27
  • Last Updated - 02/25/2011 15:55
Nash Server Down (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • It would seem that there is a problem with the server, Nash, currently. We are in touch with the data centre to try and ascertain the problem and to get an estimated time of resolution.

    We will update this as regularly as we receive updates.

    00.45 GMT

    The Data Centre became aware of power issues affecting UK servers. The power has now been restored and the servers are now coming back online. We will keep you updated here when Nash is fully powered and retored.

  • Date - 02/03/2011 18:22 - 02/04/2011 01:43
  • Last Updated - 02/03/2011 18:49
Quota Rebuild (Resolved)
  • Priority - Medium
  • Affecting Server - Goron
  • The quota on the Goronserver has become damaged and needs to be manually rebuilt.

    During this time websites and email will be down.

    We've scheduled this for 1am EST (6am GMT) on Wednesday 3rd November to cause minimum disruption for users.

    Update: This has been completed

  • Date - 11/03/2010 00:00 - 02/03/2011 18:26
  • Last Updated - 11/03/2010 04:42
Quota Rebuild (Resolved)
  • Priority - Medium
  • Affecting Server - Hermes
  • The quota on the Hermes server has become damaged and needs to be manually rebuilt.

    During this time websites and email will be down.

    We've scheduled this for 1am EST (6am GMT) on Wednesday 27th October to cause minimum disruption for users.

    Update: This has now been completed

  • Date - 10/27/2010 00:00 - 10/27/2010 06:26
  • Last Updated - 10/27/2010 06:26
Credit Card Gateway Issues (Resolved)
  • Priority - Critical
  • Affecting Other - Card processing
  • We've had reports from customers about credit card payments failing to complete, we've found that this is an issue with our merchant gateway and the payment module our system uses. We are currently working on fixing the issue.

  • Date - 10/01/2010 10:24 - 10/01/2010 13:21
  • Last Updated - 10/01/2010 10:25
Server Reboots (Resolved)
  • Priority - Critical
  • Affecting Other - All Servers
  • As any Linux geeks will already be aware, a new Kernel vulnerability has just been discovered which can cause big security issues if it isn't patched.

    To stop this from happening to us (and you!) we are going to be patching the kernels on all affected servers right away, to complete the kernel upgrade/patch the servers will need to be rebooted, during this time your sites and email will appear offline.

    The servers usually only take a couple of minutes to boot back up, but can take up to 1 hour in some cases.

    We've got this scheduled to take place at 1am EST on Weds 22nd September to cause minimum disruption for you.

  • Date - 09/21/2010 16:23 - 09/22/2010 03:19
  • Last Updated - 09/21/2010 16:27
Intermittent Connections (Resolved)
  • Priority - High
  • Affecting Other - Network
  • Some users have opened tickets reporting websites being unavailable and connections dropping out, this is only effecting certain users, so if this is effecting you, could we please ask you to do the following on your computer, and then send the results to our technical support department so we can track down the issue:

    To perform a tracert on a Windows machine:

    Start > Programs > Accessories > Command Prompt

    Type the following command and hit the "enter/return" key:

    tracert your_domain_here.com

    Can you then copy/paste the results of the test and send them to us via a support ticket.

    To copy the results to your clipboard, right click anywhere in the black space and choose "Select all", then push the enter/return key on your keyboard. You can then come back into this ticket, and push CTRL + V on your keyboard to paste in the results.

    Update: This issue has now been resovled

  • Date - 09/12/2010 14:56
  • Last Updated - 09/13/2010 03:49
Server Reboot (Resolved)
  • Priority - Medium
  • Affecting Server - Wilde
  • We are going to be rebooting this server at 1am EST to complete the setup of a NFS. During the reboot sites will be unavailable, it should be for more than 20 - 30 minutes.

  • Date - 09/09/2010 11:18 - 09/10/2010 04:28
  • Last Updated - 09/08/2010 11:19
Node Migration (Resolved)
  • Priority - Medium
  • Affecting Server - Wilde
  • We are going to be moving the server wilde.d9hosting.com onto a new hardware node to improve performance, we are going to do this at the same time as the VLAN reconfiguration.

    There is expected to around 1 hour of downtime for sites and email on the server wilde.d9hosting.com as a final rsync takes place to ensure the integrity of the data.

    This is scheduled to take place at 1am EST (6am GMT) on Thursday September 2nd to cause minimum disruption.

    Update: This has now been completed, we are now going to tweak the firewall settings on the server

  • Date - 09/02/2010 00:00 - 09/02/2010 09:04
  • Last Updated - 09/02/2010 04:09
VLAN Reconfiguration (Resolved)
  • Priority - Medium
  • Affecting Other - Network
  • We are going to make some tweaks to our VLAN configuration in the early hours of Thursday morning. Some users may experience some downtime during this time, but we don't expect this to take more than 1 hour to complete.

    This is scheduled to take place at 1am EST on Thurs 26th August to cause minimum disruption.

    Update: This has been moved back to Thursday September 2nd, at 1am EST.

  • Date - 09/02/2010 00:00 - 09/02/2010 09:04
  • Last Updated - 09/01/2010 09:09
Issues with credit card processing (Resolved)
  • Priority - High
  • Affecting Other - Payment Gateway
  • We are currently having a few issues with our credit card processing system, so we are unable to process card payments. We are aware of the issue and are working with the gateway to get it resolved.

    Update: This has now been resolved

  • Date - 08/20/2010 03:16 - 08/23/2010 10:25
  • Last Updated - 08/23/2010 10:25
cPanel & WHM Loading Issues (Resolved)
  • Priority - High
  • Affecting Other - cPanel & WHM
  • We are aware that at around 7pm GMT, clients have been experiencing loading issues when logging into their cPanel, only half of the page will load, if that.

    We are aware of the problem and are currently working to get this fixed. We will update this page when we have more information.

    This has now been resolved, please contact us if you face any further issues. 

  • Date - 08/18/2010 13:15
  • Last Updated - 08/18/2010 16:50
Sites not resolving (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • Sites are currently not resolving correctly, we are aware of this and are currently working on the server.

    Update: There was a firewall misconfiguration on the server. We have fixed this and the DNS zones are currently reloading, once the zone for a domain has been reloaded the site will appear online, so if some of your sites appear online but others don't, this is the reason. Once all the zones have been re-loaded the sites will be back online.

    Update: All sites are now back online, please let us know if you have any further issues.

  • Date - 08/09/2010 05:42 - 08/09/2010 08:17
  • Last Updated - 08/09/2010 08:15
Multiple Server Network Migration (Resolved)
  • Priority - Medium
  • Affecting Other - Network Migration
  • **The migration has now been completed and all sites running from the new servers**

     

    ==========================

    Status Update 1: A few UK users are having routing problems with the new IP's, we are aware of the problem and the DC are working on it

    ==========================

    Status Update 2: We have moved all sites back to the old servers whilst we sort out the routing issues on the IP blocks, if you still see a "Site is being moved" page, you can either edit your .htaccess file and remove the redirect statement, or open up a support ticket and a member of our team will be happy to do this for you

    ==========================

    Status Update 3: The IP routing issue has been resolved and we are now continuing with the migration, apologies for any inconvenience this has caused you so far

    ==========================

    Status Update 4: A small number of scripts are having problems with blank pages and depreciated functions, this is due to the PHP version used on the new servers, we are currently manually compiling PHP and downgrading it back to the version we had on the old servers - During this time you may encounter a few "500 internal server error" messages when visiting your sites, this is temporary and will be fixed one the recompile is complete.

    ==========================

    Status Update 5: The migration is now complete for all servers except Bullseye which is still transferring. If you find you are missing any files or websites then please let us know ASAP and we will re-sync everything for you. We are currently installing RVSitebuilder on all the servers and will then proceed with the Fantastico installation, so both of the addons should be installed today.

    ===========================

    During the next few days, the following servers will be migrated to new hardware in a new data center:

     

    wilde

    bullseye

    goron

    hyrule

    shaw

    There should be no downtime during this move, however we like to keep you all informed on what's going on just incase.

    The move to the new DC will also incorporate a new hardware approach that we've been researching for some time now, whilst there are many technical details that we wont bore you with, one of the main differences with the new hardware compared to the old is the Disk system we will be using. No longer do you have to put up with SATA drives that bottleneck at peak times, our new hardware includes enterprise SCSI drives in a RAID 10 array to bring you ultimate performance and reliability.

    We are also utilizing an external backup system that will provide us not only with account level backups, but also full file system backups so we can get back up and running much quicker if any disaster occurs!

    The geographical location of the new data center is in Florida, so any of you that are based in the Florida area may even be able to see the new machines from your bedroom window ;-)

    We will send you all an email when the migration is complete, but until then, please let us know if you have any problems.

    Kind Regards,
    D9 Hosting

    P.S. This server migration does not effect clients hosted on our UK servers or Dedicated server clients.

     

  • Date - 07/13/2010 14:28
  • Last Updated - 07/20/2010 09:13
Sluggish Servers (Resolved)
  • Priority - Medium
  • Affecting Other - Servers: Goron, Bullseye, Wilde, Shaw, Hyrule
  • We are currently running the 1st backup on these machines, as it's the 1st backup all data on the server needs to be backed up, so this is quite a resource intensive process and uses quite a lot of resources.

    When the backups finish, the load will be back to normal.

    Future backups are incremental, so the high load is only due to it being the 1st backup run, rather than something that'll be an ongoing issue.

  • Date - 07/20/2010 03:48 - 07/20/2010 09:12
  • Last Updated - 07/20/2010 03:50
MySQL & Website problems on Bullseye (Resolved)
  • Priority - High
  • Affecting Server - Bullseye
  • Update: The problem is now resolved, an abusive user was found on the server which was causing 100% disk usage and services to fail. We have suspended the user and freed up some space. The MySQL problems were due to databases becoming corrupt when there was no more disk space for them to work in, if you still face problems please let us know.

    Update: We've got sites back up and running, but we are having problems bringing up the cPanel service so you'll be unable to log into your cPanel/WHM areas, we are working on this and will update you as and when.

    A number of users are reporting "Cannot connect to Database" errors on their database driven websites, we are aware of the problem and are currently investigating.

    We have also discovered some sites are not resolving correctly, we've escalated this issue to the DC and update you when we have more info.

  • Date - 07/18/2010 05:00
  • Last Updated - 07/18/2010 11:58
httpd Service Not Running (Resolved)
  • Priority - Critical
  • Affecting Server - Goron
  • After setting the PTR record for the server and tweaking the hostname, Apache is refusing to restart and as such sites are not resolving, we are working on the problem and will have everything back up ASAP.

    We have resolved this problem, and all sites are now back up.

  • Date - 07/16/2010 12:04 - 07/16/2010 12:33
  • Last Updated - 07/16/2010 12:30
Server Down (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • The Hyrule server has become un-responsive, we are currently working on the server and will post status updates on this page.

    Update: The server has been rebooted and is now back online

  • Date - 07/12/2010 04:39
  • Last Updated - 07/12/2010 05:04
Network Issue (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • Sites on the server nash.hyliahub.com are taking longer to load than they should, this is down to a network problem at the Datacentre, we have opened a support request with them about the issue and will update this page when we have more info.

    Update: This has been resolved, please let us know if you have any further issues

  • Date - 06/27/2010 04:18
  • Last Updated - 06/27/2010 05:17
Server Not Responding (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • The server hyrule.hyliahub.com has become unresponsive. We are going to reboot the server and will update this thread when we have more information.

    Update: The server is now back online after a reboot, we are investigating to see what caused the issue

    Update: The /var partition got clogged up and became 100% full, we've cleared the partition and everything is running fine.

    Please let us know if you face any issues.

  • Date - 06/23/2010 04:35
  • Last Updated - 06/23/2010 05:00
Drive Replacement (Resolved)
  • Priority - Critical
  • Affecting Server - Bronte
  • We are going to be taking the server down to replace a drive, during this time the D9 Hosting websites will be unavailable. (The server is used for the D9 client area, no customer websites will be effected)

    This has now been completed

  • Date - 05/21/2010 09:34
  • Last Updated - 05/21/2010 10:59
Files Hacked (Resolved)
  • Priority - Critical
  • Affecting Server - Wilde
  • It has come to our attention that quite a lot of accounts on the Wilde server have been hacked, and had their index pages replaced. We are going to restore the accounts from our backups, we will update this page when it has been completed.

    Update: After some investigating, we have discovered that the hack isn't as widespread as we 1st feared, so we don't need to restore every single account. What we are doing at present is running a report to list all files that were modified on the 16th May (when the hack took place) so we have a full list of every account that has been infected, we can then go ahead and restore those accounts.

    Update: We've now located all of the hacked accounts, there are 8 in total. The hack occurred during the last backup, so we've got some clean backups from the last daily backup and some from the weekly. They are currently being restored, once restored we'll be changing the passwords on the accounts and notifying the owners via email.

    Can we also take this time to remind users to ensure all scripts are kept up to date at all times.

    Update: The restore is now complete, please let us know if you find any more defaced files.

  • Date - 05/16/2010 04:39
  • Last Updated - 05/16/2010 10:32
Houston Network Issues (Resolved)
  • Priority - High
  • We are currently facing a network issue which is effecting clients with VPS, and Hybrid servers in our Houston facility. The DC techs are aware of the problem and are currently working on it.

    All of our servers in the DC are back online, please let us know if you continue to have problems.

  • Date - 05/03/2010 08:07
  • Last Updated - 05/03/2010 09:05
Power Cable Switch (Resolved)
  • Priority - Low
  • Affecting Server - Hyrule
  • We are going to switch power cables on the rack, to connect the server to a new APC remote reboot device, during this time sites on the server will be down. Downtime shouldn't be more than a few minutes.

  • Date - 04/28/2010 01:24 - 04/30/2010 03:45
  • Last Updated - 04/28/2010 01:25
Email Issues (Resolved)
  • Priority - Critical
  • Affecting Server - Hyrule
  • There are currently problems with the email service on the Hyrule server, inbound mails are not being received. We are currently looking into this.

    Update: The issue has been resolved, an automatic Exim update caused the AV scanner to malfunction, you should now be able to receive mails. Please contact us if you face any issues.

  • Date - 04/24/2010 14:18
  • Last Updated - 04/25/2010 04:34
Server Down (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • The server wilde.d9hosting.com has stopped responding. We are aware of the problem and are currently working on it.

    Update: A file system check is currently taking place before the server can boot back up, as long as no errors are found during the check the server will continue to boot up and sites will be back online.

    Update: The server is still fsck'ing.

    Update: The fsck is now complete and the server is back online. Please accept our apologies for any inconvenience caused.

  • Date - 04/13/2010 06:42 - 04/13/2010 09:28
  • Last Updated - 04/13/2010 09:28
UK Datacentre Network Problem (Resolved)
  • Priority - High
  • Affecting Server - Nash
  • There is currently a network issue at the UK datacentre, which has been caused by the Cisco firmware on the switches. Sites were unable to resolve for a short period of time but are now back up and running and the issue has been corrected.

  • Date - 04/09/2010 12:51 - 04/09/2010 13:13
  • Last Updated - 04/09/2010 13:13
DNS problem (Resolved)
  • Priority - High
  • Affecting Server - Bullseye
  • There is currently a DNS issue on the server bullseye.d9hosting.com which is causing sites not to resolve, we are aware of the issue and are currently working on it.

    Update: There was an issue with the firewall on the server, this has been corrected and all sites are resolving again. If you continue to have problems, please clear your ISP and browser cache.

  • Date - 04/07/2010 04:34
  • Last Updated - 04/07/2010 06:33
Server Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • We are in the process of upgrading the Bullseye server, we will be moving to a new, higher spec quad core machine, with double the RAM of the old server. The data is currently being sync'd between the 2 servers, so the load on the Bullseye server will be higher than normal at present due to the sync that is taking place.

    No action is required on your part, once the data is sync'd we'll update the nameserver, and hostname IP's and sites will start to resolve from the new Bullseye server.

    Update: We had added an additional drive to the server, and the upgrade is now complete, all sites now resolve from the new server. Please let us know if you have any problems.

  • Date - 04/02/2010 09:56 - 04/06/2010 16:04
  • Last Updated - 04/06/2010 16:04
HDD Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Wilde
  • The primary drive in the server is failing, so we are going to create a snapshot of the drive, and restore it to a new drive. We will then replace the old HDD with the new one, downtime will be 10 - 15 mins whilst the drives are swapped.

    Update: This has now been completed

  • Date - 04/02/2010 09:55 - 04/06/2010 16:03
  • Last Updated - 04/06/2010 16:03
IP's Not Responding (Resolved)
  • Priority - High
  • Affecting Server - Bullseye
  • All of the IP's on the server have stopped responding, apart from the main server hostname IP. We are not yet sure if this is to do with the move to a new box we told you about earlier in the week. We are currently investigating and will let you know when we have any further information.

    Update: The issue has now been resolved

  • Date - 04/04/2010 04:10 - 04/04/2010 04:48
  • Last Updated - 04/04/2010 04:48
RAM Upgrade + Drive Replacement (Resolved)
  • Priority - Low
  • Affecting Server - Wilde
  • During the next few days we'll be upgrading the Wilde server, adding more RAM and an additional hard drive. The server will be taken offline to install the new drive and the additional RAM, down time shouldn't be more than 30 - 60 minutes.

    Update: We have now started the upgrade. (31st March, 4.15pm GMT)

    Update: This has now been completed, downtime was less than 15 minutes.

  • Date - 03/26/2010 15:35
  • Last Updated - 03/31/2010 11:17
Drive Replacement (Resolved)
  • Priority - Critical
  • Affecting Server - Bronte
  • We will be replacing a drive on the server bronte.d9hosting.com

    During the replacement the site will be offline. No client sites are hosted on the server, only the D9 Hosting websites and client area, so clients sites will remain unaffected.

    This has now been completed. 

  • Date - 03/30/2010 09:26
  • Last Updated - 03/30/2010 12:16
High Server Load (Resolved)
  • Priority - High
  • Affecting Server - Bronte
  • The server that hosts the D9 Hosting client area and websites (Bronte) is becoming overloaded, causing the D9 Hosting sites to load very slowly, if at all. We are aware of the issue and are investigating.

    No other servers are effected by this, and all clients sites are loading as normal, the problem is isolated to the D9 Hosting websites.

    Update: We've installed a new Kernal and this has rectified the issue, please let us know if you continue to have problems.

  • Date - 03/23/2010 17:19 - 03/24/2010 14:17
  • Last Updated - 03/24/2010 14:17
Rack Connectivity Issues (Resolved)
  • Priority - Critical
  • Affecting Server - Wilde
  • Hello,

    The rack that houses the server wilde.d9hosting.com is currently having connectivity issues, as such the server is unreachable. The datacentre are aware of the issue and are currently working to bring connectivity back to the rack.

    Other servers, on different racks remain uneffected.

    Update: There was a power issue with the rack, power has now been restored. We need to wait and see if there server boots gracefully or requires a file system check

    Update: Powered is restored but the server hasn't come back online gracefully, so the onsite admins are currently working on the server to bring it back online.

    Update: The server is still being worked on after failing to boot itself after the power was restored

    Update: The server is currently fscking, updates will follow

    Update: The file system check has now completed and the server is back online, thanks for your patience.

  • Date - 03/18/2010 09:18
  • Last Updated - 03/18/2010 14:11
Server Down (Resolved)
  • Priority - Critical
  • Affecting Server - Nash
  • The server nash.hyliahub.com is currently unresponsive.

    We have put in a reboot request with the DC and will update this thread when we have further info.

    Thanks for your patience.

    Update: The server is failing to come back online, the onsite NOC techs are currently checking over the server to diagnose the problem.

    Update: 12.47 GMT: The server nash.hyliahub.com is up and running fine now

  • Date - 03/13/2010 03:14 - 03/13/2010 03:14
  • Last Updated - 03/13/2010 06:49
Drive Replacement (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • One of the drives in the server is failing, therefore we must replace the drive ASAP.

    To do this the server will be taken down to have the drive replaced, the Operating System and cPanel will then be installed, and the data copied over from the bad drive.

    During this time your sites and email will be offline.

    Downtime is expected to be around 1 hour.

    We appreciate your patience during this time, and we will update this status when complete.

    This has now been completed, downtime was around 20 minutes. 

  • Date - 03/01/2010 00:00 - 03/03/2010 04:43
  • Last Updated - 03/03/2010 04:44
Changes to backup procedure (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • We have found that the server is having problems keeping up with the daily backup schedule, each backup is taking longer than a day to complete, so a new backup is starting whilst the old backup is still going on, causing an endless loop of backups, which is causing the server load to remain high at all times.

    To combat this, we have decided to set the backups to run every 3 days, rather than every day, to see how this effects the backups and the server performance.

  • Date - 02/12/2010 04:11 - 03/01/2010 08:20
  • Last Updated - 02/12/2010 04:14
Shaw server downtime (Resolved)
  • Priority - Medium
  • Affecting Server - Shaw
  • There are currently issues with the Shaw server. Our level 3 tecnicians are working on this, so please watch this space for further information

     

    Update:

    The Shaw server is actually `up' but there have been problems with faulty IPs so the IP pool is being rebuilt. We are currently waiting for an estimated timescale and will update again when we hear from the datacentre.

    Update: This issue is now resolved.

     

     

  • Date - 01/30/2010 11:20
  • Last Updated - 01/30/2010 12:21
cPanel Updates (Resolved)
  • Priority - Medium
  • Affecting System - All servers
  • We are currently updating the cPanel version on all servers to the latest stable release, during this time you may have intermittent problems when trying to access your cPanel whilst the upgrade is still in progress.

  • Date - 12/17/2009 08:00 - 12/18/2009 16:10
  • Last Updated - 12/17/2009 08:01
Domain Name Registry Problem (Resolved)
  • Priority - Medium
  • Affecting System - Domain Names
  • There is currently a network issue with the registry that is causing the following actions to fail:

    - Registering / renewing domain names

    - Changing Name Servers

    The registry is aware of the problem and is working on bringing the system back online.

    Apologies for any inconvenience caused.

  • Date - 11/19/2009 03:42 - 11/19/2009 11:02
  • Last Updated - 11/19/2009 03:44
Network Issues - Sites Down (Resolved)
  • Priority - High
  • Affecting System - Entire Atlanta Network
  • You may have noticed your sites and email have been unavailable for around 1 hour earlier today, this was a network issue and now looks to be resolved. Will post more info as soon as we get it!

    Update: The problem was down to a router failure

  • Date - 10/16/2009 12:11
  • Last Updated - 10/16/2009 17:08
Sites not resolving (Resolved)
  • Priority - High
  • Affecting Server - Bullseye
  • Sites have stopped resolving on the Bullseye server after an emergency Kernel upgrade. We are working on this issue and will have the sites back online for you ASAP.

     

    Update: The problem is due to the re-allocation of IP's in the block, the IP's which have been incorrectly allocated by the DC include the nameserver IP's as well as the main shared IP. We've escalated this to DC level to get the issue resolved.

    In the meantime you can still access your email either via our webmail service:

    https://bullseye.d9hosting.com/webmail

    Username is the email address, password is the password for that email account.

    Of if using an email client (Outlook, Outlook Express, etc) you can change the incoming and outgoing server names to: bullseye.d9hosting.com

    More updates to follow.

    Update: The network settings for the server have now been restored and all IP's now resolve as they should. Please let us know if you are still having issues and apologies for the inconvinience this has caused.

     

  • Date - 08/26/2009 03:30 - 08/26/2009 09:53
  • Last Updated - 08/26/2009 09:53
Network Issues on Hyrule Server (Resolved)
  • Priority - Medium
  • Affecting Server - Hyrule
  • 16.59 GMT There are currently issues on Hyrule server and some sites on this server are not resolving. We are working on this issue and will update as soon as possible.

     

    16.01 GMT This issue is now resolved.

  • Date - 08/06/2009 10:58
  • Last Updated - 08/06/2009 11:02
DNS Issue on Wilde Server (Resolved)
  • Priority - Critical
  • Affecting Server - Wilde
  • There is currently a DNS issue on the Wilde server, we are aware of the problem and are will update once we have more info.

    The server is still up and running but no sites will be resolving.

    *Update*

    The DNS server is up and running, but some sites are showing an Internal Server Error message and some are not resolving at all. We are continue to work on the problem.

    Thanks for your patience, we hope to have the sites back up and running ASAP!

    *Update*

    All sites should now resolve from the server, rather than bringing back a generic error message from your browser. They will however show a 500 Internal server error message, we are aware of this and technicians are still working on the issue.

    The problem was caused when a site transfer went wrong and "messed up" (technical term!) the ownership rights of a number of core files.

    *Update*

    All non-PHP files (.html, images, etc.) are now resolving as normal. Still working on the PHP config.

     

    *Update*

    The issue is now resolved, if you continue to face problems with your site or email please get in touch with us. Thankyou for your patience, it's appreciated!

    *Update*

    There are still problems with the email server failing to come up, we are aware of the issue and are working to get your email back up and running ASAP.

    *Update*

    The mail issue is now resolved. Please contact us if the problem persists.

     

  • Date - 07/24/2009 05:30 - 07/24/2009 13:08
  • Last Updated - 07/24/2009 13:08
Hard Drive Replacement (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • The backup hard drive on the Wilde server is dying, we are going to take the server down and replace the bad hard drive with a new one. Sites and email hosted on the Wilde server will be offline during this period.

    Update: This has been completed and backup started to the new drive.

  • Date - 07/10/2009 14:53
  • Last Updated - 07/10/2009 16:49
RAM Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • The Bullseye server will shortly be taken offline so we can add some more RAM to the server, downtime it likely to be between 10 - 20 minutes.

  • Date - 07/09/2009 13:56
  • Last Updated - 07/09/2009 16:25
Server Offline After Reboot (Resolved)
  • Priority - Critical
  • Affecting Server - Wilde
  • After performing graceful server reboot the server isn't coming back online, the issue is being looked into.

    Update: The server is now back online. 

  • Date - 06/25/2009 06:15 - 06/25/2009 08:21
  • Last Updated - 06/25/2009 08:13
Bullsye Taken Down For Investigation (Resolved)
  • Priority - Critical
  • Affecting Server - Bullseye
  •  

    The server Bullseye will be taken down in the next few minutes whilst we check the config. Only 1 CPU is being detected so we need to see if it's a BIOS config problem, or a problem with the CPU itself.

    Updates will follow.

    **The issue has now been resolved**

     

  • Date - 06/23/2009 12:40 - 06/25/2009 03:53
  • Last Updated - 06/25/2009 03:53
Network Upgrade (Resolved)
  • Priority - Medium
  • On Friday (19 June 2009), we are going to perform an upgrade of our core router. That will improve the following:

    - redundancy by helping position to remove single points of failure
    - network response times

    The expected downtime will be 20-35 minutes.

  • Date - 06/19/2009 00:00 - 06/20/2009 03:00
  • Last Updated - 06/18/2009 03:54
Apache Update (Resolved)
  • Priority - Medium
  • Affecting Server - Bullseye
  • We are currently performing an Apache update on the Bullseye server, during this time websites may become unresponsive. We will update this issue when complete.

  • Date - 06/05/2009 04:58
  • Last Updated - 06/07/2009 06:59
Server Down (Resolved)
  • Priority - Critical
  • Affecting Server - Bullseye
  • The Bullseye server has stopped responding, we are aware of the problem and are currently investigating.

    The Apache server encountered problems due to tempfs (a memory based system), we have fixed it and the server is up and running again.

  • Date - 04/28/2009 04:52
  • Last Updated - 04/28/2009 05:55
Hyrule Port 80 Issue (Resolved)
  • Priority - High
  • Affecting Server - Hyrule
  • There is currently an issue on the hyrule server preventing site visitors from accessing the http port (80) and this is causing your websites to appear as offline despite the server being up.

    Our admins are currently aware of the issue and are working on the server. We will let you know when the problem is resolved.

    Update: The problem is because of the shared IP address no longer resolving. Working with the Datacentre to correct the issue. Apologies for any convinience caused.

    Update: This issue is now resolved

    Update: The problem has returned this morning and we are investigating

    Update: Sites are now resolving from the server again, we will keep a close eye on the server for the next few hours to see if the problem returns.

    Update: Sites have been resolving on and off all day now so we are taking the serer down whilst a full analysis is performed. (16.28 GMT)

    Update: Sites are now resolving the problem was due to a network misconfiguration.

  • Date - 04/06/2009 17:14
  • Last Updated - 04/08/2009 12:24
Possible Hard Drive Replacement (Resolved)
  • Priority - Medium
  • Affecting Server - Wilde
  • We received a warning message stating that one of the drives on the server may be having problems. We are scanning the drives for errors and if any are found the drive will be replaced.

    Both drives have been checked and no errors have been found. 

  • Date - 04/03/2009 09:42
  • Last Updated - 04/04/2009 04:56
File System Check (Resolved)
  • Priority - High
  • Affecting Server - Wilde
  • **Latest Updates Posted At Bottom Of Page**

    The "Wilde" server is currently experiencing a much higher load than normal, we are aware of the problem and are working to resolve it. We will update the network issues page with more info when it becomes available.

    Update: The load average is now back to normal and all services are up and running, if you have any problem accessing any sites on the server please open a support ticket.

    Update: The problem is re-appearing daily at around 9am GMT when the daily backups are running. Whilst the backups are resource intensive they shouldn't be causing the problems they are, so we are continuing to look into the problem. Stopping the backups from running would solve the problem, but we feel that it's better to continue to let them run whilst we look into the issue, that way we always have a recent backup of all your sites.

    Update: A file system check is now going to be run within the next few hours, this will cause the server to be unreachable for the time when the file system check is running. We will keep this page updated with information.

    Update: We thought we had found the problem last night when we found too many IPCS processes running on the server, we killed them but the problem re-occured this morning so we are still investigating. No downtime has occured as yet.

    Update: The file system on the backup drive was extremely "messed up" (it's a technical term!) so we have reformated it. This should now resolve the problem.

  • Date - 03/26/2009 04:37 - 03/29/2009 03:04
  • Last Updated - 03/29/2009 15:05
Bullseye Downtime - LAN Card Replaced (Resolved)
  • Priority - Critical
  • Affecting Server - Bullseye
  • Users on the "Bullseye" server experienced around 3 hours of downtime on January 22nd.

    After issuing a reboot request the server failed to come back online. After inspection by our onsite admins we found a LAN card was causing the problems. This was replaced and the server was brought back online.

  • Date - 01/22/2009 11:45 - 01/22/2009 13:45
  • Last Updated - 01/22/2009 13:49
cPanel/WHM Update (Resolved)
  • Priority - Medium
  • Affecting System - All Servers
  • We are currently updating cPanel/WHM to the latest RELEASE build, during the update users on all servers may be unable to access their cPanel/WHM. We will update this notice when the update is completed.

  • Date - 12/30/2008 08:22 - 12/30/2008 08:41
  • Last Updated - 12/30/2008 08:23
Server Down (Resolved)
  • Priority - Critical
  • Affecting Server - Bullseye
  • Users on the "Bullseye" server were unable to connect to websites and email. The server was rebooted and is currently functioning as normal.

  • Date - 12/30/2008 00:00 - 12/30/2008 00:00
  • Last Updated - 12/30/2008 08:20

Server Status

Below is a real-time overview of our servers where you can check if there's any known issues.

Server Name HTTP FTP POP3 Server Load Uptime
Baracus
Bronte
Bullseye
Fleetwood
Goron
Hermes
Hyrule
Kipling
Nash
Shakespeare
Shaw
Wilde