Monkey Tree Hosting - Datacentre Incident Affecting Hosted Services – Incident details

Datacentre Incident Affecting Hosted Services

Resolved
Degraded performance
Started 6 months agoLasted about 5 hours

Affected

Web Hosting

Degraded performance from 2:43 PM to 7:13 PM

Plesk

Degraded performance from 2:43 PM to 7:13 PM

cPanel

Degraded performance from 2:43 PM to 7:13 PM

Server Hosting

Degraded performance from 2:43 PM to 7:13 PM

Hosted Apps

Degraded performance from 2:43 PM to 7:13 PM

VPS'

Degraded performance from 2:43 PM to 7:13 PM

Updates
  • Resolved
    Resolved

    August 23, 2024 20:13 BST

    [Resolved] This incident is now closed. Should any customers still be experiencing any connectivity issues, please raise a support ticket via your regular support channels. A RFO will be circulated via the ServerChoice portal as soon as it becomes available.

  • Monitoring
    Monitoring

    The teams investigating have now provided an update regarding this. Please find it below:

    August 23, 2024 18:50 BST

    [Monitoring] Resilient ports have been brought back online for most customers We will be monitoring the network over at least the next hour before closing this incident.

  • Update
    Update

    The engineer teams have reported back in to notify us that they are still working on this:

    >August 23, 2024 18:36 BST

    >[Identified] Engineers are still working on bringing all ports back online in batches.

    We do apologise for the delays in resolving this on their behalf.

  • Identified
    Identified

    Please find the latest set of updates below regarding this matter. We thank you for your patience so far:

    August 23, 2024 16:56 BST

    [Identified] When bringing up resilient ports, further connectivity issues were seen and these have been now been brought down. A further period of monitoring of no less than 30 minutes will take place before any further changes are made during this fix.


    August 23, 2024 17:31 BST

    [Identified] Engineering team are starting to bring resilient ports online in batches now.


    August 23, 2024 18:01 BST

    [Identified] The engineering team are still working on bringing back resilient ports in batches.

  • Monitoring
    Monitoring

    Please find the latest update below regarding this:

    August 23, 2024 16:43 BST

    [Identified] Customer ports on a single upstream router have now been enabled again. The team will monitor this for a further 15 minute period before bringing up resilient connections in batches.

  • Identified
    Identified

    The teams have provided us with another update just now. Please find it below:

    August 23, 2024 16:32 BST
    [Identified] Some customers primary links have been shutdown during this monitoring period. Engineers are still working through the connectivity issues

  • Monitoring
    Monitoring

    The teams that are investigating have now provided this update. Please find it below:

    August 23, 2024 16:29 BST

    [Identified] All customer ports on a single upstream router have now been enabled. The team will monitor this for a 15 minute period before bringing up resilient connections in batches.

  • Identified
    Identified

    This is the newest update provided at 16:15. The engineers there are currently working to bring these services back online. We expect this to be resolved shortly. Further confirmation will be provided soon:

    August 23, 2024 16:15 BST

    [Identified] Engineers are still working on restoring full connectivity services. Customers ports are currently being brought back online in batches.

  • Update
    Update

    The relevant teams have posted another update regarding this. Please find this below. Once we have further information regarding this, we will post it here:

    August 23, 2024 15:55 BST

    [Identified] Engineers are still working to restore connectivity to all customers.

  • Investigating
    Investigating

    We are currently aware of an issue with one of our datacentre that is affecting different servers that are provided by us. This may affect performance and access to any services, such as websites and remote applications that are currently hosted on these servers.

    This is currently being investigated and worked on as a top priority with the relevant teams. Any updates received regarding this will be posted here, including further information.

    Please find the latest update below:

    August 23, 2024 15:35 BST

    [Identified] Our engineers continue to investigate the incident and will provide further updates here.