CS New Media - Server Status

We will be undertaking significant maintenance on key elements of our Linux hosting infrastructure on 22nd August 2008. Whilst every effort is being taken to minimise the service impact, unfortunately there is a necessity for a period of downtime during the work. We have scheduled this period to occur overnight at a weekend to reduce the inconvenience caused to all of our valued customers.

WORK TO BE PERFORMED:

This essential maintenance is across three vital areas:

1) Major reorganisation of our rackspace to facilitate further growth, improved cooling (critical for continued reliability), and additional functionality;

2) Significant network upgrades including updated network hardware, additional transit partners, and enabling us to provide enhanced network capabilities in future;

3) Following a review of our power resiliance, we will be improving this aspect of our service with more servers utilising redundant power supplies, PDU hot spares, and dual power feed options.

MAINTENANCE WINDOW:

* Start @ Friday August 22nd 2008: 22:00 BST/Western European Time/UTC+1
* End @ Saturday August 23rd 2008: 06:00 BST/Western European Time/UTC+1

Estimated maximum outage per server: 4 hours

DURING THE MAINTENANCE WINDOW:

No email will be lost during this maintenance period, all mail will be stored on our off-site backup mail server located in the USA and then re-delivered to mailboxes once all our services are back online. No loss of data will occur during this maintenance period.

Helpdesk support during this maintenance period will be limited while we focus our attention on this maintenance work. Our own email and helpdesk systems will still be available throughout this maintenance period should you need to contact us for any reason either during or after the maintenance period.

If you have any questions or concerns regarding this maintenance work please do not hesitate to contact us at
linux-support@csnewmedia.co.uk or submit a ticket via our web based helpdesk.

Thank you for your co-operation & understanding during this maintenance period as we continue to improve the quality, performance & reliability of our service.

00.01 / 22nd June 2008 - We are currently investigating an outage with Apache on the Web1 Linux Web Server. Please accept our apologies for the loss of service during this time.

UPDATE 00.07 - Apache is now back online, please accept our apologies for any inconvience this outage caused.

Our network provider has informed us that a potential hardware issue has been discovered on a crucial piece of network equipment. As a precaution this hardware will be replaced at 1 minute past midnight tonight (17th June 2008).

During this time all Linux based services including customer dedicated servers will be unavailable for up to 2 minutes.

Please accept our apologies for the short notice of this work.

05/06/08 / 15.57 - We are currently receiving a high volume of support tickets with regards to failures when sending email to @hotmail / @msn and @live email addresses. We are aware of this issue and working on a resolution.

08.45am / 3rd June 2008 - We have identified some hardware issues on Linux Web2 Server. We are performing an emergency migration to replacement hardware as a precaution now before things get busier later today. This should involve a few minutes downtime instead of risking a substantially longer period of downtime.

The downtime is expected within 30-90 minutes, and to last an estimated 5-10 minutes.

Please accept our apologies for the inconvenience.

13.20 - Monday 19th May 2008

We are currently investigating an outage at our Manchester Hosting Facility which our Linux Servers are located in.

Please accept our apologies for the loss of service, we are investigating the problem and will update this post shortly.

UPDATE: 18.30 - We believe the cause of the problem is a switch all servers are connected too, and 1 server in particular maxing the switch out and eating all the provided bandwidth. We are still working on resolving this, during this time access to our Linux servers will be intermittent. Please accept our apologies for any inconvenience this may be causing.

UPDATE: 20.12 - All services are now back on-line and we do not anticipate any further problems. We will be issuing a full RFO (Reason for Outage) within the next few days. Please accept our apologies for today’s problems.

We will be performing essential maintenance on the Linux Mail Server on Saturday 17th May 2008 from 12.00am - 2.00am. During this time the mail server will be unavailable at certain periods while updates are carried out.

On Monday 12th May 2006 between 6.30am - 7.30am , we will be adding an additional 2Gb of RAM to the current Web1 web server. During this time all services on Web1 will be unavailable for up to 10 minutes.

We are currently investigating communication issues between the Web2 Linux Web Server and the MySQL5 central database server.

Please accept our apologies for the loss of service during this time.

Friday 25th April 2008 - 15.04

We are currently looking into problems with the Web1 Linux Server, this is resulting a number of websites being unavailable.

Please accept our apologies for the inconvenience this may cause.

UPDATE: 17.15 - All sites are now back on-line. The problem was caused by a bug in the control panel software we use, Plesk. It was sending the incorrect IP address to our centralised DNS servers which resulted in a temporary period where the wrong IP was being served. We repaired this as soon as we noticed it by writing a workaround ourselves, It will not happen again, as we are now aware this bug exists and we will be reporting it to the developers shortly.