123 reg web host deletes customer websites

[mikerr]

In a major fail, 123reg have accidentally wiped all customers private webservers (VPS), due to a badly written script !

Customers sites have been down for 30 hours, and they spent most of today and yesterday telling customers it was a "connectivity issue":

https://www.123-reg.co.uk/support/system-status/

but have now admitted to data loss:




Currently they're still scrabbling to restore "some" data ...

[BobF64]

At least they're trying to restore data.

I'd suspect that most hosting providers say you are responsible for all the data, regardless of why its lost as there is no technical difference between hardware or wetware failures, or that you can pay them for backups.

[mikerr]

Thing is, they sat on the reporting of the problem for 30 hours, just calling it "connectivity problems".

If they'd announced earlier many could have restored their own backups to a new blank VPS and had minimal < 1/2 hr downtime.

As it was thousands of websites were down and still are (some quite major) - having lost the whole weekend

However, if their backups were stored on 123reg servers - that may be gone too

[shaithis]

At least they're trying to restore data.

They might as well wound-up the company if they hadn't!

[pp05]

BBC news: link

Anyone struggling to access their website?

[pp05]

Don't they do a dry run of their scripts just to make sure it works as intended?

[DanceswithUnix]

Don't they do a dry run of their scripts just to make sure it works as intended?

Either "no", or their test environment and their production environment aren't quite the same.

[peterb]

Users of cloud services take note!

[Guy]

http://forums.hexus.net/networking-b...-vps-data.html

[peterb]

(Duplicate thread from GD merged into this one)

As always, keep backups. The people who have lost data were using an unmanaged service and neither used 123's backup service nor kept there own backups.

Moral of the story, if you entrust data to an entity outside your immediate control, keep a backup, and if you control your own data, keep a backup.

[Marcvs]

Pretty sad but funny at the same time. People laugh at me for having a quadrouple leveled backup solution.

[scaryjim]

Kind of glad I canned my VPS a while ago. Only thing 123-reg host now are my domain names and a few mail forwards. Good warning about the ephemeral nature of cloud services though; we do get used to them just being there, and I know there's a few important emails that I really should make sure I have an offline backup of....

[jessadecena]

This terrible company deserves to go bankrupt! Everything I had in my business account is gone. I lost already hundreds of revenues since disaster day! On the other hand I'm a lucky guy because I made backups on my own devices and extra on Dropbox. Yesterday I moved my backups to a new VPS provider as recommended by a few friends who have been hosting with them for years. The company seems to have the world's lowest account cancellation. Account activation was instant and the transfer was much faster and smoother than I expected.