It happened again. A power outage hit a datacentre in London, and things cascaded from there: multiple banking services faltered, court hearings were put on hold and patients reported temporary glitches on the NHS app.
The datacentre in question was in Google’s europe-west2-b zone, one of over a dozen “zones” that the US search giant maintains in Europe to service its clients.
The outages – which have affected, according to the website Downdetector, Royal Mail, Revolut, HSBC and Lloyds bank, among others, are so far not of the scale of a similar cascade this time last year, which saw Amazon Web Services go down for nearly half a day.
That outage hit a far bigger node – Amazon’s us-east1 – in a concentrated region of Virginia, which hosts the greatest density of datacentres on Earth. It saw cascading effects worldwide: taking out the Roblox game and Signal messaging app and causing smart mattresses worldwide to suddenly heat overnight and toast their occupants.
It lasted about 15 hours.
Today’s problems appeared to peak around 9am, and by mid-afternoon were somewhat resolved – according to reports submitted to Downdetector. Because Google Cloud stores data and runs operations for a number of businesses, there were widespread knock-on effects – mostly confined to UK businesses.
Revolut confirmed “a service disruption affecting multiple services”, but added “all our customers’ money and data are safe”. Lloyd’s said its app was working, but “we understand some [people] are having issues with certain online services this morning”.
HM Courts and Tribunal Services, which handles remote video hearings in courts across England and Wales, said it was “aware of an issue affecting Cloud Video Platform,” and was “working with our technology partners to restore service”.
Like the far-bigger outage of last year, it’s a reminder of the physical infrastructure – often creaking and provisional – that underpins most of the virtual world.
Michał ‘rysiek’ Woźniak, an internet infrastructure expert, said the outages were a reminder of the increasing concentration of the internet and vital services on the infrastructure of a few key companies – the “hyperscalers”.
“Hyperscalers have become too big to not fail. But worse, they’ve become so deeply ingrained into the very fabric of our digital lives that switching away from them seems unimaginable.”
The Guardian has reported that British police keep vital data on Microsoft’s Azure cloud, despite being apparently aware that this exposes sensitive information to US government insiders, hostile foreign actors, and outages like today’s. Up to 60% of the UK government’s infrastructure is hosted on cloud services, mostly Amazon, Microsoft and Google.
“I am sure Google’s engineers are doing everything they can to mitigate them, but it is unreasonable to have a handful of gigantic companies whose outages time and again cause public institutions or large swaths of the digital economy to stop working en masse,” said Woźniak.
Last year’s AWS outage was later revealed to have been caused not by a power outage, but by a bug in automation software. A separate outage was caused by an AI tool that Amazon had adopted after laying off tens of thousands of staff last autumn.
Woźniak pointed at an announcement earlier this year from Google indicating that it was also embracing AI, with 75% of all new code “now AI-generated and approved by engineers”.
“One can only wonder how that affects the reliability and quality of Google’s services. Multiple large outages at another hyperscaler, AWS, a year ago have been linked to internal AI tooling.”
Google has been approached for comment.
Disclaimer : This story is auto aggregated by a computer programme and has not been created or edited by DOWNTHENEWS. Publisher: theguardian.com
