05-14-2019 01:57:15 PM -0400
05-09-2019 05:01:30 PM -0400
05-09-2019 01:41:48 PM -0400
04-18-2019 10:46:35 AM -0400
04-18-2019 10:18:40 AM -0400
It looks like you've previously blocked notifications. If you'd like to receive them, please update your browser permissions.
Desktop Notifications are  | 
Get instant alerts on your desktop.
Turn on desktop notifications?
Remind me later.


Google Cloud Melts Down, Leaving Millions without Services: Nest, YouTube, iCloud, Snapchat, Others Affected

On Sunday, Google suffered a major outage affecting such sites as YouTube, Google Adsense, Google Analytics, Nest, Google Home, YouTube and apps like Snapchat.

According to a series of status updates at the Google Cloud Platform website, Google began investigating an issue that began at 12:45 Pacific. At 12:59, the company reported that the "issue is related to a larger network issue."

Approximately 30 minutes later there was another update: "The issue is still ongoing," Google explained. "We are experiencing high levels of network congestion in the eastern USA, affecting multiple service [sic] in Google Cloud, GSuite and YouTube. Users may see slow performance and intermittent errors." Areas of the West Coast also appear to be affected, and reports of outages are coming in from around the world:

IsItDownRightNow, a website that monitors web outages, is reporting widespread problems:

As of the time of this writing, the Google Cloud Platform dashboard shows outages for Google Compute Engine and Google Cloud Networking

Google says it has identified the root cause of the problem and expects to "return to normal service shortly."

Complicating matters, a Google employee reports that the outages are affecting the tools Google engineers need to communicate so they can fix the problem.

PJM first noticed the problem at around 4 p.m. Eastern, when our Google Analytics stopped reporting data:

The term "the cloud" refers to programs and services that run on the internet, rather than on your computer. If you use Gmail, for example, your email can only be accessed when you are connected to the internet. The same goes for the Internet of Things devices like Nest or the Ring doorbell.

While some of the outages, like the Google Play Store, are little more than an inconvenience, others, such as those affecting Nest and Google Home could cause major problems for homeowners who utilize those services, rendering their Smart devices useless. Users may have difficulty adjusting the lighting in their homes and may find it impossible to adjust the heat and air conditioning, for example. Worse, home protection devices may also be affected:

I received this message when I tried to access my Nest device:

Sunday's issues follow high-profile YouTube outages in January 2019 and October 2018. In addition to some Google services being offline on Sunday, other websites that rely on Google servers have also been affected, including many iCloud systems, according to Apple:

The outages could be the result of anything from human error to intentional sabotage by state or private malefactors. Sunday's problems highlight how dependent we've become on cloud services and smart defices in recent years. If one of the major providers of these services -- namely Google or Amazon -- suffers an attack resulting in large disruption of services, it could cause major problems for individual users and businesses that rely on them. Everything from banking services to public safety could be affected. When talking about terrorism, law enforcement authorities often say that the terrorists only have to be right once. The same goes for hackers. While terrorists and state-based spies employ teams of hackers looking for vulnerabilities, a single hacker who finds a way into a system could also wreak havoc on the internet, sending the world into a tailspin.

PJM will update this article as more information becomes available.

Update: 9:25 p.m. EST: Google announced this evening that the issue has been resolved: 

The network congestion issue in eastern USA, affecting Google Cloud, G Suite, and YouTube has been resolved for all affected users as of 4:00pm US/Pacific.

We will conduct an internal investigation of this issue and make appropriate improvements to our systems to help prevent or minimize future recurrence. We will provide a detailed report of this incident once we have completed our internal investigation. This detailed report will contain information regarding SLA credits.

Follow me on Twitter @pbolyard

This article has been updated to include additional information about the outages.