Amazon AWS down again, this time it was a man-made disaster

Downtime is one of the nightmares of major Internet companies. Once downtime occurs, the official website and APP will not work properly, and large-scale customer complaints will follow.

However, any public cloud provider, in the course of its development history, has encountered various outages and failures due to human factors, severe lightning, power outages in the computer room, dug up optical cables, or incorrect code input.

This time, it's Amazon that's in trouble.

According to foreign media reports on October 22, Amazon was attacked by DDoS, causing some Amazon Web Services (AWS) to go down, causing customers' websites to be on the verge of collapse.

Due to the continuous interruption of AWS services caused by the attack, unfortunate netizens suffered the painful experience of intermittent failure to access AWS sites and related services on the Internet.

Honestly, this intermittent experience is more annoying than not being able to open the page at all. It feels like...

Presumably, companies that have access to AWS services are being forced to engage in “psychological warfare” with the Internet at this moment.

In response to this, Amazon's technical support agent was the first to speak out - this is not a natural disaster, it is a man-made disaster!

He said the AWS DNS servers were hampered by a distributed denial of service (DDoS) attack, in which attackers attempted to flood the system with junk network traffic, rendering the service inaccessible.

Some customers reported that the attack seemed to start at 9 a.m. US time, and Amazon's AWS services were down for about 10 hours afterwards.

In this case, Amazon's DNS system was clogged with a large number of packets, and some legitimate domain name requests were released and used to alleviate the traffic congestion.

That is, websites and applications attempt to contact Amazon's backend hosting systems (such as S3 buckets), which may fail, resulting in error messages or blank pages for users.

For example, if your web application or software attempts to communicate with your bucket at mycloudydata.s3.amazonaws.com, the DNS query that translates that readable address into an IP address might not make it through Amazon, causing the code to fail.

One solution is to insert the region of the bucket into the address, such as: mycloudydata.s3.us-east-2.amazonaws.com, so that the code can be resolved correctly. This process is not stable, once the service is running normally, it means that the cached DNS query is working properly.

“The mitigation was initiated due to intermittent DNS resolution errors caused by the attack. ”

This affects not only Amazon S3 customers, but also prevents connections to any Amazon services that rely on external DNS queries, such as Amazon Relational Database Service (RDS), Simple Queue Service (SQS), CloudFront, Elastic Compute Cloud (EC2), and Elastic Load Balancing (ELB).

These are services that countless sites and applications use to handle visitors and process customer information.

Amazon Cloud Support Console tweeted that it is currently investigating reports of sporadic DNS resolution errors. At the same time, AWS DNS servers are being attacked by DDoS, and mitigation measures are not only absorbing a large amount of such attack traffic, but also marking some legitimate customer queries, which makes it difficult to sort out the source of the attack.

Twitter wrote:

Amazon S3 customers affected by this incident can mitigate the impact by updating their configuration to specify the specific region where their buckets are located.

For example, instead of "mybucket.s3.amazonaws.com", a customer would specify "mybucket.s3.us-west-2.amazonaws.com" for a bucket in the US-WEST-2 region. If you are using the AWS SDK, you can specify the region in your Amazon S3 client configuration to ensure that requests use the region-specific endpoint name.

The DNS resolution issue also intermittently affects other AWS service endpoints that require public DNS resolution.

Amazon has not provided further information since then, with the cloud support desk tweeting that it is investigating reports of intermittent DNS resolution errors related to Route 53 and external DNS providers.

Reference source: heregister

▎Internet giants gathered in Wuzhen, what did they talk about?