Hacker News new | past | comments | ask | show | jobs | submit login
Is AWS S3 having an outage?
93 points by GGO 3 months ago | hide | past | favorite | 44 comments
AWS status https://health.aws.amazon.com/health/status shows all green but I am having internal server errors returned from S3. Anyone else having this issue? Downdetector has spike for AWS outage coincidentally https://downdetector.com/status/aws-amazon-web-services/



AWS has posted the outage: https://health.aws.amazon.com/health/status


Something related/non-related, it's still painful to read specific timezone not UTC.


I've long wished for built-in browser functionality that converts times to the user's preferred time zone, with perhaps a dotted outline indicating that a change was made by the browser to the page.


I’d always hoped this is what HTML’s time tag would become. Unfortunately it does almost nothing.


same with gps - the user should deside how to open direction / location links.


This is worth proposing. A tag where fallback text is provided within which can be overridden by the browser with a formatted date string would be excellent.


It exists: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/ti...

As a nearby comment indicates, it's not clear any browser makes use of it.


I used the <time> element in a subscription dashboard.

All times on the page are in UTC and are wrapped in a <time> element. The raw UTC time is on the datetime attribute and a fallback more easily readable time is inside the tags. Some JS on the page removes whatever text is inside the tag and replaces it with the user's locale specific format (no libraries required, it's a part of the browser standards).

User friendly times wasn't actually the reason we implemented it.

Our largest concern was our E2E testing. We needed to make sure that the dates/times displayed were always right (and since subscriptions involve money, we wanted to make sure that part of the website was the most tested.) The E2E testing simply ignores whatever is inside the tag and reads the datetime attribute instead. Then we can fastforward or rewind the simulated time inside the test to make sure everything is working as expected.


Yeah, but, I mean, the least you could do is to say UTC-7. I'm sure everybody living in PDT knows it's, well, their time, but how the fuck should everyone else know what time PDT is.


Also, some timezone labels are ambiguous, e.g. "CST".


What I learned recently is the way Python strptime deals with this. It only parses the current time zone or "GMT" or "UTC" and all have the same effect (of returning a tz-naive object). This behavior is amazing because I don't think it's ever what you want.

$ TZ=Asia/Shanghai python -c "import datetime; print(datetime.datetime.strptime('4CST', '%H%Z').astimezone(datetime.timezone.utc))"

1899-12-31 19:54:17+00:00

$ TZ=America/Chicago python -c "import datetime; print(datetime.datetime.strptime('4CST', '%H%Z').astimezone(datetime.timezone.utc))"

1900-01-01 10:00:00+00:00

$ TZ=America/Havana python -c "import datetime; print(datetime.datetime.strptime('4CST', '%H%Z').astimezone(datetime.timezone.utc))"

1900-01-01 09:29:36+00:00

$ TZ=America/Havana python -c "import datetime; print(datetime.datetime.strptime('4CDT', '%H%Z').astimezone(datetime.timezone.utc))"

1900-01-01 09:29:36+00:00

$ TZ=America/Los_Angeles python -c "import datetime; print(datetime.datetime.strptime('4CST', '%H%Z').astimezone(datetime.timezone.utc))"

ValueError: time data '4CST' does not match format '%H%Z'

That last error was real fun to debug when something worked in production but not locally.


How is CST ambiguous? Are you thinking of all those counties in Indiana that don't observe daylight savings? In theory at least, a specific instant in CST maps to a single specific instant in UTC, GMT, CUT or Zulu Time, whichever you set your watch to.

And it does seem like people sometimes forget that CST is not the same thing as CDT. Several months ago (after the 2024 PST -> PDT crossover) I had someone across town tell me they wanted to set up a meeting at something like 9AM. But when the invite came, the time zone was set for 9AM PST instead of 9AM PDT. I assumed it was PDT since we were in the same locality. But I did make sure my schedule was clear for the hour before on the off chance they had some weird software bug that picked time zones at random.

If your point is something similar, then I heartily agree. In theory, converting between time-zones and daylight and standard times is easy, but in practice there are several situations where it's made more difficult than it should be.


I was referring to the multiple timezones that are abbreviated to "CST", as described in a sibling comment.

But you bring up another good example. I've worked with people who write times as "PST" or "EST" all year long. This bothers me in its simple incorrectness -- I do know what they mean but I hate autocorrecting them.

But I've also worked with people in parts of Indiana, and Mexico. When they say "CST" at any time of the year, they might mean it precisely!

My strategy is to write, e.g. "8:30 AM US/Pacific". Which probably annoys other people, but at least it's precise and unambiguous. I think it's less awkward than the equivalent "America/Los_Angeles".

Full list of US lower 48 timezones: US/Arizona, US/Central, US/East-Indiana, US/Eastern, US/Indiana-Starke, US/Michigan, US/Mountain, US/Pacific.

And the +2s: US/Alaska, US/Aleutian, US/Hawaii


> How is CST ambiguous?

- CST: Central Standard Time UTC−06

- CST: China Standard Time UTC+08

- CST: Cuba Standard Time UTC−05


Yes, this can be super painful, especially when correlating information from several sources, or when traveling. It can become almost impossible when reviewing historical data.

My solution to inconsistent time rendering in AWS (sometimes UTC, sometimes localtime) is to run a separate browser profile for AWS (and third party status pages) that that's always running in UTC.

But seriously, status page people: there's no excuse for being vague about timezones!


https://health.aws.amazon.com/health/status

Rhetorical question (I know why, humans manually involved, incentive to not report SLA breakage, etc.): How does their status page not auto-update when one of their core APIs goes to basically 99% 500 (or even above 5% 500) status error?


"Show me the incentive, and I'll show you the outcome."


us-east-1 gang rise up!!

For once I can feel mild pleasure at seeing the tables turned…

To be honest I don’t know why all my projects are always in USE1, I guess it’s just because that’s where we have always had them for my lab so I’ve stuck with it for no good reason…


US East 1 was the default region until a few years ago. If your account is older than that, then most likely all your stuff is there.


Yeah the account is from like 2019 or so. But still, I’m spinning everything up with Terraform or AWS Copilot, so it is still an active choice on my end to put it in us-east-1.


Unless - like most people - you and your audience are not in the US.


This wasn't an "everyone is in the US" type of post. Even for non-US folks, for a long time the default region was us-east-1, until they switched it to eu-west-1, and about 1/2 of all of AWS customers are in the US, or at least used to be until recently.


"Where's my new server gone? ... oh, it's hanging out in us-east-1, damnsit."


They launch all the new fun toys there!


It’s usually always cheaper there.


Elastic Beanstalk, Lambda, and CloudWatch returning errors for us. us-east-2


Can confirm, S3 backed CloudFront experiencing random errors in us-east-2 for the past 15 minutes at least.


Cloudfront is: my small, not even yet profitable SaaS landing page and javascript frontend, served by CloudFront, are down since about 17 minutes ago (3:28 US Eastern)

the API whcih sits behind ELB is working fine

edit: us-east-2


I wonder if this is related to the NHC outage?

https://news.ycombinator.com/item?id=41771629


Looks back up: https://www.taloflow.ai/is-aws-down/us-east-2


I am seeing on multiple AWS accounts CloudWatch log groups are not loading. It even says you have no log groups, that's not the case.


S3 on us-east-2 stabilized for us as of 2 minutes ago


Yes. I can't tell if listing/reading is fine but putting (uploading) for sure seems to not be working (500 error).


Have issues reading (500).


We're seeing CloudFront and S3 issues in us-east-2 in multiple accounts. No errors in us-east-1 or and ap-south-1.


No issues in us-east-1 (for once....)


Thank you! I was misreading our logs. Can confirm no errors in us-east-1 for us as well.


Next time you can also check https://cloudstatus.page/cloud/aws/issues

I guess I'll increase the frequency of some probes to get better aligned with the official timeline :)


Also seeing Cloudfront failures on my end, both in us-east-1 and us-east-2.


Seeing CloudFront + S3 errors in us-east-2 on our end as well.


And now we're back up.


us-east-2 seeing tons of S3 errors.


This is why I use Hetzner's Object Storage. Proven and tested! /s


Next time my boss complains about uptime, I will tell him, even AWS have downtime. Boom!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: