The software engineering paradigms used within the company create brittle rube goldberg machines of events flowing everywhere in the company. Almost all of them are on maintenance mode, where the oncall burns out the engineers and prevents them from creating new products. There is no knowledge sharing between team members. Legacy team members guard their technical platform knowledge to solidify their place on the team.

The engineers themselves are not students of computer science, but just crunch out tickets.

If Amazon wants to change they need to remove a significant amount of tenured employees, and actually promote young engineers into decision making positions.

AWS hasnt released an innovative product in a really long time

Sometimes I try to talk to my 83 year old dad, who was a Teamster at the same company for his entire 35 year career, about the software industry. He's so surprised how often people like me change jobs, how we switch companies as a way to get a raise, and just generally how different expectations are. When I told him about how I'd left a company partly because they didn't promote me after 18 months, he didn't say anything, but I knew what he was thinking. In his world, 18 months just isn't long enough to feel entitled to a promotion. Promotion is primarily seniority-based, and the company rewarded loyalty as much as, err, value creation. It's a different world we're in, but I'm not sure exactly what makes it different: is it the 21st century, something fundamental about the industry, or the fact that software people feel they have more mobility, and thus less loyalty and patience?

One thing I'm fairly certain about is that companies don't treat us worse than they treated blue collar workers in the 70s. I think we'd all be surprised by the poor selection of candy in the catered employee cafeteria they had over at the shipping depot.

Given that betrayal of the old social contract, why on earth would anyone choose to be loyal to such companies? Work loyalty still exists with individual managers or owners of small businesses that actually interact with their workers, but these days most workers correctly assume that most corporations are being run with little regard to anything aside from profit; and those workers act accordingly.

I can't tell you the amount of good colleagues I've seen leave because management was too stubborn to give them a 7% raise. The most baffling thing is that they'll have no qualms (or perhaps no choice) to subsequently replace them with someone else for at minimum double of whatever 7% would've cost them. This causes a cascading issue where people are annoyed that new hires get such a big boost while the current staff have to fight for every measly percentage increase, so the older employees slowly trickle out.

I've had it happen myself too. A place I really enjoyed working at, I found the work great and had a great relationship... But the CTO stubbornly refused to give me a 9% raise. I referred my friend and started looking elsewhere. I got a roughly 30% paybump, and my friend got hired to replace me for roughly 20% more than what I asked for.

So why on earth would I be loyal? The people running the show are hilariously short-sighted and can only see the beacon emanating from their VC buddies shoveling money their way, so whatever, I don't care ultimately if something I worked on dies or not.

They are either greedy or stupid. You have to pick one.

but boy howdy does it work, and make a lot of money.

There are plenty of people who are greedy and not smart. I'm not sure what basis this statement has. Plenty are greedy and not smart for their entire lives.

They are not smart. For many stupid would be an understatement.

The amount of stupid people who fail upwards is far larger than the few and far between who get there on merit.

My direct boss at the moment is one of the smartest people I've ever met. The people running the plant are some of the stupidest I've ever seen. Constant mistakes, short-sightedness, asking questions anyone who works at the plant should know in their first week, etc.

Best part is it's a relatively small city, so everyone knows everyone kinda. Our plant manager (top guy on site) was in a middle management role at his old plant (the worst in the city), where he was fired for incompetence. He wound up with us and within the year was the highest role.

We have lost countless years of experience due to people leaving over this guy and his cronies. He has no formal education, hasnt been in the plant life long enough to know anything, constantly makes idiotic decisions, etc.

These are the people who wont get out of the way of the people doing the actual work, and make their lives/work measurably worse due to their stupidity, greed, and incompetence.

A tale as old as time, yet for some reason we keep letting it happen.

Nah. Some people are definitely both, and it's the "stupid" bit that causes the "greed" thing to be applied blindly therefore achieving the opposite of what they wanted.

ie they make poor decisions and lose money instead

Greedy but too much short term thinking is stupid.

Talent has to be hired at market price or why would they join your org?

Their bet is that they won't actually leave if they get rejected on the raise. This is where the stupidity comes in, they actually think people would be loyal and stay despite requesting a tiny raise.

What happens instead is that people are, obviously, annoyed and start looking elsewhere, where they'll get a much larger pay bump anyway. Now the idiots are forced to look for new people at much above what they could've paid to keep the old person, because no one is going to accept a new job below market rate.

So yes, stupid and greedy.

It is easier to make the justification in a lot of orgs that you need to go out and hire someone to replace someone who left at "market rate", than that you need to pay someone x% more, based on work that is probably nebulous to whoever you need to make the case to.

I don't think there has ever been a time where this has not been the case outside of very small companies or niche operations. The same sort of incentives are endemic in a business of any scale, because ultimately org structures end up as pyramids and people will intrinsically compete to be at the top

(Note I am American and work for American companies. I've had good experiences and terrible in the ol' USA. We have the fundamentals here but damn it too many management people just don't listen to it.)

I think more incentive structures naturally emerge over time. Managers have other managers who have other managers... etc who have some different set of incentives. There are also shareholders

A lot of time the right smart thing is done anyway, but usually that's luck in having a couple of good smart people in your org chart chain, or just the rising tide of the economy making it easier for generosity to prevail

Management "blew it", but there were probably no negative consequences to them in the case. There are negative consequences for the shareholders when this sort of stuff happens over time in the aggregate... which I believe you get to call the "Principal Agent Problem" if you earn an Economics degree or understand game theory or something...

Part of that skill is knowing what can go wrong. And to that end there's no better short read than Ishikawa's tqm the japanese way.

I also agree with your other point: defects are often seen in aggregate when unfortunately the damage is done and cynasysm and infighting are almost unstoppable.

Let's see if you still hold that opinion in a few years when you reach your 40s/50s, can't find a job because ageism, and you have to compete with people who work 16h/day surviving on cold pizza and Red Bull.

> I

"It's not happening to me, so it's not happening" says the programmer type, who isn't carrying 25kg loads up stairs 10+ times a day

The article is specifically about tech workers in Amazon, so I think it's pretty clear from that and the rest of the context that I was discussing unions in relation to tech workers.

I was disputing the claim that you a) it's hard to get a job in your 40s or 50s, and b) if it is hard, it's because of ageism and not something you did (or didn't do). And as others have pointed out, there is certainly a class of employee who is going to be chugging energy drinks and working 16h a day but if you're 40+ you're not competing with them.

Which isn’t to say that everyone who doesn’t retire by then is bad, just that the ratio is different than 20s and 30s and this probably colors people’s view of the group. Especially with the crazy industry growth we’ve seen over the career of the average 50 year old in tech.

The median developer salary in the US is in the neighborhood of $110k/yr and includes $0 in bonuses and $0 in stock. Outside of the San Francisco bubble simply being able to type JavaScript into a computer does not put you on a trajectory to retire decades before your peers in other industries.

I thought it was typical for, say, a regional bank software department, to be staffed with a mix of ages up to retirement age, not mostly people in their 20s and early 30s.

All of the ageism concern I’ve seen discussed has been about big tech and startups. Are 48 year olds being pushed out of working at mid sized insurance companies?

It's not a meaningful reference point for this discussion.

If you're going to say something like 90% of the workforce is under 30 you're going to need to cite something because I just absolutely do not believe that.

Even at age 36 that’s no contest. If they’re producing crud twice as long that just means there’s more crud.

I’ll admit that that might be harder to see for management than the fact they spend twice as long in office.

Unions forcing companies to pay him higher due to his age makes ageism worse, not better.

In the case of this example, the manager was given a budget and that meant choosing staff. So she told my friend that he could go the easy way or the hard way. There are plenty of causes that the manager could dream up. It's not hard. If you want a cause, they can find one.

That's avoiding union contracted processes. Union representation can help a worker when management is concocting bullshit issues. The "hard way" might make things more difficult for a worker (closer supervision, tedious training) but it's definitely harder for the manager than if the worker quits.

Do you think none of us are working in our 40s and 50s?

There are plenty of unions that aren’t seniority based.

(Roughly speaking) the old-timers stick together because they can and because there are old-timers. And few people get promoted because there is only a very narrow path upward (say, 6-10 lower to a ground-floor supervisor, 6 of these to a higher rank). That necessarily creates a very thin path up. And there is space up only when the previous person is promoted or leaves. There is respect for the skill of a senior employee when it has been proven but respect will only buy you so much. Mostly that skill is put to good effect without massive salary change. There is nominal seniority-based salary increase because that's affordable and easy to manage - and is captured. That is, this salary scale was designed by the previous seniors. Same for promotion order and layoff order.

Seniority then becomes a way to "force" loyalty: If you jump, you may not be one of the top dogs who think they have captured the system. No need for (much) higher salaries if the staff themselves support seniority.

Are you active in your union? Have you voiced concerns over this policy of theirs?

If you are truly valuable, no amount of union anything can prevent you from getting a good deal.

You are arguing against the very idea of a Union. The whole point of a union is people sticking together for long to tend to each other's interests.

Why should a union reward someone known to be a grifter?

While the newcomers have little stake in the game, perhaps don't even vote, vote in dispersed order, etc.

I spent almost a decade doing union labour in Canada (auto industry). The idea that they "stick together" any more than any other group is not my experience.

During one of our negotiations, the company proposed making everyone hired after a certain start date a "temporary employee" (no benefits, lower pay, can be laid off or recalled with 24h notice). This would affect 25% of the employees. In return, the company offered a bigger signing bonus: $2500 vs the usual $1000.

The union leadership recommended voting against the contract; the fact they even brought it to the table is crazy. And the union membership voted it in anyway. Screwed over fellow workers for $1500 a head. It was that easy. Us "temporary workers" were now working beside people making 30% more money, doing the same job but facing more uncertainty. We were gradually laid off over a couple years, all in the name of "corporate viability". I had spent years hearing about "solidarity", but the second an extra paycheque was on the table no one cared about their "union brother".

Joke's on that union, I guess: the plant closed during the 2008 financial crisis and everyone lost their job. Hope that $1500 lasted!

I have no interest in rewarding Adam just because he happened to be born 3 years before Bob.

The word 'Union' itself means a association of people with a purpose of taking care of each other. You are free to not be a part of such an arrangement. But that doesn't mean that the association is wrong, or its members shouldn't look after each other, or they are wrong to do so.

In fact you are free to do whatever you want, and they are free to what they want. Its just history shows that if you can win big individually(outlier) you are not likely to love unions. But very few people can be outliers so for the bulk of the human race unions work just fine. You tend to do well on the average on the longer run.

Doesn't help against cargo culting and reinventing wheels Just Because We Can And Have The Money.

Just because Google did Kubernetes everyone else followed suit despite there being competitors available (e.g. DCOS, but that one sadly failed because it was utter bananaware). Everyone followed Google for Angular and then went over to Facebook's React, and on the backend side it's Go here and Rust there.

This is not really a question, yes. But... they're sledgehammers and most of the people used them because "the large ones are doing it" - for a loooot of use cases simpler solutions such as plain old jQuery+SCSS / Portainer or Docker Swarm/Compose would have been more than enough.

This results in a situation where a lot of different places have logical, reasonable reasons to pick k8s because it matches reasonably well 80% of their needs, and while everyone's 80% is different, there's enough overlap. This pulls in, like frame dragging, others who might not actually need it, but it's not because some scaling memes.

In practice, a lot of initial intake for k8s was not bananas scaling arguments, but it actually solving problems for people who either had growing complex messes, or considered the spend associated with "good practices" that are often thrown as argument against kubernetes, like running separate cloud instances for every application.

Similarly, everytime I look at Nomad and related stack, it turns out that various bits are missing, and DIY will be harder than with k8s-like stack because it's not properly designed to be extensible.

[0] Mediocre unit tests are as good or better than pristine API docs that are slightly out of date.

So in addition to having better continuity of institutional knowledge, there is a benefit to staying on longer (at least for me).

But no boom lasts forever. In other creative fields or games development, where supply far exceeds demand, the situation is much less favorable for workers and that's eventually where the software industry is going to be as well.

Do remember that there is a whole bunch of software that needs to be created for our household appliances, any electronic device (those tiny little SoCs that control a sensor or voltage or whatever), or tools used to make anything else. We are also in the midst of a push to get everything "digitized", including our entire homes, cars, buses, libraries, movie catalogs, music libraries, battery charging, communication infrastructure etc. And everyone keeps a computer or two or three (phone, smartwatch, tablet, fitness trackers) on them at almost all times — all these need software to do something useful.

Basically, I think software developers are going to remain in need as long as we rely on computers. And just like farming has never gotten obsoleted because we continue to need food (though increasing the brute force of the tools we have has allowed us to rely on fewer farmers), we'll continue to need new software (it's still an open question if there are tools like Copilot that will enable us to achieve similar efficiency improvement and rely on fewer software developers, but my guess is no).

Well unions are pretty well known for prioritizing seniority above all else.

It's one of the biggest negatives that come up against them.

Because truckers don't "create value" the same way that software engineers do. If you had a company of 100 of the "top 0.1%" truck drivers, maybe you could charge a bit more because your insurance would be lower and you'd have fewer accidents and late deliveries. If you had a company with 100 of the "top 0.1%" software engineers there's a good chance you're on your way to having a company worth billions of dollars.

If you have the top percentile of drivers chances are you’re making heaps of money moving oversized loads, nuclear materials or running time sensitive contracts.

Then California came for my contracting job.

I would rather phrase the question as: why should any employee have loyalty towards a given company? The main goal of companies is to make money (if they could make money without employees, well imagine that). The main goal of an employee is also to make money. People don't usually work for free (there are exceptions, of course, I'm talking here about the vast majority of works and employees). Given these statements, it's natural that:

a) companies get rid of employees that don't make money (unproductive employees). Layoffs happen every single day

b) employees switch to other companies for more money. Obviously, this is easier to do in some industries than others, but the essence is the same in all of them

It didn't use to be that way, which is what OP was trying to get at. Companies used to reward loyalty with bonuses that scaled by tenure, and tried to retain workforces that they spent considerable resources on training, even through lean times.

So I wonder if this is either the end-stage capitalism everyone is talking about, or the community aspect got thrown under the bus somewhere.

Companies reward loyalty because they can keep you behind the market rates. Hiring a new person costs you market rates. Keeping you around with 2-3% increases every year with occasional promotions of 5% will ensure you stay behind the market.

As a uniom fellow he might well be overpaid if he at 30 years in makes a lot more than the 10 year guys while doing the same exact job as them.

Basically people like your dad were getting tithed by management for loyalty while wondering why you don't tithe your company for security in a market where the most important goods require 2-5x as much.

Many blue collar workers in the 70s could afford a house, a car and a wife who didn't work to take care of the home and kids.

Thank you for saying this. I have interacted with various seller-facing Web software platforms (KDP, Ads, Advantage, Seller Central, Transparency, Brand Registry) for years. Hurriedly designed features, broken processes, false-positives, and kluged-together garbage are the norm. Automated support scripts, overwhelmed human CSRs, and the siloed nature of the company compound the problems.

Here's one small example: Amazon forced many sellers in the US to start selling in Brazil. It was automated. People's accounts were enabled for Brazil by default. It was poorly communicated, including no easy way to opt out (redirecting links on the help page). Many sellers in a panic started requesting support to shut down their Brazil accounts, which led to entire global accounts being shut down.

https://sellercentral.amazon.com/seller-forums/discussions/t...

I have never worked at Amazon. But I did work at a company that decided to implement microservices a la Amazon, even going so far as sharing Bezos' famous 2 pizza team memo. This effect essentially happened over night. As people spun up more and more microservices, things got more and more siloed, cross team collaboration was significantly more difficult, and things became an increasingly more complicated rube goldberg machine that just destroyed people with on call schedules.

At HBO Max every incident had a full writeup and then real solutions were put in place to make the service more stable.

My team had around 3 incidents in 2 years.

If the cultural expectation is that the on call buzzer will never go off, and that it going off is a Bad Thing, then on call itself isn't a problem.

Or as I was fond of saying "my number one design criteria (for software) is that everyone gets to sleep through the night."

The customers win (stable service) and the engineers win (sleep).

Was there any cost considerations associated with prioritizing the implementation or even limiting the scope of the solution?

That typically didn't happen because engineering reviews had to occur first.

A single command created a new repo, setup ingress/egress configs in AWS, and setup all the boilerplate to handle secrets management, environment configs, and the like.

All that was left to do was business logic.

If the issue impacts tens of millions of customers, then yes, get it fixed right now. Extended outages can be front page news. Too many in a row and people leave the service.

Ideally monitoring catches outages when they first get started and run books have steps to quickly restore service even if a full fix cannot be put into place immediately.

Speaking as a technician whose seen 3 AM at work many a time.

One possible bonus, being on call operating your own software also gives you a solid incentive to not wake yourself up in the morning by writing bad code, and fixing those issues that do arise quickly.

Unfortunately, my software interacts over network with software written by other people; if something goes wrong at 3 AM the users don't know which part caused the problem, so they wake up a random person.

Now I see young engineers from top-tier school working "on call" without complaint. I've found ways to avoid such roles, but it always seemed ridiculous and completely unnecessary in a world where there are software engineers around the globe that could easily work full time support positions.

When you're making real money, they own your ass.

Both salary and hourly gigs have income hanging by a thread with plenty of work, yet only one can get overtime.

Be given responsibility/salary for something (aka hired) by a particularly needy manager/org and be 'undependable'. Read: not at their call. See how it turns out.

The worst/eventual outcome: bye-bye money. Hopefully one has a more reasonable environment. Workers have little on their side.

As someone who does SRE (not AWS, elsewhere)... I would absolutely prefer pay as an hourly rate over salary. I don't like putting in more hours/making less money because Developer Kelly had a bad launch... but I have to, The 9s (and bills) Must Flow.

Fortunately, my current place takes this into account. I don't actually need bonuses or structure change... but the larger trends remain. The employer is buying you, salary opens the time box.

If your product is really so important that it can't be down, hire more engineers and pass the markup to your customers.

I'm glad I live in the country where you have to have 11 hours between end of work and start of work(except for special cases afaik).

If the company cannot afford that, then the product is not that important and can remain broken until the morning.

Even 24h fast food places hire 3 people (each working 8h)!

Manual fixes should never be done in a hurry, and if your system is that fragile, I really wonder about the competency of your senior employees and leadership.

Keep your fancy valley salary (with the ridiculous rent prices attached), and I'll keep my European workers right's protection—including undisturbed sleep after my 8 hours workday.

Mostly things went smoothly so that's a pretty nice bonus.

Something ridiculous like that is luckily impossible in (most?) EU countries.

Is the effect unformal harm?

In the EU when you are on call this is a contractual thing.

That is not common in Europe. Generous compensation and additional time off is quite typical for engineers handling on-call burdens.

At one company, I was technically on call 24 hours a day 7 days a week for over ten years. Did I get called that often? No. Did I get called at the worst possible moments? Yes.

And it’s certainly industry-specific. Some doctors have this, firefighters—and software engineers. Contrary to the first two, they usually don’t save lives, but revenue though.

There is a cost to having on-call. Whether it's in the extra hours you are paying your engineers or other technicians, or sleep deprivation, dwindling motivation and performance, the cost is always there.

In a business, cost is always balanced with the return on that investment.

So it trivially follows that on-call only makes sense where the return is bigger than the investment. If you are having your $100/h engineers become $20/h engineers during the day because of the on-call rotation, and you lose $200 of sales over night when things are down (even your customers are asleep) — you are actually investing that $80/h difference for 8 hours ($640) to recover $200, for a net loss of $440.

Yes, there are cases where it's fully acceptable to simply have your service down for the night. Eg. imagine a service that provides the amount of energy sun is providing for a location (to combine it with solar farm production): is it really that bad if that's down at 2am? Sure, it might be nice to get it back up before the sun is up, but this is just a trivial example where an uptime of ~70% (fluctuates) is perfectly acceptable.

I don't understand your take. Every single time I had a job with an oncall rotation, that oncall was paid. I was paid a bonus for being oncall, I was paid a bonus if during oncalls a pager fired outside of office hours, I was paid a bonus if I was pulled into an incident response outside of my oncall rotation. There was always a cost, and we were paid for it. Being oncall represented loosely a pay bump of around 15%.

If that's not your case then I'm sorry but your problem is not the oncall rotation.

So, hire a team in another time zone.

This is a problem of management not prioritizing the health and wellness of their employees, simple as that.

Even if your app is critical infrastructure (it isn't, and 99% of you shaking your head and saying it is are objectively incorrect), you don't need a software engineer to fix it. You need an SRE. That's completely different.

Micro services solve a problem for companies that have already reached a certain scale were cross team communications have become unfeasible and now communicating by well defined API contract is a better choice.

Also ideally micro services should reduce the blast radius of outages. If an outage is not easily traced to what team is root causing it, then proper monitoring is not in place.

Sure I've had times where on call went off and it wasn't my team's fault but it was no more than 10 minutes to determine that and reroute the call and then to back to sleep.

The other fact is that when designing a new microservice it should be done in conjunction with the primary consumers of that service just like designing any other part of software. Stakeholders need to be brought in and consulted.

The advantage of microservices is that new code is only going to impact direct consumer s and downstream services.

It also allows you to upgrade a service in place or completely rewrite it so long as you adhere to the original contract.

I've seen impressive rollouts of security updates across a huge code base that was only possible because of a microservice-based design.

I've seen giant monoliths fall apart as multi-year long efforts are undertaken just to update the build system.

The hard part of a microservices is they require discipline in an organization and basically assigning an engineer for 1 to 2 weeks to write run books and add monitoring.

Of course, those runbooks should be written no matter what paradigm someone uses!

Nothing stops someone from simply enforcing the same division in a single large code base. Your API contract can be your public API in whatever programming language, and this would allow you to work with the same assumptions from the SOA.

It would only be easier to break out of the recommended way of doing things, but you can provide simple tooling that does static analysis to prevent that (I remember using Zope3 security configuration to achieve exactly that with Python code in ~2006).

If you are concerned about a performance from such a large monolith, you could be using a functional language (or at least the pure functional paradigm) that allows easier infinite horizontal scaling.

I'd say nothing except human nature.

The article comes to the exact opposite conclusion.

The article repeatedly cites problems associated with culture breakdown, and tenured employees are a key source of continuity for culture. The article goes to some lengths to underline that the culture is a positive and key differentiator for Amazon.

The more interesting point is that the cultural element looks like it is highly durable and effective, but has broken down as Bezos (the most tenured employee!) has pulled back. So it's not as durable as everyone thought.

Tenured employees help maintain culture, but they don't guarantee that it stays static.

Culture developed by the employees who spent their first 5 years growing with the company is not necessarily the same culture developed by those same employees who have stayed in the same role for the last 5 years with a vanishing potential for growth.

On the other hand if the culture you want to maintain is "constantly be trying out new technologies, new ways of working and fresh ideas, at the cutting edge nothing is set in stone" you might benefit from new employees who know firsthand how other companies solved whatever problem you're facing.

And the conditions for most big tech companies have changed radically over the last few years. So the subtle devaluation or reinterpretation of culture is kind of inevitable when forever-growth turns into a zero-sum game. As you say, it might even be the veterans who are doing it.

This is not specific to Amazon, it's endemic to the whole industry (and probably outside of the tech industry, for all I know): Very senior leaders and engineers are sometimes where they are because they've clung to their role for a long time, and they don't leave or die at a rate sufficient to frequently bubble up new senior talent through internal promotion. These people may also deserve their position because they are brilliant, too... or they may not. I've worked in a place with technical leaders where nobody knew what they did--their job was apparently "Being Employee Number 3" and nothing more.

One of the smart things they managed to do was recognize that they were top-heavy with senior+ talent who were fixed in their thinking & behavior and aggressively began recruiting new graduates and reaching out to CS/Engineering/Science seniors. They also added some pretty impressive mentoring programs to bring the new people up to speed quickly.

Then they switched to doing stupid shit that mostly undermined all that and the newer people started leaving. But it was pretty good for a couple of years.

The reality was that in a fairly short time, focusing hiring on new grads actually had the intended effect of improving the way we did things. However, the company started becoming more unpleasant to work for soon after that. And a lot of those new hires, now with a couple years' experience, left. As did I, so I can't say how permanent the changes were.

Why does there need to be anything more? They probably know how the systems work and are ready to step in if necessary.

Also, they busted their ass before other people were at the company, enabling the company to grow and create the jobs that all the other people are now in.

To me it's not anywhere near as egregious as the founder who worked for a couple of years, then outsourced all leadership to a CEO for hire, and now does nothing, but still maintains 50%+ ownership of a company.

Seems like this is also the problem with Congress.

That's not true, at least not in AWS. In fact, I'd argue that a big problem in AWS is that promotion is too fast too soon. Getting to L6 used to be a big deal, and many people were content with being on L5. But now everyone expects that L5 gets promoted to L6 in 2 years or even less. And L6 to L7 in 3 years or less. What many directors do is that they place their favorite L6 to lead a project that launches a new product. If the launch is successful, the L6 will most likely be promoted. We even joked that L8 is the new L7. And of course, this creates so much anxiety to the point that more than 50% of the 1-on-1s I'm aware of involved some discussion about "moving to the next level".

This is a nice metaphor that should become widely known and discussed!

It can be applied to general management and administration was well as software engineering. Organizational procedures tend towards complexity over time: everyone is involved in 'improving' systems by creating new procedures, and soon enough a new person is needed to deal with or manage the increasingly complex procedures. This is a well known micro economic phenomenon. It's hard to simplify and streamline and so being aware of the tendency towards brittle rube goldberg machines is a useful concept indeed, should become Amazon's 17th management principle.

OH MY GOD!

Thank you for saying this, this is __exactly__ what I've been experiencing. I've seen a bunch of people pursuing their personal (promo) agenda ruthlessly. Delivering on their own projects and initiatives and nothing else.

Hire and develop the best my hairy ass!

I'm sorry if I sound harsh, but if that's your take them either you haven't been paying attention or you're not in the company for that long. The whole org was designed so that all employees are ephemeral. Managers are encouraged to promote employees up and out. At each career level you're evaluated against your direct colleagues in the same level, and those lagging behind are bound for PIPs.

Check out the old fart tool. The average tenure is below 3 years. Do you it screams upward mobility?

I have worked on major tier 1 services in every major part of the company: Retail, Alexa, and AWS. I know many of the people running the important orgs

I have to call bullshit on this take. Last time I checked, less than 1% ever make it to L6, let alone L7. because it is extremely competitive.

It takes a very special person with all the stars aligned to even be considered for a L7 position. We're talking about Principal Engineer/Senior Manager.

Claiming they are not promoted to L7 because they don't want to screams of Aesop's the fox and the grapes.

You also seem to be inadverdently supporting their point. If it takes "a special person" to be an L7, doesn't it also require "effort" to become that person? The person is simply saying that they are not even making the effort they think is necessary to be considered for an L7 since they think there is too little upside to justify that effort. To support their point they mentioned that another friend we hardly see anymore became an L7, their intense schedule and workload being the reason we don't see them anymore. It seemed pretty believable in that context.

You claimed they were purposely lounging at L6. I called bullshit on that, and explicitly mentioned Aesop's "The fox and the grapes". It's unbelievable to claim that they are not L7 because they don't want it. You need to outrun everyone in the whole company, and even then you are limited by circumstances and track record.

It seems like optimizing for innovation makes sense with new upstart products and optimizing for maintenance makes more sense with established products.

You started with if Amazon wants change and I think that's exactly what a product with a position of relative market dominance doesn't want

Both exist in industries that aren’t innovating much. Amazon doesn’t enjoy such a luxury.

That... seems about right? I wouldn't expect to get to L7+ in much less than that.

Firstly:

>AWS hasnt released an innovative product in a really long time

They never released anything innovative, ever. Amazon creates value through optimization. The original AWS came about because they had extra servers lying around that they needed for holiday traffic, and they decided to rent those out.

The problem with Amazon is that its too bloated, which goes against their optimization bread and butter.

The cycle goes like this. First you have an entry level engineer, that hasn't been taught really how to solve problems algorithmically - instead its basically just education in the form of memorization about problems and how to solve them, without anything more fundamental.

Then, companies like Amazon need engineers to actually build the products, so they are forced to tailor the interview process to them, thus the prevalence of leetcode style questions, because they know that the engineers are studying those for other companies.

So the candiates get selected are basically those that have shown that they can memorize processes better than others. When these candidates work, they do the exact same thing internal to the company, focus on doing the existing processes in hopes to get promoted.

As a result you get lots of bloat both time and technology spaces. So no, promoting them into decision making positions is not the right thing to do.

The best thing Amazon can actually do long term is to focus on more automation, and reduce headcount. You want fewer, more talented engineers who are able to solve problems autonomously without barriers in the way, who don't need handholding or don't need to handhold others.

There are newer companies, like temporal.io, whose platform capabilities make AWS look outdated at this point.

I think your understanding of leetcode is not correct. It filters for:

Can learn computer science concepts

Is willing to sit in front of a screen and learn said concepts for a while

Is willing to follow directions and recipes

Is willing to follow the rules

Has some minumum bar of intelligence

Unfortunately, these interviews do work. Some percentage of people that pass the interview are actually very good. The amazon firing process will weed out the rest.

Meta has released the best open source software in the last 20 years, and their hiring process is all leetcode memorization

For example, lets say I have a server rack at home. There is nothing in AWS that exists that I can't replicate on my server rack at home. The only difference is that time required to implement all of it on my server rack vs in AWS, which is optimization.

If AWS comes out with some closed source ML model that is good at doing something, thats innovation - i can't replicated that at home without knowing specifics. Likewise, if they have super cheap EC2s with RISC or dedicated ML hardware that are cheaper per hour to run than anything I can build at home with power costs, thats also innovation.

The interviews aren't fully useless, I agree that leetcode provides valuable insight. However IMO it should be the bare minimum.

This isn't true, it's a bit of marketing nonsense.

There was a paper proposing to create the service, it got workshopped quite a bit, and got funded. The servers used weren't extras, it was dedicated capacity from day one.

At the time, Amazon was not using the same hardware (or anything) as AWS. Even today, there are still critical Amazon services running on the older corp fabric, because the Native AWS migration isn't 100% complete.

The other key parts of AWS were also innovation, and not mere optimization.

You have demonstrated that you have a near complete lack of understanding of how AWS innovates. Your ignorance is thorough, and that gives you confidence; but it's unwarranted. You're just another Dunning-Kruger twit.

The corp fabric does run on AWS. Its just the legacy tools that are used to manage the virtual machines instead of them being liked to the AWS account. The hardware is in the same data centers, and legacy tools wrap AWS stuff under the hood.

But yeah flagging for unnecessary personal insults.

what's the evidence that it's better run? continued growth could just as likely (or more likely) be a result of its dominant position

Your pay is equal to three card monty. You are promised something big when you get hired but it gets spread over 4 years and most of it only vests at the end. Then you find out most employees are pushed out within 2 years and you realize only 10-20% will ever survive to get the amount you expected when you joined. If you make it this far they do it again for more money. The odds of you staying are even less...

While this is happening they give you 8 hours of meetings and expect 8 hours of development. You end up working 16 hours a day. You end up so tired and confused you keep going until you burn out. At that point they ppe you.

The best way to survive at Amazon is find dirt on your boss. Hire a pi. Coast and use that information to keep you employed. Rinse and repeat for new bosses.

It is not run well. Taking these practices and putting them on a regular business wouldn't work. Amazon (Google, Apple, Facebook..) will keep increasing in value regardless of management style (each one of those companies have different culture) because of factors outside of this. They could require everyone to be left handed.. or speak French or be able to ski and they would still succeed.

At the end of my first year I got "exceeds" and a large comp bump (almost all of it from RSUs).

Those RSUs would start vesting one year after this "exceeds" conversation took place--April 1st (start of Q1 a year later).

So I'd get a (small) vest ~13 months later. Considering the average tenure (typically 18 months from what I heard from a higher-up in my org), the deck is stacked against actually making the big comp numbers. You work very hard to make more if you can survive another year after you worked hard. Odds are good you won't.

how do you get anything done with such a huge turnover? Maybe it's because we're small but it would be impossible for us to accomplish hardly anything useful at that rate of training replacements

in other words, "better run" != "employees are treated better"

This thing comes up in a lot of companies, but really the issue is your 2-year term job tourists/hoppers, and there are quite a few of you going around. Its tiring after a while to keep teaching and training every new member who joins your team. This look even more pointless when you look at the fact that they are likely to leave in a few months anyway.

You mostly have your own personal knowledge management system, where all the tribal knowledge goes. Giving people a run of such a large corpus of knowledge/information, especially when they are not like to work with your for more than a few months is pointless.

Eventually you only get as much information as much as you have commitment.

>>If Amazon wants to change they need to remove a significant amount of tenured employees, and actually promote young engineers into decision making positions.

Why would you promote short term people, who are likely to make bad decisions without stake only to fire people who have demonstrated serving in the long term interests of the company?

>>AWS hasnt released an innovative product in a really long time

The whole point of AWS is scale and stability, not newer products.

You say there isn't any innovation: May be you don't need that in a utility company that moves boxes and bits? You need reliable utility that customers expect and love.

In an alternate ad-supported company where the primary product (ads) is not the one people are most familiar with, you could play all these games of 'upward mobility', 'promotion' etc because money runs freely and your job is detached from the reality of product, customers or even the larger company goals (assuming, you are not directly working on ads).

However, in a company that is truly customer focused (and obsessed - to over use a Bezos word) as is Amazon: all of what you said seems like a good thing?

I do take issue with how they treat the low-level employees with ruthless nearly-dictatorial oversight - Amazon burn out is a real thing. OTOH, having talked to Amazon engineers/PMs who quit due to burnout still have some sort of Stockholm syndrome where they have fond attachment to their 'burn out' time and how boring/unfulfilling their new job at X/Y/Z is! It's Complicated.

Known feeling

   There is no upward mobility at the company, unless you have been in some org 5+ years.

> If Amazon wants to change they need to remove a significant amount of tenured employees, ...

Doesn't this include yourself as well then?

It's brilliant and it always works. There are no downsides whatsoever.

Does anyone know? Meh, let's just hand the task to HR and hope it all works out.

What does this even mean?

Does it mean they do not have proper CS background? If it means what I think it means, how is that even possible? Amazon recruits at top tier schools, not po-dunk university. Do that have a large % of their Engineering staff from bootcamps?

The biggest slap in the face about this LP is that the word "strive" used to be a bad word at Amazon. If you ever said you were going to "strive" to do something, you would be told that's not good enough. If you put it in a doc, you would be told to remove it. "Strive" was a weasel word. We never accepted "striving" as part of our tenets or our goals. Our goals never were to "strive to build a good service", our goal was just "build a good service"!

And then they went and made the new LP "Strive to be the world's best employer". Why isn't it just "Be the world's best employer"?

It may seem like a small thing, but it's an example of the ethos that has been chipped away inside Amazon. That particular LP has always been the butt of jokes because it's been clear over the last 5 years that leadership isn't actually serious about it, but the use of the weasel word "strive" really just put the cherry on top.

And of course, everybody understands there is no way you can force people back into office -- many people commute over an hour a day -- and also be "best employer" at the same time. That's just obvious.

One of the most surprising things in my brief experience at Amazon was how much of a shitshow it was. The two-pizza rule and self-sustainability for each team led to huge overlaps, with teams doing the same thing.

With such a huge organization, you had to go through 15 different stakeholders to get a single thing done, and there was an ingrained middle management whose only function was to connect you to the right person.

Just figuring out who was responsible for what and how to get things done was a challenge in itself.

Despite all this, Amazon still succeeds, and their process of PR/FAQ, leadership principles, and one-pagers is one of the best I've ever seen.

But I wonder if at some point, like with any philosophy and just like Bezos predicted, it will become too much and the whole thing will cave in on itself.

They had this monstrosity called Code Commit that I saw recently is getting canned. Good riddance. What an ugly pos.

This is false. Otherwise I'd be interested in any reliable sources.

As far as I can tell this is some sort of urban myth since AWS first stepped onto the scene.

I don't think that's myth, though I've heard it criticized variously as outdated, or just one person's take.

Amazon certainly uses AWS but I think it's foolish to think they use every product (they obviously do not) nor that they must. For the foundational pieces -- EC2, SNS, S3 -- sure, of course. They spent a long time migrating workloads.

I don't expect that with every new service they launch that they have already widely adopted it by then. Maybe after their customers widely adopt it, if ever.

Really? From the link:

> "The Golden Rule of platforms is that you Eat Your Own Dogfood."

and from Yegge's interpretation of Bezos' mandate:

> "All [mandated internal APIs to be used for all interservice communication], without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions."

Of course Amazon doesn't necessarily use every single AWS feature internally; I don't think dogfooding implies building all your features for internal customers first, and then selling them. Rather, it implies refraining from selling crappy reimplementations/duplications of internal tools.

If you're able to recall, can you list who these 15 stakeholders were? Are they like legal department approval, etc?

I'm not saying that's not true, but we shouldn't take it at face value that Amazon's internal values drove its success. The values stuff seemed to me to be mostly propaganda when I was onboarding at Amazon.

Could it just be basic market forces that caused Amazon to grow, attracting the best talent due to high compensation, which caused a (to borrow terminology from Bezos) "virtuous cycle"?

People talk about Amazon's corporate culture in a way that I don't see them talk about Google, Facebook, Apple, or Microsoft. Which makes me inclined to believe that it's mostly hocum used to justify where they ended up, instead of just attributing it to being in the right place at the right time.

Today I learned that teams in Amazon should contain no more than 2 people.

I guess Amazon has a problem with specifications as well.

Or just me.

The Amazon "2 pizzas" are Two Large American pizzas. Something with 8-12 slices where 2 slices is enough for a meal.

De Mesquita, B. B., Smith, A., Siverson, R. M., & Morrow, J. D. (2005). The logic of political survival. MIT press.

Which is echoed in the article:

> One current Amazon senior manager said they’ve consistently observed bosses new to the company use the principles “as weapons to put those who aren’t favorites in their place,” while employing other principles to build up their favorites.

I'd assume that the older management style that the article points out (from Bezos) was similar, but with different words; As these rules about power are consistent from governments, to companies, to home owners associations, etc. But the article paints a rosy picture of them, so it's not clear if there's something novel there, or just a dressed up version of the same old rules.

I have said many times, and will continue to say:

Amazon is a 2 trillion dollar company despite their policies, not because of them.

This is literally what they are doing lol.

My use cases are: play songs via bluetooth, turn on/off things (it's almost scary how fast it is), and set kitchen timers.

It's perfect, fast and gets to the point (just says 'Okay' as a response).

No gimmicky answers to stuff like 'meaning of life, universe and everything' or 'how many tennis balls would cover the surface of Mars etc'. Just. Gets. The. Job. Done. No need for fancy LLMs here.

Regardless of whether you think about the RTO mandate, internally Amazon leaves it quite clear that the whole point of these leadership principles is to reduce ambiguity, which means drive decision-making in the absence of a clear direction.

This is absolutely not the case with RTO. There was a very clear and specific diktat from the very top of the organization. There was no ambiguity or void to fill in the decision-making process.

For that, Amazon also has a leadership principle: disagree and commit.

Also, prior to the pandemic Amazon was very clear that remote work was the exception and not the norm. For example, L4s were explicitly barred from WFO, and L5s and above had to explicitly request an exception in order to work partially from home.

It's ok to frame the debate on whether it's in the company's interest to enforce RTO. It's complete nonsense and utterly absurd to quote leadership principles to justify their position.

Also, I should add that Amazon's RTO mandate was pushed when the company was trying to cull their workforce. The general feeling was always that RTO was designed to push people out without paying severance. Citing "hire and develop the best" when ignoring how the company was decimating teams doesn't build up the critic's credibility.

At Google they cut back office space and made every desk a shared desk for many Bay Area teams because hey you work from home now right? We’re not giving you a designated desk to work at anymore. At the exact same time they mandated RTO so you had to be in the office more than half the time.

Essentially the policy is to share a desk that you and another person must both be at for more than half of the time each. You can probably see the stupidity in the policy pretty easily right? Sundar still hasn’t figured this out. You can’t divide a desk in a way that gives two people more than half. If you want us there the majority of the time give us a freaking desk already!

As someone who came into work full-time post-pandemic because it turns out that living in a 1-bedroom apartment doesn't give you effective home office space to work from home, I was consistently aghast at how much the office planners were fully "oh, we're going to make your office environment much less conducive to work because no one coming to the office to actually work" (and then the return-to-office mandate comes in...).

What hasn't helped has been the most notable change from pre- to post-pandemic: no one takes meetings in a conference room anymore. As I'm typing this, I'm getting half of another meeting thanks to someone sitting two cubicles behind me.

The full trail starts from https://archive.google/jobs/lunar_job.html