Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You're ignoring the cost of unrestrained development without standards. The impression I've gotten is that a AWS papers over their lack of development gatekeeping with unreasonable oncall loads. All things considered, the "cost" of readability in terms of development velocity is fairly low. If anyone on your team has readability [in the language], they can review your code. This can be problem if only one person has it, but usually once one person gets through the process, you can get more people through fairly quickly.

I'm a bit of a polyglot at Google (I have readability in C++, python, and Javascript/Typescript, and am part-way through the Go and Java processes, which covers basically every popular language at the company), and IMO, the readability process for each language has been a net positive, although it was painful for a while when I had no readability and none of my coworkers did (but there are procedures for getting around that today).

For example, I "donate" my readability, and review random changes from people who don't have readability but need a quick turnaround or aren't yet working to gain readability.



> You're ignoring the cost of unrestrained development without standards.

I'm not. I'm not saying that code should be merged without any stylistic commentary; I'm simply proposing that stylistic commentary be provided by other stakeholders in the service you're contributing to, on top of whatever linter or sanitizer is being used. That is, after all, the goal of a human reviewer. To add _another_ layer of gatekeeping is what I'm questioning.

> The impression I've gotten is that a AWS papers over their lack of development gatekeeping with unreasonable oncall loads.

I don't agree with this perception at all. AWS spends a lot of time building frameworks and libraries which attempt to minimize human gatekeeping to only the necessary areas. So think retries, backoff, timeouts, circuit-breakers, etc. Instead of adding gatekeeping at the code level (with things like code style), instead gatekeeping is added at the architecture and operations stages, which is where most of the work of keeping a production level service up, available, and fast. Operational level gatekeeping in the bounds of a given organization is a much more neatly constrained problem than stylistic gatekeeping and this gatekeeping happens on a per-service basis rather than a per-CL/review basis. This keeps overhead a lot lower.

My impression from being outside of Google (and having a Xoogler for a partner) is that Google loves bureaucracy and process, and that engineers that continue to stay at Google are fine with navigating these processes. Its answer to problems is to add more bureaucracy, another item for the checklist that a person/committee/reviewer needs to check for. That reflex is hard to shake, IMO.


> That is, after all, the goal of a human reviewer. To add _another_ layer of gatekeeping is what I'm questioning.

I guess I'm not following the distinction. For most people, that's exactly what happens. Either you, or the someone on your direct team, will be the person with readability for your code. The readability process just ensures that you don't end up with silos where style diverges too much, and that the stakeholder who proposes stylistic commentary has a baseline knowlege of linguistic best practices.

With few exceptions, the thing people complain about is attaining readability themselves, not getting changes approved.

> I don't agree with this perception at all. AWS spends a lot of time building frameworks and libraries which attempt to minimize human gatekeeping to only the necessary areas. So think retries, backoff, timeouts, circuit-breakers, etc.

I think we're talking a bit past each other. These things are all table stakes defined by frameworks and updated by centralized processes as best practices change. Human reviewers, for the most part, are only going to say "why are you diverging from the default".

But importantly, that sidesteps my statement entirely, which, rephrasing, is that amazon carries along more technical debt than Google. Whether that's a good long-term business decision remains to be seen, but the impression I get from ex-amazon friends is that maintainability is less prioritized, and maintainability is one of the goals of the readability process. Global consistency helps anyone understand and improve anything else, and to an extend avoids haunted graveyards (though...not entirely).

> instead gatekeeping is added at the architecture and operations stages, which is where most of the work of keeping a production level service up, available, and fast. Operational level gatekeeping in the bounds of a given organization is a much more neatly constrained problem

I'm not sure what you mean, but if anything, this sounds more bureaucratic.

> My impression from being outside of Google (and having a Xoogler for a partner) is that Google loves bureaucracy and process, and that engineers that continue to stay at Google are fine with navigating these processes. Its answer to problems is to add more bureaucracy, another item for the checklist that a person/committee/reviewer needs to check for. That reflex is hard to shake, IMO.

FWIW I don't get this impression. It's honestly difficult for me to tell what you even are referring to. Like, other than readability and promo, there's very few processes or bits of bureaucracy that are consistent across the company, and those two have gotten decidedly more streamlined since I joined the company.

Well okay that's half true, I can think of a bunch of places where bureaucracy has increased, but they're all spurned by legislation.


> I guess I'm not following the distinction. For most people, that's exactly what happens. Either you, or the someone on your direct team, will be the person with readability for your code. The readability process just ensures that you don't end up with silos where style diverges too much, and that the stakeholder who proposes stylistic commentary has a baseline knowlege of linguistic best practices.

Separating these two out, even if one person may fulfill two roles, IMO adds to bureaucracy that I don't care for. Ramping up onto a new language sounds like a pretty fraught experience if nobody on your team has readability and the service you're contributing doesn't have a member who has the time to actually work with you, especially if your work is not relevant to theirs. They even touched upon this in the linked video. The whole idea that you can enter a bureaucratic catch-22 at a company seems like an anti-pattern to me, a moment to stop and think; a "modern" engineering organization should try its utmost to accelerate development, with only as many checks as needed for safety and no more.

> With few exceptions, the thing people complain about is attaining readability themselves, not getting changes approved.

That's what I mean by bureaucracy. Even having "another eye" on the code should work. Having a certification process for readability is even more bureaucracy.

> But importantly, that sidesteps my statement entirely, which, rephrasing, is that amazon carries along more technical debt than Google. Whether that's a good long-term business decision remains to be seen, but the impression I get from ex-amazon friends is that maintainability is less prioritized, and maintainability is one of the goals of the readability process. Global consistency helps anyone understand and improve anything else, and to an extend avoids haunted graveyards (though...not entirely).

Ah, that makes a lot more sense. In my career I've never worked at a place that prioritizes code style that much, and I've worked at other high scale places in the past. This seems like a Google specific need. You're right that global consistency helps to save off "there-be-dragons" codebases, but I feel like this is a case of YAGNI. Unless you have engineers constantly cross-cutting across the company, most engineers learn the style guidelines of a team/product/service by ramping up on the team. The optimization for global consistency seems premature to me.

> FWIW I don't get this impression. It's honestly difficult for me to tell what you even are referring to. Like, other than readability and promo, there's very few processes or bits of bureaucracy that are consistent across the company, and those two have gotten decidedly more streamlined since I joined the company.

But promos drive _so much_ of the company culture, at least from what my partner tells me. And the video in the OP certainly makes it sound like Google has lots of bureaucracy, though I'm not sure how many of the requirements outlined in the video were there for hyperbole or not. But especially for services that have low SLOs, it makes no sense to put thought into failover, evacuation, or anything like that.


> Separating these two out, even if one person may fulfill two roles, IMO adds to bureaucracy that I don't care for. Ramping up onto a new language sounds like a pretty fraught experience if nobody on your team has readability and the service you're contributing doesn't have a member who has the time to actually work with you, especially if your work is not relevant to theirs. They even touched upon this in the linked video. The whole idea that you can enter a bureaucratic catch-22 at a company seems like an anti-pattern to me, a moment to stop and think; a "modern" engineering organization should try its utmost to accelerate development, with only as many checks as needed for safety and no more.

Keep in mind this video is ten years old. While yes, trying to write something from scratch in a language no one on your team has any experience in can be fraught (although this raises other questions: what exactly are you doing, in every case I've used a new lang, my team has been able to find people to review if needed), even in that case, the organization has help for you (see again my prior comment about "donating" readability, and my understanding is that the readability granting process actively prioritizes people who "need" the readability because they have few potential reviewers).

> That's what I mean by bureaucracy. Even having "another eye" on the code should work. Having a certification process for readability is even more bureaucracy.

I think initially the primary driver of readability is C++ (but Java + Guice also...needs it). Take a look at https://abseil.io/tips/. That's 70 tips, which is around a third of the internal ones. I have C++ readability and have internalized, some of those. The readability granting process ensures your code is reviewed by someone who knows all (or nearly all) of those tips, and will recognize when you're doing wrong things in your code and help explain how or why you can improve. When I personally went through the C++ readability process, I got mentorship on how to fix bugs in my code, some of which a peer on my team would have caught, and some of which they wouldn't have. I can now recognize those bug prone patterns (which are hard to lint for, in my case they mostly had to do with parallelism) and know about tools to debug them (msan and asan, which an experienced C++ user should know, but on my team of people who had basically no C++ experience at Google? Nah).

That follows into other languages. Being granted readability means you have a familiarity with the language that suggests you'll be able to effectively mentor others and not mislead them. You don't have that otherwise.

> Unless you have engineers constantly cross-cutting across the company, most engineers learn the style guidelines of a team/product/service by ramping up on the team.

Google uses a single repo and builds stuff from HEAD. This has up- and downsides, but one of the upsides is that its very easy to investigate unusual behavior in your dependencies. I was doing something similar just this week, and fixed obscure bugs in like 7 other teams tools that were causing downstream problems that those teams weren't aware of. No bureaucracy. I found the bug, fixed them, and sent the change to the owning team. Imagine other situations, where I'd need to maintain a fork, or file a bug to have them fix it, or coordinate an update with them.

You'll pay the cost no matter what, choosing to do so in a way that also provides value and mentorship makes a lot of sense to me.

> But promos drive _so much_ of the company culture, at least from what my partner tells me. And the video in the OP certainly makes it sound like Google has lots of bureaucracy, though I'm not sure how many of the requirements outlined in the video were there for hyperbole or not.

I mean the entire thing is tongue in cheek and again is a decade old. I joined Google ~5 years ago, and even then most of the problems with resource acquisition were solved ("flex") and borgmon and its associated readability were gone, replaced with monarch, which is centralized and used a python-based (though still admittedly arcane) query language that doesn't require readability or managing your own instance. And since then things have gotten even more turnkey for any service that's...reasonably shaped. And, well, the level to which promo drives company culture is consistently overstated (at least in some ways).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: