No surprise to me. At one point when they had an API, I could follow and see posts from "locked" accounts that I couldn't see from the website. I reported it and they said something like "oh yeah we're making a lot of changes right now" and of course never fixed it (until they decommed their API which effectively fixes it).
They're up against the world, and they're spreading themselves very thin, aiming to replace all of the "big tech" use cases, for better or for worse. They hate Silicon Valley but they "move fast and break things" more than anybody, arguably by necessity. I'll cut them some slack if their public platforms have some bugs, but they shouldn't do anything with sensitive user data.
I think a lot of it also speaks to the pool of quality engineers who are willing to work for a company like Gab, Parler, etc. Companies like that appearing on your resume will no doubt cause friction with would-be employers down the line, for better or worse.
I certainly wouldnt work there unless I was absolutely desperate and couldn't look more than a few years down the road.
I'd like to think people would not work for far-right hosts of hatred, racism, sexism and often terrorisms for other reasons than just "it would look bad on my CV". I would hope people would have enough of a moral compass to be able to tell that these companies are, in fact, not morally good.
It doesn’t say anything like that. I guess you missed the part about him being a facebook employee. Go ahead, say facebook hires low quality engineers...
Not saying they do (not the poster you responded to) but why would you assume they wouldn't hire low quality engineers? No recruitment/HR/filtering process is perfect, no candidates are perfect, no hiring decision is perfect.
There seems to be an assumption that the big tech firms are inherently better at picking the better 'engineers'. Having experienced a lot of what they produce (not specifically Facebook, but Apple, Microsoft, Google, etc) I'm inclined to assume the opposite personally.
> I'll cut them some slack if their public platforms have some bugs...
Your generosity naively assumes that given enough time they would resolve your concerns. That is never the case with these anti-social social media companies.
It's taboo to talk about, but the correlation between political conservatism and overwhelming amounts of technical debt leading to non-existent operational security is hard to overlook.
Voat wasn't viable [1], 4Chan wasn't viable [2], 8Chan was screwed over by their own moderator [3], Gab obviously isn't secure [4], and Parler was neither secure nor viable [5]. Daily Stormer survived by the skin of their teeth [6].
While I realize that all major platforms have had their dark days, those days usually pale in comparison to a dark day at a conservative-based platform.
Why are we now conflating business viability with technical or security chops?
I think you’ll find extremists of all stripes perform poorly, mostly due to the much smaller talent pools involved. Can you name a handful of highly successful extreme left tech companies?
Moreover, assigning any random highly successful tech company as ‘leftist’ seems silly. What’s less leftist than a vehicle for minting billionaires and further enriching existing ones?
>Why are we now conflating business viability with technical or security chops?
Technical ability has helped many startups leapfrog others in a similar space and time. It's the lore which HN is based upon. In a scenario where lots of actors are trying to censor your site, having great tech chops can help with resilience. Technical ability unlocks new revenue streams and optimizes existing ones.
Viability starts to approach zero if the business faces security issues. Its as if the hull of a ship is breached. The safety of everybody on board now is at risk. People literally get murdered when the wrong DMs or photos leak. Exposing your users digital life to grifters and governments a huge risk to them and your business as a result, let alone the hard cost of managing a data breach. Companies recognize these risks and put checks and balances in to ensure that these things get caught. Some even have a fiduciary responsibility to do so. That said, when a CTO decides to push code, they generally have the ability to override any of those safety systems.
I think you're spot-on that extremist companies can't grow very large and struggle for talent.
It's entirely possible to generate lots of wealth while advocating for wealth to be distributed more broadly. One can argue and support reforming how Trusts are taxed, and may make a bigger dent in the problem than they could by holding up a sign on a street corner. I don't see entrepreneurial success as antithetical to progressive causes. Using the position of power that your success affords to change the rules of society to benefit you and yours... that there is conservative, commonplace, and where leftist opposition should be laser-focused.
Those sites, other than maybe Parler, aren’t anything to do with conservatism. 4chan...really? I don’t think there is a less conservative site on the whole internet. Half of it is anime/hentai and “anything goes” porn, including of the LGBTQ variety.
The cto should focus on strategic thinking and have one or two devs that lead daily work - and code review for such basic issues (or use a code analyser) to detect sql, xss, xsrf, session management, password based user data encryption, message encryption, and other trivialities. Not that i like gab, but i wonder how many rookie mistakes like this one are then blamed on ”foreign agents” and other media bollox.
I'm of the general mindset that the CTO should be the most technical.
I think we see a lot of tech company failures when the CTO is just a manager type that doesn't understand the technology and doesn't have the ability to contribute to it.
In every company I've ever worked for, and any company I know these kinds of things about, the position of a CTO is a high-level managerial role, with lots and lots of budgeting strategizing communcating yadda yadda, so you had better have someone on that seat who is a good manager and good at high-level, big-picture thinking and good with people and a very strong communicator.
If you require that person to also be the most technical person in the room at all times, whew. Either that's a very tall order or recruiting/hiring needs work. More technical than that person who's spent a career getting absolutely ace in their niche? More technical than the data scientist who spends 8 hours a day digging really deep into bleeding edge neural networks?
I'm totally with you that a manager should be able to understand the thing they manage to a certain (not too shallow) depth, but you hire experts for a reason, and in ordinary circumstances, and in an ordinary IT department, a C-level manager should not strive to out-expert the experts under their purview. That kind of micromanagement/smartassery will just drive away the experts, and then you need a new CTO and new experts and maybe go bust.
Exception to the rule: If you're in a startup where the IT departmend consists of a CTO and one guy remoting in from Bulgaria who runs the servers and is really really cheap, you as the CTO actually should be the most technical person in the room. But once you hire the first actual expert, you should take a step back and get more big-picture-y.
I am the CTO of my company and the "most technical" for sure, but I'm not the "most up to date". I haven't built an JS frontend with Vue, React, and Svelte like one of my team has.
Basically I set the team values, priorities, and processes, but I do need a lot of my team to have deeper knowledge of certain things than I do.
> I'm of the general mindset that the CTO should be the most technical.
For a decent size company I'd hope the same, but with the proviso that over time any specific industry experience will degrade to a more general understanding, which is fine.
And if they don't have the experience they delegate.
So bearing in mind the original story, either the CTO should be at least decently technical enough to avoid composing SQL like that commit does, or the CTO should not be committing code in the first place.
My point is that if someone hacks and steals all the data on your social media site then they can expose random people by doing analysis on the site's data and make individuals lose their job. Whether that is something good or bad depends on your point of view. Should the consequences of having opinions against mainstream views and venting about them in public make you liable for those views to your employer? Sometimes the venting is honestly stating your feeling about the subject which causes gasps, the firing of the recent Mandalorian star comes to mind.
Yes, absolutely, some of those views should cost you your job. I don't want to work with people who want to eradicate Jews, or think black people are subhuman, or believe women are beneath men. If I found out someone on my team held those beliefs, I would definitely act to remove them. That's just not compatible with the type of environment I try to create at work.
I haven't seen a single example of someone who was fired because it turned out that they were a fan of trickle-down economics or opposed Obamacare. Those aren't the kind of opinions people are being "oppressed" (insert eyeroll here) for.
But really, if they get fired from one job because of their awful opinions, they can try to find an employer who shares them. I guarantee there's someone out there who will hire people who believe the kinds of things I mentioned above. Maybe their new job will be worse, or have lower pay or benefits, but that's the consequence of sharing those opinions in public.
And let me be very explicit: I can happily work with people of all sorts of political persuasions. I couldn't care less if someone's a Republican, Democrat, or neither. If they're a decent person who treats the people around them with respect, then welcome aboard! Again, I've never heard of anyone losing their job just because they thought William F. Buckley had some interesting ideas.
So it's not about cancelling, it's about who you want to work with. Fair enough.
There's a gap between genocide and tax cuts. What about not believing in gay marriage and trans identity? Or even having strict cultural (I won't even go to racial) standards for immigration.
The answers for that are going to be different for everyone. For me, personally: if you can come to work and be nice to Joe and his husband Steve, and don't talk about their air-quotes "marriage", and you're not a raging asshole about it online, fine. If you're not sure about trans identity but still call your coworker Jane by name as Jane, and refer to her as a woman, and aren't a raging asshole about it online, then fair enough. And by the online behavior part, I mean I'm personally not going to hate you for discussing the topics civilly and in good faith. If you're nice to Joe at work but on Facebook talk about "the gross gay guy at work", and it got back to me, I'd definitely have issues with it.
Some people will have a much lower tolerance for such opinions, just like I know for a fact there are people in the midwest who wouldn't hire me because I'm an "out of touch coastal liberal" or such. Like, I could name names. Similarly, there are people who wouldn't think a thing of it.
Also, I truly don't believe there's such a thing as "cancelling". In every case I've seen of it, the root cause was someone who had been making an ass of themself and the people around them had enough.
Cancelling is a vague term I agree. I meant it here in the sense of going out of your way to discourage anybody from hiring the person in question, which is different from having your own hiring preferences.
Oh, Rowling was "cancelled," you say? What does that mean? Did she lose her publisher? No: her last book came out in September 2020. Her movie contracts? Nope, still has those, too. Can she still find places to publish whatever she wants? Yes. Can you still read the essay that she was supposedly "cancelled" over? Absolutely -- it was nominated for a prestigious award. Is she still a literal billionaire? Why, yes, from all appearances, she is.
Or is what you mean by "cancelled" is that a lot of people said mean things about her on Twitter? Yes, I suppose they did at that. Perhaps she can console herself by rolling around in her giant bin of money.
> Or is what you mean by "cancelled" is that a lot of people said mean things about her on Twitter?
Generally, yes that is what I and much of society means. An organized bullying campaign that tries to destroy someone to enforce their version of politically correct behavior.
And the fact remains that the behavior she was attacked over, whether or not you want to regard her as a human capable of being victimized or suffering nagatively from such things, was not related to any neo-nazi or otherwise objectively "evil" behavior we can wave away as deserving. Or was it? That was actually my question.
JK Rowling's billions of dollars taken away? Were her books removed from sale? Was she fired from writing the screenplay for the upcoming 11th major Hollywood movie featuring her work?
She lost a lot of fans and faced a lot of criticism due to her extremist views. That's a consequence of saying things most people don't like.
So she's an extremist? If she wasn't cancelled does she deserve to be? Seems like you're trying to have it both ways here.
We were talking about what it takes to get someone cancelled. I'd say the uproar that ensued qualifies. And what's with all the snark and cross-examination I'm getting in return for even asking the question? Why are you guys emotionally invested in this argument about JK Rowling?
What evidence do you have that "we are emotionally invested"?
What is "the uproar", some posts on Twitter?
All of the above is weasel words to try to get around the fact that "cancel culture" really just translates to "people facing the consequence of saying unpopular things".
Rowling, who, again, was not "cancelled", faced criticism for her TERF views. TERFs are considered extremists, one that will get you kicked off of platforms for hate speech. This is because at its core they *very unpopular* in all kinds of polls.
Having unpopular views will get you backlash. That is on you.
"You guys" means you plus the other poster who responded to me.
Cross-examination such as this is precisely my evidence regarding being emotionally invested. And despite the fact that you agree with my point ultimately. I have no idea why you care so much to disagree with all the people (certainly you know it isn't just me) that think Rowling was "cancelled", but clearly you care.
The thread is about the dangers to ordinary people in being doxxed by hackers, and none of these angles you come at me with are making the case against it. Of course you're not here for "curious conversation" about that, are you?
If "being canceled" means that people say mean things about you but you face no material consequences, maybe it isn't the awful thing it's made out to be.
More snark? Presumably people are referring to the active attempt to make someone suffer consequences as canceling too. for example the electrician who got fired would still be described as a victim even if he later got his job back. Though boycotts do have material consequences depending how many people participate. I understand the number of people was greater than zero in this case.
> What about not believing in gay marriage and trans identity?
The problem is usually with what backs that opinion up, e.g.: not believing in gay marriage because at its core they don't find the relationship legitimate in the first place. Even discussions on immigration don't have to be bad at their core, it's the "why" that's usually the problem.
Trans identity is a different matter because it's essentially: "what difference does it make to (figurative) you?". (i.e.: one's identity shouldn't matter to someone else)
You could ask that hypothetical about any of a million different things that aren't a protected status.
Should you in theory have a hard time finding employment if you can't be what any employer anywhere wants you to be? Sure. That's on you. Protected statuses are rare, we should encroach as little as possible on the freedom of employers to employ who they want.
Should you starve to death? No, that's why we maintain a social safety net.
In the US at least, there is a welfare system in place that you cannot be "canceled" from if you have the need.
I'd be surprised to learn there are instances of people being canceled and starving to death.
As for going broke... The US system has never and continues to not guarantee people employment. Guaranteed employment is a property of another socioeconomic system that starts with a 'c', but it's not capitalism.
Who's doing the "allowing" in your question? No one has that kind of power except for the state, and suggesting that the government should step in and make "asshole" a protected class seems silly.
Your standard sounds pretty reasonable. I think that there's a lack of trust that people really mean what you're saying. It's a motte and bailey thing. What about when people start saying that denying that transwomen are women equates to "denying the humanity of trans people"? Which puts trans people's lives at risk, they claim. That's practically incitement if you want it to be.
I mean heck, they tried the previous president for incitement of violence. As best I can tell he irresponsibly weaved a false reality for people which (I think inadvertently) lead to the storming, but by no legal standard is that incitement.
Maybe people in my corner are imagining things, but there's enough fuzzing of the lines that we're uneasy about it.
I do think the moderate conservatives to have a little bit of work to do as far as political theory goes in this area, specifically the synthesis of the positions "companies should not be allowed to fire employees for far-right beliefs" and "companies should be able to hire and fire employees for nearly any reason* and at any time."
*save for a Civil Rights Act violation, which is usually recognized as a harm across the board
What about someone who thinks males and females are biologically different? Or that surgery can't change that fact? The line keeps moving. I see people even on this forum attacking others in the worst language for opinions that look mainstream and benign to me.
I don't care either way, but, my question is, why not just legislate this? If you really want to see this as a permanent thing, it should be in the law..
I'm not going to tell a company they can't employee bletcherous cretins, let alone make it illegal. I strongly support the right of people who don't want to employee them to not to have to.
As someone else here said, "asshole" is not a protected class.
She posted in public purposefully, and under her own name, whilst having a job where she's representing Disney with her name
That's a very different case from some private individual speaking privately in a situation where they don't represent their employer
For a more generic case, during your interview, you tell your prospective employer that you agree with their values and want to work together, then after getting hired, you tell them you disagree with their values. Should they still have to work with you?
Eg. You become a teacher at a Catholic school, but you only pretended to be a Catholic in the interview
The poster was talking about hackers exposing anons however. The Disney example doesn't need to prove that particular facet of the danger can happen because we already know it can.
So you think Disney would acted differently if they knew the person had been making an effort to stay anonymous before being exposed? That's not how google handled the gender-differences guy.
What are corporate values? The only thing you are legitimately saying to your employer is you pay me and I do the work and we both make money. That is all. Anything more than that infringes on your personal rights outside of your employment, as these comments are my own. The fact that we have the reach of corporations way outside of work in public life is not right in my opinion, it's akin to slavery. This corporation employs people with these sorts of thoughts??? That does not really register in my mind as any sort of sound reasoning for dismissal, for having the wrong thoughts and saying them in public.
I _guarantee_ you she, and any other actor with Disney, or any studio, has contracts specifically stating that she would not make controversial remarks that may damage her reputation, the show, or the studio.
You can argue whether such things are palatable, to be sure, but she absolutely knew and agreed up front to this.
And realize part of the enticement to accept such clauses is the entirely healthy salary that comes with it.
There is no right to a job. There is no right to immunity from consequences from saying things online. The end.
If you don't like how Disney operates, then vote with your wallet. That's literally the only thing companies care about, and is the only reason why she was let go in the first place.
Saying we should force companies to retain hires despite what they say online, is ridiculous.
If you were to make a general rule, it's saying if you want employment then don't say anything online, period. So it is in fact depriving any employee the right to speak about things publicly all the time. It's like putting a muzzle on a dog so he does not bark. If that's not slavery I don't know what is.
As for right to employment, well how far can you push it. And how does that not violate white privilege principles or corporate privilege principles. We have a corporate class which holds most of the wealth and if you say anything you will lose the right to make a living. That smacks of privilege, in fact white privilege.
Only the corporate class may speak. It's not sound in any way.
You are always free to choose another employer. She literally already found work with another employer.
If enough people speak with their wallets, then disney will follow that. It's really that simple.
You are suggesting that corporations are one monolithic entity, which is simply not true.
I have no idea what you mean here by white privilege or corporate privilege. Nobody has the right to a job. Saying dumb things online and getting consequences is applicable to everyone, because corporations only follow the money.
Why is there this pervasive notion that there shouldn’t be any consequence to your words? You are free to vent all you want, and others are free to react to your vents.
For example, do you really expect someone with white supremacist view to show up to work the next day with a bunch of POC and have everyone pretend nothing happened?
"why on earth should my actions have consequences!?!? this is cancel culture" is easily the most disingenuous cringe thing in the last year, and that is saying a lot.
Anything and everything is apparently cancel culture now. There are no "consequences" anymore, it's all just "cancel culture".
This phenomenon is not new to humanity. There is no immunity from the consequences that can occur from saying things online. If one doesn't like Disney's behavior, then I suggest speaking with one's wallet.
Exactly. Especially when your actions are your own right to say what you think. It's akin to blasphemy and the problem is that the doctrine that you are violating has not been taught to anyone for them to even know that you are not allowed to say that.
Consequences arising from communication outside of work would make you liable to your corporation all of the time. Most of the arguments I see here are people should just know these things they are common sense, and that they have not seen any extreme examples, only things I personally believe are wrong so it's all good. Further some people here have claimed that they would actively work to get people fired that don't share their opinions and their opinions are discriminatory. Though their heart is in the right place, they don't live in the real world where people are raised in religions and cultures which actively promote those ideas. Therefore, they may believe their ideas to be right or generally acceptable. Holding people liable for "correct" ideas in an ever changing democracy is not sane. It's insane, you must be very dogmatic and authoritarian to believe these things just like the people who would work to get gay people fired for being gay, it's the other side of that same coin. It's wrong and I am happy to take all the down votes in order to state that point.
As for assholes being a protected class, it's called right to religious freedom. It's a human right. And many religions teach a lot of things that modern left who would use this sort of tool to power their way into winning. Which then will lead to the right seeking political solutions in the form of their own extreme power grabs.
It's not that hard to learn what one shouldn't say in public.
If anything, the most significant change the Internet has brought upon us is that people lose track of whether they're in public. Hacker News for example? Very public space; everything here is searchable and indexed.
I'm not terribly surprised on their hiring practices. I got a Gab account a few years ago for kicks, and I was banned within a week. I never got any justification for my ban, and after scouring my locked account for any offending posts I can only conclude that their moderators are horribly corrupt.
I love decentralized software, but don't let Gab fool you. They're just out to grab power right now.
Idk when this guy worked there, but today a Facebook product dev likely doesn't get a chance to make a mistake like this. Which is good. But it also means people who've only worked there can have some really surprising gaps
It's always worth remembering that there's a difference between employment and a university degree. Universities don't grant degrees until someone has proved they have whatever competency the degree indicates. Employment means you got paid by the company, but what you did to get paid could be lots of things... And it's pretty easy to lie on a résumé. Universities have a reputation to lose if they grant a degree to someone who goes on to mess up badly; corporations have no such ties to a former employee, and Mr. Marotto making a major error when he's no longer employed by Facebook doesn't reflect poorly on Facebook.
"Former Facebook engineer" means Facebook paid him... No more and no less.
Not to be that guy but some of the worst security vulnerabilities I've seen has been from senior devs with college degrees. Having a degree =/= you know your shit or being "immune" to absurdly stupid mistakes like this.
You'd be surprised how often I've found leaked API tokens, passwords in plaintext etc on github for instance.
> Universities don't grant degrees until someone has proved they have whatever competency the degree indicates.
Universities grant degrees based on the successful completion of course requirements. I don't know if it's fair to conflate that with what you said. As long as you can get C or better in every course required, you have a degree. Does that necessarily mean that you are competent?
It's an even further stretch if you think that at University cares all that much if one of their Computer Science graduates makes a mistake.
> As long as you can get C or better in every course required, you have a degree. Does that necessarily mean that you are competent?
It means that the university granted you a degree (and, indeed, the med student who got a D average in every class is still called "Doctor" when they finish the process). The point is: universities have a vested interest in the question "Does that degree indicate you have competence?" and incentives to change if the answer, on average, trends toward "no" for their graduates. Corporations have no particular vested interest in the question "Does the corporation's name on an ex-employee's résumé mean that employee has competence?" That's some competitor's problem, not theirs.
Software Engineer and former SRE-SE here; also a two year drop out.
> Universities don't grant degrees until someone has proved they have whatever competency the degree indicates
That is not a rule. I have interviewed countless folks with CS and CE degrees that can't get past basic scalability questions in abstract systems design. I've started plenty of hour long interviews only to learn five minutes in that the candidate has a four year degree and cannot correctly identify a use case for binary search nor implement it in plain code.
> Universities have a reputation to lose if they grant a degree to someone who goes on to mess up badly
They absolutely do not lose reputation. We don't publish on a website where the degrees came from that allowed Cambridge Analytica to exploit the American people. Everyone will easily write off the offending programmers lineage as a fluke and move on with their lives. Nobody really cares where you came from in this world, just what you're up to now.
> Mr. Marotto making a major error when he's no longer employed by Facebook doesn't reflect poorly on Facebook.
I agree that this would be a pretty trash take and a poor precedent to establish long term.
> "Former Facebook engineer" means Facebook paid him... No more and no less.
Simultaneously, this is not true. Working for Facebook carries some prestige, as we can see from all the folks who proudly sport "x-Facebook", "x-Google", "x-Netflix" show us. Obviously some people make use of this market and obviously someone is foolish enough to believe the sheen. The value in that perception rests with the viewer; if they hired him because of his work at Facebook then it was significant, if they would've hired him regardless then it matters very little.
Have you ever worked on any really large platforms or databases? IME it's pretty common that the idiomatic Arel/ActiveRecord code you might normally write can become hopelessly slow when the query gets complex and the datasets get large. I'm not trying to pick on ActiveRecord here, pretty much all ORMs I've ever used will do this sometimes. The only way to get the speed up to something acceptable may be for someone who knows SQL and the DB Schema well to write a custom SQL query that makes the most of the database indexes and optimizations.
When a large product in Production is having performance issues, it's very possible that you'll write something like that in a time crunch to get the site back up. The code might not be the cleanest possible because you aren't sure how well it will work yet. It might be checked into Git because any remotely sane deployment procedure requires it. And maybe somebody years later will pore through the Git logs and come back and drop snark on you without knowing anything about what was involved in writing and releasing that code and they've never had to deal with something like that.
At a glance my first though was "there's probably SQLi here", but it's not obvious to me that attacker controlled parameters could actually be used to exploit this. The ID and pagination parameters all seem to be ints, and it's not obvious to me that an attacker could turn them into arbitrary strings. Is custom_filters.phrase the attacker controlled string?
There's nothing in that code that says that any of those parameters have to be ints. They're certainly expecting ints, but there's no casts in that code that would enforce that. Presumably somewhere upstream they're just passing directly from the POST/querystring into these API functions.
You'd expect the ID parameter to be typed (at a minimum from the database). It's not clear from the code in the article whether the pagination parameters are type checked (or whether they'd raise a type error at any point if set to strings).
It's definitely bad code, but if you're going to write an article which so heavily implies "this idiot's bad commit is why a breach occurred", then whether the code is actually exploitable seems worth investigating first imo.
Again, it's Ruby. Unless you do unusual things, there's no type checking beyond duck typing.
We can't tell from just that commit whether that code is definitely exploitable, but they'd have to be doing non-default things to make it not exploitable.
It'd have helped this specific case where we can know that all the arguments must be numbers, but you're going to hit the same injection issue as soon as you need to allow a string argument.
The general fix is bound parameters, not typed arguments.
You could probably use a static type system to ensure you're not allowing string arguments from untrusted input and possibly even allow string constants to be used as arguments. Not that I think this provides any sort of advantage (or that I've even seen this), but it seems like it would be possible and work.
Not an argument per-se, but there were cases where I had to manually build raw query parameter-values when using the `IN(v1, v2, v3)` operation. Many libraries still do not support passing in a list parameter as of 2021, even newer ones. For example, the Rust MySQL library.
I felt much safer doing that knowing the parameters I'm concatenating are integers.
A better answer is both not trusting it and validating it: parameters prevent data from being interpreted as code but validation is also important because allowing unexpected inputs could lead to other vulnerabilities (e.g. XSS in an error message or a partially-generated page) or complications when making changes in the future if you learn that common clients have come to rely on your sanitization logic cleaning up their requests.
Please no. Trying to validate your input for any format that currently exists or may exist in the future is really wrong. You need to handle this at the edges - for SQL this is using prepared statements, for XSS its about using a framework that makes the correct choice (default escape HTML entities).
Anything else is madness. Of course, if you're dealing with legacy systems you might have to do something tragic like not allow people to put quotes in passwords.
I think you might have misunderstood my comment even on the second reply after the ruder version you deleted: validation at edge is exactly what I was suggesting. If you're writing your code assuming a numeric ID, using placeholders means that it won't be exploitable if someone submits a non-numeric value but you're still at risk as described and likely other problems. If you validate it immediately and send an HTTP 400 for anything which isn't a valid number in whatever range you support, you don't have to worry about later finding out that someone came up with a clever way to do XSS, log splitting, etc. attacks even if they couldn't get at the database directly.
This leaves a bad taste as to what the "forensics" could uncover just due to the open development model. It's nice to be the prophet of evident mistakes when the trail is easy to follow, even if you can't exactly master the lingo.
From the article:
>The change, which in the parlance of software development is known as a “git commit,”
It was a change. Parlance commit. Parlance, per tool used, "git commit" (and then check as to tool standard parlance).
My point being, what do we routinely hide thanks to not coding in public? What do engineers routinely hide, when possible?
I would rather, as an engineer, discuss core issues we can fundamentally address: compromising on inadequate workflows (including core architecture and paradigms), commitment to over-delivery, and the ever-dooming deadlines. What victories of the CTO went ignored, as part of "the job"?
It's nice to be smart when you have regular 8 hours of sleep. I've had enough stress to remember just how idiotic many of my decisions were, as a "leader". Most of them went ignored just because we were covered by being invisible, by design. Morally, I can't judge this CTO. If you look at your coding history, can you?
Yes, but then, really, the fundamental popular software paradigm is not just unsanitized but unsanitary. The models for sanitary-by-design are there. It's just math.
The core leadership behind inadequate decisions, often above the CTO, are frequently of the "don't care about the math, just the numbers" type.
Perhaps the CTO raised concerns. Maybe, not. But if we want an open engineering culture in software, unlike "applied engineering" in other industries, we should actively oppose punishing those that embrace open-to-peer-review models, even when the openness backfires and the history gets removed by the open workflow participants.
We may still have a fragile and unique culture in software, that perhaps contradicts past history such as engineering in construction (look up "corruption construction") or the unique corruption of medicine ("sugar lobby", "food pyramid").
Despite bad decisions and the fumbled cover-up, the attempt to perform in public on their part is valuable to me. We don't have easy access to which of the doctors took money to publish "research" that "calories are the same", pushing for more carbohydrates in diets. This may translate to multiple people, people you might personally know, dying of diabetes.
With open software, we get the names. This should not reward click-bait media witch-hunting.
I can 100% judge him. It is the CTO's job to put in place processes and safeguards that reduce the possibility of one of the most common widely known security vulnerabilities. Either he didn't put in the safeguards or he bypassed them, either way it's a fireable offense that put the whole business in danger.
Do you have your commit history available in a public repository? I don't. Honestly, i'm paid for being a professional fuck-up. I just fix things quickly and support my team enough for us to bear the mutual guilt in silence.
There are SQL injection fuzzing tools that will have no problem catching this. This is not the kind of security defect that would depend on "white box" testing.
If you're suggesting that the obscurity of closed source would have prevented the hack then I very much disagree. There are countless examples of sql injection attacks in closed source software.
I am commenting on the core foundation of the "article", to quote:
> "A quick review of Gab’s open source code shows that the critical vulnerability—or at least one very much like it—was introduced by the company’s chief technology officer."
What would the writer have without the open source?
Ok that's true. With a closed source process the company gets to more carefully control the narrative. That might be better for the company and for protecting reputations, but it's not better for the public at large.
Further, with a closed model, one can always peruse the emergency clause, force majeure, the ever popular "state actor".
"Independent experts indicate (fee undisclosed), a powerful malevolent actor was involved in the recent malicious attack on our infrastructure. This aligns with the recent series of threats identified by the State Department and other US government agencies as enemy state activity to undermine Democracy! They hate our Freedom!"
> commitment to over-delivery, and the ever-dooming deadlines.
Surely the difficulty in recruiting people to work for their shitty website with shitty politics should illuminate for you and everyone also in denial that politics is also engineering.
> „ Specifically, line 23 strips the code of “reject” and “filter,” which are API functions that implement a programming idiom that protects against SQL injection attacks.“
This is not correct. The mistake was to use ‚find_by_sql‘ without parametrizing the query. The mentioned reject and filter methods are merely skipping some of the data the query returns.
The previous code that used those functions probably did prevent a SQL injection as a side effect, as using them avoided making a direct SQL query at all.
But you're of course correct that it's not the replacement of an ORM call with SQL that's the problem.
> “Sadly Rails documentation doesn't warn you about this pitfall, [...]" said Dmitry Borodaenko, a former production engineer at Facebook who brought the commit to my attention wrote in an email.
Maybe they were new 15 years ago in Rails, but I was using them in Perl in the mid-90s.
There is no excuse for writing an SQL injection in 2021. Zero. None. And if you're in the position to write code that'll be deployed to production, you darn well better have it reviewed by peers before it's merged.
If the CTO did this and worked around the developers, he's a freaking idiot. If the engineers saw this and signed off because he's the CTO, they're freaking idiots. I wouldn't ordinarily be this harsh about it, but come on. SQL injections in 2021? That's astoundingly bad.
And Visual Basic itself had them 23 years ago as well.
Though, the inflection point, at least in my circles, was in 1998, when rain.forest.puppy talked about "piggy backing" SQL commands.
Even there they mentioned using parameters with stored procedures to protect yourself.
It's just clear no one in IIS land did at the time, so there was a wide open exploit playground for everyone that saw that release of Phrack, once they got done playing with directory traversal exploits.
> Specifically, line 23 strips the code of “reject” and “filter,” which are API functions that implement a programming idiom that protects against SQL injection attacks.
That's not true at all. If you are going to do technical analysis that calls out mistakes, the publication should have multiple technical people review the article.
Unfamiliar with Ruby/Rails, why's that untrue? Is it because `reject` / `filter` are just array mapping functions rather than ActiveRecord methods? (Best explanation I could come up with on a quick skim.)
Yeah, at a glance it seems they had a performance problem, possibly from their use of reject and filter which you correctly guess operate over the results instead of becoming part of the query.
I'm guessing he was not familiar with the codebase and maybe not even ruby/rails in general and just wrote the whole sql query by hand, not bothering to figure out the correct way of parametrization , using string interpolation instead.
I had the same reaction, but upon reflection, the reject/filter actions themselves aren’t what’s safe, it’s the usage of functions that accept unsanitized input and safely wrap the SQL that’s going on here.
> Besides the commit raising questions about Gab’s process for developing secure code, the social media site is also facing criticism for removing the commits from its website. Critics say the move violates terms of the Affero General Public License, which governs Gab’s reuse of Mastodon, an open source software package for hosting social networking platforms.
Lmao, that's not how AGPL works. Although, it would be pretty funny if a license explicitly required you to post all your Ls for public ridicule.
The GPL family, including the AGPL, defines source code that must be provided as “the preferred form for making changes”; if the form the distributor uses is a VCS repository with full history, publishing either static snapshots or a repository with an expurgated history could arguably be a violation of both the letter and the spirit of the licenses. The language of the licenses makes the requirement dependent on actual development practices, not fixed, and that's not an accident.
I'm guessing you're referring to the first sentence in section 1:
> The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work.
I don't think that's referring to the method by which I make changes or manage those changes. It's simply trying to make it clear that source-code is something that should be in a form you can actually change, and not something obfuscated. e.g. minified, transpiled, object-code, etc. The second sentence gives the first more context.
The spirit of all GNU licenses is that the end-user should have the freedom and ability to understand or modify software that's *distributed* to them for themselves. It has nothing to do with having a full history of the source-code available. What does full history even mean? I'd like every auto-save to every LGPL licensed file, please.
> My opinion is that it's more likely than not that these sites are little more than honeypots to troll the gullible into making asses of themselves so that we have another subject for our Two Minutes of Hate.
Or maybe, instead, some people really do hold extremist views and post them on the internet? I don't understand your theory here - the majority of Gab content is public already (well, behind an authwall, but otherwise public). The hack achieves nothing for these masterminds that just want to make fun of people being bigoted on the internet because they already could.
> Now that data can be used against the most ferverent Republican supporters and there's no one to blame, officially, because they were "hacked". Oops! Maybe you should think twice before publicly supporting Republicans online, I guess!
Do you seriously think the problem people had with some of the content on Gab is that users were "supporting Republicans"????
GP has edited their comment like 10 times so here's the version I'm replying to:
> I'm one of the world's biggest fans of Hanlon's razor, and I've done so much damage to it with Gab and Parler that it looks like a hairbrush now.
> My opinion is that it's more likely than not that these sites are little more than honeypots to troll the gullible into making asses of themselves so that we have another subject for our Two Minutes of Hate.
> I do a lot of research on social media mechanics, and I've used both of them. I can't quite put my finger on it, but they performed and felt differently from any other sites I've used over the past 20+ years. For one thing, you could not read or see ANY comments without registering first.
> I could just as easily be imagining it, of course. And they did paint a huge target on themselves. But they also made it easy to steal the data.
> Now that data can be used against the most ferverent Republican supporters and there's no one to blame, officially, because they were "hacked". Oops! Maybe you should think twice before publicly supporting Republicans online, I guess!
> (Not affiliated with any political party, apolitical, have friends and family who are into all different flavors. Just think it's fucked up to throw thousands of people under the bus like that, if it's true.)
I think you're encountering Poe's Law here - at some point it becomes difficult to tell the difference between a troll and someone genuinely expressing an extremist point of view.
Are there trolls on Gab and Parler? For sure. Are there purestrain rabid culture warriors on Gab and Parler? Yup. Are Gab and Parler honeypots? I don't think so, but that also depends how you define "honeypot."
Both exist because other social networks cracked down on distasteful content. Gab erupted during the era of cracking down on ethnonationalists and is full of wannabe nazis and 4chan-grown edgelords. Parler erupted during the era of cracking down on dangerous misinformation (QAnon, COVID denialism, the things that we now know lead to election denialism, etc etc) and if full of people that choose to believe that nonsense rather than reality. Parler was also signal boosted by political voices that intentionally framed crackdowns on distasteful content as an aspect of the ongoing culture war.
Both naturally courted their respective audiences.
The guy running Gab is a True Believer. Check out Gab's Twitter (heh) for some pretty questionable recent posts that blame the hack on "mentally ill trans hackers," only with a slur instead of trans, because of course there's a slur.
The Mercer family bankrolls Parler. They have a long history of throwing money at causes that follow their political viewpoints. We don't have a reason to doubt that their board and and management are True Believers in their own cause as well.
So, is it really a honeypot if it's honest by design?
He's definitely not but he knows how to play to his audience. Instead of owning it and saying 'we messed up' it's easier to blame the 'mentally ill trans hackers'!
But the 'hack' was almost certainly intentional, it's the exact same play Parler did a few weeks ago. They're both honeypots, this is the easiest way to get the data out there.
I do like your theory that it was a honeypot from the left created to produce people to attack.
I would be surprised if our intelligence agencies and law enforcement didn't have a role in creating/running extremist sites. Seeing as the FBI likes to run child porn sites and infiltrate political groups that could cause instability, they are the most obvious perpetrators.
FWIW Gab very proudly proclaims that they report things to law enforcement if they cross a legal line. I could be wrong but I would think that if the FBI was trying to entrap people they would not want to give such a warning.
I think you are wrong, stating it prominently and repeatedly makes the warning disappear into noise for users so there's little downside and a lot of upside if it were in fact being operated by the FBI.
I doubt it is though, I don't think the FBI has the capability to be frank, but even more if the FBI were operating it as a honeypot somehow I doubt that the websites would have been banned from AWS and such.
I'm sure that if it were true the FBI could have easily just told AWS that the presence of extremist content was being allowed by law enforcement specifically to identify terrorists.
I think the idea that it's a honeypot is a very poor fit to the available information.
Occam‘s razor would disagree. While it‘s certainly thinkable that it‘s a honeypot of the FBI, set up from the left (just like it‘s possible that it was Antifa who stormed the capitol posing as Trump supporters), it‘s far more likely this was a job done by nutjobs, for nutjobs.
If you also think the earth is flat or the election was stolen despite literal mountains of evidence, how are you supposed to properly weigh any other rational evidence like the existence of SQL injections?
Or maybe they just didn't build a strong enough engineering team and quickly ran into the limits of their experience and skills levels.
There are numerous examples of start ups and even mature companies making basic mistakes. This is easily explainable without resorting to conspiracy theories.
> maybe they just didn't build a strong enough engineering team and quickly ran into the limits of their experience and skills levels.
"Free Speech platform Gab has announced Fosco Marotto as the company’s new Chief Technical Officer (CTO). According to a blog post from the company, Marotto was formerly a software engineer, production engineer, and developer advocate during a seven-year career at Facebook.
Marotto reportedly brings 23 years of industry experience to the platform along with extensive knowledge in backend infrastructure and insight that will help Gab scale as it becomes increasingly popular."
"If your app is compromised, it’s not only you as the developer who suffers, but potentially the users of your app as well. Continue reading for our suggestions for sensible defaults and precautions to take before releasing your app into the wild."
*A strong engineering team should've had vulnerability scans at some point in their build process. That SQL injection vulnerability would've been easy to spot
*That SQL injection probably should've been picked up in a code review
*They should have been using parameterized queries in the first place. The fact he removed input sanitization methods is besides the point, that they shouldn't have been relying on those in the first place.
*A senior engineer or CTO should've known better. But I've seen very senior people make bad mistakes before. These mistakes are much more likely with immature processes and safeguards.
*Sometimes someone can get to a 'senior' level without necessarily knowing how to do some aspects of software engineering well.
A strong engineering team is about the performance and practices of the whole group. You can have individuals who are experienced at particular skills but have big gaps in the skills and experience needed to build a strong team.
I mean, "worked at Facebook for a while" isn't a guarantee of competence. And some security boilerplate text isn't a guarantee of being good at security.
You want guarantees in software resources? Have you heard about the famous interview torture for jobs?
Your OP wanted to paint a picture of inexperienced development team. Both items in my post show that both the team (who initially had it right) and the CTO (who inexplicably removed the two lines) are definitely experienced.
That Parse Server looks like a substantial piece of code and there is a community around it, so let's be fair and grant the CTO a measure of competence, shall we?
I mean, given a choice between "an incompetent programmer with seven years work experience exists", and "Gab as a whole is some sort of weird honeypot", Occam's razor probably suggests the incompetent programmer.
23 years of experience + Existing codebase demonstrating care for matter such as filtering input == experienced. Certainly not clueless.
I said absolutely nothing about a honey pot. Review my posts. Occam's razor strongly suggests to not make un-necessary assumptions. My post challenges your assertion of incompetence. You need to remove that assumption from your analysis.
The thread was about the claim that it _must_ be a honeypot because it's so incompetent. What's your theory for how this happened, if neither incompetence nor deliberate?
> What's your theory for how this happened, if neither incompetence nor deliberate?
Honestly no theories, just rubber necking here :) but since you ask, here is what my shaving session with Occam brings up:
- cognitive impairment (alcohol, various fun chemicals)
- emotional agitation (relationship, personal matter, etc)
- rushed for time (silly mistake)
- arrogance ("I can do C++, Scala, JS, and PhP. This Ruby stuff will be cake, and I do not need to ask my team why they had those 'funny' filters on top of the function")
- malicious coding ("Sure, I'll help you scale it. I hate SV too. I hate it so much I spent 7 years at FB.")
So re. that last possibility, I also do not agree with your GP that this indicates that that platform was a honeypot. This specific event can be construed as placing a backdoor in their code.
True. I worked at a company that once hired this former Yahoo engineer when they were hot and he sang the Smurf song often and wrote shit code. All the engineers were happy he was the only dev in his section because he also smelled.
Even with all that, I would still expect him to not make any commit similar to this CTO.
Ok so if in 2021 a “cto” or developer, let alone former facebook developer, doesn't use bound and escaped parameters as a minimum then that developer needs to take a break and catch up on things.
On a related note, I've become a gigantic fan of Slonik, https://github.com/gajus/slonik, which simultaneously (a) makes it extremely difficult to expose SQL injection attacks while (b) still letting you write SQL in a very "natural" way using template literals.
It takes advantage of a somewhat-not-well-known feature of Javascript template literals in that you can apply functions to them, e.g. sql`SELECT foo FROM bar WHERE id = ${userEnteredValue}`. No need to manually escape userEnteredValue, the sql template literal function does it for you.
You get the same benefit of "template literals" but the data and query are separated at the DB level. The DB knows what is data and what is query therefore any attempt at escaping from SQL will be quashed there.
That's what the library does. It just uses tagged template literals[1] so that it "feels" like you're writing a template string, while really it's a prepared statement.
Fuck this story, and shame on ArsTechnica for publishing it.
We shouldn't call out and shame individuals for making a mistake like this. Any company which even does something like this internally is toxic; the blame lies on the company and the process, not the individual. The flimsy excuse "oh, theys a bawd bawd company filled with icky people" doesn't hold water; we're talking about basic decency here.
You want to write a story about the commit? Hold the company responsible? Encourage processes and procedures which would have caught this? That's fine. Blur out the names. Ars isn't a tabloid, but they're acting like one; there are real, serious consequences for shit like this. You didn't set out to create the perfect ammunition for someone within a company arguing against the open sourcing of some project, but here it is.
> We shouldn't call out and shame individuals for making a mistake like this.
For a random line worker, sure. For the CTO, the position is absolutely relevant. You can't have a conversation about what processes or procedures might have caught this without taking that into account; in every small company I've worked for the CTO was either tacitly or explicitly able to bypass any amount of code review etc..
The fact that it's the CTO is informative, educational, and important, and should be reported. Their personal details are irrelevant but hiding them would also be meaningless.
Especially in Rails it is. Since there have been so many iterations on how to sanitize input strings and typically you have explicitly choose not to sanitize it to allow for this to happen. Which is exactly what the author of that code did. So yes, I'm in general against mocking coding mistakes, but in this case I'd say its warranted given that the person at fault actively tried to cheat his way out of using the framework provided primitives.
I don’t know if we have comparable life experience.
I have seen more shitty code than high quality code in my life. It’s hard for me to empathize with anyone who thinks startup code is going to generally meet some decent quality bar.
People like Brian acton and Jan Korum exist, but they are the exception. If you always expect people to write good code, you will be fucking disappointed.
The only solution that scales is having adequate speed bumps and guard rails that prevent people from pushing code till after it’s been properly tested. And just because it’s in a git log doesn’t mean it gets pushed to production.
This article reminds me that journalists only make money when they squeeze your amygdala. Reading content from ars is the equivalent of an intellectual jump scare. Add 127.0.0.1 arstechnica.com to your resolv.conf file for mental health. Fuck off ars
Right, but the point is that either Gab had normal, reasonable safeguards in place (for the reasons you point out) and their CTO bypassed them creating a very serious security hole, or they didn’t have the safeguards in the first place meaning their internal processes are poorly set up and controlled.
What’s important here is that the CTO is ultimately responsible for the failure either way. That’s the point of the article and it’s why the article is valid (even if you don’t like it for some reason): the engineering buck stops at the CTO, especially so if they personally create a bug that exfiltrates all of the company user data, regardless of how they managed to screw it up.
Brian Acton and Jan Koum weren’t exceptions either. WhatsApp had some hilariously bad vulnerabilities. Like no authentication on their APIs and trusting the client.
To be fair, Rails has a lot more baked in base security than erlang did. Yes Rails had it's fair share of vulnerabilities. I remember there was a major eval issue in some deserializer code, but things such as sql injections or using POST over query strings is just basic knowledge almost every mediocre+ rails dev knows. They picked erlang for their scalability, but as a result had to write a lot of other things from scratch. WhatsApp is still an interesting engineering case study.
it's not a rails vulnerability though. This would happen irrespective of the language/framework if the developer decide to forego basic sanity checks. This would have been prevented with basic input validations (in any of the layers before it hit model, and model itself), or the organization did basic security testing.
You miss the point. If you use rails provided primitives it absolutely wouldn't happen. So it's fair to categorize this as a rookie coding mistake. The author took code that had these security primitives baked in the framework and REMOVED them, replacing it with insecure custom sql, which in turn led to the vulnerability.
This is totally untrue, considering the caliber of companies which have been hit with SQL injection vulns, including Google, Facebook (your employer), Reserve Banks, governments, etc.
I don't know what experience you have, but senior engineers constantly make security mistakes. Linux kernel developers are some of the smartest programmers in existence yet they continually introduce and fail to patch security vulnerabilities.
No senior engineer should blatantly concatenate strings to make an SQL query. That is a junior mistake, no doubts about it. If you haven't learned that user input is not to be concatenated into code unless you think long and hard about it and you're really sure there's no better option, you are not in a position to lead.
Seriously, if this is a mistake a senior engineer could do at your company, why would you work there? I'm not one for gatekeeping but this is learned within a couple of code reviews in your first years.
By your logic, nobody should work at Facebook, Google, Apple etc etc etc.
Unless you're claiming that those SQL injection exploits were introduced AND review approved by junior engineers, a practical impossibility according to policy.
Your claim that senior engineers don't make whopping security blunders is absolutely false according to all available evidence. The smartest programmers working on kernels make them constantly.
Former Facebooker, on the storage team—if a senior engineer* wrote the supplied code, well, I can't say they'd be fired, but it 100% would not reflect well on them come review time. Also, all their peers would know they made the most boneheaded of mistakes, and they would definitely be getting the hairy eyeball come code review.
Raw untrusted string substitution into a SQL query is just unacceptable. It's on the level of storing unhashed passwords, or 1000-line god functions: the kind of mistake you'd expect from a self-taught developer writing PHP in 2002. Literally every popular SQL client and ORM provides quoted substitution; please use it. And, dear reader, if this was news to you, and you write SQL, please go get educated, because it's not the last nor the sneakiest pitfall in this space!
* Senior enough to be writing raw SQL code at Facebook
Facebook has had multiple injection exploits, including SQL and JS RCEs. It's possible no senior were the git blame behind them, but it's absolutely impossible that no seniors reviewed and then maintained that code.
WhatsApp had a raw JavaScript RCE literally last year.
Also, blaming colleagues for making a mistake is extremely toxic behavior. I've fixed dozens of >9.0 CVSS vulnerabilities at multiple billion-dollar organizations. I'd say senior engineers were involved in their introduction at least half of the time.
I suspect that the mockery comes at least in part because it's Ars reporting on Gab. If it were Breitbart reporting on MoveOn.org, then I have a feeling folks would be a bit more receptive to your call for empathy here.
I'm with you in the sense that it always is the process, rarely the dev. It's complicated that this mistake was made by the CTO, who is responsible for being sure such processes are implemented. The lesson for me is that we all must be continually vigilant against both shitty code and evil, both without and within. Anyone can write terrible code, irrespective of political stance. Likewise anyone at all can do evil.
Bit of an aside, but it's hilarious that we've gone from SQL as a language targeted at non-programmer business analysts to something that apparently only senior devs should even consider writing out manually. The whole 4GL experiment turned out to be a disaster.
I think that's more of an OLAP vs OLTP difference here. Say you're an international shipping company: you would be perfectly happy to have your analysts write SELECTs versus your shipment database (or a replica), but you wouldn't want them writing the UPDATEs executed from field data.
I've been on teams building web applications for 14 years and not once have I seen someone simply rip out a query builder like this in production code and replace it with a non-parameterized string—let alone someone with 20+ years of experience.
Accidents and SQL injections happen, usually because of non-obvious query-building but this is a different level.
When swift was still young the only mysql library available for it did the same mistake. The memory safety sure didnt help here... Sure enough i had to roll my own code.
Well not all injection vulns deserve this level of scrutiny.
I'm not familiar with that vuln, but I don't see how it could be the "same mistake". My guess is there simply wasn't support for parameterization or there was a non-obvious concatenation problem when building the query string—but please correct me if I'm wrong.
This instance is novel because the parameterization protection was removed in favor of concatenation and that the vuln is so obvious a first-year CS student wouldn't struggle to identify it.
It happened to your employer. You might not have been on the team but this stuff is rife. WhatsApp in particular has made some absolutely rookie-tier security blunders [1].
Yeah, I beat this dead horse every time the armchair security quarterbacks climb on their high horse in these comments. These “dumb” mistakes happen to everyone, at every company. Even security experts. Even one of tptacek’s companies had a well known incident, and people said all the same things about their “dumb” mistakes.
Companies are very porous, people take shortcuts, temporary workarounds get forgotten and left in place, etc.
You can tell who actually has real experience by who is saying this and getting downvoted for it, ironically. All of the downvoted comments in this thread are correct.
I take the same standpoint as you most of the time. But you can't fucking interpolate user input into SQL queries without physically cringing and call yourself an experienced engineer. Can you really pull up a code editor and write that out without your brain sounding off a thousand alarm bells that you're massively messing up something?
I can imagine myself doing this, having not slept in the last 20 hours, after 7 martinis and one especially long island. Maybe he did that too, but he should have the common sense as CTO of the company not to!
Who knows what was going on. In an ideal world we have the time needed to do things well. In reality, you've got twenty developers asking for something, users requesting features, various government agencies asking for documents, the lawyers inform you about some case and... oh your wife is calling on the other line because... etc etc etc.
This is the textbook example of what not to do with SQL queries. You can't even learn SQL or an SQL library API without being warned against this a dozen times.
If the CTO pushed this code, and the processes the CTO themself is responsible for designing did not catch this, it is safe to say the CTO is incompetent.
> These “dumb” mistakes happen to everyone, at every company. Even security experts.
Dumb mistakes happen, true. This one should never happen to security experts, ever, unless their expertise is fake/bluff.
Stripping out safe parameterised code and replacing it with injected user input is only something that "happen(s) to everyone" at a very early stage in their career.
- For an intern or a junior dev, it's just a learning experience and no names.
- For a dev with a few years experience, it's a cause for concern but still no names.
- For a senior dev, I'd wonder if they lied on their CV but still no names.
- At CTO level this is not acceptable. Maybe no names, I'm on the fence, but they are a semi-public figure by nature of their role and their name was on the commit so they are the ones that made it public.
Either way, the CTO should have lost all credibility with their colleagues and be out the door. They represent the norms, processes, and safeguards the company employs and should not be in that role.
PS. I understand that there is no proof this commit caused/allowed the data exfiltration. It doesn't matter; the commit is heinous enough on it's own merits.
While it doesn’t make sense how a mistake like this could happen, I also don’t think this could be intentional either. I don’t see why anyone would throw their reputation into the gutter like that.
Imagine trying to explain this to a future employer.
There’s really nothing stopping one developer from pushing a commit purporting to be from another developer. You can put any name or email address you like in each commit. However, that’s something that should be caught during review, so it’s not much of an excuse.
Even a small company has a huge amount of things going on at the same time. Any one thing in isolation is simple, but put together they amount to a huge amount of work where you can't put the time needed into become an expert on everything.
That applies to anyone any employee, including the CTO, especially so. They're just people.
Publishing an entire article to shame them is just disgusting. Like, why? The reporter if anyone deserves being shamed for apparently making a career of flinging shit at people.
Yeah most management books (See: Radical Candor) say that executive’s should own up to their mistakes publicly and receive public criticism so that it makes it easier for subordinates to give criticism and even receive it themselves privately.
On another note, how do we know that the new CTO of Gab didn’t just do this on purpose so that there would be a breach?
It’s easy to be armchair sql injection expert and point fingers.
I can guarantee that any system that you’ve worked on has numerous OWASP security bugs. You’ve probably looked at the bugs countless times and never noticed it.
Every software engineer of all levels has overlooked obvious sql injection bugs in their code base. Most likely you’ve added to the bug list.
Software bugs are simple part of any development effort. All major companies, Microsoft, Google, Facebook, etc. has very simple bugs like this in their systems.
That’s why they pay out bug bounties, it’s cheaper for them to add the bug and have some random security researcher find the bugs for them.
I was of the same opinion as you before reading the article.. but really? Genuinely just putting user input in SQL queries? I mean, everybody starts somewhere.. but how the hell do you get to be CTO of anything when nothing goes off in your brain when you're interpolating raw user input into SQL queries? WHAT? How? Everyone slips up, concatenates strings when they shouldn't, whatever.. but this genuinely looks like he didn't know that SQL injections are a thing. That is unacceptable for a technical leader.
I barely know much about programming. I have finished LPTHW years ago but I'm not a coder by any stretch. Even I know to google how to sanitize user inputs before putting something into production.
The only thing experience will do is decrease the frequency of errors. I mean this is why there is a movement to prepared statements and safer language. None of us is smart enough to sanitize correctly or avoid buffer overflows.
I can talk about the commit, and I can talk about the article.
We were getting crushed with scaling from 3M to 30M visits a month, and every day was a 16 hour day. The site was falling over on a daily basis, and the performance of the home feed was atrocious. The APM traces were full of loops. The muting and blocking features were incurring tons of additional queries. As I remember it, one night I decided to write a SQL query to replace it. I believe I thought the values were already being sanitized, but I have no problem agreeing that I should've looked into that further and made sure. I'm not a rails dev, and I'm generally negative towards rails and ActiveRecord. I wrote SQL for many years in the past, and as the folks who went through my stackoverflow saw, I am not unaware of the importance of sanitizing user input. I've written much code that properly sanitized user input, in various languages.
It's really easy to imagine that Gab is a large company, considering the breadth of their platform and products, but you'd be shocked and impressed. It's still really early days, like this is only Chapter 2 for us.. we're growing and will keep getting better.
This article is interesting to me for other reasons though: 1. there's been no confirmation this commit was related to anything that has happened. and 2. the person who reached out to Ars with the story, a link to the commit and a quote, is a former co-worker with a personal grudge. I can only imagine the glee with which he wrote that email.
For the last 2 days we were being extorted for nearly half a million dollars. I received a death threat today, first one ever. I think we're doing something important. Go ahead with your code critiques.
I've been on the other end of a journalist with a grudge, and it's not something I'd wish on anyone.
It's interesting that the technical language in the article is so clearly mangled. Either Ars Technica has no editors that know how to edit (unlikely), or that know the basics of git (again, it seems unlikely). It reads like a journalist taking notes from a dev ("uh-huh, a 'git', right. Oh, no a 'git commit'. really?").
Thanks for the honest description of what really happened there. I've also had the experience of needing to rewrite a query that used a bunch of safe ORM statements but was painfully slow into a big, messy raw SQL query that was probably less safe in order to restore performance. It's easier than many would think to miss whether or not some check was performed somewhere along the line when it's in the middle of a complex process.
Meanwhile, Ars still has a few decent articles/departments, but it seems like 50% of their content is tabloid-grade cheap and mindless bashing of anyone who disagrees with the elite mainstream opinion. Probably like 80% of the mainstream media right now is even worse.
I wonder how Ars would fare if somebody was to tally up all of the boneheadedly stupid mistakes they've made in journalism, grammar, spelling, site administration, etc, and publish articles trashing the specific employee who did it each time.
That explains a lot, and shame on ArsTechnica for enabling that person's petty revenge.
Anyone, including a CTO, can make mistakes and introduce accidental vulnerabilities. It's not productive nor healthy as a culture to blame someone for being imperfect.
The best thing to do, as a company, is to learn from such a mistake, and improve processes to catch and fix it earlier, before it gets released.
It's disappointing to see Ars participate in a low-ball personal attack, calling out a "rookie coding mistake". They're encouraging a toxic mindset.
It could be in anything. I’m nobody and I’ve had death threats uttered about me on Gab; it’s the only thing the site is good for, talking about who to kill. If this guy’s only had one it’s frankly perplexing.
Hi Fosco! Thanks for this breakdown. It’s helpful to know that you’re still incapable of accepting accountability. Can we assume from your cagey “there’s been no confirmation,” defensive sign off, and immediate retreat from transparency as confirmation that you did indeed shit the bed? Maybe you got a little rusty after 5+ years of Facebook paying you to post garbage on Workplace.
And don’t give too much credit to Dmitry - he didn’t think of you differently than any other fascist white supremacist. You earned this coverage through the sheer force of your own incompetence.
Best of luck to you all heading into Chapter 3! Can you give us any exclusive hints as to what’s coming? I think it’s even odds between a wave of mass shootings and being overwhelmed by child porn.
Lol. Whatever you say, clueless dude. I assume you still work for the company which is responsible for, what is it, two-thirds of all child porn reports online? Yes that was it... 12 Million cases in 2018. It's also the premier destination for live streamed violence... But, gotta stay focused on attacking those evil Trump supporters right?
HAHAHA you worked there in 2018, genius. And yes, FB catches and removes more child porn than any other platform, but that’s because other platforms aren’t catching it and removing it, a pretty basic concept that literally anyone who has any business being CTO of a platform should know. LOL, Gab is so fucked.
You’ve changed your tune since your bootlicker “we should all be Team Facebook” comments on every Zucc post were a mainstay on Workplace.
You predicted Trump would win; he lost. You said Qanon was harmless; they stormed the capital. You thought you could do this job (and you started while you were still pocketing $ from Zaddy); you ate it just a few months in with a cock-up so basic that it would be laughable if it hadn’t seriously hurt your users, company, and reputation forever. Honestly, you should get therapy.
Yeah, we should not blame software engineer Fosco Marotto for introducing a rookie error into the site, we should blame the person in charge of setting up standards and processes so such errors never make it into production, Fosco Marotto!
> You didn't set out to create the perfect ammunition for someone within a company arguing against the open sourcing of some project, but here it is.
One important note: Gab is currently running a fork of the Mastodon software, licensed under the AGPL-3.0. It's not their right to decide whether they want to publish their source code.
It is however their option whether they want to squash, rebase, and/or rename committers when they share the source code as they are obligated to do under AGPL.
Normally I'd be inclined to agree with you on this, your bottom-line software engineers shouldn't be outed and publicly shamed. In this case, however, the CTO, a person at the top of the org chart, must be cognizant of these sorts of vulnerabilities especially at a company with a growing userbase and a target on its back (mind you I don't support Gab or anything they stand for, it's just the reality of things). More importantly, you have the CEO actually, really trying to scapegoat the incident on 'mentally ill <t-slur> demon hackers' when its their own incompetence in the top ranks to blame. It's an exception to the rule.
You also seemed to miss this part of the article, which is very telling beyond just shaming someone
> Besides the commit raising questions about Gab’s process for developing secure code, the social media site is also facing criticism for removing the commits from its website. Critics say the move violates terms of the Affero General Public License, which governs Gab’s reuse of Mastodon, an open source software package for hosting social networking platforms.
Gab is likely breaking license agreements by performing this sort of cover up. So yes, this is newsworthy.
Gab is likely not breaking license agreements, as long as they provide it upon request. Which was never tested.
That's the problem with reporting like this; it jumps to conclusions to justify outrage, while clearly being written by someone who is overworked, underpaid, underexperienced, and only has a cursory understanding of the systems at play.
Of course, most people are underexperienced in domains like this, so they rely on traditionally high-quality reporting sites like Ars to provide accurate, precise, and fair reporting. The wording Ars used is accurate; your interpretation of their wording isn't. That's the responsibility journalists have; what they say matters, a lot, because people interpret what they say in different ways.
Ars said that they're "being accused" of breaking license agreements. You said they're "likely" breaking license agreements. That's a world of difference; if for no other reason than because one is correct, but useless, and the other is (probably) wrong.
> We shouldn't call out and shame individuals for making a mistake like this.
Yes we should. This individual is the company. He's the CTO. "Chief" Technology Officer. The buck has to stop somewhere. How can you possibly suggest with a straight face that he should not be held professionally responsible for the code that:
A) He personally wrote
B) He is responsible for professionally as the CTO
CTO of a hot company, knowingly exposing user data (only 1 year CS students can claim that they don’t know that you have to sanitize any input into hand crafted SQL queries) totally deserves a public scrutiny.
You're using "knowingly" wrong, unless you can prove intent. You're also totally incorrect, considering how many companies have been hit with SQL injection vulns.
If I design a bridge without calculating load rating I’m knowingly, as an engineer, putting public safety at risk. Hand crafted SQL statements without sanitizing input in 2020 are same level of negligence from software engineers - you know you’re putting your system safety at risk.
If you were writing SQL queries and not sanitizing inputs in 2010, shame on you. If you're doing so in 2020, you have no excuse. Even if you never read any tech news, all of the major languages and frameworks make it easy to parameterize inputs and hard to do it wrong. Every SINGLE guide about writing SQL on the internet will give this exact code as an example of something not to do. If you're this ignorant, that kind of flagrant disregard means you don't deserve the title engineer - much less "CTO".
Having no clue and doing nothing to remedy it is negligence. SQL injection vulnerabilities in particular are so infamous that parameterized queries have become the default interface. It's the path of least resistance and people have to go out of their way to avoid it.
Unpopular opinion - we should stop pretending that reading few tutorials and competing coding bootcamp, or taking few CS classes makes you an engineer.
It can make you a developer, aka code monkey, but proper engineering requires more comprehensive training, that focuses a lot not only on basic coding skills, but teaches you about design, managing lifetime of products, security/safety/ethical considerations, etc.
A lot of people who call themselves engineers in IT fields are much closer to code monkeys than engineers.
How many have been hit by SQL injections introduced by their CTO, who also has previously written [1] about the need to use parameterized queries instead of string interpolation?
Incredibly unfair, the character assassinations are totally uncalled for. The infosec community has a real problem with tearing each other apart. So much so, that journalists now also think it's okay.
* Just a reminder, they were tearing into the CISO for having a liberal arts degree (as if college degrees actually prepared you for anything..).
Totally agree here: blame the company and the leadership for not investing in the right processes and tools. People will always make mistakes.
Blame the people at the company who didn't make sure the right processes and tools were in place? It seems like while CTO made the error, he was also responsible for making sure he wouldn't be able to make such an error.
He failed at doing his job as a CTO and then failed at being a moderately competent entry level software engineer.
So you don't think it's newsworthy that the most sensitive financial info held for every American was supposed to be secured by someone with no security background?
People's lives were destroyed by that breach. It wasn't a victimless mistake. The public wanted to know how it happened, and hiring the wrong person is part of how it happened.
I think it's unfair. A single person doesn't bear the whole responsibility of a breach like Equifax's. The breach is just the final symptom of deeper problems with company leadership & policy allowed to fester by ineffective public oversight and bad incentives. The witch hunt will change nothing and absent policy, it will probably happen again because the specific people were never the problem.
A single person is not "the leadership". Blaming just corporate leadership is also unproductive; if incentives and lack of oversight encourage this outcome, punishing and replacing the leadership is just playing musical chairs. Saying "the buck stops with them" is pretty useless unless your goal is simply to declare a target for emotional rage.
Isn't the (nominal) reason the people at the top get paid so much is because they're supposed to take responsibility? If not them, then who? They're supposed to set the incentives and culture so that the people below them in the org chart do the right thing. It's their job to know that their suppliers aren't using sweatshop labor to produce their goods. It's their job to make sure managers don't lean on the rank and file so much as to incentivize fraudulent behavior (but the managers would never outright say to do these things, oh no). "Don't ask, don't tell" isn't an excuse. And if the greater environment is such that "everybody does it", well maybe we need better regulations and/or policies, but that still does not absolve them individually.
The people at the top might not be at fault, but they sure as hell are responsible.
Hold the company responsible and let its stakeholders and internal processes figure out how to course correct. Maybe the CEO or CTO or whatever screwed up and need to go, or maybe it was a rare accident that is only human; given the right incentives & punishments, companies will work to identify and fix whatever is causing it to be punished as part of its profit maximization goals. All orgs of that kind of size operate as complex systems, inviting uninformed mob pitchforks into the process is counterproductive.
No, I don't think people at the top get paid to be a voodoo doll of responsibility. They get paid a lot because good executives are hard to find and can produce huge benefits, so the market values them very highly. It can feel good to throw around moral judgement like "that still does not absolve them individually" but if a set of incentives/environments consistently produce bad outcomes, the people involved are not responsible. It would be unproductive to punish the people involved when their replacement would do the same (especially considering that punishment is notoriously less effective at deterring human behavior). I personally think it is also morally wrong to do so, similar to punishing a thief for stealing food in a system that consistently deprives him of the ability to acquire food legitimately.
Well gee, in this framework it seems like there is no way for anyone to ever be negligent or liable for anything they do, no matter how ill-considered.
The very few leaders are paid enormously in money and status for all the value they are supposedly adding, if they screw these things up why should the hit they take not also be expensive?
At our company, we would not blur/obfuscate the actors' names for internal communications about technical errors.
The key is that humans make errors, errors of commission, errors of omission, errors of misplaced/almost brilliance, and errors of outright apparent stupidity. IMO, you should dive deeply to understand exactly what happened (which includes who did what and why), but you don't use that as ammunition against the individual humans.
Pretending that nameless, faceless committers did this thing isn't as useful as "even our CTO can make a stupid error [and not get fired for it]; here's how we seek to eliminate this entire category of error reaching production: <1> <2> <3>"
Internal communications are pretty different from a news article. A company can enforce a culture & policy for internally discussing mistakes. Publishing details in a popular outlet like Ars Technica opens up individual employees and the company in general to unproductive and unfair polemic, especially if the company in question is already unpopular for other reasons. It's no surprise that few companies open up their internal communication to public scrutiny.
The CTO of an org should be technically competent. If you are not technically competent, you have no business being a CTO.
If a civil engineer makes an egregious mistake and a bridge collapses, it's not toxic to say an egregious mistake was made.
And, in an organization, if you are unable to recognize and call out mistakes (in a respectful, constructive manner that is), you're a failed organization.
Rookie mistake is certainly not a constructive way to report on this problem, but it is true.
You know what's really a shame? People overusing the word "toxic".
You used it here a couple times in here so far.
I've been on the wrong end of a SQL injection once and I was called out publicly multiple times for it to my endless shame from my customers who bought the software and were hacked.
Was that toxic for them to call me out?
Heck no. In the 15 years since that time, I made triple-sure to never code something that could lead to a SQL injection again.
>... the blame lies on the company and the process, not the individual.
What about when the individual is responsible for establishing the processes? And when you blame "the company," are you not ultimately referring to those in charge, i.e., the C-level executives? After all, don't these same top executives claim a large share of the reward by arguing that their positions carry more responsibility and risk? So it seems entirely fair that they should expect to be held personally responsible when there are failures under their watch.
Technically, the CTO is an officer of the company. In a very real sense, he is (a piece of) the company. Furthermore, how would one "hold the company responsible?" -- review their security culture? Where do you think that culture comes from?
Take a step back and consider for a moment that these coders replaced their gitlab repo with a zip file with the password JesusChristIsKingTrumpWonTheElection; sounds more like over grown kids who never got an adult lesson run this site so a bit of name calling at the CTO level is warranted.
On the contrary, more of this please. Personal accountability is sorely lacking these days, from the heads of Fortune 500 companies getting away with golden parachutes after ruining countless lives in 2008, to shit like in the Ars story.
Sure, hold the company responsible, but don't just let the person get away with it so they can run off and do the same at another company. Crooks and arseholes go from company to company causing damage everywhere they go because firing companies won't tell anyone while they let a person go. This seems to be a result of just that culture of silence, because how else can you explain someone becoming CTO of an org like this without know basic stuff about their own area of expertise?
When minions make mistake, let the company take the heat. But when the person screwing up is in leadership and therefore directly responsible, it ought to fall on them.
I mean ArsTechnica is owned by Condé Nast, which in turn owns Reddit. Reddit is very anti-conservative, and has been this way since at least 2014. It's not far-fetched to think that maybe Condé Nast, the parent company is lead by people with similar or compatible values, which in turn affects their other platforms as well (such as ArsTechnica). Another thing to consider is that Gab is sort of a competitor to Reddit, so why wouldn't they fight it.
Reddit investors:
"Reddit raised $50 million in a funding round led by Sam Altman and including investors Marc Andreessen, Peter Thiel, Ron Conway, Snoop Dogg, and Jared Leto."[1]
- Sam Altman is an open democrat and has donated 250k USD in support of the democratic party.[2]
- Peter Thiel is pro-LGBT and pro-Trump.[3]
- Snoop Dogg is known for liking (and singing about) weed.
I don't know about you, but this explains Reddit's front-page and their overall content policy quite well. There is not as much information about ArsTechnica as there is about Reddit, but if we do some research I think the realities of ArsTechnica's potential biases will reveal themselves as well.
If I were a journalist who wanted to rake in money it wouldn't occur to me to include dull screenshots of diffs and commit messages. For that matter, it wouldn't occur to me to work for a niche outfit like Ars Technica either. My dentist probably earns more than a tech columnist.
> Gab had long provided commits at https://code.gab.com/. Then, on Monday, the site suddenly removed all commits—including the ones that created and then fixed the critical SQL injection vulnerability. In their place, Gab provided source code in the form of a Zip archive file that was protected by the password “JesusChristIsKingTrumpWonTheElection” (minus the quotation marks).
Hopefully this eliminates any doubt on Gab's real mission. It isn't free speech.
It's also clearly not intended as a password (why would you password protect a public zip archive?), but instead a "yeah you got us, but f** off" kind of message.
It may seem juvenile, but it is the kind of thing people do in an adversarial situation when they have no better hand to play, AKA they "got nothin'".
> why would you password protect a public zip archive?
I have no idea why they did it in this case, but I've seen it done to stop overly aggressive antivirus or antispam software from looking inside when the archive contains stuff likely to trigger false positives.
It is always interesting when people complaining about having their speech suppressed turn out to hold (in the former case) extremely orthodox, standard views or (in the latter case) fairly common views, instead of obscure or actively unpopular points of view. (I'd certainly appreciate it if the latter weren't common, but it polls shockingly well in the US right now)
How would you functionally suppress the majority? What would that look like? What would the actual negative effect of it be? If 40% of the population believes something, keeping them from posting on Twitter isn't going to magically cause their belief system to evaporate. They'll at most feel persecuted and go somewhere else. These mainstream views are represented all over broadcast TV and in the US house and senate, even if someone gets their account shadowbanned on Twitter for saying it using some expletives.
Naturally, most of these majority belief systems and viewpoints aren't functionally being suppressed, because many employees at these companies hold the same views. It's usually just the extremist stuff, and some of the people who bail from Twitter to head to Gab or Parler end up causing trouble there too by being so extreme they run afoul of those sites' incredibly lax and generous policies.
I could understand this complaint if the government was applying force or other punishments (financial penalties, etc) to suppress a majority viewpoint, but in this case it's private corporations exercising their rights to exclude majority speech from their platform. This is an important right for them to have regardless of how popular a view is.
It wasn't that long ago that polling numbers showed the average American supported the Kent State massacre, and it wasn't that long ago that interracial marriage was illegal in parts of the US. The nature of "majority views" and the impact of reducing their visibility is difficult to grapple with.
Look up gell-mann amnesia effect. The article doesn’t know much about OWASP practices.
SQL injection bugs are not rookie mistakes, it’s prevalent in many current and future applications. Look into Vtech sql injection hack, a large company with lots of resources had similar bug.
Look at previous hacks, solar winds hack, Sony hack, were all preventable common hacks.
I'd amend to this "... or composed of local string literals". Programmatically-generated SQL can be advantageous in terms of maintenance, readability and even performance, depending on the situation.
> The reason it’s so prevalent is because it’s not a rookie issue and very difficult to fix properly, without impacting significant changes.
Neither of these claims is true. Placeholders have been the recommended way to do this since the 1990s (I remember having this same talk with Perl & PHP 3 newbies) and one of the points of using a framework like Rails is that these are much easier to avoid if you use an ORM. The problem in this case is that they found a problem they (probably incorrectly) believed couldn’t be expressed in the ORM _and_ ignored the placeholder support _and_ didn’t validate their inputs. None of those require advanced experience to fix and at least the latter two are trivial to implement.
You would be amazed at the number of "developers" that have no idea what an SQL injection is when I ask them as part of my Full Stack developer interview.
SQL injection bugs of this fairly trivial type are. This is literally what web tutorials were pleading with PHP developers not to do 20 years ago, and they weren't new then.
It's a failure of tooling. The library, or the compiler, should stop you from interpolating stuff into SQL strings. We've already seen things like this keep happening over and over until it's made impossible.
The whole situation is similar to having a construction scaffolding without safety railings, and calling someone falling off it a rookie mistake.
> a large company with lots of resources had similar bug.
Large companies are even more likely to run into this sort of issue.
It's still a rookie mistake. Any given well established company will contain a large number of 'rookies'. It's up to the company and everyone involved to make sure these are caught before going into production.
well, even if it's rightwing idiots it seems you can go relatively far. We just deleted all our labs data because a colleague didn't get that you had to unmount and destroy the existing (benchmark) zpool to create a new one when finally setting up some new drives. Didn't ask for help or anything but went straight to "fix" it with `nvme format` on the raw devices. Hope the backup works at this point (that's the other colleague who accused me of paranoia when I mentioned 2FA with yubikeys for admins and then silently removed his smart outlet from the network once I ran a powercycle on it...).
People like to slag on other people's programming skills, but are you sure that all code you wrote doesn't have some absolutely boneheaded mistake in it?
I like to think I'm pretty good, but I would never claim that and certainly wouldn't put money on it. I'm sure I've got boneheaded mistakes scattered through my code. Sometimes you're having a bad day. Sometimes the reviewers are a little pressed for time. It doesn't take much for really stupid code to slide through as long as it appears to work.
I am 100% sure that none of the code I’ve written in the past 20 years interpolates user input into SQL. There is absolutely no excuse for this, and there hasn’t been for a very long time. Prepared statements have been a thing for longer than half the users of this website have been alive.
I've completely given up judging technical decisions as representations of the person who made them's skill or ability.
I still judge the technical decision on how it's impacting the system today (specifically around whether or not it's sustainable), but trying to ascribe intent or brand something as "dumb" is almost always done without the relevant information that went into the decision being made.
This is the problem being in a field (programming) that doesn't actually value experience.
I got this knocked into me by a very senior engineer when I complained about some really silly circuit on an older version of a microprocessor. His comment: "Okay, let's design that adder better. Oh, by the way, you only have two layers of metal in that technology. Let's walk through the options." My response: "Uh, no. We don't need to. There are exactly 3 options and only one that works." His response: "And now you know why the engineers prior to you made that decision. Don't assume your predecessors were untouchable geniuses, but don't assume they were moronic idiots, either. Just like you, they had a job to do, did it, and, most importantly, shipped it."
I've had some bone headed and careless mistakes make it into production, especially when I was a junior dev, but it was always something that messed up performance or functionality, e.g. a front end widget not responding or an unhandled API parameter.
When it came to user security I _never_ fucked around. I have about 6 years of experience now and consider a trivial and catastrophic mistake like this unconscienably reckless. Never mind 20 years and experience working at a FAANG company.
They're up against the world, and they're spreading themselves very thin, aiming to replace all of the "big tech" use cases, for better or for worse. They hate Silicon Valley but they "move fast and break things" more than anybody, arguably by necessity. I'll cut them some slack if their public platforms have some bugs, but they shouldn't do anything with sensitive user data.