Interviews are really a dumb game these days so if you want to really game it you can go with a statistical approach:
* Practice questions by company on LeetCode, sort by frequency of last 6 months and work down the list, do maybe 75-100, the list updates once a week
* Search for the company on the LeetCode forums and sort by most recent. If a question is not on LC yet it will likely get posted there, so you can get really fresh intel.
I've read of people doing this and getting like 6/6 questions they've seen before.
No, if you really want to game it you sign up for membership on Chinese forums where people post the questions word for word minutes after completing the interview. That or work exclusively with private recruiters that tell you the questions verbatim because they have a vested interest in you passing.
Interview questions don't rotate that frequently, especially for smaller companies or more specialized roles, and a $60 membership for a month will buy you internal referrals and potentially land you hundreds of thousands of dollars of value in a new position.
As an interviewer, this is so incredibly frustrating - we don't change our questions often and because of that the questions and answers are all over these forums. With that said, it is incredibly easy to spot someone cheating - they often write the most perfect optimal solution from start to finish, helper functions first, often with the same function names as the forums themselves. The trick I've learned is to ask "why" - "why did you decide to use a linked list for this problem?" - the real cheaters often freeze up and can't give an explanation at all. It's rare, but still too common.
If you've seen the question/answer before just say so! I will totally appreciate the honesty and it goes a long way.
> If you've seen the question/answer before just say so! I will totally appreciate the honesty and it goes a long way.
Why? You're testing their ability to produce the right answer to a given problem - not their problem solving ability. To that end it shouldn't matter if they've seen the problem or not.
I always find it hilarious when recruiters say that "getting the optimal solution isn't everything." I've failed numerous interview rounds where due to time constraints or implementation details I'm not able to completely code the optimal solution, but I am able to talk/walk through each step of it in pseudocode with the interviewer. By your own criteria, being able to clearly explain the solution and demonstrate an understanding of the different tradeoffs should count for much more than just being able to copy/paste the solution from memory, but I've never advanced in any round that finished without a "working" piece of code.
Honestly, the one thing I appreciated about FB/Meta's recruiters is that they were always honest about the process and what was expected - 2-3 Leetcode mediums/hards in 45 minutes and they only care about optimal solutions. I much prefer that to disingenuous sentiments of "getting the right answer is important, but we also want to see your thought process and how you might work with another engineer on the team."
> "getting the right answer is important, but we also want to see your thought process and how you might work with another engineer on the team."
That's how it works in all the companies I've hired in.
It doesn't matter if you don't get to the end of the problem, I just need to see you can think and that you know how to code.
Do you think your daily job will require you more than that?
And believes me, this is enough to filter out plenty of bad apples.
Poor performance in my experience was never about not being able to solve a technical problem, it was always personal issues / not having motivation / hating the environment.
> the one thing I appreciated about FB/Meta's recruiters is that they were always honest about the process and what was expected - 2-3 Leetcode mediums/hards in 45 minutes and they only care about optimal solutions
A counter data point: I recently passed Google's interview a few months ago. In one round, I was asked to solve a certain kind of bipartite graph matching problem; not being a fresh new grad with ACM medals, I obviously couldn't solve it in polynomial time and just implemented a brute-force solution in the end. In another round, I came up with a solution the interviewer said they've never seen before, although it could be slightly slower than the "optimal" solution he had in mind depending on the input size.
As an interviewer in my last company, I always made sure the solutions were well motivated, and have rejected candidates who could write down the perfect solution but couldn't explain why. If I were to be asked by the candidate for the specific runtime I was looking for, I would probably just reply with "don't worry about it, just try your best" or "let's focus on correctness first and worry about efficiency and other trade-offs later".
Testing for problem solving ability is hard, but that's still one of the key signals we wish to extract whenever possible.
> Why? You're testing their ability to produce the right answer to a given problem - not their problem solving ability. To that end it shouldn't matter if they've seen the problem or not.
Pretty sure most people want to test problem solving ability, and hopefully your problem solving ability solves the problem correctly. If you method to solve the problem is to find the answer online and repeat it... that may not be how the company wants you to solve their problems.
Really? In my position as a senior member on my team, one of the biggest mentoring costs is just teaching junior members how to search and find answers for themselves.
Yes, day to day I'm not copy and pasting huge blocks of code from Stack Overflow, but when I need an answer and I don't immediately know it my first move is always to search internally or externally for others who may have already shared it.
Why is being able to effectively search for for answers not considered good problem solving?
> Why is being able to effectively search for for answers not considered good problem solving?
Memorizing interview questions is decidedly not "effectively searching for answers".
Effectively searching for answers requires breaking the problem down into separate pieces that you can actually search for. This is one of the skills that can actually be demonstrated during the interview. And then showing that you can also come up with the solution (or be guided towards it in discussion with the interviewer) is the natural way to round it out, instead of "ok, now google for this sub-problem while I'm watching you".
Go ask your manager if you should copy and paste something you found on a chinese forum as 100% of the output you generate. Nothing original, no actual thoughts for yourself, JUST copy paste.
Search and find answers for themselves is not the same as blindly copy/pasting, which is basically what the scripted forum answers are.
Why not just throw out the questions. They seem useless, easy to cheat, and as commenters are suggesting there's a "pay to win" component. What are you really getting from these questions? Why is software engineering so different to other industries when it comes to interviews? Is there really a positive to this type of interviewing? Why not do what everyone else does and look at previous work (which in software we have a lot more considering GithHub), ask some questions relevant to the job, and see if the person has a personality match. Most jobs don't need the best of the best.
I'm not convinced the coding interviews have improved upon the standard interview style in any meaningful way.
> If you've seen the question/answer before just say so!
I did that once at FAANG interview, instead of honesty credits I felt like the interviewer just got annoyed by having to come up with another question.
I recently interviewed with a startup. They had "outsourced" the first round to a 3rd party firm that specializes in taking tech interviews (Mostly Algorithm rounds).
The interviewer posted an LC question and asked me to read it. Since I was already logged into my LC account, he first asked me to show if I had solved it. I said I did. He then posted 9-10 LC questions one by one, all of which I had solved (I was doing LC regularly then). In the end he got tired, and posted a question from another website (Hackerearth) which I hadn't solved. We ended up taking ~5 minutes just going through different LC questions and he was disappointed that I had solved all of them.
I have also faced situations where I have seen an LC question that I solved but couldn't solve it in an interview setting, mostly because of the pressure.
> If you've seen the question/answer before just say so!
In my experience failing to answer the alternative question you give me has (on average) a much more negative impact than pretending I don’t know your question (especially when I can explain it).
Yep, I did it a couple of times and I didn't get any credit, just got harder questions that the interviewer did not practice in a long time. You should only disclose this if you're getting the same question in the same interview round from the same company, or maybe if you're back for another round a couple of years after failing.
As an interviewer though, there were a few instances where I just told people "let's do this problem anyway, don't worry" and the candidates didn't always do a good job.
> As an interviewer though, there were a few instances where I just told people "let's do this problem anyway, don't worry" and the candidates didn't always do a good job.
This is what I've done as an interviewer. But then, my questions actually lend themselves to that. Something that the sample LeetCode questions just don't.
In my experience (as interviewer) LeetCode kind of questions are "gotcha" kind of tests: Either you know "the trick" or you don't, but there's no real constructive value.
In contrast, I prefer tests like let's write a Tic-Tac-Toe, Snake, Twitter-clone (no DB in memory), etc. in 30/60 minutes together in your computer with your IDE, your software, google and language of choice. I am able to do a quick coding session with the person, see her weaknesses and strengths, and even if he has done it before, looking at his real project coding style is super useful.
> there were a few instances where I just told people "let's do this problem anyway, don't worry" and the candidates didn't always do a good job
Which should be a clear indicator that interviews aren't only judging problem solving skills, but also the candidate's ability to withstand the pressure of being watched and judged while they solve complicated problems.
For the interviewer, it's just another day and sure, they "want the candidate to succeed" and all that, but for the candidate, their future and livelihood are on the line and that's tough for some of us to just ignore while we focus on the not-easy school quiz problem of merging overlapping intervals or whatever.
Yeah and often the interviewer won’t be prepared with a backup question so you waste time for them to find one. It sucks that it puts the interviewee in a worse position for being honest.
A advantage to having standard interview questions. Plus the questions we ask lend themselves to extensions (also predefined) so you can still see how they reason about and solve those. And given that our interviewers are familiar with them, we can calibrate across many candidates. We also have a pool of people who maintain the standard interview questions.
Yeah I worked for a company that did all of those things as well, it didn’t change the fact that the interviewer was always less prepared with the second question, and that there’s always wasted time switching between questions. The true nightmare was when a candidate knew two questions in a row, or when they didn’t realize they knew a question until 5-10 minutes in.
In a perfect world every interviewer would calibrate for those minutes lost, but that calibration is fuzzy at best unless it’s built in to your scoring rubric.
>If you've seen the question/answer before just say so!
In reality no one will do this though. There is way to much of an incentive on the candidate side to lie and work through as if they haven't seen it before.
>I will totally appreciate the honesty and it goes a long way.
Not far enough to get a job in most cases, at which point, after the rejection, you and I are likely done and won't talk again. On the remote chance we cross paths again, the honesty might be worth another interview (if you're still in that position), but not much more. Person-to-person, I know the honesty is appreciated, but here's why candidates are not likely to be honest and simultaneously have no moral/ethical failing during interviews:
You're an agent of the company with a power dynamic over candidates when interviewing. You can't be honest with us via feedback, even if you wanted to, because there's potential legal problems for the company if you are. Many companies don't give feedback as policy for these reasons. So, knowing that I know that you can't be completely honest with me, I only hurt myself by being honest by telling you I've seen the problem before.
If I already knew the answer, but can't answer 'why?', then that's on me - I likely don't fully understand the solution in that case. Reject those people, sure. But they have no obligation to be honest with you. Similarly, if I already knew the answer and disclosed that fact, I might get a tougher question. So there is real a risk that I hurt my chances by being honest.
The risk of getting a tougher question is important. For it to be really fair, you'd have to grade these questions and determine if the next question is of similar difficulty. But your interview and problem grading process (if you have one) is not likely to be disclosed to me. The honesty would be appreciated, but we both know that interviewers aren't going to share those details chiefly because it defeats the purpose of the test, but also it'd end up on a forum if you're a well-known company leading to more cheating.
If you were giving a well-known test developed by professionals (GRE, GMAT, LSAT, SAT), there are significant resources to show me that those are fair tests developed by academics and other professionals. I would trust those testing admins to substitute questions of a similar difficulty. Were that the situation with your interview, the risk of getting a tougher question by being honest is significantly diminished because I know you have a bank of questions with accurate difficulties attached.
---
I'm not arguing that you should change your process. A candidate unable to answer 'why?' is a perfectly good reason to say the candidate failed the question and maybe the interview. I'm arguing that this really shouldn't be considered cheating or a moral/ethical failure.
If you had honest experience with the question at hand, it seems a little weird, to go into an interview, and basically say "I already know I can answer that one. Give me a new one."
Google translate is sufficient. They’ll do it pretty carefully, complete with “pretend you get stuck at this specific point and if you get asked why to use a hashmap, act baffled for a moment, and then say X”
It’s ridiculously obvious when people have seen the question before.
The way we do it is like this: we have like 3 or 4 different small variations on each question. Such that the solution is measurably different, in quite telling ways, but that the given problem looks almost identical.
In one specific case the given is identical, but there are 3 variations to the question based on how the candidate asks questions about it (if they don’t ask questions the question is not possible to solve as we don’t give all of the information you need.)
We started doing it this way precisely because we kept running into people who would have 3 nearly perfect interviews and one “hard fail”, and we eventually realized it was because they were so good at faking that they’d seen it before but if they hadn’t seen it before they bombed it hard.
So now that we have the “variations”, at least once a month someone will “hard fail” the interview because they’re obviously cheating and will quite literally give the right answer to the wrong question, just rote memorizing it.
Your candidates show that after learning how to solve a problem, they can demonstrate they're able to solve it.
Have you considered just hiring candidates and then training them, or expect them to learn approaches that are new to them?
Right now, you're pretending that your company needs random puzzles solved, and they're pretending that they're able to solve random puzzles without looking them up in an algorithms book.
What's the point of this whole theatre?
I get that your ego is enjoying that, but is that providing your company really any value?
Along with what the sibling said, I'm not interested in tricks and puzzles, but I am interested in how people take in and handle new information. I'm not going to pretend like my job is bleeding edge or remotely novel tech. I do CRUD with SQL.
It's impossible for anyone to be an expert in every application that my team handles. The key for us is that we try to keep our applications relatively simple with how data moves from point to point. Orienting yourself with new environments and applications significantly increases productivity here. It's always good to have people who can recognize and apply logic to patterns, but knowing how to ask questions is important. It isn't about the "gotchas". It's about what happens after the person is stuck. We try to make sure our applicants can make some assumption or ask clarifying questions about ambiguous portions.
Being able to implement a solution after having already been shown how to do it is sufficient for many roles. But not all. Some roles require that you are also able to figure out solutions to problems you have never seen before.
If your company requires only the former that's fine. But if you also require the latter, that's fine too and it's ok to test for it in your interviews.
If a company generally requires candidates to be overqualified for their intended role, that's a bit dumb. But I imagine that such a problem would eventually be fixed by the free market (supply of workers at various qualification levels vs. demand for said qualifications).
> Your candidates show that after learning how to solve a problem
The way I read it, they’ve shown they can learn the solution to a problem.
It’s like asking ”what’s 43 × 57?”, getting “2451” as a reply and, from there, assuming they’ll be able to calculate ”42 × 58” or “41 × 59”, too. If they memorized just “43 × 57 = 4251”, that conclusion may be (and likely is; most people who know how to multiply won’t memorize such products) incorrect.
"Your candidates show that after learning how to solve a problem, they can demonstrate they're able to solve it."
The parent of your post is talking about a situation where they've demonstrated they've memorized a solution to a particular problem, not that they can solve it. And that it was the wrong solution, which they didn't notice. It's that last bit in particular, combined with not being able to adapt or create a solution for the actual problem, that is the hard fail.
I can't imagine my personal interview questions & style are common enough to have shown up on any of these sites, but I have personally witnessed two people (out of a few dozen) who knew only and exactly what was on their school curriculum, but were completely incapable of stepping outside of it. I come at interviews from the point of view that I'm trying to discover what they can do, not what they can't, so when I say that, bear in mind that I didn't hand them a problem and smugly watch them fail for 30 minutes, I actively tried to formulate a problem they could solve and failed. I've also witnessed someone persistently applying a book solution that was the wrong solution to the problem. Perfect depth-first search, but it should have been breadth-first, they couldn't seem to understand my explanation of that, and they shouldn't have put it in a tree in the first place. (It would have worked if the correct search was applied but it was massive overkill. It was something like finding a substring in a string... yes, you can represent that as a tree problem, but you're only making things worse.) They nailed the definition of graph and basically just wrote out the depth-first algorithm as quickly as they could write... but it was wrong, and the moment they stepped off the path they were completely lost.
I also don't do brainteasers like this, I focus a lot more on very practical things, so we're talking more like "failing to write code that can take the string 'abcd,efg' and split it on the comma without hardcoding sizes, either handwritten or by calling library code". I really want to start an interview on some kind of success but every once in a while a candidate gets through that I simply can't find the place to start building successes on at all.
(You have to remember the "evaporative cooling" effect when reading about interview stories. Of the good candidates that even go through an interview process, they do one round of interviews and get snapped up immediately. People who have good-looking resumes but, alas, graduated with insufficient actual skills, interview over and over again. The end result is the average interviwee, anywhere, is not very good. One of the preceding interviews I'm referring to emotionally wrecked my day, I felt that bad for someone who had clearly put in immense amounts of (all the wrong kinds of) work but I couldn't even remotely hire. But there's nothing I could do about it.)
I should also mention I run a style of interview where if I ask you a question and you blast it out of the park in a few seconds, great, you (metaphorically) get full points and then I move on to the next harder variant on it. And I can always make up a harder variant of something we can talk about for an hour or two. If I can't find something you can't blast out of the park in an hour or two, well, unless you're completely insufferable or your salary expectations are something I can't meet, expect the offer shortly. But what you'll be "blasting out of the park" will be something like a full round-trip delivery of a service with code-as-infrastructure and a database and user logins and a solid security design and so on and so on, not solutions to Project Euler problems.
Having been on both sides of interviews but fortunate enough to no have to do leetcode interviews I have a genuine question for those that do.
Why do them?
Are you really facing those problems frequently enough at FAANG to have know them? Is it uppity engineers? Gatekeeping? Or are you just getting so many applicants that you have to filter somehow and leetcode interviewing has some nice properties (easy to apply remotely, can be done by engineers, strong pass/fail criteria).
Genuine interest. Ignore the negative subtext, that's just me on this subject.
Its probably partially gatekeeping. "I had to invent a unique sorting algo in this interview so you will too".
But also, its a good measure of someones ability to take an abstract problem and solve it. Lots of mini events in a LC problem to critically think. Like you said, we need a filter and its really easy to use LC to be that.
That said, I've worked at fang with co-workers who had trouble using iterators or properly assessing complex Boolean logic (and I'm not talking about needing de Morgan), so sometimes LC skills are needed on the job. So getting a signal that "this person can't write loops" means "we don't trust this person not to write an infinite loop", however rare that day comes.
There's enough programmers who want FAANG jobs and its easy enough to apply and the pay is high enough that you should be free to gatekeep by someone who understands intro-to-java level data structures and algos. Maybe leetcode-hard is unnecessary, but easy should be doable.
Because if you otherwise like the candidate you can help them along and if you don't like them you can just watch them sweat, but then claim your interview process has some level of technical rigor.
I really don't get shareholders at big tech companies.
They could hire a few teams of talented engineers and trove of cheap developers and forget all of this.
Almost nobody needs leetcoding engineers, 0.1% of their tech is hard, the rest is all forms and api, as is most of the industry.
The lack of self-awareness shown by this interviewer is staggering. You're right, they seem to relish wasting their (paid) and applicants' (unpaid) time. I won't hold my tongue. The people that do this are total assholes, if not downright sociopaths.
Why do you call it cheating? Did the candidate actively copy a solution by deception or were they just very well prepared - to the point that they quickly recognised and solved the problem which they had seen before?
Perhaps I should throw out my CLRS and Skiena and invent everything in those books from scratch?
It's an Arms race because people like you have turned it into one. You're not solving Nobel prize winning problems, so stop expecting people to magically invent on the spot novel algorithms for things they've never seen before.
As someone who has studied and passed before in this manner, and is now an interviewer, I have a simple solution that other companies should follow:
for at least one round of interviewing, let me (the interviewer) use my own custom question, where the goal is not so much to solve it but rather to reason outloud collaboratively about many different aspects of the question.
I like to use 3d graphics as a domain that candidates most likely havent seen before, but sufficiently motivated/smart ones can hold their own in. If someone doesn't quickly and intuitively grasp that a shape is a collection of faces, and a face is a collection of points, and a point is a collection of vertices, I'm not sure that they have what I am personally looking for (even though we dont do any 3d graphics in our project)
There's a certain irony to exploiting an ethically questionable method to obtain a job and, once you've obtained that position, using your authority as a gatekeeper to attempt to prevent others from entering the same way.
I can't speak for them, but I would assume that jsiaajdsdaa still considered themselves qualified for their positions - even if they employed ethically questionable methods for passing the interviews. In the end isn't that all that matters given that's what these interviews are supposedly trying to measure?
What part of jsiaajdsdaa's process do you think is more efficient? To me it sounds less objective ("the goal is not so much to solve it") which seems like it would make the process less efficient when assessing candidates.
I think as a general rule the idea that you give someone something that they are unlikely to solve because they are not completely familiar with the problem space does measure some things well - specifically, how one performs with something new and unknown (a critical programming skill) and their ability to abstractly reason as opposed to having memorized some information.
Although for other things related to the job not that useful.
That said I was talking about your comment that it was somewhat unethical; they didn't so much think that they gamed the system but that the system was so inefficient at doing what it should do that someone had to fit themselves to the system to get in.
Or tyrants gatekeeping future tyrants, or mature bears killing/eating younger bears to make their own life easier in the future (food and access to sex).
This is the first time I’ve come across this definition of a point. Geometrically point is defined as a zero dimensional shape (or something similar) if I recall correctly.
Besides, I don’t see how it’s intuitive at all! It isn’t to me at least.
Larger point being, if you pick such random domain without calibration you will run into such argument/discussions during an interview. Not sure what data point could be derived from such discussion when you are looking for someone who can write a decent maintainable code.
I must say that I’m terrible with geometry/graphics which hasn’t stopped me from creating value through software development in my domain, online payments.
If your intention is to gather signal for collaboration then I suggest picking something you are likely to collaborate on a day to day basis. Let’s say code review or architecture review. You could discuss why such and such an architectural pattern is useful, under what conditions, what are the pitfalls to watch out for etc etc.
A vertex is a point. So a point being a collection of vertices doesn’t make sense. A point in the sense you mean is a collection of real numbers equal to the number of dimensions of the space it’s in.
> A point in the sense you mean is a collection of real numbers equal to the number of dimensions of the space it’s in.
Not necessarily. In 3D graphics, it is common to represent points with homogeneous coordinates, where points in N-dimensional space are represented by N+1 real numbers. Using 4x4 matrices [0] to describe affine transformations of 3D points is very convenient.
(Agreed with your overall point though. Just goes to show how different some fundamental perceptions/definitions can be.)
> The extra real isn’t really part of the definition of the point in space though
It actually is. It's generally assumed to be equal to one, but it need not be.
> isn’t necessary to store to apply a 4x4 matrix
...if you assume it is equal to one, yes.
However, actually representing the fourth component is both mathematically sound and occasionally useful. For example, the midpoint of two affine points, such as (1, 2, 3, 1) and (3, 6, 9, 1) is actually just their sum: (4, 8, 12, 2), which represents the same 3D point as (2, 4, 6, 1). The fourth component can also be zero, in which case you describe a point at infinity in the given direction.
But yes, if you only use homogeneous coordinates for chaining 3D transformations, storing the extra component it pointless.
I don't understand how a point is a collection of vertices. Are you talking about X, Y, and Z coordinates?
This is a good example of how ambiguity kills an interview and reduces it to quickly figuring out what the interviewer is talking about so I might have a decent chance of solving the problem with the time remaining.
My experience with interviewing at Google, Facebook, and Amazon can be reduced to "What the hell are you talking about?"
> If someone doesn't quickly and intuitively grasp that a shape is a collection of faces, ..., I'm not sure that they have what I am personally looking for
I was doing the same to weed out the bad candidates - asking them something they should know, something logical and basic - but got bad feedback once, been asked to instead focus my questions on the strong points described in the CV. I mean, for the practical part the candidate wasn't able to count the unique words in a text file in 30 minutes. I thought opening files and reading strings and splitting are so basic anyone should know them.
I program in C++ daily and wouldn't know the currently accepted way to read lines from a text file in it off the top of my head. It's simply not something I ever have to do. A good candidate should still manage to figure it out in 30 minutes, but your programming experience is most likely a lot less universal than you think.
They didn't specify needing to read the file line-by-line. You could read the whole file at once. There might not even be any new line characters in the file. You invented a requirement.
They specified "strings", which I interpreted as lines in a text file. But it doesn't actually matter, because I could make fundamentally the same comment no matter how words (or "strings") are separated.
Are you very knowledgeable in 3d graphics yourself, some of the replies here suggest that you may have a superficial knowledge (which is what I would have), if you choose an example domain that you have some knowledge about but not deep knowledge what happens if by accident your interviewee has significantly deeper knowledge than you? I worry that person might end up sounding overly technical or even like they're BS'ing their way through the interview.
There is also another way of creating 3D graphics that does not involve the use of faces and vertices, so developers with that background would be at a loss (or win depending on your viewpoint.)
This is exactly the type of question that is the worst for interviews. It's a completely uncalibrated, completely subjective, esoteric type of question where you can't say exactly why you liked a candidate or why you didn't like her. There's no data underneath it except for "I liked how the conversation went."
It completely gives an advantage to candidates who know 3D and completely gives a disadvantage to candidates that know nothing about it. Even worse, they are relying on YOU to explain it to them. Who's to say you are sufficiently qualified to teach people the basics of 3D graphics enough such that they can answer your questions? Have you been calibrated or judged on your ability to teach even the basics of 3D graphics? Or are you assuming that you're good enough?
It's completely inappropriate and relying completely on you to determine a candidate's qualifications based on nothing except your feelings. It's a horrible question and I seriously hope this is not entertained at all at your company.
You have some good points, but can you please make your good points without crossing into personal attack? I'm sure you didn't mean to, and I'm sure you have good reason to feel strongly, but what you posted here is already aggressive and a step away from what the HN guidelines call for.
I do not believe the hard data you are looking for exist for real.
Interviewing is assessing what value an individual will bring to a project. Ideally we want this assessment to not depend on the interviewer and as a unidimensional real number so it's easy to do comparisons.
But the reality is:
- future performance depends on the team (and more largely on everything else in the business) and that varries unpredictably and is usually not part of the evaluation anyway.
- future performance also depends on how the project itself is going to evolve, which is hard enough to evaluate in itself (not just the timeline but oftentimes the involved technologies)
- assessing the value of anything is its whole can of worms (it's intrinsically subjective, you can't measure a physical "value" in SI units)
I prefer data over opinions like everyone else, but this may be a case where it is eventually safer to rely on as many as possible individual opinions and weight them, with just the amount of process in place to avoid common biases? (friends-of-friends, judging on look, country of origin, gender, ...)
I've seen big companies that takes hiring very seriously rely equaly on some preset metrics (based on prewritten questions thus easily gamed) as well as the gut feeling of several interviewers who are free to ask whatever additional questions, and I think it's the correct approach.
So I'm not a part of this world at all and I'm super fascinated by this perspective. I hear your point entirely. How do you counter the issue with pre-structured interviews where the questions get distributed and you end up with candidates who learn how to answer your exact question (but lack the skills to actually be dynamic in their job, or even do their job)?
I used to interview for a big company, and I could tell some candidates already knew the questions, but I'm a bad interviewer myself as I tend to rate most candidates as 7+/10, I very rarely found coding-illiterate candidates.
When I detected the candidate knew the questions, I'd switch order, introduce new questions I didn't usually ask.
The interview we did was quite extensive on the java basics and if the candidate somehow managed to learn all that stuff by knowing the questions, that was a pretty good sign anyway.
It was just this one time I found a candidate who aced every question I asked, even OOP, Design Patterns, I made up on the spot a code-design challenge, the guy suddenly was not so brilliant, but still managed to hold his own and I didn't penalize him for my impression that he knew the questions, he was way better than the usual candidate anyway.
Perhaps the stakes were not so high as in the USA, but I can say I've never found someone who would have failed without knowing the questions in advance; I did fail people who were trying to cheat on the phone interview, but those were rare cases.
My impression is technical interviews just filter programming-illiterate programmers and people who don't give a f#ck -- as a general rule.
There's also the 30% of cases where the interview is made up to show the candidate he's not really all that senior as he thinks and to lowball his salary request.
And some companies have very high technical interview standards because of some cultural inferiority complex (we are just like Google, you see...).
If I could answer that question, I would be a billionaire already.
Programming interviews are almost exactly akin to actor's auditions. Just because you flunk an audition doesn't mean that you're a bad actor. Also, auditioning takes a special skill and it's not very much like being a real actor. But they still do it to this day. Programming interviews are similar.
The best hiring model I can come up with is the Netflix model. Pay top of the market, hire and fire people quickly if they don't meet expectations, with a generous severance. Have high expectations, reward the ones that can fulfill those expectations, and quickly get rid of those that don't. It's ruthless, but the Netflix engineers I know love working there.
Yeah, I would say a much bigger issue than algorithm questions in interviews are interviewers who assume all programmers follow a particular path (usually the one they followed) and discriminate against those-that-did-not-follow-particular-path.
Computer science is a massive field that people enter through many different and unique ways. If you're trying to gatekeep and force everyone to enter through the same gate that you entered, you should not be an interviewer.
3D graphics is not a good moat, I agree.
There are much better moats -- such as recursion, there's no way I'd let someone in my team if the don't get recursion, even though we almost never use it :)
Interesting. I had never expected 1P3A mentioned on HN. It's mainly a forum for Chinese overseas to discuss their life and so on. Surrounding this topic they have also developed side features like COVID-19 statistics and some system for school applicants.
Is this common among immigrant communities? For example do Vietnamese or Indian or Nigerian communities exist to give each other exclusive support on finding jobs or other such advantages?
Yes. And American immigrant communities in other countries. The size and formality will generally correlate with the size of the immigrant community. From real-world social networks to Facebook groups to dedicated sites and forums.
Many people say this. But the reality is that solving 100 LC questions and actually understand the solution enough to solve a variation of the problem is a lot of work. Especially if you are working full-time. I wouldn't call that "game it", just usual study and hard work.
When gaming it is just passing by studying, a repeatable process that anyone (with a CS degree) can do, just means the interview process is quite well designed.
The interview process is a test of endurance, not intelligence. And it should be exactly that, since software engineering is mostly an exercise of endurance and focus.
Every time a friend of mine QQs about failing a FANG interview, I give them the study prescription that worked for me. The reaction is always the same: they don't do it. If you can't get past these interviews, you probably won't make it past a year in these high performance companies anyways. Because you actually have to exhibit the same work ethic that interview studying requires.
I felt the same when I failed a leetcode interview. The reason behind my annoyance was that I had spent considerable amount of time over the past few years improving my skills in the areas I thought mattered (practicing TDD, design patterns,learning FP etc.,) I felt disappointed when none of these mattered compared to an artificial set of challenges setup in an unreal environment.
I can't say for sure that such challenges (array manipulation, backtracking etc.,) are not typical for a software job but I know that they aren't relevant for my line of work. Even if they are, the challenge of solving problems in a live coding session is not the same for all. Different people need different settings to perform or excel.
But now I am committing myself to leetcode and getting better at it, but it has become one among hundred other things I want to get good at. At some point of time, I will feel the burnout. But I sincerely hope the acquired skills are useful in my professional setting.
Skills such as design patterns etc do matter. There are separate rounds for testing just that, think Design rounds (LLD or HLD) but you usually have to pass the Algorithm rounds before to get to that stage.
I don’t know if it’s as useful a signal as you imply. For me, it’s easy to be motivated studying leetcode: small, self-contained puzzles with just the right amount of challenge and immediate gratification. Actually doing a FAANG job can be a slog where it takes months to see results from your work.
I can get hired as a software engineer wherever, but I’m only mediocre at doing the job. I’m not the only person I know like this.
I don't want to underestimate the skill of being good at LC. In the past couple of months, I have solved about 200+ LC and I have been interviewing vigorously too. Its a tough skill for sure but beyond a certain point, you start seeing patterns you have applied before and once you are in the flow, it becomes slightly easier.
I only wish the real Software Engineering job was as simple as being good at LC, because it clearly isn't.
Sure. But I’m (by my own estimation) a mediocre software engineer who is very above-average at solving fun coding puzzles, and therefore at interviewing. Usually these threads are full of people complaining that they are good engineers who aren’t good at interviews; I’m suggesting that the opposite isn’t uncommon, even if it’s more rarely admitted.
Getting something working in the beginning feels very different from a coding puzzle. During the next phase of optimizing something is where it gets interesting. Understanding the hardware and figuring out the right way to make full use of it via memory tricks or specialized processor instructions always felt more like a puzzle. Unfortunately, I feel engineers are seldom given the opportunity to go deep down the optimization route once something is working reliably, since it's already getting the job done.
> I can get hired as a software engineer wherever, but I’m only mediocre at doing the job.
I feel exactly the same.
Mind if I ask how you deal with this?
I recently left software (not sure if temporary or permanent yet) and I'm pursuing tutoring in an unrelated field. So far I'm liking it more because I feel better than mediocre.
Haven’t really figured it out yet. Working on something I care about seems to help, although it’s not always easy to find. Having a strong, trusting relationship with my coworkers also helps. Easy to get detached otherwise.
Having other sources of meaning in life keeps me going during periods where my career isn’t going as well, gives me perspective and keeps me from getting depressed (I’m prone to it).
Working at top companies has helped me meet a lot of amazing people, including many of my closest friends, so I’m grateful for that at least.
Also, this gets thrown around a lot on HN, but if you’re brilliant at hard programming puzzles but not good at engineering jobs you might have ADHD. I do. Medication and/or ADHD-targeted treatments and accommodations could help. They’ve been modestly helpful for me.
Software engineers that QQ about how unfair algorithm interviews are are clearly out of touch with how difficult and truly unfair interviews are for other high paying industries like law or medicine (in the US) where getting interviews is based on pedigree and getting one or two rejections can permanently deny you from top firms/positions.
There are always things that can be improved about interview processes, but many software engineers display great immaturity when it comes to trading a few weeks of time studying content that should already be partially familiar to them from school for six figure salaries/raises.
I don’t think anybody is complaining about the initial barrier to entry. These interviews would be more like asking a surgeon who’s worked for 10 years in one hospital to pass an anatomy multiple choice quiz when transferring to another hospital
While it's true that asking an experienced surgeon basics of anatomy is worthless, please take into account that all surgeons at least operate on humans. If you take 10 random developers, chances are most of them worked on completely different levels and domains throughout their careers (e.g. embedded dev, backend, frontend, devops will have vastly different skills).
> but I’ll take it over how finance jobs are where it’s all about connections and where you interned when you were 19
Is this different than Big Tech really? If you do not go to the right schools, internships, whatever it can take years to have the right employers in your CV to be called for an interview.
Not true in my experience. I went a no-name state school, graduated with a mediocre GPA and an unremarkable local internship and I still got the same interview and job opportunities as my friends that went to Berkeley and other top schools. I know people at top tech companies that never finished their college degree.
I don't know of anyone at a top law/medicine/consulting/finance firm like this.
The average employment length at a faang is 2 years or less. A job in a single top law firm is an endurance battle that can span your career. Switching firms is looked at differently a lot of times negatively.
Once you land that job in finance or as a lawyer you are on track to be set for life. At a faang after a few years you'll get pushed into management or pushed out.
You are just as "set for life" in big tech as big law. In fact, if you're looking at the last decade, big tech won hard considering tech stocks and how big law froze (and even slightly cut) salaries during the great recession.
What is the relevance of this whataboutism? Some other industries have bad or worse interview practices as well. That doesn't make this process good. What is the point you're trying to make?
Its immature to think we can do better because others have it worse? Rubbish.
That's pretty presuptuous. Do you truely believe that working at a FAANG company requires the same amount of 'edurance and focus' as working at a non FAANG company _and_ spending several hours every week practicing coding skills?
The major frustration is that its endurance in an mostly irrelevant task. If the problems were more relevant, active software engineers wouldn't need to study outside their day to day duties, no?
If somebody has been doing something for 20 years and an assessment test can yield dramatically different results with 20 minutes of preparation, then the assessment test is total bullshit. It's going to yield a false characterization and is fundamentally garbage input
I know that's pretty strongly stated and it matches the strength of my convictions here. I've tried many methods. Asking someone to figure out something is a sliding window is a garbage test
Problem is, there is not a lot of incentive for people looking for a job to use LC and similar to understand algorithms and data structures, and with good reason;
Many interviewers do not ask the questions to check for thought process or problem solving ability, they treat it like some TV quiz: Ask question, get answer, compare answer to note in hand, applaud if its the same answer, next question. Why? Because its a lot easier to sit there watching the candidate squirm at the whiteboard, while thinking about what's for lunch, than engaging the candidate and gasp talking to him/her.
This creates incentive for people taking these BS interviews to learn-for-the-test: Get a list of the current top50 questions asked regularly at interviews (there are resources for that) and memorize them.
Why? Two reasons:
1. It is alot easier than understanding the concepts and purpose of different algos and data structures.
2. Trying to solve them by applying actual understanding, runs the risk of getting stuck on an unfamiliar problem, or producing a slightly sub-par solution instead of the "correct" answer, and getting booted out despite demonstrating the exakt thing aka."problem solving ability" the interviewers allegedly look for
And, unsurprising, because there is money involved, an industry has sprung up around this: Pay-For-Tech-Interview-Training is a thing, including regular updates on popular questions.
The result of course: Companies running the risk of hiring people who are great at answering LC questions but fail when they actually have to solve a problem where they cannot copypaste the solution from SO.
I’ve gotten interview questions I’d recently solved and my problem was it was too easy. I had trouble acting like it was the right amount of struggle. Is there a trick for that?
I've interviewed folks for FAANG roles. If you know how to solve the problem already, just tell the interviewer up front. Either they have another question or they will go deeper into a discussion about why and how you solved it the way you did, testing it, other approaches and why they are or are not good tradeoffs, etc.
It's pretty obvious to interviewers if you've solved a problem before, and we appreciate the honesty. Interviews are not adversarial; they're to see if a candidate is a good fit for the role and dishonesty is never a good fit.
"We expect you to study and be prepared for algorithmic questions, BUT NOT TOO PREPARED! Only just enough. We will give you a random question of our choosing, but if you already studied this, then you will be deducted points, unless you tell us, so that we can ask you a question you've never studied before."
A true interview would give the candidate their choice of question with the expectation that they know how to solve the question. It makes it a lot fairer for everyone involved.
I was always kind of confused about how advice for interviewers always comes along with warnings of the type "Some people talk a good game about programming, but they can't actually do it. You have to watch out for these people. Never hire one. This is the biggest trap you can stumble into as an interviewer."
After getting some interview coaching, I think I understand how this complaint arose. The whole problem is an artifact of the interview format, in which the design intent is for the candidate to be unable to solve the problem on their own. So you get scored based on how well you can charm the answer out of your interviewer while making them feel like everything they said was your idea. Instead of a test of how well you can program or solve math problems, it's a test of how good you are at cold reading. ( https://en.wikipedia.org/wiki/Cold_reading )
And unsurprisingly, a test of cold reading will end up delivering people who are good at cold reading without reference to whether they're good at other things. If you want to avoid this problem, just start giving assessments that don't involve interaction with the interviewer.
A good way to avoid this conundrum is to ask questions that are worth solving in real life.
If the candidate breezes through the discussion because they've actually had to solve the problem before, then their victory is well earned.
If on the other hand its an academic question in the same vein as the data structures or algorithms puzzles you find on $interviewprepforum, then the fact that they've solved it before tells you very little.
I don't think there's much difference either way. In both cases, a candidate gained a large advantage in a way that tells you very little about their ability.
I think this is why contrived questions gained popularity in the first place - they eliminated noise due to candidates randomly having solved similar problems before (that and "real life" problems usually can't be explained and solved in 1 hour).
I think this is on the right track. The best interviews I've been a part of involve the interviewer asking a question they don't fully know the answer to. Then the interview turns into a conversation where both parties are trying to work together to formulate an answer. The candidate should be scored based on how constructive that conversation is.
Be honest: a candidate that works through a seemingly novel problem correctly is getting a much better review than another candidate that admitted they've done the question before and breezes through it. And this is assuming your interview round doesn't get thrown out entirely to begin with.
The interviewer's goal is to evaluate the interviewee accurately.
The interviewee's goal is to be evaluated inaccurately.
If you really need an example, then look at it this way:
1. The interviewee's goal is to be hired.
2. Assuming there is no conflict of goals, then the interviewer's goal is to hire the interviewee.
3. This immediately implies that the interview is a pure waste of time. You can just make the hire without having the interview.
If you don't believe that interviews are -- in every case -- nothing but a waste of time, then you must reject one of the two premises. You either believe that (1) The interviewee does not wish to be hired, or (2) there is a conflict between the interviewee's goals and the interviewer's goals.
We can make a similar observation purely by knowing that interviewers sometimes reject interviewees.
>> The interviewee's goal is to be evaluated inaccurately.
> Only if the interviewee doesn't think they should be hired.
Nope. The interviewee would always like to be evaluated as better than they actually are, regardless of whether they meet the notional hiring threshold.
> These aren't necessarily adversarial.
The goals you list are still necessarily adversarial, because the interviewee's goal is always to get an offer and the interviewer's goal is to stymie them.
The interviewer's goal is to evaluate the interviewee accurately.
More specifically: the interviewer's goal is to minimize (1) false positives and (2) the expenditure of the company's resources.
Meanwhile, candidates hope to be evaluated "fairly", which is in direct conflict with criterion (1). They also naively expected to be treated "decently", which is in direct conflict with criterion (2) and which explains why employer-side ghosting is so widespread, along with other abusive practices like piling on lengthy take-homes, etc.
> which is in direct conflict with criterion (2) and which explains why employer-side ghosting is so widespread, along with other abusive practices like piling on lengthy take-homes, etc.
I don't think concerns about resource expenditure actually explain ghosting. I think that happens despite what the company would prefer, because the people involved find it unpleasant to notify candidates of a rejection.
Lengthy take-homes are easier to explain by reference to resource concerns.
Minimizing false positives is not a fundamental goal of interviewing -- accuracy is a goal in all settings, while minimizing false positives isn't. But minimizing resource expenditures is; you're right about that.
> It's pretty obvious to interviewers if you've solved a problem before, and we appreciate the honesty.
> ..
> dishonesty is never a good fit.
If you rejected every such “dishonest candidate”, the paid services offering candidates practice questions would have gone out of business long ago.
Given the popularity of such services, have your considered the possibility that you overestimate your ability to spot candidates who have solved the problem previously?
I understand the sentiment about honesty and finding a legitimate good fit. Neither interviewers or candidates are perfect. But there's something off about this, isn't there? If you ask a candidate a question, and they answer it well, they've done what is asked of them. Why should they put themselves in a position of vulnerability just because they're well prepared?
To some extent it paradoxically sounds like you guys want applicants to not have studied basic algorithm problems, since the problems you find if you try to study are the same problems you guys ask in interviews.
Just tell them it’s familiar. Depends on the company, but they can often tell. If you blast through a tough question and then struggle on easier ones, it’ll show up in hiring committee (or equivalent).
It could easily cost you the offer if they think you’re being dishonest.
If you say it’s familiar and they ask you to do it anyway, just solve it very quickly and thoroughly. They’ll give you points for your performance and probably ask you another.
Just go through the steps out loud, start with a really naive solution, say what's wrong with it, go on to the next. You can usually go through at least 3 levels of that. Also spend some time verifying the specification of the problem, ask about any potential edge cases. If you really know the problem well, go into maybe some extensions of it or harder versions.
The interviewers are ideally trying to get a sense of how you think through problems, not just that you can spit out an answer you know. At least that's what they say.
It's not the struggle, it's the questions you ask, and what you share about your thought process trying to got to a solution. You might work yourself into a hole solving what you think is a simple array manipulation problem, but then realize some edge cases require you to switch to a heap/priority queue, etc.
Please don't cross into personal attack, no matter how wrong's interview preparation strategy is or you feel it is. Perhaps you don't feel you owe them better, but you owe this community better if you're participating here.
* Practice questions by company on LeetCode, sort by frequency of last 6 months and work down the list, do maybe 75-100, the list updates once a week
* Search for the company on the LeetCode forums and sort by most recent. If a question is not on LC yet it will likely get posted there, so you can get really fresh intel.
I've read of people doing this and getting like 6/6 questions they've seen before.