"Then, acting as though this was the first time I'd seen this problem, I would ask if it was ok if I thought aloud as I worked my way through the problem on the board. I'd mumble to myself about moving-this-piece-over-here and-now-we're-going-to-get-this, and lo-and-behold, I accidentally solved it in constant memory space, in C - a language I didn't even claim to be particularly good at! Only someone with amazing problem solving skills would be able to do that their first time thinking through this problem so earnestly!"
All this tells me is that tech interviewing is horribly broken. What we have here is a story of a guy with a little bit of experience who has (if his story is true) successfully learned how to game the interview systems at dozens of startups. If you interview people, this article should terrify you. You can argue that it isn't "gaming" if the guy turns out to be a good employee, but that's wrong. If he can trivially game the system, so can anyone else with the ability to go to dozens of interviews and remember questions.
There's a fix for this problem: make the interview about more than the answer to the stupid question. If the candidate answers your question a little too quickly, ask something harder. Chase down a detail. Pick an experience on the resume (and not an obvious one), and dig as deeply as you can. Good candidates know the details. Bad candidates don't have details you can chase.
If you're interviewing correctly, no candidate should be able to come in and snow you with some regurgitated C code from previous interview experience, because you'll be able to out-flank them at every attempt to cough up a line. Unfortunately, there's a corollary: you need to know the subject better than the person that you're interviewing. If you don't, you have no hope of screening for good people.
It's not really gaming the system. He solved the problems, himself. He asked questions, learned and then came up with more and more effective solutions. Aren't these exactly the traits an employer should be looking for? Didn't he put a lot of effort and time?
Here's my own example. I've been recording some of the problems I have to deal with at work, and some of the things I come across in my spare time in the form of a blog, for the last 3 years (with some gaps here and there). I recently started looking for a better place to work. I included the link to my blog in my resume. I got an offer recently, it will be a substantial increase in salary. I only interviewed with ~5 companies to get an offer, that's my easiest offer and shortest job search ever.
Now, would you say that I 'gamed the system' because I had a blog, and most don't? I guess you can say so, because when I started my blog, I had that in mind - it may be useful next time I look for a job. But I did not mindlessly type stuff in - I made sure I understood it as much as I can, and quite often writing the solution down helped my understanding. Same with this guy - he did not just memorize how to do it.
I had one YC company ask me to create a linked-list structure in ruby, and then reverse it. I did it recursively, mentioned that it would blow the call stack, and then re-implemented it iteratively. I also remember there was some problem where I used symbols rather than strings and that would cause some bad memory leaks over time in a long-running process. - From this passage, he might not be the "top 1 percent" everyone tries to hire, but at least top 5%, that I'm sure of. Or maybe he gamed my impression too.
The definition of gaming the system is different here. These days unless you are implementing a library yourself. You are probably never ever going to even reverse a linked list ever in your life. Or deal sit down with a math text book when you code. Those days are probably gone in the 80's.
That means asking these questions is akin to you taking an approach to judge an candidate on things which have nothing to do his job. That's why so many programmers don't about all this. The reason is for 99% of the work they are likely to do they don't need to know it.
Under such circumstances when you ask these sort of questions the only way candidate can deal with them is not by actually learning how to solve them. But learn them just for the interview sake.
So you end up hiring some one on a criteria doesn't matter. And the candidate manages to beat you by system of work which defeated your original means of judging him.
You could just ask: How is lists:reverse(List) implemented in Erlang, or if you don't know it, how could it be implemented? What are the trade-off and limitations for different implementations?
Since most abstractions are leaky, you can make a good argument, that learning how libraries tick, even if you are never going to rewrite them, is a useful skill.
Understanding the limitations of libraries might help you use those libraries better. Using libraries can be a part of delivering business objectives.
I don't think that list reversal is a good interview question. At least not as a positive discriminator. Though it might help you weed out people: I'd probably not want to hire people who can't figure out how to reverse a list. If they'll stumble upon such a simple problem, they might also stumble over the more complicated problems we have to solve to deliver business objectives.
I wrote `can't figure out' deliberately. Not knowing how to reverse a linked list, but being able to come up with an algorithm after a few minutes thought is perfectly adequate.
I agree with you - it is a weeder-outer, but not an includer-inner - there are lots of people who can write code to reverse linked-lists that I don't want to hire.
In particular I don't want to hire the people who would rather write their own implementation than use a library (think Not-Invented-Here and Yak Shaving).
Indeed. By the way, in purely functional implementations of a list reversal there are non-trivial trade-offs to make. (See Janestreet's standard libraries for OCaml.)
If you don't need to know how to solve algorithmic problems or use mathematics and you're programming, why the fuck is your job still not automated or outsourced?
Tell me about it. I have nearly 20 years of experience in CS, and hold an advanced degree. I have 5 years of experience in a very niche field which is relatively hot. I was approached by Facebook for this specific purpose. They invited me in for just a "chat", asked me if I'd be interested in working with them.
Obviously I said yeah, I'd be willing to talk. They set up an in-person interview. Just 45 minutes they said.
So I show up, fully expecting a discussion about that area, my background, etc.
Instead, I get a random coder who wasn't even making eye contact when asking questions. And the questions? Elementary CS stuff that any undergrad would know.
Now, obviously I know them too; heck, I taught Algorithms courses in grad school.
But why waste my time with this crap? You can hire almost any fresh graduate from a top-40 school who would do a halfway decent job at answering those questions. What does asking a PhD "how to print the levels of a binary tree, one per line" tell you about his work? (That wasn't the question, but that was the level of questioning).
The sad part is: I can waste a couple of weeks of my life, go to Glassdoor etc. and basically memorize these questions. I'll probably stand a good chance of blowing the interviewer away by pretending to solve them on the spot. But what does that tell you about how I am at real-life problems?
I've interviewed a few people who thought the elementary questions I was asking were beneath them. I insisted, politely, that they humor me, because it's something we make everybody do, nothing personal. They failed. Hard.
Some people are probably bullshitters, but my theory is that some are just people who used to code (a little) but have since moved on to making architectural diagrams for so long they've forgotten what it's like in the trenches. Unfortunately, if you're interviewing for a job in the trenches, you need to demonstrate the ability to do the job.
It's entirely possible they screwed up, depending on why they asked you in, but once a company starts letting in people they "know" can code without verifying it first hand, things go bad fast.
"I insisted, politely, that they humor me, because it's something we make everybody do, nothing personal. They failed. Hard."
I'm sensitive to that argument, but I also think the parent has a point. Coding the solution to a toy problem on a whiteboard has only limited correlation with the ability of a candidate to solve real-world problems. We use the tool because it's one of the best we have, not because it's a great tool. And if your interviewers can't summon the energy to make eye contact with a candidate...well, they're one-tool cavemen. Most software engineering demands communication skills and culture fit, yet we're still screening as if individual mental horsepower were the most important factor.
The danger in dismissing someone because they can't stand at a whiteboard and barf out the code to print the levels of a binary tree is that it has next to nothing to do with real work. That's just a fact. I probably couldn't do it correctly on a whiteboard, on my first try right now, and I write a lot of code. And I'm sure some hiring manager would eagerly dismiss me as "another frickin' PhD that doesn't know how to write code."
The point is twofold:
1) As in all things, good judgment is key. Sometimes the candidate can bork the technical question, and still deserve a hire.
2) If you find yourself depending exclusively upon the outcome of coding puzzles to screen candidates, you've already failed.
I was talking to a friend of mine about just this issue recently. He and I have been on both sides of the interview table at different times. I have gotten interviewees to solve toy programming problems and I've asked them the funny little logic problems, just as I have been asked the same sort of things in the past. But eventually I too came to the realisation that these kinds of things had little if any real significance to the work that the interviewee was going to be doing. Is it usually going to be the case that the guy you hire will have to write his code with unusual speed, under the pressure of three or more people looking over his shoulder and judging his answer, without the benefit of Internet access or Intellisense or what have you?
My approach is to throw out the whiteboard coding and logic questions whenever I can and to look for examples of a prospective employee's real world work, namely contributions to open source (e.g. GitHub makes this easier) and if not that, then perhaps they can directly provide me with private real world examples of code they've written.
I realise that not everyone will necessarily be able to provide such examples of real world work and yes, looking over someone's pre-existing project will probably take more time than seeing if they can reverse a string. But maybe this is also part of the problem: hiring practices in some places have gotten lazy.
I can't think of a way to entirely, in 100% of cases get rid of FizzBuzz, and the guy who has to escape the fire spreading from one side of the island etc. but I really wish I could, because it just doesn't feel right.
I'm constantly surprised when I read articles like this one, or when I take interviews myself, that noone considers how the employee deals with a "project as a whole". Toy logic problems or algorithm optimisation tell you nothing about how good that person is at actually working as a software engineer. Do they care about code unit tests and code coverage? If they find a process missing - no continuous integration or something - do they just mumble and make do or do they find out why and, if necessary, implement it? Are they capable of delivering quality software, or can they merely write neat, optimised algorithms?
Those are questions a manager should be asking, ie, outside the scope of the interview I'm conducting. They're also the easiest to fake. Everybody knows the answer is to say they write test cases for every change before they check it in. That's way easier to memorize than any coding puzzle solution.
I've had some frustrating experiences where the obviously talented candidate was nearly passed over because they didn't use the right software engineering buzzwords while at the same time the manager came close to overruling the technical veto because they felt some other idiot sounded like a good fit.
Teaching someone to write unit tests is, imo, far more likely to succeed than teaching someone to understand recursion.
I can waste a couple of weeks of my life, go to Glassdoor etc. and basically memorize these questions. I'll probably stand a good chance of blowing the interviewer away by pretending to solve them on the spot. But what does that tell you about how I am at real-life problems?
Most of the so-called algorithm experts I've seen fit into this knows-theory-but-no-practical-stuff category. The fact there are some list of questions and their standards variants. All you need to do is just crawl over interview forums. I assume they even have books, as in pdf ready made for these purposes. Just ensure you read all those questions before hand. May be spend an hour a day to get familiar with them.
Go to the interview, when the interviewer asks the question act as though you've never heard the question. Act like you've been having it tough. But then suddenly like a hero emerging from a crisis present the solution to the interviewer.
You have not clue how many people do this. I've seen candidates, whose only job is this. They change a company every year. Spend half their day everyday studying and collecting salary and interview trends. Apply to a big brand, game the interview collect the x% hike stay for an year and then move on.
These kind of people are an online club. And they tend to hire only their kind.
No regards for the guy's actual work knowledge, his ability to solve real world problems. His knowledge of a programming language, tools, techniques. His productivity all that is irrelevant.
All that matters in these large web companies is a Ivy league brand and theoretical knowledge and some arcane facts memorized at college. Close to 98% of the so called algorithm experts fit into this category.
The genuine 2% are at places they like to work at and surely they pick the place they want to work at and not the other way around.
The cost of hiring someone who can't do this stuff makes it worth the risk of putting off someone like you. Sometimes you have to ask rather than be faithful if you haven't personally worked with the person you want to hire. A shitty programmer will kill an entire team.
There's a line here - I think the interview should cover a lot of bases, but by no means should a hire be a concrete thing. The first two weeks should be an intensive, fully-paid interview/trial-by-fire.
I suppose, I just don't see why a shitty programmer couldn't be filtered out almost immediately through this technique. Maybe it's a company size/age difference - what size company do you have in mind?
In the real world, people often have to relocate to work for your particular company. So you're out $50,000 if they don't work out in the first two weeks and you fire them. Also, people that will work for you are working for you (rather than themselves) because they value stability. You owe it to them to get it right the first time.
(Also, at big companies, people cannot be fired. If you fire someone, their manager "loses an open" and they are less likely to get promoted. Remember, you have to deal with the bad employee, not your manager.)
As a Facebook employee who holds a PhD, I can say we are constantly preoccupied about improving the quality of our interview process. In particular, the topic of how to adapt the interview process advanced degree holders comes about quite often. For example, I recall we recently discussed that we should offer PhD candidates the option to give a talk about their research. (The option is indeed available.)
That being said, I think a dismissive or patronizing attitude towards a coding interview would be mistaken. Allow me to explain.
It is quite well known that the ability to code is poorly correlated with holding an advanced degree or other formal pieces of evidence. A candidate with strong credentials understands this and should, therefore, not be in the least offended by being asked coding questions. As a good professional, the strong candidate would handle the questions with panache and should be ready for more difficult questions, discussing advanced topics linked to the question at hand, or discussing her particular area of expertise. Either way, she must be able to code her way out of a paper bag, and I see nothing wrong with the interview process probing for that.
I've interviewed a few dozen engineers while at Facebook, many with advanced degrees (because recruiters pair as well as they can interviewers with interviewees for areas of expertise), some from top 10 universities. I ask 2-4 coding questions extracted from my daily job, starting from undergraduate level and on rare occasions ending with a difficult complexity question. Nobody has ever done perfectly well, and for a variety of reasons holding an advanced degree doesn't correlate strongly with doing well.
All in all, I think you'd be mistaken to believe two weeks of rote memorization would be enough to pass the Facebook interview, which is very difficult. Overall, an attitude of professionalism, modesty, and focus helps a lot in any interview.
I have some simple baseline questions I ask everyone. Getting it right just means we continue the rest of the interview, but doesn't get you a job.
I almost always have a programming question I just made up 30 minutes before the interview. I make sure that I can easily do it in 10 minutes and then ask it. It really isn't important to me if they get they answer, but I want to see what questions they have, and how they think. Good candidates will usually get it coded, but some may get hung up on something early on, but still make good progress.
If they come in with a precanned response to my question then they're either a mindreader or can time travel. In either case, I still went them on my team.
>> I almost always have a programming question I just made up 30 minutes before the interview. I make sure that I can easily do it in 10 minutes and then ask it.
This is a lot easier said than done. Do you really do this and, if so, any tips on how? (I'm guessing variations on a core set of questions.)
I usually don't mind asking a questions someone may have already heard because there's not really a right answer, but I just want to discuss a problem to see how you think through it.
I actually don't have an explicit set of core questions, but I'm sure there are clear themes that run through the problems.
Probably my only real tip I can think of is not to have them code something related to stuff you're currently working on. You'll be far more likely to underestimate how hard the problem is.
But the problems aren't that hard to think of. They're not like ACM programming contests. They're more like:
You have two lists. One is a list of IP addresses. The other list contains IP addresses or IP addresses with wildcards. E.g., 192.168.$.1 or $.255.255.255. Return the set of IP addresses from list 1 that are matched in list 2. Additionally return the index from list 2 that matched -- I want the index that is most specific (as defined by having the fewest wildcards, or wildcard furthest to the right when there is a tie).
If you are implementing, than you are right. If you are interested in studying algorithms and how they work, asymptotic considerations can tell you a lot about the algorithm. You have to decide what you want to ask for in your interview.
My usual question on the interview is to implement an atoi() function. All I care is that a person doesn't do stupid shit like switch case for characters or a strlen() for iterating through the input. It really is a 10 to 15 minutes task and boy it shows whether a person can write code. Usually then I continue with "how'd you test it?" question, then move on to multithreading and a quick design question. All in all I spend maybe 30 minutes to tell if I'd make a hire.
Unless the end result is a completely incompetent applicant getting an offer, I wouldn't call the system "horribly broken". Very few companies are in the business of reversing linked lists, and hence make binary decisions depending on the outcome of just that one question.
I don't think gaming the system is that trivial - an interview is a conversation, so things like computational complexity, runtime restrictions, data structures, implementation differences in various programming languages, and other things are bound to come into conversation. Now, if one can maintain that conversation as it moves along, and seem like a pleasant person to work with, it seems that they're no longer purely "faking".
Exactly. I've had lots of candidates basically pass the initial question and then sink their own battleship by volunteering some running commentary that happened to be totally wrong. Memorizing enough crap to impress an interviewer about even a basic reverse the linked list question without understanding it is actually quite hard. It's not enough to remember to drop the terms stack and heap, you have to remember which one is which.
I also don't think that what the article describes is "gaming" the system, except maybe for the part about pretending to have not heard the questions before.
Generally, the reason I ask such questions of candidates is to make sure I'm dealing with someone who can actually analyze a problem and get a reasonable result. So if they nail the questions because they took time to study and get good at interview-style brain teasers, that's actually fine with me - it proves the person can execute a plan to get correct results.
I'm not sure I'd call anyone who successfully "games" the system incompetent.
Going through the details is the way to estimate candidate's true knowledge of a subject. Ask an easy question, then ask a very hard one, repeat depending on answers, i.e. basically do a binary search:
- Do you know what SSL/TLS is? general
- What's the maximum size of SSL protocol record? specific
- What's a cipher suite? general
- How is DH-anon suite is set up? specific
...
- What is a linked list?
- What is offsetof (and how is it relevant)?
...
- What is TCP? general
- What is TCP/Vegas? specific
...
This is the quickest way to converge to an actual level of expertise, and it really helps with filtering out people with bullshit resumes and those who cheated their way through an offline problem solving. We would say something like "You are not expected to answer all questions, we are just trying to understand what you know and what you don't know."
I recently interviewed for a position with this sort of technical interview. I felt it worked very well. The interviewer was able to cover a wide range of topics in varying depth, and could see what I could easily recall and what I struggled with.
With the right set of questions, and some hints when the candidate struggles, you can very effectively gauge a candidate's limits on a subject, and see how they try to puzzle things out when they don't know an answer but know enough to take a good guess.
The key distinguishing factor of this method, IMO, is a candidate cannot memorize enough answers to bluff their way through a good questions list. If a candidate really is an N out of 10 with technology $FOO like they claim, they should have no problem answering most/all of your 50+ N/10-level questions for $FOO (my technical interview consisted of ~2 hours of this sort of interrogation for a straightforward junior web dev position. Interview length, in this interviewing style, is important for ensuring you've exhausted a candidate's book knowledge).
There's always a downside to every approach though. Some people get flustered if they have to say "I don't know" too often, so instead of bouncing from tough to easy, I'd probably prefer to progress from easy to tough.
People will feel confident until they hit their limits. Their limits are what you are looking for (presumably), not their ability to think clearly when they are scared they just blew the interview on a question you didn't really expect them to answer.
Yeah, that was more or less the point of the post. For applicants, don't feel nervous, it's a silly game just to see if you know some of the puzzles.
For people hiring, go ahead and use this as a first-pass filter, but realize that it's easy to game (by someone who's trying hard and learning, so there is a positive signal in that). It's going to be important to go through and work with them for a period of time in a let's-date-before-we-marry relationship. As an employee, I always insisted on a contracting relationship before an employment relationship.
Like I said in the footer, Heroku really seems to do this the right way, and it shows. Kudos to them.
In all my history of interviewing people, I've always found there's almost no info to be had in correct answers to any kind of question. You learn about a candidate once you guide them into territory where the fail.
Both where that line is, and how they handle the failure tells you a lot.
(Caveat: Make sure you're keeping it on the fail/no-fail edge, and you treat the candidate with respect. The point is not to show them that they're wrong, but to explore the boundaries of their skill set. Without being mean is a major goal here)
And if you can't push the candidate there: HIRE THEM. Right away. If they know that much more than you do that you can't get them beyond their limits, they are a fantastic addition to your team. (Provided they match personality-wise, too)
I've done the same - keeping on pushing until the candidate is on uncertain ground. (I interview for consulting/ analytic /business/ senior management roles.)
As most of a job is about doing things that haven't been done before, getting the candidate into unknown territory is when you really find out how effective they will be. Some people just stop, others switch seamlessly into problem solving mode and come up with an answer that may or may not work, and also with a plan to test whether it does.
One of my favorite interviews was like this. The interviewer started by asking me some of the basic logic problems. I had heard most of these many years before. He actually appreciated it when I stopped him mid-question and finished the question for him. He asked another one, I finished the question for him. He saw that I had a math degree and asked about a particular problem involving infinity that he had heard about, and I gave him the summary of a half-hour talk I had given on the topic. Finally he sat down and he told me about a startup he had worked on when he was young and we talked through the technical problems. He stumped me most of the time but we both agreed that it was a very enjoyable interview. Funny thing is, that was also the only interview I had ever been on where I didn't get the job. Probably because I didn't where a tie to the interview and this particular company is apparently known for requiring ties.
That's why I always asked for dress code before the interview. You may feel slightly stupid when you ask, but you'll be dressed right when you come to the interview.
This sounds like a great technique. It also gives both sides the chance to talk a problem through, bounce ideas off of one another, and see if there are any interesting insights to be had on the other side of the table.
A candidate and interviewer both have to be fairly relaxed and comfortable to do it properly, but that's more of a problem for people new to the interviewing process (probably not senior management roles)
It also gives both sides the chance to talk a problem through, bounce ideas off of one another, and see if there are any interesting insights to be had on the other side of the table.
Some companies do this intentionally without hiring anybody in order to double-check their plans. It's basically free consulting.
That can be gamed as well. The OP could have let himself be pushed further by getting it done faster, but he did the "silly but adorable" thing of solving it with some mumbling and hesitation, which seemed to convince the interviewer they had seen his learning process. Someone more devious — betting on an unusual interview style — might have failed some answers on purpose.
"you need to know the subject better than the person that you're interviewing. If you don't, you have no hope of screening for good people."
I think it is not necessary and not possible that the interviewer know more about every subtopic than every candidate. It is enough that they have comparable knowledge in most topics. Good candidates can very easily know more about some parts of the subject (and maybe less about other parts). Being a good interviewer is a really tricky business, because good interviewers should see the strength of candidates and should see if the candidate know even more than them about some subtopics. I've seen some bad interviewers who asked only the very narrow topics they knew well, and basically wanted exact copies of themselves.
It sounds like in this case, this guy might be a good candidate - look at all the effort he put in and his ability to remember things, think critically, and learn.
On the other hand, there are plenty of competent programmers not willing to put up with this nonsense, and the companies are going to miss out on them.
All that is true, but there's a simpler fix. If they're not reusing each others' questions then it should be significantly harder. Nobody should be using "reverse a linked list" anymore - that's a really old one.
That's not a simple fix. If you don't use "reverse a linked list" anymore, then you start asking increasingly harder and harder questions, and the bar gets ridiculously high, to the point where everyone needs to know how to solve NP-class problems within 45 mins. And even then the system still can be gamed by simply memorizing more and more questions. It's a lose-lose situation.
You just have to make the problems so diverse (not necessarily hard), that genuinely understanding the solutions is the easier strategy than rote learning.
Can you give me an example of a question that isn't located in glassdoor.com and is "diverse but not hard". As far as I'm concerned, the bar is already ridiculously high to the point where I feel like the only people that get jobs are the ones who randomly get lucky enough to get 3 or 4 interviewees that ask easy questions.
There are people I know personally that have gone through every single one of the 1000+ Google interview writeups in glassdoor in order to prepare for interviews. One of my former co-workers showed me 10 pages of questions he distilled from glassdoor of various questions that were reported, and he memorized the answers to all of them, ranging from "find the common descendant of two nodes of a directed graph", to programming an AVL tree on the spot, to pure dynamic programming questions.
If they asked these types of questions 15+ years ago when I started, there's no way I could have gotten into programming. Even now, the caliber of those questions are simply too hard for me unless I sit down days before the interview to get the answer. There's no way I would be able to figure those out within 45 mins let alone 2 hrs, and I'm not dumb. It's ridiculous the level of expectations people have these days.
> Can you give me an example of a question that isn't located in glassdoor.com and is "diverse but not hard".
Sorry, `diverse' is an attribute of the distribution of questions you ask. Not of any single question. If for each interview you take a random question from glassdoor.com (and perhaps even tweak it a bit, if you feel like it), that might even meet the definition of a diverse but not too hard selection. (Unfortunately, I know almost nothing about glassdoor.com, and what kind of questions they have.)
When applied to me the problem is probably the other way round: I like all those silly questions and solve them in my free time for fun. (I did study mathematics, because I like this kind of thing.) But honestly, that doesn't help me much in writing non-trivial programs, and there's a bunch of people I know that are worse in those interview-type questions, but better at developing software.
But the clever thing is the work required to answer these questions is a filter in itself. Firstly it filters for the seriousness of the interviewee - have they done their (days of) preparation? And even then only a few people will have he ability to actually understand the quiz questions, and even if the answers are there in front of them.
The only problem is the filter for being good at cramming interview questions is not quite congruent with the filter for being a good programmer/developer/engineer/$role_youre_hiring_for.
as someone who successfully built a great dev team at a startup, i can say that this is a sign of sheer laziness on the interviewers' parts. which is sad, because in a small company, hiring is one of the most important things you can do. there is no real excuse for having an interview so generic that someone can game it in this manner unless you're so large that you can afford the odd charlatan.
> there is no real excuse for having an interview so generic that someone can game it in this manner unless you're so large that you can afford the odd charlatan.
I'm also part of a small start-up (as an employee), and I can certainly vouch for that. I for one put a very high emphasis on the personal/professional projects the guy/dudette in front of me has been involved in, on how best s/he can explain them to me, on how passionate the person seems when explaining all this (and no, I'm not talking about the enthusiastic approach that can be easily faked, I just think that when you're a programmer you can feel what the other programmer in front of you is very interested in).
Also, this mania a lot of interviewers have with the interviewees "always ticking the right boxes and giving the correct answers" sort of scares me. It's probably because I work in a start-up, but I can assure everyone that at least half of the time we ourselves don't know the answers (assuming that we even know the correct questions), and there are lots of times when we seem to be knowing in what direction we're going but truly speaking we don't.
All this tells me is that tech interviewing is horribly broken. What we have here is a story of a guy with a little bit of experience who has (if his story is true) successfully learned how to game the interview systems at dozens of startups. If you interview people, this article should terrify you. You can argue that it isn't "gaming" if the guy turns out to be a good employee, but that's wrong. If he can trivially game the system, so can anyone else with the ability to go to dozens of interviews and remember questions.
There's a fix for this problem: make the interview about more than the answer to the stupid question. If the candidate answers your question a little too quickly, ask something harder. Chase down a detail. Pick an experience on the resume (and not an obvious one), and dig as deeply as you can. Good candidates know the details. Bad candidates don't have details you can chase.
If you're interviewing correctly, no candidate should be able to come in and snow you with some regurgitated C code from previous interview experience, because you'll be able to out-flank them at every attempt to cough up a line. Unfortunately, there's a corollary: you need to know the subject better than the person that you're interviewing. If you don't, you have no hope of screening for good people.