Wow, it appears my point was completely missed (and I was otherwise painted as a bigot), so let me try to explain better.
> It's a solved problem, and something an employee would NEVER do in your company, let alone search for the optimal solution in 10 stressful minutes while you breathe down his neck.
Actually, just a couple of weeks ago we had something almost equivalent, that involved chasing "pointers" through databases and files. It was for less than 1M cases, but still - any O(n^2) solution would have been infeasible (and an nlogn solution about 20 times slower than an O(n) solution; in this case, it was a one-off, and the difference would have been between 1hr of runtime and 1day of runtime).
> When's the last time you ever did anything as low level as reversing a linked list?
Please re-read what I wrote. I wrote, in fact, "you (almost) never need to reverse linked lists in practice". I also explained that it is an abstract version of a kind of thing that DOES happen often.
> Major red flag. I don't mean to be rude, but this sentence comes off as very arrogant.
Perhaps. For all you know, I'm a dog, but I'll say that I'm soon entering my 4th decade in the industry, I'm well respected, I've hired (and fired) tens of people through the years (and interviewed hundreds if not thousands); and I'm well respected among my peers for having very accurate evaluations of people's abilities. I do not aim to be politically correct.
> Why? Because they're in a lower caste? If you can't handle direct feedback, you're a terrible manager.
No, because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now. I meant to say "direct immediate feedback", which is perhaps better wording.
After I know a person, I ask them (and usually get) the most direct and raw feedback I can get.
> it was a one-off, and the difference would have been between 1hr of runtime and 1day of runtime).
What's wrong with 1h vs 1d for a one-off thing? Your effective delivery time would be close enough to equivalent after all the administrative BS and customer expectations, and a naive approach piss simple to verify.
If it was going to mean the difference between 1h and multiple days, here's what I'd do:
- Fire up my trusty browser.
- Navigate to google.com
- Type in "reverse a linked list"
- Evaluate all of the solutions I find and pick the best efficiency / verifiability match to solve the problem in a reasonable amount of time.
Are you going to test that in an interview question? No, you're going to effectively "whiteboard" me, asking me to reinvent the wheel on the spot. And I'm going to resent you for it.
> but I'll say that I'm soon entering my 4th decade in the industry, I'm well respected, I've hired (and fired) tens of people through the years (and interviewed hundreds if not thousands); and I'm well respected among my peers for having very accurate evaluations of people's abilities. I do not aim to be politically correct.
And I'm in my third decade in the industry, all the respect, blah, blah, half a million lines of open source in the wild. So what? You can make the same mistakes over and over in any industry and never think to question if you're doing it wrong. This is ESPECIALLY problematic when it comes to people skills.
> No, because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now.
It's the PERFECT time to ask them, because it's fresh in their mind. A skilled interrogator can get information despite the subject's frame of mind. I'm actually surprised that you assume everyone you accept or reject is either going to be a sycophant or hold a grudge.
You WANT them to talk while the frustrations they encountered are still fresh in their minds. That's the most raw feedback you'll ever get.
You really do need to start treating them like human beings.
Wanna know how I evaluated potential game devs in person? I asked them what dev environment they were most comfortable in, gave them a computer hooked up to the internet and a picture of a tweetie bird, and said "Make the bird jump. Use google to look shit up." Worked amazingly well.
"If it was going to mean the difference between 1h and multiple days, here's what I'd do:
- Fire up my trusty browser.
- Navigate to google.com
- Type in "reverse a linked list"
- Evaluate all of the solutions I find and pick the best efficiency / verifiability match to solve the problem in a reasonable amount of time."
And now we're getting into absurd arguments. End of interview. Thank you for your time, but I don't think this will be a good fit.
> Are you going to test that in an interview question? No, you're going to effectively "whiteboard" me, asking me to reinvent the wheel on the spot. And I'm going to resent you for it.
Unfortunately, the real world business problem that needs to be solved is not "reverse a linked list", but rather "discover the relationship between the widgez and the widgix that the customer is after". That relationship, once you understand the problem domain, can be discovered by (doing something quite but not exactly like) reversing a linked list. And unfortunately for you, a google search for "discover the relationship between the widgez and the widgix" yields nothing. If you can't (do something similar to) reverse a linked list, you can't deliver the solution.
And it's fine you are going to resent me. I do not want to employ anyone who thinks "reversing a linked list" is an inappropriate question in a technical interview.
> What's wrong with 1h vs 1d for a one-off thing?
I was just giving this for completeness - in this case, it doesn't matter. But if it is 1h vs multiple days, or it happens often (say, every 3 hours), then picking the correct solution -- which your method will NOT generate -- is actually important.
> It's the PERFECT time to ask them, because it's fresh in their mind. A skilled interrogator can get information despite the subject's frame of mind. I'm actually surprised that you assume everyone you accept or reject is either going to be a sycophant or hold a grudge.
I ask again, have you hired people? because the "skilled interrogator" mindset makes me think you haven't. I've just negotiated with a person, whom I've had less than 2 hours to know. If I don't want to strike a deal with them, I want to be fair and tell that to them as soon as possible, at which point the vast majority of them just want to move on (and I happily oblige the very few who do want to continue the discussion). If I do want to strike a deal with them, we start the salary negotiation phase, at which point (assuming they are interested) they have a vested interest in me offering them as much as possible; which some people express by trying to please. These things are often subconscious.
I do not assume anyone is a sycophant or holds a grudge. But I do assume, at that point in time, that they are not exactly objective; and I cannot, in fact, calibrate their response since I don't really know them.
> Wanna know how I evaluated potential game devs in person? I asked them what dev environment they were most comfortable in, gave them a computer hooked up to the internet and a picture of a tweetie bird, and said "Make the bird jump. Use google to look shit up." Worked amazingly well.
Sometimes, you can do that. Often, you can't. Especially if the person has no experience in the field they are interviewing in.
"Here's a stock exchange API, write a bot that makes markets" is super simple for someone from the field, and would take hours (perhaps days) of grokking for someone who isn't. And making a bird jump would not necessarily tell me someone is apt for writing trading software.
> Unfortunately, the real world business problem that needs to be solved is not "reverse a linked list", but rather "discover the relationship between the widgez and the widgix that the customer is after". That relationship, once you understand the problem domain, can be discovered by (doing something quite but not exactly like) reversing a linked list. And unfortunately for you, a google search for "discover the relationship between the widgez and the widgix" yields nothing. If you can't (do something similar to) reverse a linked list, you can't deliver the solution.
How does testing a candidate on how to reverse a linked list help evaluate his ability to understand that the implementation is an appropriate solution for a given problem?
It doesn't. A different question will give me indication about that.
What reversing a linked list does tell me is, once the solution IS arrived it, that this person can implement it. And I repeat again, I treat this as an abstract version; real world problems often have solutions with many corners and are seldom if ever "Eureka! i just have to reverse the list".
Another real world problem I just recall is analogous in many ways to a "git merge" which requires tracing through a huge DAG. In these kinds of things, even if you realize that it's a git-merge, it is often not possible to generate a git repository just so you can run git-merge and observe the result.
There are two independent skills: abstracting a problem, and solving an abstract problem. I test for both but the linked list question only addresses the latter.
> There are two independent skills: abstracting a problem, and solving an abstract problem. I test for both but the linked list question only addresses the latter.
The problem (heh) with this is that, unless one is actively doing research on improving an existing technology or method, all abstract problems that might come up are already solved.
Every solution is one search in google away (including your git example) so there's no point to it. Paraphrasing someone else: if the time spent in coding a problem is expressed as mx + c, m is the time spent in figuring out what the problem is and c is integrating a solution. That c tends to be insignificantly small.
Frankly, as someone holding a degree, if you were to interview me and asked me to reverse a linked list, I'd do it, laugh in your face, and walk out.
On that point, interestingly, in your earlier examples of people you got feedback from, you didn't mention the most significant one, the one more likely to point out the problems in your process and that would give you actually actionable feedback: the people who rejected the job offer.
Didn't happen many times, I did ask for feedback, but rarely got any; mostly along "your interview was fair, but I prefer a different offer I received". Which is what I would expect - unlike what some people in this thread expect, the candidate is maximizing their revenue and future benefits; saying anything nontrivial after rejecting the offer is not helping towards either, and potentially harmful.
> Frankly, as someone holding a degree, if you were to interview me and asked me to reverse a linked list, I'd do it, laugh in your face, and walk out.
That's good, actually. Means we have a culture mismatch, identified early. Win win.
> unlike what some people in this thread expect, the candidate is maximizing their revenue and future benefits; saying anything nontrivial after rejecting the offer is not helping towards either, and potentially harmful.
I had already noticed a very strong reality distortion field in your other comments[0] but this one is just weird. I have been a candidate, most people in this thread have, exactly why their saying what they'd do is not a reflection of what a candidate might do? For instance, I cannot conceive why saying "anything nontrivial" matters in regards to revenue and future benefits. I've done it before (probably not "laughing in their face" but, then again, I have yet to meet an interviewer with such a clear misplaced arrogance) and so far, rejecting a company and giving feedback has not hurt me in any way; they have no impact on my future because I rejected them.
For that matter, how did this even address what I said? I see not how it isn't a non-sequitur, other than you believing that you have a strong enough pull in the industry that you could badmouth me into not getting a job if I rejected your offer and told you it was because your interview was terrible. Of course, that is not the case.
> That's good, actually. Means we have a culture mismatch, identified early. Win win.
Yeah, no, that wouldn't have been deduced from that interview; one could get a culture mismatch by exposing the applicant to the actual company during the process. What I said would only bring to light that your interview process is terrible and doesn't actually account for people who aren't desperate for getting the job (which, in itself, explains why you think everyone you hire is a sycophant, they probably have to).
Of course, I understand your inability to see that it might be a problem with you specifically; no, it must be that the interviewee has a problem with "the culture" of the company.
[0]: For instance, there was no majority saying that the creator of homebrew "was in the wrong" regarding his tweet. At most, there was an even split, although there was far more complaining about interview processes like yours.
> other than you believing that you have a strong enough pull in the industry that you could badmouth me into not getting a job if I rejected your offer and told you it was because your interview was terrible. Of course, that is not the case.
It is obvious our different experiences give us different views of the world. I would just add that I have seen this happen, mostly in business settings. A lot of people take things personally.
I don't particularly take things personally myself; it's all business or math to me. But when I was younger, I used to give brutally honest feedback about everything, and there were a couple of investment opportunities that it cost me (and I cannot say that it created others). I learned to play the politics game as I matured. I hate the game, yes, but I needed to start playing to break a glass ceiling.
> Yeah, no, that wouldn't have been deduced from that interview;
No, it would have surfaced immediately. I try to hire people whose attitude is more of the "yes, I can do anything that's needed. But why?" rather than "I laugh in your general direction since I think you are wrong".
> with you specifically;
Me and google. And facebook. And hundreds of other companies. You are entitled to your opinion that e.g. Google's culture is the problem.
Which is not to say my company is as successful as google. Just that I share the idea that culture fit is important. I recommend reading "Peopleware" if you haven't yet -- they have some empirical anecdata for that.
Also, I'm amused at how my "I prefer to ignore noisy data" comment (which is all my comment said, explained several times) was taken to mean "I think everyone's a sycophant or crybaby". I think it reflects on the readers' ability to consider that I'm calibrating my stats differently than they expect, or rather lack of it. And I calibrate it that way based on decades of interviewing -- I didn't start that way.
> It's a solved problem, and something an employee would NEVER do in your company, let alone search for the optimal solution in 10 stressful minutes while you breathe down his neck.
Actually, just a couple of weeks ago we had something almost equivalent, that involved chasing "pointers" through databases and files. It was for less than 1M cases, but still - any O(n^2) solution would have been infeasible (and an nlogn solution about 20 times slower than an O(n) solution; in this case, it was a one-off, and the difference would have been between 1hr of runtime and 1day of runtime).
> When's the last time you ever did anything as low level as reversing a linked list?
Please re-read what I wrote. I wrote, in fact, "you (almost) never need to reverse linked lists in practice". I also explained that it is an abstract version of a kind of thing that DOES happen often.
> Major red flag. I don't mean to be rude, but this sentence comes off as very arrogant.
Perhaps. For all you know, I'm a dog, but I'll say that I'm soon entering my 4th decade in the industry, I'm well respected, I've hired (and fired) tens of people through the years (and interviewed hundreds if not thousands); and I'm well respected among my peers for having very accurate evaluations of people's abilities. I do not aim to be politically correct.
> Why? Because they're in a lower caste? If you can't handle direct feedback, you're a terrible manager.
No, because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now. I meant to say "direct immediate feedback", which is perhaps better wording.
After I know a person, I ask them (and usually get) the most direct and raw feedback I can get.