Dogfooding their process is going to give them adverse results here: Microsoft is paying them to take all the time they want, but good candidates won't be so willing to spend so much time. Already, they're up to a full day of interviews, which is ridiculous. One of the worst parts of current FAANG interviewing is how long it takes to get to the day of interviews, needing to take off from work to attend the interview day, and then getting rejected nonetheless. Not only does this process ask for too much commitment on the part of candidates, it selects for candidates for whom such a commitment is less relatively costly - i.e. people who are currently unemployed, students, etc, and while there are many such people with great talent and potential, respectively, they're also less likely to be strong candidates than people who are currently employed.
One of the key issues in recruiting is how to make hiring decisions quickly. Good candidates don't have the patience for your internal politics to resolve over a month or two to figure out whether they're getting an offer or not; they're just going to interview with other places in the meantime who will beat you to the punch.
Take a resume off the pile. Email or text to schedule a call. During the call, talk about relevant experience / past projects, what each side is looking for, etc. Make a decision during the call whether you want to invite the person for a two-hour interview, and schedule it during the call, first offering an evening time slot for the interview. In the first hour, talk about culture, problem-solving and teamwork approaches, engineering attitudes. In the second hour, pull out an actual current problem and solve it collaboratively. If all goes well, send an offer, and offer to set up a dinner to talk about / negotiate the offer.
It should not take more than a few hours to make a decision on someone; it should not require a candidate to take paid time off. Your hiring pipeline's average lead time should be measured in days.
One company wanted approached to interview me for the position of a "scientist". Their initial step of the interview process was to spend 4-6h on some "challenge" they designed. I told them, I can't spend 4h of my time on the 1st step where I have no clue what my hiring manager even does! I made a counter to their proposal as all my research projects are publicly available. My counter was similar to your proposition. I asked them to pick any of my research projects and talk about it, I told them I just need a 5 minute heads up. Of course, they didn't take me up on it because they required "commitment from the candidate". I chuckled on the irony in that last phrase. Interview is a 2-way process and it seems a lot of people have bought into some of the kool aid.
I can share with you one hiring insight I have that goes against your sentiment. I've been hiring for hedge fund research roles for the last three years interviewing ~30 candidates over that period.
What I've learned is that it is a very weak signal to talk to candidate about "their research" unless it's immediately and directly applicable to what we were doing as a company, so that we can talk about it as equal experts, which never really happened in our case. Previous research done by candidate was, like research almost always is in a narrow and specialize niche. Every candidate I was talking to was very good with talking about their research and from my perspective at the time of the interview it was impossible to validate and evaluate their claims. That conversation could convey the character of the candidate but I couldn't evaluate their ability to do new research.
Instead, presenting candidates with the same problem, that in our case took around 3 hours to solve, in our case proved a good way to compare candidates a bit more objectively and test how they approach solving problems they haven't seen before which is more applicable to what they'll be doing on the job.
The downside of that process is that during three hours you can only test so much, but hiring is a really hard thing to do right and we didn't want to take more of candidates time as we did value it highly and found that to be the sweet spot for us.
The upside of asking candidates about their research is it does a good job of verifying what their contribution is in often highly collaborative environments. Too often I've seen candidates who've worked on super exciting stuff, but not actually been the key contributor. This is way more common in junior roles though.
> That conversation could convey the character of the candidate but I couldn't evaluate their ability to do new research.
It all depends on the questions asked to the candidate. You don't ask them about their research which is publicly available in any case. You ask them questions that let them reveal their thought process about their research. Now, it's up to the interviewer to come up with good questions to make this happen. The interviewer has to be well prepared and think about questions. You don't need to be an expert to ask good questions.
I'd like to reiterate that this applies to research/scientific position where pushing the boundaries is a crucial aspect of the job. In case you are looking for a very specific skillset, things may be different.
>>I told them, I can't spend 4h of my time on the 1st step where I have no clue what my hiring manager even does!
You had a lucky escape. I had the same from a company. Which was arguably a 2 full nights worth work if done right.
I took the challenge. Did it, it passed their functional test suite which they provided. Unit test cases, git repo, documentation etc etc all in place.
They came back saying it was not enough. So it's not like if you do the take home project, they let you in.
Also its a tad little irritating to make people work for so long, make them do everything. And even then with perfectly working solution turn them down.
I had one which took a similar effort, passed all their unit tests, then they went into radio silence and after trying to understand what was going on, the only feedback I got was "we were expecting something else".
Further inquires about what was wrong, given that it was working as per specification got zero answers.
For a long time I worked for a company that used a coding test as the first stage of the selection process. We advised candidates not to spend more than about 90 minutes on it, although I'm sure spent more. There was also no time pressure: we had some people take weeks to return the completed test due to exams, holidays, family commitments, or whatever, and it was no big deal.
Anyway, this all seemed perfectly reasonable until I went for a job I'd been approached about by an agency with a similar test at the beginning of the application process. They'd asked me to build a simple calculator and said it should take no more than 2 hours.
No big deal, but it was the weekend before I had 2 hours free where I could do it due to other commitments. Pushy recruiter didn't understand this but, whatever[1]. Should have been a red flag straight away.
Anyway, I put together a working solution with comprehensive tests, send it off to the recruiter and on Monday hear... nothing. That's right: nothing. Gave it a few days, got in touch with the recruiter and still nothing. No, "sorry, it wasn't good enough." No feedback on what might have been wrong with it. Nothing.
I was absolutely incensed. The whole incident gave me a different perspective on pre-interview tests, to the point where I'm no longer really interested in doing them for "unknown" companies. I made an exception to work with an organisation where I had several friends, so was confident that even if I didn't pass I'd get some decent feedback on why.
The most significant effect, however, has been on my own approach to recruitment. At the moment we don't, and I have no plans to introduce, programming homework as a pre-screener. Rather we take a much more personalised approach. We're also in the situation where we get few enough applicants that we can afford to talk to all of them, although this won't scale and will have to change.
Still, it's hopefully uncontroversial that before you invest the 5-6 hours (across 2 people) required for a 2 hour technical interview (in our case an exercise where we work together on a problem), you do need to some sort of pre-screener to weed out the people who really can't code: https://blog.codinghorror.com/why-cant-programmers-program/. I wouldn't say this is 99 out of 100 programmers, nor anything like, but it's certainly the majority. We therefore do a half hour telescreen, which is a fairly informal chat about what the candidate has been working on, what we do, and then - of course - there's a very simple programming task at the end. This takes an investment of roughly 90 minutes on our side (again, 2 people). We're explain the format to candidates in the invitation, which gives them an opportunity to decide whether they want to go through with it or not.
Even if candidates don't pass, and it's usually on the technical task that they fall down, we give them feedback, including pointers to how they can improve their skill level.
(Apologies, this came out much longer than I intended.)
[1] There's some merit to the "well, do you want a new job or not?" mode of thought but not much. E.g., in my current role I advise candidates not to do our technical interview after a full day of work because they're not likely to perform at their best. Obviously not always possible, but it's about ensuring they set themselves up for success: if you dislike your current job enough to be looking for another one don't you want to make sure that when you get an interview you do everything you can to do well in that interview? This is obviously very different to expecting candidates to jump through hoops under time pressure just because "commitment".
Once I wanted to interview for a company that wanted me to implement a FULL chat client, including the full range of auxiliary features, for step one...
>Interview is a 2-way process and it seems a lot of people have bought into some of the kool aid.
What too many people choose to forget or ignore is employment itself (at least in CA) is a 2-way process. The employer is not doing you a favor by employing you, despite how many people have drank the kool-aid to the contrary. You and the company have entered into a mutually beneficial agreement. Unfortunately, many people don't have the means or desire to walk away when an employer veers into "you should be thankful we let you work here" territory.
Yeah, if only it was all unicorns and rainbows. What you describe is how I imagine they interview for director / VP positions. With hundreds (thousands?) of applications per day for software engineering positions, all with perfect resumes, what you describe just doesn’t scale. Software engineers do 1-2 interviews per week each, it’s part of the job. I might be a minority, but I would refuse to spend two of my evenings per week interviewing people. I’d rather spend time with my family instead. I think asking candidates to get a day off for interviews, which happens like once per year or even less often for each candidate is more reasonable than asking your employees to work evenings each week.
> Software engineers do 1-2 interviews per week each, it’s part of the job. I might be a minority, but I would refuse to spend two of my evenings per week interviewing people.
Why are software engineers doing interviews at all? What makes software engineers qualified to interview people (especially when you consider the need to avoid hiring bias, problematic statements by interviewers that expose the company to legal suit, etc.)?
I'm not saying that companies should go to the other extreme and have all interviews conducted by HR - that presents its own set of problems in that HR doesn't know how to evaluate candidates for technical skill. But expecting hiring managers to open up a few evenings per week in the irregular and relatively uncommon circumstance (if it's common, then you have problems with churn on that team, and you should consider firing the manager) in which positions open up on their team to evaluate potential direct reports is not even remotely unreasonable.
i am right now preparing to fill two positions, and i am looking to interview at least half a dozen people maybe more. due to my schedule i can't really interview more than one per day, so this is going to take a week or two.
i do want the opinions of the people already on my team, because a big part of the interview process is to find out how well they work together. for me hiring is a team decision, not a top down mandate. so yes, all of my software engineers will be doing interviews. there is no way around it.
on the other hand i agree with you. these rounds of interviews should not happen to often. if our team grows, i expect to add two more positions every quarter. (i don't know how realistic this is, but this just to give you an idea of how much interviewing workload i expect.)
if i am looking for a different job, i'll not just interview with one company, but several. even more so if i still have a job and i really need to weigh the options to decide if the new offer is really better.
there is only so many days i can take off. with two weeks holidays, i might spend all of my holiday budget for this year.
I'm not sure what you think is a reasonable alternative. In general, having some F2F on-site interviews is in everyone's interest. I'll agree that there should be some reasonable pre-screening before everyone makes a significant time investment. But the reality is that if you're job hunting you may well have to take some time off--especially if the opportunities aren't local.
i was mainly responding to this: I think asking candidates to get a day off for interviews, which happens like once per year or even less often for each candidate is more reasonable than asking your employees to work evenings each week.
by stating that it is most certainly not just once as i am likely interviewing for multiple jobs.
i don't know what the solution is, but i think it is reasonable to say that some compromise is necessary, and as much as i value your (and my) time with family, if we want to grow our team so that we are not having to work overtime as much and actually get more time for our families, then we'll just have to make some sacrifices.
apologies for being a bit polemic.
actually, i do have a solution for myself. i only hire young people fresh out of university who don't have a job yet. higher positions are filled from within. ask me in a few years how well that's going.
>i think it is reasonable to say that some compromise is necessary
I'm honestly not sure it is. Half-a-day to a day of on-site interviews (possibly after a phone screen) is how most companies have been filling professional roles for decades. Certainly it's an easier process for people doing it out of university. And it's easier to do locally when a candidate can often just take a few hours off. But, while many debate some of the specifics of how companies screen for technical talent, I don't see widespread pushback against candidates coming in for on-site interviews of some sort.
it depends on how many candidates you have. a few months ago i was hiring for a position, where i didn't get a single candidate who was able to take time off work. (ok, i had one who took an extended lunch break, but the rest were evening or weekend only)
I was referring to this:
> invite the person for a two-hour interview, and schedule it during the call, first offering an evening time slot for the interview.
I completely agree with this. It's a process that's practical, fair and scalable.
I'm currently interviewing and so far I have had a 48 hour weekend project, a 3 hour code assessment, and I'm expected to endure 3 days of interviews. I'm fairly close to throwing in the towel and saying I can't be arsed with this.
I very much disagree with you: the more experienced the candidate is, the more the interview is about the company and the work environment, not just the candidate. I.e. it is a two way street, and the candidate has to decide in the end whether he/she wants to work for that company or not. There is simply no way a 1 or 2 hour interview can provide this information - this is why you need more time, and this is why I do think the process described in the post is a great way to interview.
This may not apply to all companies or situations, but for a Microsoft-sized organization this is the way it should be done.
If you're Microsoft-sized, then if anything, your interview process should be less strict, not more strict. At organizational scale, enforcement processes become more important than culture. That means building out custom standardized entrance exams, regular "induction days" with mandatory internal courses for new hires (wherein it is your job in the first few weeks or even months to just go to those courses during normal working hours), deployment pipelines which guide new junior hires by enforcing minimum test coverage, style conventions, passing static code analysis, etc.
Your hiring needs should reflect the size of your organization, and your hiring process should reflect your hiring needs. If you hire relatively infrequently then it makes sense to go over individual resumes and have infrequent evening interviews. If you need to hire hundreds of people this month just to negate churn then you have the resources to build out the kind of enforcement-process-oriented above since you surely need that kind of infrastructure anyway. Everything in between is just a matter of degree and where your company is on the process-vs-culture balance as a function of its size.
I sorta agree on the latency although there are a lot of unavoidable reasons that happens including but not limited to travel schedules of the decision makers and the need to interview multiple candidates.
However TBH an in-person day with 4 or so interviews (perhaps post a phone screen) is absolutely standard for all sorts of professional positions and has been for many decades. I can only think of one case where I just had an informal in-person lunch instead--and that was a case where I knew the company's owner very well.
I sometimes wonder if a shortcut would be to hire & Fire quickly but only for short-term contract work. After X time as a contractor devs can upgrade to a full position.
> But the aha moment for me was that not everyone does well in those fast-paced brainstorming sessions. A lot of people (including me) prefer to sit with a cup of coffee and some data and try to think things through.
This is me. Hard to get that across in an interview, but once people work with me, they're cool with me coming back an hour later in an email with some thoughts on the last meeting topic. They know and respect that that's the way I work[0].
[0] I'm not the only one on the team who does this, so that helps.
I'm like this and I know why it happens for me. I cannot think my thoughts while listening to others. Either I'm listening, or I'm thinking, I cannot do both at once. I can follow others and sometime I can make a contribution, but it would be a small contribution, a deal less than I could do given a time to think.
I tried to find some way around it, and the only way I see is to learn how to think loudly. The only way is to supress others from thinking by speaking aloud my thoughts as they come. Let others follow my thoughts, not the other way around. I figured it out while making some group work with ~20 yo students being a student myself. I'm older (I was 35-40 over that period) and therefore smarter and in the most cases I was able to do this group work myself a way better then they do.
I tried different approaches. I tried to take a leader role in a discussion and to talk everytime I feel like this. I tried to let someone else to become a leader and to follow her. I tried to just get out of the discussion, to sit quietly with my own thoughts and to come with my own solution to compare it with the group one. The conclusion is: I cannot think AND listen at the same time.
Given that many people report that their best thinking is done while walking or taking a jog, it might not be too wild to try out interviews which allow the candidate to take a walk before they give an answer.
that's an interesting idea. how can we implement that without it becoming awkward?
the problem i see is this:
it is unexpected, so the interviewee doesn't know what to make of that offer.
i don't know how much time to give them, and they don't know how much time they need. they also don't know how much time they should take.
i don't even know if they are the type that likes to take a quiet time to think. if they aren't then that offer could come across as a suggestion that their answer is wrong and they should think about it more.
at best it would be some institutionalized coffee-break, like we do the interview in two parts, first we are doing a bunch of exercises, then we take a break and after the break we do something else, but the interviewee is offered to add thoughts on the exercises that they may have come up with during the break.
The solution to unexpectedness is to tell them up front what you're going to offer. With that in mind, I like the idea of the "coffee break" where you're allowed to wander off.
Of course, you can also just leave the room while they're working on it.
Hijacking this thread to say that blog post isn't about Microsoft as a whole, but only about Microsoft's Developer Division (Visual Studio, etc.). The rest of Microsoft still uses the standard whiteboard/Leetcodey approach.
My goal when interviewing candidates is to ellicite this response at least once. I want someone who is honest that they don't know the answer and won't bullshit me about it. Also like to hear how the candidate would go about figuring it out, e.g. what resources they would use. I'm also fine with speculative answers as long as they're clearly stated as such, "I dont know, but I think it'd be something along these lines..."
Yeah, in an interview, I would certainly expand on that succinct motto; and attempt to do exactly this. But, as others have suggested, people respond too much instead of just thinking about a problem/issue/comment/etc. from all the angles (this skill was natural for me; but honed during the Ph.D. process.)
That same nugget was in The Pragmatic Programmer which I recently have been reading. In regards to estimates or questions about things you don't know, if you can't give a good answer on the spot the second best answer is "Let me get back to you.". The bad answer or estimate will just cause pain for your team/org.
I’ve never done any leetCode style interviewing or studying, but from what I have heard, it’s all about “pattern matching” and knowing how to solve similar problems.
I have done plenty of architecting. A large percentage of architectural problems that you are asked to solve have already been solved. After enough years at enough different companies and reading enough architectural books, it’s just pattern matching and figuring out the correct tools.
This is actually something I've encountered outside of interviews in day-to-day work with teams.
When the topic is something where I know some underlying complexity exists, or the stakes are high, I often want to spend a bit of time thinking it through by myself because that's how I think best about those sorts of problems.
I've encountered friction at times with those that aren't familiar with and/or care about the underlying complexities and just want to talk high-level and make decisions and move on. Sometimes it is important to make quick decisions to avoid analysis paralysis, but often it is worth giving those who do best thinking through things on their own a chunk of time to go off and do what they do best. I wish more people took that into consideration when conducting brainstorms.
I'm not the best with Python, but if someone gave me that answer I'd feel like I avoided a land mine. Dicts, and analogous data types, are incredibly useful.
Yes! I avoided the landmine. I can't imagine how I'd have felt working with a woman who interviews for Python role and doesn't know or understamd dictionaries!!
Plus they were giving me (x%) a raise, and I got (x-5)% in my current company. So thank God I didn't join there. Contacts are more important than salary.
The most charitable explanation I can think of is that they have a very narrow idea of "practical application" where the performance of looking stuff up in hash maps all the time is not good enough. If that's the case, they should have been more specific, and asked for specific improvements. Otherwise, they're full of shit.
Exactly! I was sitting there wonderinf if they understood doctionaries at all. My guess is that they came from Java background and thus don't use dicts as default as HashMaps aren't native basic data types
This is fine in most situations, but sometimes companies want people who actually have experienced in a given problem domain, and generally speaking, if you have to "sit with a cup of coffee and some data and try to think things through" that implies a lack of experience, which may not be desirable to the interviewer.
I'm one of a couple domain experts on my team. I am well respected as such. The reason I "sit ... coffee.. think things through" is because I am doing a mental impact analysis. How are the suggestions going to affect the system as a whole. One simply cannot know these things off the top of their head in a meeting for a system of any meaningful size.
It is precisely because of my experience that my team is OK with me taking an hour to ponder.
Edit: Let me clarify one thing. I'm not talking about situations where, say, the PO comes and asks, "Hey, JustSomeNobody, why does the system do this under these conditions?" I know the answer to this. If they want logs for that specific instance, I can get them. What I'm talking about is, when the PO calls a meeting and says, "I just got off the phone with <Entity>, they want these changes to the system. What will it take?" I don't answer these questions, definitively, without thinking about them. That does NOBODY any good. I'll sometimes give a few initial thoughts for the sake of the meeting, but that's it.
i’m like this but admittedly, in some cases, you’re interviewing for roles where it’s critical you identify that candidate can think on their feet (eg devs that might need to serve in 24/7 ops roles), where “let me get back to you” isn’t a feasible option
Usually those who optimize for thinking on their feet do so at great compromise in quality. Maybe you can afford that, but it's kind of "shooting from the hip" so to speak.
It's probably not that these candidates can't think on their feet, it's just that they generally aren't willing to compromise so greatly on accuracy and quality of the answer.
That's reactive thinking and it honestly sucks.
While you might encounter unexpected situations from time to time, a good ops person will be preventing problems a lot more than he firefights. If you only hire a good firefighter, he will do the only thing he knows ad infinitum.
I've worked with people that were reactive and would love to be "the fixer" when, in reality, 99% of the issues he fixed were caused by his ignorance/incompetence in the first place.
Specially for crucial 24/7 roles, you need people that think ahead and not reactive ones.
in a situation like that i'd like to see both happening. start the emergency procedure, and while it's running think through if you missed anything or if there is an alternate approach.
just had that happen. we had an overloaded server, and while we tried to figure out how to make the server faster, after some thinking we realized that the better long term solution was to change some of the architecture so that the server would get less load to begin with.
Great, now can Netflix, Google and Facebook do this, too? Not because I want to work in these places, but because they influence everyone else and as a senior engineer in the systems space I feel I shouldn't need to study days or weeks for fizzbuzz sorting algorithms questions that are designed to test comp sci recent grads. I have a proven career, and was never suddenly stumped in a project due to not being able to.. sort letters in some obscure order based on arbitrary rules. How about we look at that, and compare the candidate's past experience to the current needs and interview based on that -- You DID look at the CV before bringing the candidate in, didn't you?
I've interviewed and been interviewing for over a decade. I have had the displeasure of interviewing many people who have very impressive resumes yet when asked how they would judge their skillset in an area, and being told from them that they would say "expert", being shown exactly the opposite when it came time to answer some questions in that area.
One data point... Senior engineer that founded the local java user group and wrote a published book on the java language. Couldn't code a simple reverse string algorithm, nor explain whether, after with help devising an answer, the method was thread-safe.
Another data point... A self-described expert in SQL with it all over their resume. Couldn't write a simple join, inner outer or other.
Weed-out questions exist because our industry is flooded with people who don't know, nor care to know, their craft. And there are plenty of employers who continue to hire these people, and promote them. I interviewed numerous "architects" (with development experience all over their resume) who couldn't come up with a naive solution, let alone the optimal, to a very simple problem. Want to fix the interview process in our industry? Figure out a way to weed out the liars? I'm all for it. I would rather not have to do basic algorithm questions, but that's the current environment.
Totally reasonable response. However, it seems like we're on an endless loop with this discussion. People talk about how they need technical interviews because someone who looks good on paper can't code reverse string, or write a basic join. But here's the thing, absolutely none of my technical interviews have involved anything remotely this simple. They are "find all matching matching subtrees in a binary search tree", or "find all possible matrices with a negative determinant within an NxM matrix of arbitrary size", and it's supposed to be done in about 45 minutes, at the whiteboard.
Really, it's not fizz buzz or string reversal that causes people to re-study for their algorithms and data structures exam before an interview.
In many ways, I think this is similar to requiring that senior actuaries re-study integration by parts prior to every interview. They don't have to, because they have a proper exam that is widely accepted in their field. We don't. So instead, we are taken through full day whiteboard exams, under conditions of great secrecy, over and over, every time we interview.
This is an excellent analogy. I've found that ability to complete esoteric algorithms problems has little relation to productivity in specialized subjects, as in network security.
Did anyone stop to think that maybe, just maybe, in the allotted time there simply isn't enough bandwidth to accurately answer the question of whether the candidate can do the job and will be a good fit and that no matter what strategy you try that perhaps this is an optimization problem for which there is no good solution.
Like this is it. As good as it gets.
I actually think all this back and forth, chopping and changing, people arbitrarily weeding companies out, companies arbitrarily weeding people out is providing just the right amount of randomness for everyone to wind up somewhere.
I have very little experience with interviewing candidates, but how can you be certain these people were liars? Maybe they simply were having a bad day, or got too much in their head with stress during the interview. There are many factors at play when it comes to a bad performance and it's not always simply that the engineer doesn't know the answer.
I give them a chance to judge their own skillset. After looking at their resume, I have an idea based on the projects they take credit for, what their experience level should be. If what they state lines up with their resume, and they are unable to answer a simple question, with multiple hints and help, then I absolutely assume they are a liar (edit: or delusional, which is even worse. An expectation I have of seniors and above is to have some idea of knowing what they don't know).
It would be unreasonable for me to pass them through an interview that the other members on the team passed through based on "oh, they had a bad day".
Since you haven't been on the other side of the table, have you had coworkers who obviously were at a position above their skill level? Who allowed the rest of their team to carry them?
I've had periods wherey personal life has affected my productivity. That doesn't preclude my ability to write an algorithm that reverses a string nor write a simple SQL join. I don't ask brain teasers. I ask questions related to what the engineers under me will be expected to solve every day.
I mean, people shouldn't be reversing strings by hand, but being unable to do so is also very worrying. Not because manual string reversal is an everyday problem, but because you need to do similarly complicated things all the time. In fact, you need to do them even if you're not a dev, it's just that devs share a common language that makes string reversal a sensible problem to ask in an interview.
It isn't that hard to isolate a block of code that addresses a small cross section of your business rules and ask a candidate to do something meaningful with it.
That could be an excellent second, more involved, question. The first, super simple 2-minute sanity check question is to reverse a string. It's literally just a reverse for loop.
Then the complaint will be "they gave me a question about a really specific and hard to understand part of their code. It would have been easy to solve if I had familiarity with the concepts involved. Why didn't they just ask a similarly complicated question about strings instead?"
That's not really actionable. Even if my skills aren't up to pytester's standards, we still need to recruit developers now. And for what it's worth I doubt I'll ever be able to explain most of the tasks our business is doing what you refer to as "properly".
I joined a team two weeks ago that built a fairly complex machine learning pipeline. One week in I was told I needed to help interview a candidate.
I lopped off a block of the code, made it run without the rest of the code base and explained what it was supposed to do. This wasn't that simple, but could still be explained in about a minute to somebody with familiarity in ML (assumed; that's what we're hiring for). I then challenged the candidate to spot any bugs and implement a feature.
This really wasn't that hard. I've done the same thing twice before in two completely different industries.
Perhaps I just don't understand what you're proposing. If it's general enough to be presentable without any domain specifics, then it seems indistinguishable from a general algorithm like reversing a string. I don't use tasks quite as simple as string reversal but not far from it.
As I read it the main observable difference between your exercise and the kind of exercise you're arguing against is that you are starting with some baseline code that the candidate is supposed to modify. That seems like a pretty good idea, but I haven't used it yet.
Since you're attacking me, do you have any experience screening resumes? Phone screens? In-person interviews? Have you had the experience of a candidate that has apparently "done it all" on their resume, and confirmed their skill level when asked in-person, only to see them struggle with the simplest question in an area they deem themself an "expert" in?
I've interviewed well over 300 people, for positions from junior/entry-level to principal. If you read many of the comments to this article you'll hear other people with experience interviewing who also call out that candidates can and do lie. I don't like it, I'd rather it not be the case. I constrain the majority of my questioning to what a candidate claims to know on their resume. Here's a tip... If you don't know it (and I'm not talking about bullshit trivia questions), then don't put it on your resume.
Yup... and I don't know, for someone who has "300 people" worth of interview experience, you sure don't seem to be demonstrating in this thread the sort of level headed temperance and soft skills I would expect.
I'm joking. (sorta)
I'm glad to hear that you tailor to the resume, that's actually better than most and my favorite approach.
But if I had a dollar for every time someone walked out of an interview with a braggadocious claim like "they couldn't even do X, they must be lying" but then answer "no" to "did you ask them anything else?". I'd be rich.
Reversing a string is a bad example, it is dead easy. But it seems a lot of people have random substitutions for it of varying obscurity and difficulty.
My interview covers coding, data structures, algorithms, system design and data modeling. I attempt to tailor to the candidate, but there needs to be some level of standardization to be able to justify to the recruiters why a candidate didn't proceed.
Just because a candidate fails or falls short in one area doesn't mean they are cut, but it's a pretty large red flag if a candidate purports to know something, both on their resume and in person, yet can't answer some seemingly basic questions. If you state on your resume that you architected and implemented a system end to end, yet can't whiteboard the result and explain bottlenecks or failure conditions, I absolutely will assume you stretched the truth a bit. And if the candidate feels the need to do that to get the job, what will it be like to work with them? Could you trust their estimates? Would you be able to back them if something they delivered fails and you trusted them as a senior to deliver?
I ask follow-ups. I give hints. Honestly I'm dismayed at the skillset of purported "seniors" and "architects", and that's interviewing for small companies up to one of the FAANGs.
I've never counted but I must have done way more than 300 interviews. I have interviewed at 3/5 of the FAANG.
Coding:
on a whiteboard or with someone watching over you has no correlation with performance on the job.
Data structures:
questions usually end up so contrite that they have no bearing to real-world situations. That or based on luck of seeing an remembering the right one. Data structures usually go hand in hand with algorithms.
Algorithms:
questions become more like trivia or reinventing something that has likely got research papers on it. Oh, you want me to invent an algorithm I've never heard of on a whiteboard with your clues?
System design and data modelling:
the kind of thing you can do in an interview lacks real-world data. No empiricism. It's all about the trade-offs.
Tailoring to the candidate introduces inconsistency and unfairness/luck.
What if these "seniors" and "architects" are frauds?! How else could they hold down a job for 20 years and not be able to answer my questions? They must be really bad!
Lucky my interview process is really great and I always hire the good ones. I track every rejection and I have proof that they never amount to anything!
Do you allow candidates to use their normal workflow (which is dependent on access to Google, github and q&a sites) or are we putting people in an idealized workflow where the dev has to know everything? I think many developers and interviewers think using Google and StackOverflow is a sign of weakness. It may or may not be, but it is normal in many developers workflows.
I tell the candidates that I will happily be their Google/stack overflow. Some candidates ask questions along that line, some don't.
I agree that we don't code in a vacuum. What I'm trying to ascertain in the interview is how someone thinks, how they approach a problem and work through the solution. The more they communicate the better, even if part of that communication is "is there a length property on the String class?"
I note, but don't give it too much weight, if there are minor syntax errors. I ask candidates to walk through their code with an example input, especially the ones that have errors. If they catch their errors and fix them, I note that. If they skip over their errors because they assume the code does what they wanted it to, I note that.
Ultimately, can you reason through the problem, come up with a solution, and talk about the tradeoffs you made in your implementation. Any senior should be able to handle that without any preparation.
I worked at a FAANG where in several loop debriefs I was challenged on the complexity of my question. One of the challengers asks a chutes and ladders question (given a random chutes and ladders board, where you can choose between 1-6 for every move, what is the minimum number of moves to get to the end).
It doesn't take a hard technical question to figure out whether someone knows how to think and work through problems.
If you cant quickly write a function that reverses a string you either don't know the language at all or you can't solve simple problems. So either a liar or a completely incompetent engineer is a safe bet.
Back when I worked in an office and conducted interviews, I made a point of always having the candidate sit in front of a computer and solve some very simple problems. It's amazing the number of candidates with apparently strong CVs who simply did not know how to use a computer. I mean, people claiming Linux experience who didn't know how to type commands at the prompt, not even "ls". How were they expecting to get a Linux developer role?? Unfortunately you have to weed these people out, and this was the only way I found.
As a hiring manager, I agree 100%. These people are everywhere. I've also given someone the benefit of the doubt -- he wrote a Java book! once we hire him, I'm sure it will work out -- only to find, nope, he can't code on the job either.
Other hand, I'd suggest instead of "lying" that people are just myopic about their skill sets. I'd been writing JavaScript code for years, for example, but on an interview learned about a whole world of "modern" JavaScript that I didn't know existed. Clearly they thought I was a fraud. But I'd just been living on the 3rd floor without ever visiting the basement where all the pipes and boilers were.
It's the start of the coding interview. It moves on from there. This candidate didn't even ask if he could use Apache commons, and I hadn't yet specified that he couldn't use a 3rd party library. So yes. He failed on many counts.
1) not asking clarifying questions
2) not being familiar with some well-known libraries
3) not being able to implement a basic reverse string
4) not being able to explain whether or not a simple method was thread safe (with multi-threading experience all over his resume)
I expect engineers, senior or otherwise, to be able to write code. If you can't do that, don't put it on your resume.
If you honestly think implementing a reverse string method is "too complicated"...
Edit: sorry, leaving the response as-is, but I recognize you didn't state "too complicated". You stated "focused on the wrong skills". I posit that an engineer that can't reverse a string, as a litmus test, will be likely unable to solve a more complicated problem.
In an interview, there is an unspoken assumption that you won't be using third party libraries to implement algorithms, and certainly not a third party library that implements it in one function call. Using that as a red flag will yield a LOT of false positives.
Also, I have over 10 years experience in Java, have used Apache commons extensively, and had no idea that a string reversal method even existed. You can't use that knowledge to judge anything useful.
I agree about your unspoken assumption, but many interviewees ask. And the post I was responding to pointed to the Apache commons library, so I hazard to say it's not universally known.
A fail is not a red flag. A senior candidate not asking clarifying questions does not disqualify them (what I would call a red flag) but it is brought up in the debrief as a potential gap. I don't expect them to state this could be implemented using a single call to Apache commons. I call those candidates out in the debrief as being aware of the library, as a bonus. But not saying it isn't a negative.
If you're interviewing for some core, low level algorithm development then fair enough but I suspect if that's the case you could come up with something a little closer to the real life domain.
It's not that I think the question is too complicated, just not the best marker.
In my mind, more relevant questions are ones that demonstrate real life experience in the field. For example, populating a tree of objects in a high level language like Java from a DB where the tree of objects are stored as many to many relationships. You'd be surprised how many times I've run across production code which does this via SQL calls inside nested loops.
This whole infatuation with algorithm questions like you'd find in SICP might be good for new grads but misses the mark for Senior level engineers.
I asked a candidate to implement an algorithm that reverses a string. I don't see that as a complicated problem. It's literally a for loop, and based on how they choose to implement it you can ask about implications of append vs. prepend, etc.
I didn't ask them to implement dijkstra's algorithm, or even something as "hard" as breadth-first or depth-first search (which I feel any senior that deals with trees should be able to do).
It's a marker of this... Can the candidate solve a basic problem? Once they solve it, can they explain their solution? Do they understand the memory/time tradeoffs they chose? Do they know whether the code they wrote is thread-safe? If you don't feel a senior engineer should be able to answer those questions, to a very basic problem, how do you expect them to tackle something more complex?
Did you know in .Net strings are immutable? Neither did the guy at Google when he asked me to do some "in place" string fiddling algorithm or that guy at Amazon who wanted to reverse the strings in a sentence (yes there's a trick to that one found in some 80s programming book)
What I'm interested in from a senior engineer is how many systems they've designed, implemented, deployed to production and supported in their careers and what their specific roles were in those projects. If I can't get an idea from their resume I wouldn't contact them for an interview.
As for the interview itself, well I've never needed to ask someone to reverse a string to determine if they're going to be of value with the criteria mentioned above.
But to be fair I've never worked on a project where implementing StringUtils.reverse() was required. If I were interviewing someone to work on Apache Commons or the JDK then I suppose that would be a relevant question.
Oh come on, argue the strongest part of their argument, would you, instead of nitpicking over string reverse. They've repeatedly, laboriously, explained that reversing a string isn't the relevant part to be focusing on.
What I'm interested in from a senior engineer is how many systems they've designed, implemented, deployed to production and supported in their careers and what their specific roles were in those projects. If I can't get an idea from their resume I wouldn't contact them for an interview.
And when their resume looks good and claims they are an expert, and you call them in and talk to them and they say they are an expert, and you start by opening a warmup question on the most basic levels of their expertise and they totally flunk it on multiple levels - unable to even handle the situation with tact or diplomacy in any way, are you still going to trust what they say about the systems they've "deployed and supported"?
I understand the class of rebuttal you're invoking here, but reversing a string is extremely easy - with or without a library. This is not an academic problem designed to see if a person can study prior to an interview. This is a first pass question designed to see if a person is even acquainted with basic programming. Any such question you could devise would be similarly patronizing or orthogonal to the exact work done on the job.
And yet, despite how easy the question is people still fail it. It's frankly absurd to me that a senior software engineer can fail this question. It's practically a freebie slam dunk for anyone with basic competency. Briefly drop that you can use your language's core function or method to do it. Then declare a new array and iterate through the string backwards, appending each character to the new array. When you're done, collapse the array to a string.
In other words, it's testing if you can write a for loop. How many times in your job do you expect to implement a for loop? I use for loops quite often.
That would be an excellent question to ask during an interview.
If you were interviewing with me, I'd ask you to do the (usually expected) ASCII/array-of-graphemes version first. If you did that, you'd pass, and get a positive shout-out in the feedback session for asking about multibyte chars.
If you, after solving the array-based form, had some ideas or even working code for how to handle multibyte characters directly, without falling back to the standard library of $language's idea of "what is an iterable character-equivalent unit in a string", and demonstrated similar insight in your other interview problems, I'd give a positive shout-out with the addendum that we might be interviewing you for the wrong (too junior) position.
In my experience, most people don't ask about multibyte chars. That's fine, if they solve the problem. It's not a positive or negative. If an interviewer is building questions with hidden "must-ask" gotchas like that, I think they're doing themselves and their candidates a disservice.
Something interesting does happen occasionally when candidates do ask about byte-width (or equivalent less-common but still very important gotchas like pointer/word size, endianness, etc. in relevant problems): some people ask the question, solve the naïve form of the problem, and then offer some thoughts as to how they'd handle the broader multibyte case. But some other people act as if knowing that gotcha means they won't or shouldn't have to continue solving the problem. I've had candidates tell me that they "figured it out" and wanted to go on to the next challenge at that point, without writing a line of code. I've had candidates tell me flat out that writing non-Unicode aware string processing algorithms was unrealistic and a waste of time, and that they wanted a problem that was more real-world or suited to their level of skill. Those things will also get you a shout-out in the interview review session, but not the good kind.
>>Then declare a new array and iterate through the string backwards, appending each character to the new array. When you're done, collapse the array to a string.
a. You can't iterate a 'String' backwards. You need to know the length first. And that requires traversing the string forwards at least once.
b. Once you reach the end you are wasting the effort of revisiting the elements, since you visited them already once.
c. A better option is build a build a double linked list, as you are traversing the list forward.
d. Once you have the double linked list ready, you have pointers to both start and end, and you traverse the way you like.
e. An even better way would be to design a data structure that contains all this meta data at String creation itself.
Class CustomString {
char* headPointer;
char* tailPointer;
int length;
bool isPalindrome;
char* string;
/* Add your interview acrobatic things here */
}
Stuff like this.
Now even though the answer in your comment is correct, they could reject you for not coming up with the answers I gave from point a through e above.
Now imagine some one telling you that, on these grounds, you don't know a semicolon worth programming and are probably a liar.
I hope you understand what's going on here. Every body can be rejected if they don't cut through your cookie cutter.
> a. You can't iterate a 'String' backwards. You need to know the length first. And that requires traversing the string forwards at least once.
That's not at all true in many languages. If a candidate asked "is this string's length known without an O(N) operation on the number of characters in it?" (or knew the answer for the default string constructs in the language they were using for the problem) I'd consider that a positive indicator. If they said what you just said, without consideration for the context, I'd consider that a negative.
This is not a case of someone expecting a given "cookie cutter" solution. Interviewers who do that are doing themselves and their candidates a disservice. This is a case where statements like "I know how strings work because I know how they work in $context" where $context is not what the interview expects you to work in will make a bad impression.
I mean, in languages like Python it can be as simple as `foo[::-1]` or some similar method. Anyone who can give a naive answer like that or
usize len = strlen(foo);
char* bar = malloc(len);
for (int i = 0; i < len; i++) {
bar[i] = foo[len - (i + 1)];
}
and explain why that won't work for unicode strings, I imagine, would pass that particular fizzbuzz test. Bonus points for pointing out the null byte off by one error.
The only competent programmers I can think of that wouldn't be able to come up with that off the top off my head work in embedded or FPGAs where strings are rarely relevant.
Can you traverse an array? Do you know that a string is an array? Do you know how the end or length of a string is represented in your language of choise? Do you know how each character is represented? Do you understand Unicode? Do you understand memory allocation? Is the string being reversed in place or you allocating a new string? What are the space and performance trade-offs between those approaches? I mean, you can fill a whole interview with just this topic. So yes if your answer is string.reverse() then expect some follow-up questions.
So, I agree you want to see evidence of what's on the resume. But why isn't an actual work project of some sort better than whiteboard coding exercises?
Why do you assume that the person is lying? Why do you start your relationship with your new co-worker on distrust?
There is a way to weed out the liars - you hire them and then when it does not work out you fire them. You would be astonished how little people lie on their CV.
They may exaggerate claims - yes, that is why you gave them a call in the first place. But then again you exaggerated your claims in the job posting of which skills are required for the job.
The one thing that I noticed is that only the US companies have this awkward parallel reality 'technical-interview' hiring process. In other countries like Germany or France the interview process is mostly focused on the person and the desire to work for the company and the willingness to learn the skills needed for the job. Just because you fail to answer immediately to a random question it does not mean you are unqualified for the job.
When I conduct an interview I try to only ask open questions like:
* What is your favorite IDE? - Whatever the person answers, tells a lot about how he works, what tools he uses, and if he is aware of the common tools
* What feature of the new Version of <Programming Language> do you like the most? - A) you may learn about some weird feature, b) you can discuss how that feature will improve code
* Do you prefer JavaScript or TypeScript (dynamic vs typed language)?
There is no right answer. It is all about how the candidate argues his answer. You learn more about the person and if you can deal with him on a day to day basis (and if he has the brains to think about things).
> There is a way to weed out the liars - you hire them and then when it does not work out you fire them.
In my experience, firing someone takes multiple months. Months in which they are mutating the codebase (with review, but it's a time sink that shouldn't exist for a decent senior), influencing other engineers, etc.
Passing on a good candidate is cheaper in the long run than hiring a bad candidate. Or said another way, false negatives are preferred over false positives.
> You would be astonished how little people lie on their CV.
This has not been my experience. From something as white-lieish as "I designed and implemented" vs. "I was a peer on the team that did it" to flat out fabrications. You learn alot by simply asking "what was your role on the project" and asking follow-ups to dig into their contributions.
There was a blog post "Adventure in the Low Status of Software Engineers" by Michael Church (apparently taken down now), basically if you're interviewing for "low status" positions you're assumed to be a fraud but for "high status" positions (VP-level) you're assumed to be a great candidate even if your resumes are almost identical in both cases.
I think you're ignoring that hiring someone onto your team takes a long time and significant effort. Say you're allowed to add one person to your team. You interview 50 people and decide on the best candidate. Then you spend the next 6-months waiting on lawyers, processing visas, moving their them and their family to the US, paying for their initial housing, etc. Might not be an issue for a small company in France that hires locals but it sure is for big companies in the US.
> You would be astonished how little people lie on their CV.
That is no doubt your experience, but I assure you it is not universal. I have heard, and in some cases personally seen, some truly shocking things!
> In other countries like Germany or France...
Perhaps the gold digging nature of Silicon Valley entices far more scammers to cook up a SWE resume and roll the dice, but I think it's a very different hiring world here.
Fizzbuzz is a trivial test to detect people who are basically fraudulent in their claims of programming ability. You shouldn’t need to study for it. Sorting algorithms, sure.
I remember a colleague who was affronted at being asked to do Fizzbuzz. I told him it was just a simple screen for fraudsters and bullshit artists and not to take it personally, they didn't know him from Adam, after all.
I think, because he didn't understand the purpose, he saw it as poor company interviewing to make him jump though that sort of crap. Once he understood it was just a simple screening mechanism because of the high number of dodgy applicants, he calmed down.
Yup, my experience is that FizzBuzz level complexity is roughly what you want for interview problems. I find that and some questions about algorithms and data structures tell you all you need to know.
Fizzbuzz is not really about testing someones understanding of a modulus operator, it would be very easy to make an alternate fizzbuzz that does not require using modulus, and arguably anyone hiring for a large company should have their own as fizzbuzz is pretty widely known.
The point of fizzbuzz though is to make someone create a very simple algorithm on the spot. In my experience, there are some people in this world who simply lack the ability to structure their thoughts and plans into a complete set of ordered instructions. That's why you sometimes find recipes online that follow a structure like:
1. Dice the onions
2. Brown the mince for 5 minutes
3. But first, add the onions and tomato paste to the mince
It's the same personality type that cleans a room by walking backwards and forwards to the cupboard of cleaning supplies as they need them rather than grabbing all the cleaning supplies they can predict needing first.
Now, everyone does these kinds of things to a certain extent, but you can't program anything significant if you can't drop into an ordered mindset on command. Fizzbuzz tests your ability to do that without requiring you to know any apis or frameworks or computer science or even a programming language, since you can just as easily do it in psudocode.
Modulus is a rabbit hole - the mod operator has different behaviour for negative parameters depending on the language (and presumably other language specific quirks?)
Here's another one: when does the abs operator return a negative number?
Why make it a trivia game? Explaining the basics of how modulus SHOULD work is a fairly generic concept that is trivial for a lot of coders to describe, but impossible for non-coders. It's an ice-breaker.
Hmm, everytime I use modulus I need to double check whether it returns the number of times wholly divisible or the remainder.
I'm thinking the former without checking now.
Meanwhile, my last project has been to set up an entirely self hosted CI/CD pipeline using digital ocean, docker, jenkins, gitea, and docker registry to make changes to my websites I build with erlang, react, html and sass, so I _think_ I can create projects! (No guarantees).
The interview process is tough, and what do you want to root out more false negatives or false positives? It may depend on the size of the company. Google can probably shell out a few bucks to some putz who sounds impressive but isn't in the hopes they'll nab a few more great engineers at the loss of a few paychecks.
An early phase startup may not have that luxury.
Edit: must be tired since just thinking about fizzbuzz for two seconds clearly gives me the answer to what modulus returns ;-)
> Google can probably shell out a few bucks to some putz who sounds impressive but isn't in the hopes they'll nab a few more great engineers at the loss of a few paychecks.
They actually take the opposite approach. Supposedly there is compelling research that low-performing members of a team provide large negative contribution to team performance from knock-on effects of their low standards and low quality work. Google has consequentially structured their interview process to minimize false positives rather than false negatives so as to minimize firings and the associated psychological stress caused by working in an environment in which people are regularly fired.
I interviewed at Facebook and the tech part was completely sane and appropriate, the technical task was a bit fizzbuzzy but not something you'd need to spend nights digging into obscure algorithms book, but rather simple task which required some thinking, but not too much of it. No experience with Google or Netflix.
As of looking at CV, I've seen people with great CVs that proved utterly useless and of course the opposite too. CV can be very misleading, both unintentionally and intentionally. You need to speak with the person.
The issue with that is that if my CV is incredibly specialized in one area that happens to match the job I’m going for (eg compilers), why test my knowledge in designing things like a distributed system and in how great I do dynamic programming (that is not useful in my area)??? It feels like there is no verification at all of what my CV claims, nor does it give me a chance to show off the skills I’ve been working on for them last 20 years, both of which are very relevant to the job. Instead, I’m tested on what I knew when I graduated with my BS, and that’s it; it’s like my PhD was a waste.
In larger corporations it may not be 100% finalized where you would end up.
Knowing that yes, you master compilers (that's usually quickly determined, potentially fast enough for you not to notice it since you live and breath compilers), have basic knowledge in smashing together enough html tags to build a simple dashboard but you never had to deal with processes that spanned multiple computers, allows more flexible allocation:
Ideally, they'll give you a compiler related task, but if they find someone even more suited for that one open position in compiler internals, they might offer you a job in the adjacent compiler-as-a-service team where you'd start out doing dashboards and then move to whatever more complex task needs work.
But they won't need to bother offering you the distributed systems position that scales up the compiler-as-a-service product across the whole world without sending you to some training first.
I’m not going to take the job if I don’t wind up in a group working on problems that interest me. That is part of the bidirectional interview right? I hesitated even applying to a big Corp until a few people there convinced me they had problems right up my ally. I don’t want to work at big corp just to be working at big corp!
The hiring pipeline at every big company I've been at is not always the same. Yeah, part of it is always just a big funnel that takes general applicants in and assumes they might end up anywhere in the company. But, there is also going to be a way for specialized teams to hire specialized candidates. If you find yourself in the big funnel, by all means, just tell them you aren't interested if that's not what you want. If you want to work on compilers, you need to find a way to get into the pipeline for a compiler team, and that's not really much different than finding a job with a small company.
Yes, I already did this with Amazon who hires for specific roles, but the recruiters I’ve dealt with seem to be into bait and switch (substitute very appropriate role I applied for with something completely inappropriate).
The job opportunities are definitely out there, but many of them really require a referral (the JDs aren’t posted), and even then you might have to do a generalist interview because the company is not capable of doing anything else.
Because at big corps engineers might get assigned interviews as a routine work. And any engineer has their favourite questions that they know the answer to. They dont want to take.the extra effort and come up with new questions just to adapt to your profile or expertise. It goes the other way around. I.e.the interviewer is not going to do.extra work to validate the candidate but the candidate has to prove him/herself. Take it or leave it.
The one size fits all interview at a big corp is game-able at least, with plenty of practice material available online. It isn’t a huge hurdle if you have a month or two to practice, but that makes the whole process even more questionable.
Meanwhile some of us are just happy if candidates know of the existence of regexes, even if they forgot the exact usage, and can mention using them rather than trying to write a parser from scratch when given a problem statement of extracting phone numbers of various formats from several thousand unstructured files of web page scrapes...
How partially? I could probably do a very simple one, without backtracking (though they'd need a big whiteboard if they want full code there) but if it's something like PCRE then I don't think Facebook headquarters have enough whiteboards to fit the code :)
Though I just realized "regex parser" can mean either "parsing some text using regex" or "code that compiles regex syntax into some kind of data structure but does not actually do anything else". The latter obviously is a much easier task.
The question was: Write a method that can take in a regex and a string and return whether the string matches it, with the regex limited to the characters . * ?
(And backtracking is definitely where I started coming unstuck)
If the regex is really just a sequence of dots followed by a question mark, asterisk or nothing, thus cannot contain literals, the only thing you need to check for is the length of the string.
The minimum length will be the number of dots without a quantifier (i.e. exactly one character). If there is any dot followed by an asterisk, there is no maximum length. Otherwise, the maximum length will be the minimum length plus the number of dots followed by a question mark.
Writing the code for this is left as an exercise to the reader. ;)
Ah, that's definitely different than I understood your first post. You're writing a regex to match a given type of string. The regex parser is in the language itself, consuming what you've written and evaluating it.
It's neither. There's a DP solution for this problem and a recursive solution without memoization (with exponential complexity). The shortest solution I have seen was 18 lines of code.
Please show me a regex parser in 18 lines of code! Even 36! I did this exercise once and it took 100s of lines, I think using C++... but it's been a long time. Maybe it's significantly shorter in Python? But still 18 lines is impressive.
I probably screwed something up, but here's a 16 line attempt in Python, where the regex specials are limited to .?:
def matches(pattern, s):
if len(pattern) == 0:
return True
c = pattern[0]
assert c not in ("*", "?")
if len(pattern) > 1:
if pattern[1] == "?":
return (
len(s) > 0 and c in (".", s[0]) and matches(pattern[2:], s[1:])
) or matches(pattern[2:], s)
elif pattern[1] == "*":
for i, d in enumerate(s):
if c not in (".", d):
break
return any(matches(pattern[2:], s[j:]) for j in range(i, -1, -1))
return len(s) > 0 and c in (".", s[0])
Pretty much something along these lines, except it's also required that both pattern and s end at the same time for a full match (this complicates the base case a tiny bit). The C/C++ solution looks more elegant because you can use pointers. Note that this particular version requires no prior knowledge of the algorithm and can be solved on the spot fairly easily.
I had a junior/intermediate level engineer start asking me minutiae about HTTP and HTTPS...
After about 5 minutes of this I literally said:
"Dude, have you seem my resume? I literally built a petabyte scale full text search engine and wrote 1.5M lines of code to do so and a HTTP framework that parses fetches more than 2PB of HTML per month. If there's some edge case that I might miss I can Google it in 30 seconds".
I no longer keep small tactical issues in my head. It's pointless. Now I try to understand the system as a whole.
It could be _because_ they read your resume that they asked those questions, to see if they believe it. People heavily embellish their degree of involvement in projects all the time.
Or they could just be asking gotcha trivia. It’s hard to tell.
A lot of people will write whatever makes them sound good on their resumes, so we have to ask questions about minutiae to make sure candidates aren’t lying.
It sucks, but this is what happens when the job you’re interviewing for has no license or certification requirements (but pays six figures).
1.5M lines is the output of a team of hundreds of developers over years. I don't believe anyone would write a million lines during a job.
2PB a month, is 125B documents at 16kB each, or 50k documents per second. That's decent. That's where it starts to be interesting in elasticsearch or hadoop.
This is a partially valid point... let me restate.
About 200K lines of code is auto-generated.
A more appropriate estimation would probably be about 400k lines per year as some of this code did a big mass migration and refactoring during this time but I wrote that code as well. Just in the past.
KLOC is a shitty metric anyway.
I'd rather write 100 KLOC that worked vs 10000KLOC that was horrible.
Nothing to take away from the original posters work. He indeed might have written that much code.
But in most cases. Its not very difficult to write 1.5 million lines of code for any project if you are writing code of the style AbstractClassFactoryFactorySingletonDispatcherFacadeInitializer
That sort of the code is basically 90% auto generated by the IDE. You write the remaining 10%.
As a matter of fact I would find it a tad little hard to believe that some one wrote that much(1+ million LOC) and didn't find a way to template it or do some metaprogramming work.
The IDE can generate "headers" for a new file, it's not gonna generate tens of thousands of new files by itself, let alone in a meaningful structure.
I don't think a developer would write a million line at work in their entire lifetime.
I'm not saying it's impossible to achieve. I've seen outsourcing agencies deliver 100k lines with just a few block copy/pasted many many times, but that's neither working nor maintainable software.
I guess people who allowed hacks like adding -1 fridge to get a free tv in earlier online stores could also claim "but they have build online stores that moved thousands of sales every minute"
> You DID look at the CV before bringing the candidate in, didn’t you?
I would like to speak to this after interviewing at a few places recently, including Google. I don’t think a single interviewer looked at my resume. I’ve developed a sizable open source project that is used by real people for over a decade. If the company really wanted to know if I could write code they could just go and look at it. Instead they ask puzzle questions that I have no interest in and end up failing. These companies expect the candidate to spend weeks preparing for their hoop jumping, but can’t be bothered to actually read the CV. Multiple people didn’t seem to realize where I lived even when my address was at the top of the paper.
Netflix already does this. Just interviewed with them for a full stack role. All interviews are dependent on the team, but generally feature less whiteboarding than FB/Google. I did a takehome technical exercise, in person interviews were tailored to my experience. They were mostly about system design, general concepts, and a heavy heavy focus on interpersonal skills. Their approach seems to be:
1. Make sure you actually know what’s on your resume
2. Make sure you have the strategic thinking and interpersonal skills to own the system you’ll be working on
3. Make sure you’re a good match for their fairly unique work culture
It’s not a “take home test”. Half the purpose of live pair coding is to see how the candidate thinks.
It’s a relatively simple project with a skeleton of a class and failing unit tests. They have to fix the code to make the unit tests pass.
If they get through that, I give them a second set of failing unit tests that they have to make pass by fixing the class without breaking the unit tests.
Ah, I also thought you meant a take home test. Do you work for a big company? I’m trying to picture how that interview style would scale. Thanks for sharing.
Well, the first time so saw it Done was for the only large company I ever worked for - what was then a Fortune 10 (non tech company). But on the other hand, the department that I was hired into was actually a recently acquired small company with four developers and the manager was the founder of the original company - so take that as you may.
But, just because a company gets big, doesn’t mean that you can take hiring less seriously. You still have to be sure that you hire the right people. A false positive - hiring someone that can’t do the job - is worse than a false negative - not hiring someone who would be a good fit.
Generally agree except for the last sentence. What if the false negative is Elon Musk? Netscape could have hired him, but they passed. You can always fire a bad hire. You can’t reinterview Musk when you realize your mistake.
By "fizzbuzz" I wasn't referring to the "divide by 3 and 5" one that seems common, but something much worse. I once interviewed at a place where they presented a jumble of characters, and about 8 rules, and I was on the phone being watched (after an hour of successfully troubleshooting a broken AWS instance with a bunch of quirky Linux issues). The 8 or so rules involved something like, "You must print each line with one character longer than the last / c must never come right before vowel / x's must be on alternate rows / f's and z's ... " while parsing all of this stuff, I had 10 minutes total since we were almost out of time when we started the questioning. The interviewer said, "Don't worry, just do it in pseudocode."
That's bad interviewing.
I can do the problem easily without someone looking over my shoulder, and I need more than 10 minutes just to think about the 8 or whatever rules he gave me.
He failed me, and I ended up working there anyway a year later after the guy quit (true story).
This sounds like you're having the wrong conversations. With all your experience, why are you still going into companies through the front door? Where's your network?
Fundamentally: Why are you having the "Can you code your way out of a wet paper bag" conversation instead of the "How much value can you add to this company" conversation?
Perhaps my experience as a consultant/freelancer is breaking my perspective. It's been several years since I last had a coding interview as an applicant.
If you apply for a full-time software engineering position at a large company today, you will be run through the all-day series-of-hourlong-interviews, even if you are a referral. It is possible for the hiring manager to work closely with recruiter and recruiting coordinator to craft a more customized interview process, but that's rare (and, impossible at a place like Google, which decentralizes hiring). The value of network is the candidate wasn't subjected to the usual resume review weeding-out, but their info went straight to one or more hiring managers for consideration.
You use the term "apply for a job", which might explain the disconnect. The post you reply to isn't about applying for jobs.
You don't want to "apply" for things. You want your google VP to be having a beer with your ex-boss and worrying that his pet project was going to fail because they just can't find a guy who can really $X.
But your ex boss knows a guy who absoulutely can $X, and just finished $X'ing his company back from the brink of disaster. Lets give him a call to see if he's available.
HR will be told to give that guy an offer and, optionally, collect a resume to keep on file.
That's what we're talking about here, and it's where you want to get yourself if you want to build a career as an individual contributor.
You've essentially described nepotism here - if FAANG would follow this model, they would never hire some of their best contributors because those contributors didn't live in the Valley their whole life to network with bosses.
I prefer a process that's at least basically meritocratic to the old boys club type of hiring at any time.
Recommending someone you know who is good at his/her job is not nepotism. Nepotism is when you recommend your family or friends regardless of their qualifications.
I think it's "uncommon", but I also think that one shouldn't confuse "uncommon" with "never happens". I've had more than one job land on my doorstep without some sort of crazy tech screen. But I also talk to front-door companies on occasion too.
> HR will be told to give that guy an offer and, optionally, collect a resume to keep on file.
No, HR will be told to get that guy into the interview pipeline as soon as possible, so he can go through the same process everybody else does for a similar position.
The sad part is that the generic generalist interview will have nothing to do with X, and if you need someone great at X-ing, might yield painful false negatives or even worse, false positives (since you can’t hire specifically for X).
I would argue that if you’re in a position to be picky you would want to hire experts in X who are also good generalists. One-trick ponies don’t do well after project X is finished and they need to contribute to other efforts.
A PhD isn’t really a one trick pony, but whatever. Even if that’s what you want (specialists who are good generalists), those interviews don’t really check the specialist part, just the generalist one.
If they buy the company that you built specifically to get access to you, do you think you'll still end up getting screened out by some junior dev on a whiteboard? All policies have exceptions. Your goal is to be exceptional.
Yes, always. It is company policy. Google takes its standardized interview process more seriously than any other company I've heard of, except maybe Palantir.
I don't know, someone would have to ask Vint. But it is a plain fact that Google interviews the engineers of a potential acquisition. For a small company, they'll pick and choose which ones to keep. For a large company, I don't know. The CEO may or may not go through the full interview process -- I've seen it both ways, seems to depend on whether they're buddies with the VP.
> If they buy the company that you built specifically to get access to you, do you think you'll still end up getting screened out by some junior dev on a whiteboard?
In this case probably not, but you'll have to do several rounds with the VPs. Of course the tenor is very different, more conversational / situtational instead of technical questions. But you're still being sniffed out. Unless you're already pals with the VP.
The problem is that exceptional isn’t a binary quality, exceptional at google is going to be another level than exceptional at some obscure contracting firm.
In my experience it's rare for even this kind of interaction to lead to an interview process-free result. The exception would be if "the guy" is brought in as a contractor (which these days should probably be your goal), but I've worked with a lot of large companies and HR almost always forces at least some portion of standard FTE applicant evaluation even for "the guy" type referrals.
That's completely inapplicable to permanent positions. The current company of a prolific employee is not gonna lay him off and send him to a competing company.
It can make sense for consultants, doing short gigs here and there. People can refer you elsewhere when the current work is completed, they ain't gonna keep you anyway.
> Fundamentally: Why are you having the "Can you code your way out of a wet paper bag" conversation instead of the "How much value can you add to this company" conversation?
Because people lie. It's that simple. Referrals lie, CVs lie, friends lie to get friends in a position. Do understand that there's a fundamentally different set of problem a 40.000 FTE company has to deal with in comparison to a 30 person startup.
Also is doing a full day interview really such a horrible sacrifice for most paid jobs in the world? Professions like laywers and doctors need to go through significantly longer hazing rituals to get to FAANG income... is a month of study to be essentially among the most paid people in the world REALLY such a horrible thing?
Being on the technical interviewer side of the table it was a bit disconcerting to see people get to me who really should have been filtered out at an earlier stage. Still, for the type of lies that are most damaging, I still ask the question "what's the easiest way to uncover the lie?" Hence, FizzBuzz and FizzBuzz style questions work just fine if the lie is "can program". Why stress people out with trivia quizzes on language minutia or implementations of algorithms that CS students usually get at least a week to do (or typically never do)? Why let a too-strong fear of a lying false positive, which isn't that hard to deal with as a big company, become a strong false negative filter negatively selecting for types of experienced candidates you might want who won't put up with your clown nose "everyone is doing it" rituals? It's also easier to grade if you keep things relatively simple.
I have to do a little more to satisfy the process, of course, but I'd be more satisfied if more interviewers didn't just accept the process (as you kind of have to do as an interviewee if you want certain jobs) and continuously asked themselves how they can make things better within their own influence. I could only throw up my hands and sigh when a coworker asked me for interviewer advice, we had a big discussion about interviews, and in the end he just grabbed a random problem off some interview problems site that he hadn't even tested if he could solve himself in the time limit and gave that.
You might be a bit optimistic here: it's not always easy to deal with "lying false positives". Depending on the
legislation it may be difficult to fire somebody, and involve going to court, sometimes going on over several years.
Vindictive personalities may engage in sabotaging the company, the team, their (former) co-workers, as
a retaliation for being fired.
(Anecdote: I have witnessed all of the above in my work.)
Summary: the cost of a false positive almost always outweighs the cost of a false negative.
I don't think there's any state in the US where lying on your resume/CV is not legal grounds for immediate termination.
And any company that has thought about their hiring process for more than 30 minutes will have a probationary period of 30 or 60 days or so, after which employment can be terminated if the employee is not able to do the work.
Are you not in the US? I've never ever come across such a thing - probably because US companies can theoretically fire you on the spot for almost any reason except specifically disallowed discrimination, and most definitely for being unable to do the work.
Many people interviewing there are already among the most highly paid people in the world (lots of them are just moving around within that group) so that argument is pretty irrelevant.
I assume Google is looking for people who absolutely want to work at Google. Making the process cumbersome allows them to get candidates who stay at the company even if they have to do work that they don't like.
At the beginning of 2017, ~12,000 of Google's current ~72,000 employees had previously worked at other big tech companies, and most of them were probably in their R+D headcount of only 27,000. I wouldn't be at all surprised if half of their engineering interviewees came from similar paying jobs, and a the majority of the remainder were college hires.
> Professions like laywers and doctors need to go through significantly longer hazing rituals
At least for doctors this isn't universally true. There's continuing education/certification, (which you have to do every $period) there's public reputation, there's lots of networking. From what I've seen the interviews take few hours in one day and people usually already know each other.
I'd honestly rather have the "bulletproof tenure" that doctors then acquire after they get through the hazing. Getting laid off every time a spreadsheet jockey in accounting makes a bad projection, is pretty disruptive. (as are the times when a competitor buys your employer just to shut them down, and etc: plenty of scenarios where you lose a job due to no fault of your own are rather the rule than the exception) - Most competent engineers can always get another job. But in that process, you end up with a different commute, or even having to relocate, deal with setbacks in financial plans, and etc. The instability some of us experience is kind of not worth the other perks. And after 20+ years, it's kind of annoying to have to apply as if I'm fresh out of school when I have a solid work record on my resume, verifiable by contacting my former co-workers and managers I've provide as references.
> Getting laid off every time a spreadsheet jockey in accounting makes a bad projection, is pretty disruptive.
This has nothing to do with how programmers are hired, and everything to do with the business cases that building software serves. Even if programmers were hired like doctors, they'd still be cyclically losing their jobs, because their jobs are closely tied to the willingness of the economy to spend large amounts of money on custom software.
If you want a job for life, then you want to be in a profession that does something unconditionally useful. Healthcare, garbage pickup, teaching, farming, being a janitor. We're going to need people doing all that long after <insert Silicon Valley fad of the month> is dead and buried.
Yes, but before that you need years of low-paid jobs and networking to come close to well paid positions. Applying at FAANG requires significantly less.
You still have to go through a standard interview loop at Google/Facebook/Netflix/etc... even if you get in via a recommendation. The recommendation (and your cv) helps you in getting a serious phone screen and in getting a good choice of the team you want to join, but isn’t considered very much during the interview itself.
Well said. It occurs to me that my last job interview that involved convincing the other party that I was a good developer was back in 1999 when I had just a few years of experience.
Since then every gig has come via an introduction by a former boss or co-worker, or by introducing myself to somebody with hiring authority and using a pile of impressive artifacts to skip the "can this guy do the work" questions.
The first several years of your career should be getting yourself into a position where nobody would ever dare ask you to sort things on a whiteboard.
Back in 2004, I interviewed for a FTE position in finance - was my first interview for a full time role post college. I was a referral from a close friend and classmate. They sent me through the ringer. Had to take brainbench tests for C++ and C# (position was for C++, but I'd been working in C# and JS for the prior 2-3 years). IIRC, only got like a 65 percentile in C++, but over 95 in C#. Also had to go through an hour plus phone screen, then a day and a half on site. Got the job, but damned if it wasn't a long process. Lots of whiteboard and verbal questions, was never put in front of a computer except for the brainbench tests ar home.
You shouldn't have to only be able to go through your network to get a job. We should judge based upon the same criteria every other damned job judges ...how well they did their last job.
The problem is that resumes can be deceiving. If you've worked long enough in the industry, you've met a few incompetent people with stellar resumes. It's why having google on your resume opens a lot of doors. Everyone respects google's vetting/interviewing system.
Also, fizzbuzz isn't anything you really need to study for (certainly not for days or weeks). It's something you should be able to work out during the interview if you truly have developing experience. But then again, I've known people with amazing resumes who couldn't even understand the fizzbuzz question itself.
The Googles of this world don't fizzbuzz you, they want you to implement (close to optimal solutions!) and use tries and heaps & co. during whiteboard interviews. Neither of which I've ever had to implement in 10+ years.
I didn't say the googles of the world fizzbuzz you. I just said that fizzbuzz is so basic and simple that a developer shouldn't be studying for days or weeks for fizzbuzz. I said people respect google on your resume because their interviewing process is robust.
Did you, in your "proven career", ever work on the systems the size of Google Cloud and having same amount of users?
Because cheap stuff you can run away with on 1mil users suddenly falls apart when you need to serve more across continents and suddenly you kinda need to know what all those words in documentation actually mean.
Also, as for CV, everytime I've interviewed people I've gotten more than I few people that lied in their CVs. Persumed "senior developers" who couldn't do a fizzbuzz in their chosen language and tooling. CVs are utterly worthless from interviewers perspective - there will always be someone with exact same CV like you have but will be utterly incompetent.
The original article [1] describes new process in the context of PM interviews, which are different from software developer interview, no matter how technical PM is.
The changes that I particularly like are: improvement in coordination between interviewers themselves, and stop sharing feedback between themselves until the very end. The latter really affects how next interviewers view the interviewee.
After reading about wisdom of the crowds, we started doing this with estimates. We always break up the project into small tasks together then seperate and estimate them individually.
We've found we're a lot more accurate because we don't influence each other, as well as don't go into autopilot on an estimate assuming someone else has it under control, or someone just deferring to the more Sr. Dev when the more Sr. Dev made a mistake.
I could see how this would also work well with interviews.
Well, FB did that at least 6+ months ago when I got trained for interviewing.
As for the questions asked, there is an internal tool with various questions you are supposed to ask, with info about what signals you can get, how to extract more info and how to formulate your feedback.
I was always told that I need to help the person I'm interviewing solve the problems.
> Under the new process, Microsoft shares the interview questions in advance so that candidates can prepare. During the interview itself, a candidate might run through a real scenario or problem the team is trying to solve.
LOVE IT.
I think best when I have a chance to actually think, not when I’m participating in some odd job interview game show.
Here's what they are doing, from the linked blog post:
> Our dev teams had taken to working with candidates to solve a bug or feature as part of the interview process. It was a collaborative effort with the candidate and the team working together to solve a real problem.
Sounds like a great idea.
Except it's so variable. What if you get lucky and you get a easy bug, whereas someone else gets something much harder. Let's standardize that by giving everyone the same task to work on.
And what if we want to hire people who aren't fluent in your project's programming language? That would be a big handicap for them. Let's deal with that by having minimally sized "projects" in all the languages, and allowing the candidate to choose their preferred language.
And we don't want the interviews to be biased against people who don't already have domain knowledge in a specific field. So let's make the task generic enough to be approachable by any good programmer.
Congrats, if you did all of the above, you've reinvented leetcode style interviews.
There's some cool stuff that MS has put into practice. I like the focus on reading/understanding existing code, instead of just writing code. I also like the relaxed pacing and "open book" approach.
But for the most part, I don't think this is really as revolutionary as people think it is.
Fixing a simple bug in your own software is an order of magnitude faster than doing it in someone else's codebase. Make no mistake, it would be cheaper for them to fix it themselves than guide someone else through a paired-programming exercise. This isn't unpaid labor, it's more expensive labor.
A little bit of unpaid labor is honestly fine. Especially since I can’t imagine this being easy to exploit in a way that really impacts a company’s bottom line.
It’s like getting a free sample of food at the store, before committing to a full purchase
Imagine being a carpenter applying for a job and being asked to make a small bedside table for free. So the employer can sample your skills before committing to a full purchase.
If you ever sat on the hiring side and had just one new hire not work out, you would understand your throw away statement is actually more absurd than asking for free work. Let's flip the scenario and go your way instead...
Imagine being a programmer applying for a job and not being asked to demonstrate your skills at interview. The interview goes well, and you're offered the job.
You give your 4 weeks notice on your existing job, and turn up to your new role. You sit down, it's day one, and you're presented with a small sprint of bugs to work through to get warmed up on the project you're assigned.
Your heart sinks.
You realise that you are totally out of your depth. The solution you're working on is hard, the documentation insanely complex, and you feel more and more despondent as the week goes on.
By the end of the week your colleagues and managers have started to pick up on your struggles. Despite their effort to help you can't make meaningful progress. To cut a long story short you're let go, and now you're unemployed.
-THE END-
As an employer and owner of a software company, it's my job to assess the skills of somebody as quickly and efficiently as possible. If getting them to "build a small bedside table for free" gets both me, my colleagues, and the candidate attending the interview comfortable that we're offering a job to somebody who can succeed at my company, then so be it.
I'm more often on the hiring side of the table than on the other. Probation periods exist specifically to address the issue of hiring false positives, and they work fine.
Your story is only possible if the interview process failed miserably. If your process sucks and results in a lot of false positives, that's your problem. Don't use it as justification for milking candidates for unpaid work towards actual deliverables.
One must not use probation as a get out of jail free card. We used to think the same way you do, we ultimately realised we were wrong.
Probation exists to be used only extreme circumstances, not as a catch all for "hey, sorry we made the wrong hire". It's more expensive to hire a candidate and kick them out on probation than to avoid making the wrong hire. This is another reason my approach works - we don't "milk" free labour, we don't need it. Our interview ensures we make the right hire first time, every time and part of that process is some "real" work.
Our interview process is awesome. Refined over a period of 10 years. Candidates learn something, we learn something, we collaborate with them. We often have candidates, even those who do not receive a job offer, compliment us on our process.
I'll write our interview process up one day and share it on HN so others can benefit.
When I interviewed at Redmond 15 years ago, it was the recruiter who asked me a bunch of techie questions then decided to bring me out for a whole day of onsite interviews. During that I recall it was only a director of SW who asked me some mentally crushing questions about subtleties in how the Windows kernel fiddles with files. As I said, I was a UNIX developer who didn't know those bottlenecks in the Windows kernel. Still he persisted. He did take me for a spin in his awesome BMW sports car after though!
My thoughts on Redmond were that it rains all the time and the inside of the Microsoft buildings is pretty depressing.
I've had something similar happen to me once. I'm DevOps, very systems focused. I was leaving Cisco and got on an interview panel with 4 network engineers at some place.
I caveated I know networking better than most but I'm by no stretch a network engineer. They agreed and explicitly stated the position was on the systems side and it was fine.
The interview was only an hour and half. About 30 minutes into they ran out of questions and started to get bored. They tell they're all studying for their CCIE's and decided to start borrowing questions from the final exam.
They decided to play "lets stump the Cisco guy". And did they? YES! Did they stop? NO! This was 7+ years ago and I'll never forget how uncomfortable and bad that interview was. They managed to dredge up all these arcane, stupid questions, some of which bore no practical function.. and listened to me "I dont know" for a solid hour
I'm sure it left them with a bad impression of me, and it certainly left me not wanting to work there.
Good riddance to all those mostly useless brain-teaser questions.. They require countless hours of practice for most people to get good at them - and I find it semi-tragic that many highly intelligent people are wasting their time on that instead of on complex real-world problems (not to mention on learning things that would make them happier and more well-rounded).
> ... Second, we run through a real problem the team is trying to solve — improving satisfaction, increasing retention, boosting usage of a service or feature. The fact that it’s a real problem that we’re working on helps foster a collaborative conversation.
maybe I'm reading too much into this but ... for some reason, none of these questions are highly programming-centric (e.g. implement a sort algorithm, build a B-Tree, or even just implement the singleton pattern in C#, or whatever). all of the questions are human behavior-centric. and I wonder if maybe Microsoft has concluded that the really important challenges are not technical challenges, but marketing ones.
>(Note: I started at Microsoft when we were still asking questions about why manhole covers were round, how many ping pong balls would fill a 747, and how to reverse a linked list. In 20 years here, I’ve yet to have to write the code to reverse a linked list (copy-paste anyone?) or fill a 747 with any kind of ball.)
The manhole cover question is actually a valid industrial design or UX question even if it's not appropriate to ask for software interviews. The linked-list question is the equivalent of Fizzbuzz for data structures. And the 747 question is a valid Fermi problem. The exact number doesn't matter but the steps required to estimate the answer are similar to other estimation problems when mapping out a business strategy and trying to determine if it's profitable. Or maybe the company is interested in salvaging sunken planes: https://www.iusmentis.com/patents/priorart/donaldduck/
I was asked to estimate Pi using a unit circle (diameter = 1) inscribed in a unit square (side = 1) around 2012-2013. In a manner that would have a controllable amount of precision.
They wanted monte carlo (more darts, more precision), I went riemann sums (smaller delta x, more precision). It totally through me off my game for the rest of the interview as I was constantly thinking about it.
while not quite a traditional brain teaser, there are a couple of insights one has to get to solve it
1. ratio of circle area to square = pi/4 (but remember, you don't know pi yet, just that it exists, the point is to estimate it)
2. a way to make use of that ratio (i.e. monte carlo, pick a random 2d point (on both x,y from -0.5 to 0.5) if the distance from (0,0) is greater than 1, it is only added to the square, otherwise it gets added to both.
combine the 2 insights and you get pi ~ 4*circle/square
(as mentioned I went with riemann sums, where I calculated the area under the curve of the circle's arc in one quadrant, but same idea you get pi ~ circle_arc_area/.25)
those 2 points aren't quite obvious (I went riemann sums as area questions made me think calculus and went totally down that rabbit hole), and in 30-40 minutes of brainstorming (first 10-15 minute discussion about the person, then actual interview question, then 5+ minutes for any questions you have for the interviewer) can be difficult for many people to get.
The problem can then be, many people can't just stop thinking about a problem they struggled with and will continue to process it (hurting their following interview Qs),
i.e. even if that interviewer thought I did sufficiently, I believe it hurt me on the following Qs as I couldn't just let it go, which in the real world is how I think many people work. If they are struggling with how to solve something, they are dealing with it even when they are supposedly focused on other things (say in meetings, eating lunch....)
Unless you were interviewing for a mathematician position, I agree that's a garbage programming interviewing question. It's focused on your math skills far more than your programming skills.
If it's a "brainteaser" is I guess a matter of definition.
Look, I'm sure people are asked bad questions at all times in all companies, but the policy at Google was clearly to not do that. At least when I was there 2006-2009.
Wouldn’t the current niche clever dynamic programming problems google asks you to work out on a whiteboard be classified as brain teasers? Sure, it’s much easier to prep for via the cracking code interview book, but is still not something you would encounter in the wild.
I think it's true for majority of questions (apparently leetcoding is the new thing at Google and also FB). IMHO interning for a few weeks is the best for both sides - unfortunately it doesnt scale.
Interning "for a few weeks" means that you miss out on candidates who already have perfectly good jobs. These are often the folks that you are trying to hire.
So imagine how a candidate reacts to, "Well, while we know you're happy where you are, what would you think about quitting your current job so you can try out a position at our company for a few weeks?" (Note that candidates can't use vacation time just to do a try-out, for legal reasons, and may be reluctant to use up all their vacation time in any event).
Having a trial period puts all the risk on the candidate, and guarantees you're filtering out people that you'd otherwise want.
Manhole question is bad, because around half manholes im real world are squared. So the correct answer is that they are not in fact round and square works perfectly fine.
Of course I know right answer, but it has zero to do with me being smart and a lot to do with me being in right culture and thus having heard about it before. Which is exactly what it test along with other brain teasers - have you encountered this sort of question before?
The correct question, obviously, would be "why in some places they make manhole covers round and which advantages and disadvantages over square ones they have?" Which, as any good question does, already contains part of the answer.
It could be coupled with following question: "why in some places they make manhole covers square and which advantages and disadvantages over round ones they have?"
There was a linked list interview question going around back then that went: In constant time, without allocating memory, given a pointer to the first element of a linked list, how can you tell if it eventually ends in a null or if it loops back on itself at any point? I actually ended up implementing the solution while tracking down a hang in a DCOM service I was working on...
Isn’t that normally the 2 pointers (tortoise/hare) thing? If yes then I agree it’s not constant time.
I personally hate this kind of interview question. It’s one that is easily answerable when one heard the exact question before (in training for interviews). But super hard to answer if not. And it even does not have big applications in practice - in 20years of programming, I never encountered a problem like this. In the end the question just wastes time on both sides and won’t really tell a lot about the candidate.
There was a blog post shared a while ago on HN where the author ranted that it was his least favorite interview question because the solution (fast/slow pointers) was - if I recall correctly - literally somebody's PhD thesis. There was a years-long window where anyone "clever enough" could have waltzed in with a one page paper.
If you've never seen the pattern before, there is a very good chance that you're hosed.
Marking items visited can actually work, because on any(?) modern OS the memory for the elements in the list would be allocated at even addresses, meaning the low order bit of the Next pointer is always zero and can be reused as a marker.
It's in a critical section. (Which you pretty much have to do with the canonical one too, since it can run off the rails if another thread modifies the list.)
I used to ask this (cycles in linked list) question all the time, and I found it useful as a way to see how the candidate worked through an unfamiliar problem. No-one gave the 2-pointer answer, ever.
For me it wasn’t a pass/fail “do you know the right answer” question, it’s more of a “can I work with this person” question.
This is a pretty standard linked list question (called cycle detection) – one of the easiest questions on LeetCode. You can use the two-pointer method to just detect if there's a cycle. If you want to find the node that the cycle starts at, you can just walk the slow pointer from the slow/fast meeting point and a pointer to the head of the list 1 step at a time until they both meet (there's a simple explanation for this that I won't bother writing out).
I must admit I didn’t figure that one out till I saw the solution which was a “oh duh” moment. I don’t think i would have ever stumbled upon it without lots of hints during an interview.
The constraints, if correctly stated, can help a lot to figure what to do
if you take a look at them closely.
In constant memory there's not that many things to do, a single pointer
clearly won't be enough as you can't recall if you already visited the
node or not, so `K>1` pointers must be used.
If any two pointers moved at the same pace they would be duplicated
and the second won't give any information.
That and the fact that you can only move forward leaves a pretty
constraint situation and acts as a guide.
It's not a bad thought problem, but I guess the fact that it's really unlikely
to happen in a real world scenario plays against it (the memory on a hash
table is probably way cheaper than the extra reasoning this takes).
Also, knowing a bit about the answer is an unfair advantage, it might be
more of a test on memory than on reasoning, which makes the problem
bad for interviews.
I don't think it is a very good thought problem because I think very few would figure it out unless they have encountered the algorithm or a similar one before.
It is one that a lot of people will go "Aha, of course" when they hear about it but not figure out themselves.
I understand the reasoning now since I know the answer but I still don’t think I would arrive at the answer even with the reasoning. I don’t use it as an interview question so I don’t know if people ever genuinely solve it or just memorize the solution.
* A round manhole cover cannot fall through its circular opening, whereas a square manhole cover might fall in if it were inserted diagonally in the hole. The existence of a "lip" holding up the lid means that the underlying hole is smaller than the cover, so that other shapes might suffice. (A Reuleaux triangle or other curve of constant width would also serve this purpose, but round covers are much easier to manufacture.)
* Round tubes are the strongest and most material-efficient shape against the compression of the earth around them.
* A round manhole cover of a given diameter has a smaller surface area than a square cover of the same width, thus less material is needed to cast the manhole cover, meaning lower cost.
* The bearing surfaces of manhole frames and covers are machined to assure flatness and prevent them from becoming dislodged by traffic. Round castings are much easier to machine using a lathe.
* Circular covers do not need to be rotated to align with the manhole.
* A round manhole cover can be more easily moved by being rolled.
* A round manhole cover can be easily locked in place with a quarter turn (as is done in countries like France), which makes them hard to open without a special tool. Lockable covers do not have to be made as heavy, because traffic passing over them cannot lift them up by suction.
15 years ago I was interviewing for an engineering job and the VP of engineering at a company asked this question. I happened to have heard this question previously so spouted off a bunch of answers and impressed the heck out of the VP. And that is the real problem with this kind of questions. They largely depend on if you heard the question before or not.
I got into Oxford like that. Basically the interviews are a bunch of interesting questions like "How could I take the log of -1?" or "How is e defined?".
If you already know the answer, your main problem is acting like you're thinking it through.
Yeah for both Oxbridge unis the grades get you the interview but they could just fill the whole uni several times over with kids who got top grades.
So they reckon a 45 min quiz/chat is what will tell them who to admit.
Gets pretty open ended, favors people who can talk well.
I ended up not applying to MIT because you get your acceptance before Christmas and the US applications are huge. But my understanding is the US school interviews are more cultural fit?
Don't forget also: round manhole covers are a Schelling point.
Making a manhole cover round is one of the first intuitive shapes you'd think to make them (along with squares and rectangles); and so you'd guess, as a construction company gearing up to sell the concept of concrete manholes with removable iron-plate covers to a municipality (back in the 1800s or whenever we first got them) that what the municipality will ask you to make is round manhole covers. So you provisionally set up your tooling for round manhole covers, and build a few for the demo. And the municipality assumes you're the expert about good manhole-cover shapes, and so just goes with the flow, rather than trying to figure out the best shape themselves. So you end up with "the first intuitive thing that worked" spreading via network/"system compatibility" effects.
More like, why we got four-way intersections for so long before anyone tried roundabouts. It was intuitive to just drive one road through another, and then we added traffic laws and signalling on top to try to make it work, rather than anyone sitting down at the start to try to work out what intersection shape would optimize for traffic flow and [non-]accident rate.
It’s actually several things. It’s stronger, cheaper, rolls, lighter, no need for alignment etc. You can make sure any shape does not fall in, but strength to weight ratio is important as you want people to open them with hand tools while supporting dump trucks.
PS: At ~250 lb you can see why a less optimal shape might be a problem.
Less raw materials to produce, easier to transport by rolling, easier to place onto the manhole, easier to stack when transporting/storing in bulk, etc
I don't think the 747 problem is very predictive of how well someone can map out a business strategy.
I think a better question might be, "tell me how you've mapped out a business strategy in the past, and what are the strategies you've used to determine how profitable it would be".
^ this person is probably a PM at Microsoft. :-) When I worked there (over a decade ago) the place was so regimented you couldn't rip a fart without having to write a specification and estimating a schedule.
The manhole and 747 questions test for obedience and bikeshedding, not problem solving. Likewise the linked list reversal question tests for lack of planning and critical thinking.
These anti-skills will have been used quite regularly at Microsoft.
Although I’m a product person (CPO) I’ve hired (alongside a CTO) maybe 100+ engineers in my time and I’m hoping my perspective is useful/helpful.
The process that my CTO and I have found works best is as follows;
A 20min phone interview (to screen for BS’ers).
A technical test that’s based on the sort of problems they will be solving in their job. How would you do X in language Y, no boiler plate code, and it doesn’t have to be perfect. (So no fizz buzz, conways game of life etc.)
We also stipulate to spend no more than 2 hours on it, and if their successful they get that time back. We even offer a discount (on the thing we sell) to unsuccessful people who don’t make it.
The test serves as not only a good high level demonstration of ones skils but, more importantly, gives us all something to talk around. It also shows us how they approach key things like testing, deployment, maintainability etc etc.
Then there is a 2 hours face to face interview. Ahead of this the test is peer reviewed by the engineering team (or part of it)
The interview is made up of one one hour technical section with the CTO/Senior, and one with a designer/product manager. A joint decision is made.
Then, if they are successful, we have a probation period where we (the candidate and us) work out if it’s all going as we’d hoped. If not, we part ways.
You most likely wouldn’t marry someone before a first date. A job is the same. You both need time to see if you like each other and your profile pics match real life.
It’s not perfect, but no system is. Sure we’ve hired a few people we’ve had to let go, but for the most part it’s a great system. We’ve managed to hire not just great engineers but also build a team and culture that fits and works.
Hiring is time consuming, hard and at times emotional. There is no shortcut.
I’ve interviewed and worked at big tech companies and they all think there is some secret sauce, a magic code to hiring.
That is, if you ask super hard questions only the “best” make it through - utter tosh.
Some of the most amazing engineers I’ve worked with would fail these type of stupid questions (for a while load of reasons) but quietly write amazing code.
Also, and more importantly imo, to be an amazing engineer you need to be great (equally) at teamwork and communication, not just be super book smart.
I read a while back (or imagined it?) that MSFT only hired people with Msc’s/PHd’s and that it created a probelem where there wasn’t any intellectual diversity.
These tests cause this too. You get cookie cutter people who are smart but often can’t work in teams and can’t communicate well.
> He thinks it would be useful to figure out if there ways to either shorten the process entirely, or at least spread it out over multiple days and gives candidates more time to reflect and think.
Please don't. I can take one day off work for a full-day interview; I'm not going to burn through four fucking vacation days for you to vacillate over whether or not I'm a "culture fit" or whatever.
IMO those interviews test whether you are smart "enough" and capable
of collaborating to solve a problem. There's not that much knowledge
required and good answers tend to be quite succint.
Depending on what you do, maybe most of the time interview-like
problems won't rise up, but not everyone is ok taking the risk of poor
algorithm design on production.
While it won't happen that a single engineer will need to do everything
without any design or code reviews, if you keep "lowering" the bar you
risk breaking the review failsafe.
Maybe scaling the review process can enable teams having more
engineers working "safely" as the hard problems will still be reviewed
by the "elite" engineers those companies try hard to get.
I hope this takes some pressure away on pretending to have "real"
interviews needlessly. There's too many problems to be solved out
there to be overly picky on who can work on them.
It would be nice to hear back from MS after they a few years.
Article used incorrect title, the original blog posts describes the interview Microsoft has for technical PM and not software developers. I bet that developer interview hasn't changed much.
I think a dumbest question thread might be quite amusing?
I’ll go first, because I’m quite sure I’ll win...
This was for a front-end mid weight software engineer position.
The candidate was smart and a 100% hire (I’ve since hired him 3x in different companies)
Me; “great thanks for your time and we will be in touch later today”
Co-interviewer; “I have one more question”
Candidate; “sure, go ahead”
Co-interviewer; “Imagine your uncle has just died and left you his pizzeria in his will, what would you do?”
Me; “what?”
Candidate; “what...? Errr... sell it because I’m a software engineer, not a pizza chef”
The quality of interview depends on quality of questions. A qualified question is one which has many solutions, then the interviewee could pick the best solution on his own.
I think the downvotes/flags were because your comment was unsubstantive and snarky. We're trying for better than that here. Would you mind reviewing https://news.ycombinator.com/newsguidelines.html and following the rules when posting here?
Also, please don't post insinuations of astroturfing or similar abuse like this. That's also in the site guidelines. We're happy to investigate if you're worried about abuse, but the thing to do in that case is email us at hn@ycombinator.com, not take a thread further off topic and/or into flamewar.
I wonder the effectiveness of the MS hiring process every time I use a Microsoft product. Teaching interviewees how to test their software locally for at least 10 mins before deploying it to hundreds of millions of people would be huge.
One of the key issues in recruiting is how to make hiring decisions quickly. Good candidates don't have the patience for your internal politics to resolve over a month or two to figure out whether they're getting an offer or not; they're just going to interview with other places in the meantime who will beat you to the punch.
Take a resume off the pile. Email or text to schedule a call. During the call, talk about relevant experience / past projects, what each side is looking for, etc. Make a decision during the call whether you want to invite the person for a two-hour interview, and schedule it during the call, first offering an evening time slot for the interview. In the first hour, talk about culture, problem-solving and teamwork approaches, engineering attitudes. In the second hour, pull out an actual current problem and solve it collaboratively. If all goes well, send an offer, and offer to set up a dinner to talk about / negotiate the offer.
It should not take more than a few hours to make a decision on someone; it should not require a candidate to take paid time off. Your hiring pipeline's average lead time should be measured in days.