If the purpose of an interview is to deflate claims people make about their expertise, sure. That's not a productive goal for an interview, though.

Is it not? Hypothetically, is it not worth knowing that someone has exaggerated or lied about their skills in something? That would seem to be useful information in an interview; i.e. can someone say "I don't know" or will they make up something instead.

It can be, but I think the idea that it should be is probably the biggest misconception in hiring, and the biggest impediment most startups have to effectively hiring engineers.

Consider: there's a difference, perhaps subtle, between "deflating claimed expertise" and "assessing ability to perform a job".

> "deflating claimed expertise" and "assessing ability to perform a job".

This is a very important point, and also why I never claim to know anything I can't prove (speaking as a Linux Admin, not even developer). Everything in my resume points back to the github so whoever is interviewing can know exactly how far I've studied or had experience with a specific subject, and acts as a reference point for the job of course. Also lets me know which companies to avoid if they just care about random trivia questions rather than how much demonstrable value you can offer them.

Oh, I agree. It should not be the only thing you're looking for. I have, however, gained very valuable information from figuring out that someone likes to aggrandize their own capabilities in a particular topic and/or is unable to say "I don't know." We specifically look for people that are able to say that, since honesty and not being afraid to learn (usually the result of saying "I don't know") is important to us.

But it is /definitely/ not the only thing we look for.

My experience is that once I figured out how to reliably screen for ability to perform the job I was hiring for, I stopped caring about psychological tea-leaves like these almost entirely.

Being a raving asshole could get you evicted from the Matasano hiring pipeline, but otherwise, if you could knock out the challenges we gave candidates, you had a mortal lock on our attention.

I was responsible for candidates from first contact to offer letter, and in the last year and a half I ran the process, I looked at a total of zero resumes. Many, many hires. Guess how many didn't work out.

So I guess my subtext is, if you have the mental cycles to reason from signals like "do I disagree with this candidate about what the word 'expertise' implies", you probably don't have the job aptitude prediction stuff nailed yet.

I think that's going a bit far. I think we're saying the same thing but using different signals; the ability to say "I don't know" is actually an important gauge of aptitude in my opinion, because anyone that is unable to say it will possibly have too much hubris to take the time to learn something they don't know. It is not the only signal I rely on, and if they truly know the topic, there are other signals I rely on either way. But it is /a/ signal which I don't believe is invalid. I don't actually care if someone agrees with my definition of expertise, I just care about their ability to learn, which starts with admitting they don't know something.

Challenges are another great way to do that, and don't necessarily need to be time-limited; MicroCorruption is a great example. We will occasionally do similar "take home" style challenges. The only time we'll time limit them is specifically when we're looking for how /quickly/ someone can get up to speed on a topic they're unfamiliar with, which is also occasionally a valid question; i.e. do they need to become an expert to build something in it or can they get dangerous enough quickly. Not everyone can. In those cases, we provide plenty of resources, make ourselves fully available, and make the time limits a matter of days, not hours.

Edit: to clarify, it is far more important to me that someone have the ability to learn new things quickly than already contain an existing bit of knowledge in their head.

I think people think I'm talking about "how well people can learn new things" when I talk about this. That's potentially a part of it, but it's not what I interviewed for. I interviewed for, literally, simply, "ability to do the job we had Matasano consultants perform".

Microcorruption and the Crypto Challenges were outreach for Matasano, but they were not part of the hiring process. Our hiring challenges were slightly more boring, and explicitly tuned to generate the exact signal we wanted.

I don't know whether there's any valid signal to recover from candidate psychology. I don't have to reach that argument, because in my universe, there's a giant, very accurate signal available that makes looking at other signals a poor use of my time. The fact that the big signal also keeps me from using dubious signals and making bad decisions is just a knock-on benefit. :)

That's great that you had a very specific job req and set of things you were looking for. Not all of us are that lucky ;)

Startups sometimes look for generalists (i.e. people who can learn new things) specifically because nobody knows what the stack will look like, how the product will change, etc. a year from now.

I was actually asked some of these for a position in a SRE team. So these are interview questions you might encounter.