Lots of the problems you mentioned sound like they were in part due to the company finding that difficult, and of course it should be noted that any process like this, about humans and done by humans, is going to have errors on occasion. The real trick is realizing and correcting them.

I've had people suggest that "willing to fire" is a bad sign, because we can be even better by just always hiring the right people! Which is a self-reinforcing style, because now you really can't admit you hired the wrong person.

[] The exception is when it does, they clearly hired someone for some bogus reason, but there will always be exceptions.

Well said. And their role definitely should not include administering bullshit voodoo science "psychometric evaluations" and making any decisions based on them.

Like everything, competent implementation and use are key, so I'm sure you have had experiences that give you reasons to doubt their efficacy. But every field suffers from some form of that in some way or another, and the degree of transparency into the process varies greatly.

Fair enough. BTW, I'm not the one who downvoted you, FWIW.

Honestly, as a PhD psychologist, they are the only useful (i.e. scientific) part of psychology.

For example, see Hunter and Schmidt (1998) http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%.... This (and if you've been around here a while, you've probably seen this) is a meta-analysis of the utility of particular selection procedures for jobs. That is psychometrics.

Unfortunately, much of the bullshit surrounding the particular psychometric tests used by HR departments has thoroughly debased a discipline that invented cross validation. Smith and Mosier, 1958 (Method 6, I believe).

cut/pasting from another comment of mine:

by saying that psychological research is itself useless, you're also throwing away things like A/B testing, UX testing (including Apple's much-vaunted usability stuff), research into grief management, team-building research, research into cognitive recovery therapy after acquired brain injury, work looking into ameliorating sexism and racism, perception research for HUDs in fighter aircraft (my honours research), some pain management research, research into dealing with PTSD, research into crowd control and management...

Someone with a PhD in the topic should be well aware of the breadth of the field that is 'psychology', and to say that the only useful thing in the field is psychometric testing just displays your myopia.

Half the stuff that HN talks about is psychology, from A/B testing to building staff relationships. It is far from 'useless', particularly given this audience.

It is ironic in the extreme that the people that shit on psychology do so because they see it as a 'pseudoscience' that 'doesn't observe things properly', yet so very few of them actually see psychology for what it is - instead just falling back on their own narrow stereotype of it.

If you use a linear regression rather than ANOVA, you will often be asked to change it to an ANOVA.

I am well aware of the breadth and depth that is psychology, and most of it is poorly conducted and irritatingly bad. I actually think it has a lot to do with applying a particular experimental model of science developed for non-reflexive systems to reflexive systems, with predictably hilarious results.

Please do not take out the rest of the commentators opinions on me, it upsets me also when people bash psychology from ignorance, but I come from a place of love when I bash it, as I do really adore the subject, but feel that much of it is so very, very awfully done.

A/B testing - psychology degrees teach pretty good experimental design, but you wouldn't know it to see a lot of published research. Additionally, rigorous experimental design owes a lot more to statistics than psychology.

UX Testing: This is definitely a good area, but again I think we don't control enough for the impact of the researcher(s) - see Rosenthal, 1969 for the problem, and note the complete lack of care regarding these effects in modern psychology.

Grief management? Seriously? I don't really think psychology has added much here, but would be delighted to read some good papers that prove me wrong.

Ameliorating sexism and racism? Good intentions, but given that I read a lot of this stuff I would have to say that they are perhaps the worst for statistical sins and errors (with the notable exception of Brian Nosek).

I know very little about perception research in HUD's, do you have some good papers?

I'm not entirely certain why you felt the need to correct me, when if you look at my comments in the overall thread it can be seen that I am pretty much on the same side as you.

It is possible that I am so sick of people who commit statistical sins for career advancement purposes (something I could never do) that I may be taking it out on the field.

Incidentally, what is psychology? I would be interested to hear your definition (as long as its not waffle like the scientific study of the human mind and behaviour, which merely begs far more questions).

Unfortunately, much of the bullshit surrounding the particular psychometric tests used by HR departments has thoroughly debased a discipline that invented cross validation.

Yeah, that's the rub, innit? The times I've worked for companies that did this stuff, and when I've seen results from them, I've seen nothing that leads me to believe in the utility of the tests. Unfortunately I can't recall the specific name-brand of the ones I've been exposed to or I'd criticize them specifically.

If an organization has a successful process of doing something and they bring somebody in who bucks their methods without regard for the establishment, that can be problematic. Sure folks can come in and disrupt organizations for the positive, but I'd say that's probably not the majority of organizational disruptions. Most of them are just obnoxious and unproductive.

Loose cannons can destroy more value than they create, even if they are brilliant in certain skills.