There are some issues I (as a psychologist) would have with your assumptions.
Firstly, fMRI and neuroscience research is probably the single biggest source of errors and shoddy research in psychology. Its a combination of really small sample sizes, poor statistical tests (brain regions are independent of one another, really?) and huge amounts of data dredging to find significance.
See for a roundup: http://escholarship.org/uc/item/51d4r5tn
Interestingly enough, most of the observable pheonomena you note in neuroscience are linked to traits that people think about (optimism, personality etc) through surveys, so even if their methods were perfect, the results still wouldn't be.
On your survey point, I would agree that the participant pools are quite limited and generalizability is quite low, but your example is ludicrous. No one does surveys of gravity in psychology, surveys are carried out to investigate the manner in which people conceptualise their experience.
For a really great roundup of the problems with typical social science participant pools, see: Heinrich et al: humancond.org/_media/papers/weirdest_people.pdf
Indeed, the entire issue of BBS that that article appears in is well worth a look for a deeper understanding of these problems.
These surveys should then be calibrated against behavioural outcomes, but this does not happen often enough, which is a major issue in my view.
Thats the major problem with surveys, well that and some poor methods accepted too uncritically within the field (factor analysis).
To summate, psychology has many, many problems and I fully support this reproducibility effort (and it will expose a lot of findings as non-replicable). But don't single out surveys for derision, neuroscience deserves as much if not more of your scorn.
As a recent Psychology PhD, I agree completely, and I'll add this: people outside the field commonly believe that neuroimaging tools like fMRI are better and "harder" science than more traditional experiments that measure human behavior, but they have it totally wrong.
Here's an analogy for computer nerds: imagine if you had an "fMRI" of your computer's operation. You'd see that different tasks result in different parts of the computer "lighting up". Tasks such as graphics, disk I/O, numerical processing, etc., would lead to different patterns of activation. And if your scanner had could resolve, say, 0.1mm voxels in your CPU, you might even learn that certain parts of the CPU are related to certain tasks. But what all this tells you is something about the gross physical structure of the computer; it doesn't tell you much about the abstract, logical structure of the computer. To draw an analogy, you learn something about the brain, but not very much about the mind.
What's of interest to most people (except hardware engineers) is what the computer does, not what parts activate.
With a "computer fMRI", you'd learn little about how, say, a filesystem works, how a programming language works, or much of anything that's of interest at the functional level of the computer. The same is true of fMRI and humans: it doesn't tell us much about the mind works. Instead, it tells us that there's a brain region associated with some task. For example, one of the major recent-ish findings with fMRI is that there's a small brain region that's associated with face recognition.
I'm not saying that traditional experimental psychology is going to answer everything. The mind is very complicated and I'm pessimistic that we'll ever have a good reductive model, the way that we do in many other sciences. I'm also not saying that neuroimaging is totally useless -- but it's certainly not as enlightening as many people imagine.
Here's a summary of a paper that found that lay people thought that psychological explanations were more convincing with neuroscience talk (even though the "neuroscience" was totally irrelevant), and that those with experience in neuroscience/psychology thought the opposite:
http://scienceblogs.com/cognitivedaily/2008/03/when_we_see_a...
Firstly, fMRI and neuroscience research is probably the single biggest source of errors and shoddy research in psychology. Its a combination of really small sample sizes, poor statistical tests (brain regions are independent of one another, really?) and huge amounts of data dredging to find significance. See for a roundup: http://escholarship.org/uc/item/51d4r5tn
Interestingly enough, most of the observable pheonomena you note in neuroscience are linked to traits that people think about (optimism, personality etc) through surveys, so even if their methods were perfect, the results still wouldn't be.
On your survey point, I would agree that the participant pools are quite limited and generalizability is quite low, but your example is ludicrous. No one does surveys of gravity in psychology, surveys are carried out to investigate the manner in which people conceptualise their experience.
For a really great roundup of the problems with typical social science participant pools, see: Heinrich et al: humancond.org/_media/papers/weirdest_people.pdf
Indeed, the entire issue of BBS that that article appears in is well worth a look for a deeper understanding of these problems.
These surveys should then be calibrated against behavioural outcomes, but this does not happen often enough, which is a major issue in my view.
Thats the major problem with surveys, well that and some poor methods accepted too uncritically within the field (factor analysis).
To summate, psychology has many, many problems and I fully support this reproducibility effort (and it will expose a lot of findings as non-replicable). But don't single out surveys for derision, neuroscience deserves as much if not more of your scorn.