The problem isn’t having variables you can’t control. The problem is not recognizing these variables, or recognizing them and being too lazy to control for them, and then drawing unwarranted, overly broad hypotheses.
There’s a reason psych has a bigger reproducibility problem than any other field.
> when you have more variables than you can control
Worse: isn't this always true? Specific breed of mice, humans working with them, temperate, trace differences in food, light patterns, sounds (including sounds beyond human range)...
This is also naive. In that case you won't be able to publish your results in prestigious journals and your career will die out.
That said, you do learn not to act 100% confident. You need to say "This suggests that" instead of "This proves that", etc.
Anyway, the solution to reliable data is simple: Have multiple competing groups try to repeat the work as closely as possible.
Now that you have something to hang your hat on, you come up with some theory/model to explain the phenomenon. Even better, come up with multiple competing such models. From these you deduce some precise prediction that is checked against future data.
All of this was well known a few generations ago...
> This is also naive. In that case you won't be able to publish your results in prestigious journals and your career will die out.
Charlatan fortunetellers aren't owed careers as scientists -- if they can't produce reliable, predictive models because they utterly fail to control experiments,then they should have their careers destroyed.
We've seen enough damage from corporate leaders and politicians relying on guides to reading chicken entrails masquerading as science.
Sure, just be aware of what you are saying. Over 99% of what gets published is like this. Generations have wasted their careers producing misinformation. It all needs to be redone since these practices began (in some areas of research this is at t=0). Etc.
Consider the consequences of what you're demanding: You'd basically outlaw any research on complex systems. Everything in biology, sociology, archeology, climate science, etc. Is that a reasonable response to the problem at hand?
No, I would bifurcate research into two classes --
Let's call it 'science' and 'hunches'. (I'm sure marketing people would think of a better name.) There's nothing stopping you from publishing hunches; there's nothing stopping the same journals from publishing science and hunches. It would just be the case that we'd treat publishing a hunch as science as fraud, because you're outright lying about how sure you are.
It is the case that several fields would get (almost entirely) moved from science to hunches, but that would be good! The label will match what's inside.
If we have this split in the product lines, if we have these two tiers (models where we're really sure and complex systems where we only have a working hunch), then we should just own it and put that on the tin.
All I'm asking is people stop labeling composite wood as lumber -- and understandably, a lot of people selling composites act like that would make the sky fall.
Hardly, there's a real demand for both products and both serve a real need. Let's just be honest in our marketing and stop conflating the two.
Just like precision without accuracy, but if you get accuracy you know you have a problem, but with only precision you are hopelessly misled.
If it was a strict trade-off, I would agree that the answer was somewhere on the middle. But it isn't. Most of the things that give you precision will also give you accuracy. The few things that let you exchange one for the other look too much like cheating, and seem to always have much greater impacts on accuracy than on precision.
> Most of the things that give you precision will also give you accuracy.
This is absolutely false - digital clocks are a common example. Precise to the second (or millisecond, or more!) but only as accurate as their setting.
Calculations also give precision without accuracy, like my package of tortilla chips - 13oz (368.5g) They took a measurement or specification precise to the ounce, multiplied by 28.3495, and got 368.5.
Accuracy and precision are independent variables, and the utility of either is usually limited to the order of magnitude of the other. With either "accurate to the second with a precision of milliseconds", or "accurate to the millisecond with precision to the second", you should only report a measurement to the second (or possibly a more precise measurement with an error range).
Do they have different meaning to you? They are the same thing unless there is some scientific meaning that makes them different of which I'm not aware. i.e. Theory in Science means a different thing than what it means to a layman person.
Precision is how fine a measurement is (i.e. being able to measure temperature to thousandths of degrees versus tenths), whereas accuracy is a measure of how closely your results much the objective truth. So you may have a very precise thermometer that measures to thousandths of degrees but it may not be accurate because it's calibrated such that it always reads exactly two degrees higher than it should. Meanwhile a thermometer that can only measure to a tenth of a degree is more accurate and less precise than the one just mentioned if the shown figures reflect the true temperature.
In common usage, these words mean the same thing, but scientists often give words specific meanings in order to make their research and ideas less ambiguous.
Definitions are even more important in non-scientific fields such as philosophy or math where meaningful reasoning of abstract structures and ideas would be next to impossible without giving them concrete definitions and stating your assumptions.
Remember that definitions are arbitrary, so to understand an author's argument or idea, you must seek out the author's definitions.
Hopefully that clarifies some of the discussion going on in the comments here.
I get it, these are specific definitions within a given field. I looked at the definition of both precision [1] and accurate [2] and each would use the other as a synonym, which made it a bit confusing as to what he was talking about.
[1] pre·ci·sion:
noun
the quality, condition, or fact of being exact and accurate.
[2]ac·cu·rate
(of information, measurements, statistics, etc.) correct in all details; exact.
"accurate information about the illness is essential"
synonyms: correct, precise, exact, right, error-free, perfect; More
Standard English dictionaries often don't have the precise definitions used in specific fields, which only leads to more confusion for laymen, but any good introductory Chemistry lab textbook should be able to get you up to speed on experimental science definitions.
