I have a box with 1000 tubes, exactly 1% of which are beyond my capacity to measure.
I take a random sample of 20, and call that a roughly 80% chance of being able too measure all of them.
Let's say the average is 50V and the ones beyond my measurement average 150V. We can use this info to solve for the average of the tubes we can measure: 25V.
So, in 80% of the cases we'll get an average measurement of 25V.
In 20% of the cases we're going to realize our mistake, buy a better voltmeter, and redo everything. In this case we can measure everything so we get the correct number: 50V.
25V * 0.8 + 50 * 0.2 = 30V
This, of course, won't be statistically significant. But let's get a lot of researchers together to measure a lot of tubes. And let's say 100V limit meters are common so half make the same mistake.
Now we have a statistically significant 40V. Wrong, wrong wrong.
I have a box with 1000 tubes, exactly 1% of which are beyond my capacity to measure.
I take a random sample of 20, and call that a roughly 80% chance of being able too measure all of them.
Let's say the average is 50V and the ones beyond my measurement average 150V. We can use this info to solve for the average of the tubes we can measure: 25V.
So, in 80% of the cases we'll get an average measurement of 25V.
In 20% of the cases we're going to realize our mistake, buy a better voltmeter, and redo everything. In this case we can measure everything so we get the correct number: 50V.
25V * 0.8 + 50 * 0.2 = 30V
This, of course, won't be statistically significant. But let's get a lot of researchers together to measure a lot of tubes. And let's say 100V limit meters are common so half make the same mistake.
Now we have a statistically significant 40V. Wrong, wrong wrong.