Maybe you think you saw a black swan around the orbit of Jupiter with your new fancy telescope, but later you figure out that the telescope was falty and the swan you saw was actually white.
Because science depends on the external world, and there might always be hidden variables that we are not considering, it can only give us extremely high confidence after we make repeated observations using different methods, but it cannot give us the absolute confidence that Math can give us.
Also, this example of faulty telescopes is actually really close to what actually happened in the history of astronomy. Stars were thought to have a visible radius that made them appear to be much larger than the Sun, which was considered evidence that the Sun was not a regular star.
Here's an interesting thing. Even high confidence can't be verified.
Let's say you examined 10 million swans and you think you observed all possible swans. But there's no way you can know whether or not the actual population is 10 billion trillion swans or a google swans.
If you observed 10 million swans and they are all white but those swans could only represent 1/99999999999999 of all possible swans. Then that means your observation is low confidence and there's no way whether we can verify what fraction of the population our sample size represents.
So actually high confidence is just an assumption. At a very technical level the confidence that science brings to the table is very very weak. We are making tons of assumptions and jumping to conclusions all the time.
Because science depends on the external world, and there might always be hidden variables that we are not considering, it can only give us extremely high confidence after we make repeated observations using different methods, but it cannot give us the absolute confidence that Math can give us.
Also, this example of faulty telescopes is actually really close to what actually happened in the history of astronomy. Stars were thought to have a visible radius that made them appear to be much larger than the Sun, which was considered evidence that the Sun was not a regular star.