Nothing in the letters even hints that it played a part of the decision-making process for hiring or firing. The husband also mentioned his hair was different and he now wears glasses. Does that mean corrective vision also played a part in the process?
If the manager mentioned it, then it very well could have. Even if just on a subconscious level. I don't see anything that shows an audit of their thought processes, so you can't rule that out.