It works fine until there's a high-profile, close election. Then you'll discover the data source was noisier than you thought. We're only talking about one in a thousand people goofing up a vote and moving on.
> Some of those challenged ballots in San Mateo County had illegible or no postmark dates and Assistant Chief Elections Officer Jim Irizarry said they were waiting on information from the U.S. Postal Service
With small margins, an election could come down to how people read a smudged postmark. It might not be an issue, but you won't know who "really" won, either. It depends how you count it.