It is very easy to accidently use a GPL library without knowing it, especially if it is a dependency of a dependency and you aren't using a license scanner.
It’s very easy to do all sorts of incorrect stuff if you totally ignore your responsibilities.
If you're not redistributing the GPL library, then it doesn't matter. If you are, then there are all sorts of other licenses which come with the same (or greater) headaches.