One of the biggest issues is revocation, i.e. you can’t. Sure, you can always keep track of JWTs and refuse to accept ones that have been revoked, but congratulations, you’ve just invented session tokens (albeit more complicated).
A slightly more sensible revocation scheme would be to keep a counter attached to the user in the database. For the cost of a single int in your database you can bulk invalidate all the tokens for a specific user: just increment the counter and make sure to not accept tokens with the counter too high or too low.
But yeah at that point you might as well go for session tokens as you need a db request anyways.