We're actually in between drag and drop libs at the moment. We currently use react-beautiful-dnd [0], but it has some major shortcomings and doesn't feel very well maintained. That being said, it's a pretty impressive lib that probably works for a lot of people as-is.
We've got a branch where we've switched to dndkit [1] which I like a lot - really nice API with hooks for everything. However, it has some annoying bugs that mean we can't quite switch yet. That being said it seems like the one that's hanging us up the most is nearly fixed [2].
You have so many layout shifts from just clicking on one element. I imagine, implementing react-beautiful-dnd will give you much better performance results.