Borges, out of anyone in history, would be the one most against getting rid of your physical books on account of them "taking up room." Anyways, having physical books on a shelf, stacked up on a corner of a room, lying scattered throughout, regardless of whether you will read or re-read them, gives us a healthy reminder of all that we do not know (like the starry heavens above), which is indispensable.
Yeah but I'm not making room for room's sake. my house is not some Scandanavian still life. I have boat loads of tools I need for projects and my wife has baskets of fabric she needs for sewing. It is a matter of choosing between what helps me be productive and grow vs. what is serving as a reminder of where I've been.