I agree with you 100%. Trouble brews when people are illiterate, but unless you're pursuing a literary career, there isn't too much use for reading fiction. Nonfiction has its own benefits that attract anyone who wants to learn in general, or has complex goals.
Remember the phrase 'bookworm'? It wasn't always a cute alternative to the word nerd. I know about it, because I wasted years of my life reading crap fantasy novels, hardy boys casefiles, etc.
You mentioned an interesting point. Is the time spent on reading crap books really wasted? I wouldn't say so. It helps telling good books apart from bad ones. Reading wide variety of books makes one's taste more sophisticated. Also, mere familiarity with some books can be helpful in some social settings, when we want or need to make friends with people who read those books too.
Remember the phrase 'bookworm'? It wasn't always a cute alternative to the word nerd. I know about it, because I wasted years of my life reading crap fantasy novels, hardy boys casefiles, etc.