For ISPs there simply wasn't enough customer usage of NNTP servers to justify their continued existance. 5 years ago when I was working at a mid-sized ISP only about 2% of our customers used our NNTP servers. We carried binary groups and offered pretty good retention/completion but by then even the pirates had mostly ditched NNTP for torrents. At the time we estimated that we had maybe about a dozen customers accessing the server for non-binary / piracy use.
Going back further to why NNTP became irrelevant for discussion I'd say it was a combination of difficult setup for the average user and the lack of good free NNTP clients. Early web forums could offer discussion for free without the difficulty / expense of a NNTP client. As NNTP groups became more insular the miserable trolls were able to take over and ruin it for everyone. Almost every group I was active in during the late 90s deteriorated in this way. Just one mentally ill and/or very lonely person posting 50+ times per day could very effectively destroy a group.
Going back further to why NNTP became irrelevant for discussion I'd say it was a combination of difficult setup for the average user and the lack of good free NNTP clients. Early web forums could offer discussion for free without the difficulty / expense of a NNTP client. As NNTP groups became more insular the miserable trolls were able to take over and ruin it for everyone. Almost every group I was active in during the late 90s deteriorated in this way. Just one mentally ill and/or very lonely person posting 50+ times per day could very effectively destroy a group.