It's perfectly possible, and not that insane. Much more possible than most people likely think. The primary pain point is mostly web browsing - for a long time Elinks was the only halfway-viable web browser, and it has a lot of pain points. But maybe brow.sh is better these days - I've never tried it (although, arguably, running Firefox through a shoddy GUI inside a terminal doesn't really count as a "GUI cleanse".)

It's worth doing, purely for the enlightenment of freeing you from the busted, "macho" notion that a GUI is inferior to a command line. GUIs are awesome.