Graphics add more than nothing. At a minimum, they are an important part of UI, just like on a web page. Colors, affordances, etc. are all important to games. Games like ASCII roguelikes exchange obvious visible indicators for speed/flexibility/pixel size/complexity. Graphics set the tone of the game and have the ability to visually influence the player.
Cramming more graphic textures or perfecting water algorithms might be secondary (or tertiary) to the game, but they can still influence the gameplay directly. Take Team Fortress 2, where a lot of care was put into making each character class have a distinctive silhouette, so at a glance you can see who everyone is from a distance or from different angles - this isn't what most people talk about when they discuss "graphics," but it is the element of graphics that matters most.
Cramming more graphic textures or perfecting water algorithms might be secondary (or tertiary) to the game, but they can still influence the gameplay directly. Take Team Fortress 2, where a lot of care was put into making each character class have a distinctive silhouette, so at a glance you can see who everyone is from a distance or from different angles - this isn't what most people talk about when they discuss "graphics," but it is the element of graphics that matters most.