That's mainly because veggies have a low calorie count in general. No one's going to get a majority of their calories from fruits and veggies alone. That's pretty prohibitive for most people.
Note: I'm not counting potatoes, rice, and other carb-rich foods here as veggies.
The overwhelming majority does not have a hard time getting sufficient daily calories, but fruits and veggies are still expensive enough that many people probably don't find it affordable to buy a lot of them.
Affordable is a relative term. Think of it this way: at Sprouts where I usually shop, which is a middle-priced grocery chain here in Colorado, you can buy a pound of chicken breast for $2 on sale. You make some rice to go with it, which is at most 25 cents, plus some spices from your cabinet. So for $2.25 you can have a meal that'll satisfy you. Now to add vegetables, you can throw in a bell pepper ($1 on sale), and 4oz of spinach for $1.5 . So adding vegetables just made your meal twice as expensive and added practically no calories, so you can't save half of what you cooked as leftovers.
Your own comment agreed that calories aren't usually the critical problem, though. Fruit and vegetables are generally associated with good mental function because of the micro- / phyto-nutrient content. You can eat all the grain and meat you can afford and still be missing essential nutrients.