Those large creatures added up to a tiny fraction of the ecosystem - almost a negligible portion. They're visible and interesting - fuzzy and dangerous etc - but of no significance to most of life on earth. Like insects, birds, worms, bacteria etc. Not to mention grasses, flowering plants and algae. No, most of life on earth never noticed the extinction of apex predators ( nor their invention either ).
This played out in the Pleistocene extinctions to a very significant degree, having tremendous effect on ecosystems everywhere. Here's an entire conference worth of the subject:
