The question whether milk is beneficial for humans is far more complicated than you suggest. The only thing we know for sure is that consuming milk is not the net positive that it has been portrayed to be.
It's most likely a net zero [1] with a number of positive benefits that you describe, in particular for infants. There are however a number of risks as well, in particular when you define milk as the product that you can buy in an American/European supermarket or if you look at dairy products more generally. And that's not taking into consideration that the majority of the world's population is lactose intolerant. Here is a link to get you started (don't take this source literally for reasons of bias but treat it as a starting point) [2].
I did not even try to approach the question of whether milk is beneficial to health. First we would need to define the question better. Beneficial compared to what? To a dairy free pre agricultural diet? Then it's probably neutral at best. However compared to a calcium deficient diet it's definitely an improvement even if it didn't turn out to be absolutely optimal.
Getting enough calcium in the diet has been linked to healthier BMI and decreased risk of stroke and heart disease. You can get calcium elsewhere but most people don't, or they didn't at the time.
And if you replace milk in the diet you now need to find many other sources of nutrition to make up for lost vitamins and micronutrients which might have been challenging in cooler climates. Maybe milk isn't absolutely optimal for health, the jury is still out on that, but likely it was much better than anything they could get at the time.
Even in the context of a modern diet, replacing milk with soy milk and vegetable milks will yield too much phosphorous and leave you deficient in calcium unless it's fortified (calcium citrate, malate and other kinds of supplemental calcium are linked to worse health, in contrast to dietary calcium which seems to be protective). It can be done by eating a mostly vegetarian diet rich in fruits and leafy greens, or grinding eggshells yourself and adding it to meals but most of the population won't be doing that when eliminating dairy.
It's most likely a net zero [1] with a number of positive benefits that you describe, in particular for infants. There are however a number of risks as well, in particular when you define milk as the product that you can buy in an American/European supermarket or if you look at dairy products more generally. And that's not taking into consideration that the majority of the world's population is lactose intolerant. Here is a link to get you started (don't take this source literally for reasons of bias but treat it as a starting point) [2].
[1] https://www.health.harvard.edu/blog/dairy-health-food-or-hea...
[2] https://www.pcrm.org/good-nutrition/nutrition-information/he...