Stripe isn't really a middleman. In fact, as they also handle the acquiring process with efficient high-volume rates, generally fewer parties are involved than normal.

Even for very large businesses, actually processing cards directly is pretty difficult and generally not worth doing. Instead you can just get a processor like Stripe or Adyen to give you good rates on an interchange-plus model.