Cmd-tab, and then while holding command, use up/down arrow to drill into the windows for a specific app. I realize that this is not great if you need to switch back and forth between the same two windows over and over again.

not true: ⌘ Command + `

This only switches between windows of the same app. On Windows, with Alt+Tab you switch between all of the currently open windows of all applications.

Nope, that doesn't do it. If I have 2 Chrome windows and 3 terminal windows and 2 text editor windows open on MacOS, there is no way to merely switch from a Chrome window to a specific terminal window. You have to switch to the terminal application, which raises all 3 terminal windows to the top of the window stack on whichever display they are on, then switch between terminal windows. To enable the basic feature of switching directly between windows, you have to install 3rd party software.

I get with keyboard only to a specific window without any 3rd party app this way... I think the question is solved in a way (not saying it's the best way)

Not quite. That will switch between windows of a single app.

The AltTab thing here will switch between all windows of all apps.

I actually like having them separate like OSX does it, even though I'm generally a Windows guy.

I don't think this affects minimized app which is another annoyance