I think it must be using the building outline data (and metadata like the heights) that people have entered, because in my city the inner-city has very complete data but a bit further out, around where I live there aren't any building outlines on OSM and there's nothing in this 3D map apart from just the streets.

The result is very impressive given that seems to be the case!

Edit: There are some more details on the github about data sources: https://github.com/StrandedKitty/streets-gl