Staying in this analogy, aren't APIs more like pictures or verbal descriptions of the facade of the building?
An API spec will not (necessarily) provide you with any internal implementation detail. Architectural diagrams/design specifications very likely will do.
To me an API is more like a survey of the land. If the architect doesn’t know the survey, then the plans might not be compatible with the features of the land... but he could still try. The builder also doesn’t need the surveyors permission to build on the land, the survey is just a description of the land. Owning the survey / API doesn’t do anything to change anyone’s rights.
An API spec will not (necessarily) provide you with any internal implementation detail. Architectural diagrams/design specifications very likely will do.