That still results in a physical change to the building, even if it is only temporary (or not, if the building material is unusually porous).
The test that I remember seeing applied in the past concerns "intention distruction or defacement". As I understand it, alterations (graffiti, breaking a window, removing ornamentation, etc.) that alters the physical object or space can apply.
You're using the word in a different sense than the person you're responding to. They're talking about the standard for charging someone for a criminal act.