My brother and I got the first two grey castles in the 1980s (the first castle yellow; grey was a fairly new colour back then). Those castles mainly relied on fairly large castle wall pieces. At first, even as a kid, I felt that was cheating; we should have gotten bricks to assemble all those walls. Later I discovered what a boon they were, because they allowed me to rapidly design new and different castles. Sure, I couldn't build a spaceship out of them, but just exploring the space of all the possible castles you could build was amazing.
Some of these specialised pieces inspire creativity. Especially with spaceships.
But I do agree that many modern sets just invite you to build this one thing out of them. We did actually turn out Millennium Falcon into an X-Wing and a TIE-fighter once, but we did eventually turn them back into the Falcon. It feels more wrong than it used to to build something else.
I don't think we ever had the yellow castle, although I do remember that flag. And we definitely had a set with a horse in the style of the yellow castle. So either we did have the yellow castle but lost the instructions, or I knew someone who had it, or that flag also appears in another set.
But I really have no idea. It was a long time ago.
The reason those flags are so rare is because the grip of the 1x1 round brick at the bottom is much stronger than the strength of the flagpole so if you would remove it by pulling on the flag you'd break it.
So these commend ridiculous prices for small pieces of plastic (about 8 euros or so last I checked).
Some of these specialised pieces inspire creativity. Especially with spaceships.
But I do agree that many modern sets just invite you to build this one thing out of them. We did actually turn out Millennium Falcon into an X-Wing and a TIE-fighter once, but we did eventually turn them back into the Falcon. It feels more wrong than it used to to build something else.