It's important to consider timeliness in these things.

On the very short timeline: messages like "attack at dawn" need only be secret til dawn, at which point the information becomes known. Any method, even if utterly broken but that takes 2 days to decipher is fine for such a thing. Many things do not need "forever" level security.

On the medium - even if they had told the Japanese that they were using the Navajo language (in this case, other American languages in other cases), it is reportedly very difficult to learn, so if we take that at face value - it would still provide a long buffer before it became insecure. The time to train up enough people to have the messages useful on the battle coordination timelines is still long. The extra bit of making the Japanese figure out what the heck the language was only added to the this timeline. Security by obscurity as a portion, not the key, to security is not a bad thing.

On the very long timeline - this is totally weak. But... most of those messages probably weren't recorded anyway, it's a very modern mindset to even consider the possibility - these voice transmissions are truly ephemeral, there just wasn't the recording technology at the time for the scale needed. Besides, there were other ciphers for textual communications. Even if there was the recording technology, it would aid in the Japanese figuring out that it was navajo, and perhaps aid in training up the people to decipher it faster, but see above.

So basically if they had kept the use of these guys' Navajo language transmissions to things that had a "keep secret" timeline that was relatively short, it was pretty secure for that use.

There are lots of interesting meta analyses in this sort of thing, but that is true no matter what the cipher.

Had some neat properties at the time though, like speeding up coded communications, and: "a speaker who has acquired a language during their childhood sounds distinctly different from a person who acquired the same language in later life, thus reducing the chance of successful impostors sending false messages" https://en.wikipedia.org/wiki/Navajo_code_talkers#Cryptograp...

Randomly, how would you know, in general?

You Sir, are correct. The only time I ever hear about "security through obscurity" is when it fails, not when it succeeds. I got so used to the open source model where everything is known and flaws are more likely to be fixed. I guess that doesn't work for every application.

I have to say I almost came to write the same thing, and then the above comment got me thinking. I guess buried treasure is amongst the most successful usually... In that it keeps it away from those the owner was hiding it from for many decades or more, before accidentally getting 'cracked': http://www.telegraph.co.uk/news/worldnews/northamerica/usa/1...