Releasing publicly AI tech that is six months behind the state of the art, might be a reasonable compromise against the fear of nefarious use. That way, more people know that the tech exists before it's available to everyone, and get a 6-month window to adapt.

At the same time, the risk is lower that a different team develops similar capabilities and widely deploy it without anyone even knowing that what they do is possible.

Wouldn't be surprised if this pattern is followed even more in the future. It's an acknowledgement that there are many angry cats that will (almost unstoppably) be let out of the bag in the next decades, but attempting at releasing the cats in the most responsible manner possible.

> They said within 6 months, however by that time there will likely be something better.

There has. BERT has come out and is better.

Erm, isn't a big part of this whole story that GPT-2 > BERT -> GPT?

BERT is bidirectional. How do you use that for language generation?

It will generate words for every [UNK] in its input sequence.

It'll generate one token because it's trained to predict one missing UNK, as I understood it. What is the scaffolding? Do you generate random sentences and iterate repeatedly? And how does that get you whole coherent paragraphs? (Has anyone demonstrated that this actually works with BERT?)

BERT can pretty easily be used to generate text. It's intended to be used as a base model and fine-tuned with an additional model on top. The fine-tuning model could then be trained to generate sentences with the underlying language model powered by BERT.