And some things are impossible even with both the dataset and weights. Say you wanted to train the same model as is released, using Meta's hypothetically released training data. You also need to know the starting parameters, the specific hardware and it's quirks during training, the order the data is trained in as well as any other preprocessing techniques used to treat the text.

Considering how ludicrously expensive it would be to even attempt a ground-up retrain (as well as how it might be impossible), weights are enough for 99% of people.