The related [0] NYTimes article mentions one pattern for fixes, which was finding gaps between design and execution:
"For the screws that popped off during the shake test, it turned out that the engineering drawings did not specify how much torque needed to be applied. That was left to the contractor, Northrop Grumman, to decide, and they were not tight enough.
“You should have a specification to make sure it’s right,” Mr. Robinson said.
The review board released its report, noting a series of issues, and made 32 recommendations. NASA followed all of them, Mr. Robinson said.
One of the recommendations was performing an audit of the entire spacecraft to identify “embedded problems” — mistakes that occurred without anyone noticing.
The engineers checked the drawings and specifications. They looked at the purchase requests to make sure that what was ordered matched the specifications and that the suppliers provided the correct items.
“There were multiple teams set up, led by the most experienced people,” Mr. Robinson said. “They really dug into the paperwork.”"
I have found it's very easy to scapegoat the "engineers" for situations like this, but in the long term not the cause of the problems nor the actual solution. For example, screws have standard torques published, and for small common diameters is left implicit. The matter is, Northrop's blue collar assemblers didn't particularly care and just installed the screw however, and now the contractor is looking to shift blame. If the drawing did actually say "12 inch pounds" or however, it's still a coin flip if the screw was actually that tight, and the end result of the shake test would be identical.
For an analogy, say the website designers just specified "add a button" and you went and made the button fluorescent yellow with white text. Then the design fails the color contrast test. Will you say it's entirely the designer's fault for not specifying the button color?
Digging into the paperwork to the extent described may have the highest assurance level, but it is extremely expensive. This spacecraft in particular was quite over budget and most likely only able to be carried over the finish line by pure government inertia.
... they are not factory assembly line workers, right?
> over budget
this is the classic problem of too small overall budget. too few launches, not enough money, everything has to be perfect, because everyone wants their pet science/dream project to succeed. so overall efficiency is abysmal.
but the data gained for/through the scientific mission is nice.
> this is the classic problem of too small overall budget. too few launches, not enough money, everything has to be perfect, because everyone wants their pet science/dream project to succeed. so overall efficiency is abysmal. but the data gained for/through the scientific mission is nice.
You've got it 100% correct.
Satellite constellations are much cheaper on a per unit basis than are the bespoke single unit buy. Now, I get above are much harder when NASA is doing a research satellite (low volume or volume of 1), but it is definitely possible for Operational satellites (e.g. what NOAA flies for operational weather requirements and they need to keep running to feed the US meteorological capability).
Delinking satellite sensors from the specific satellite so you always have an excess of sensors you can use, and integrate as needed would also be a huge win for cost avoidance.
When I worked in aerospace using properly calibrated torque tools at the fastener's torque spec is drilled into our heads. How does an oversight like that happen?
"For the screws that popped off during the shake test, it turned out that the engineering drawings did not specify how much torque needed to be applied. That was left to the contractor, Northrop Grumman, to decide, and they were not tight enough.
“You should have a specification to make sure it’s right,” Mr. Robinson said.
The review board released its report, noting a series of issues, and made 32 recommendations. NASA followed all of them, Mr. Robinson said.
One of the recommendations was performing an audit of the entire spacecraft to identify “embedded problems” — mistakes that occurred without anyone noticing.
The engineers checked the drawings and specifications. They looked at the purchase requests to make sure that what was ordered matched the specifications and that the suppliers provided the correct items.
“There were multiple teams set up, led by the most experienced people,” Mr. Robinson said. “They really dug into the paperwork.”"
[0] https://news.ycombinator.com/item?id=32109760