That NASA Management Video

I recently became aware of this video on NASA management problems (via zdnet and npr). It was apparently was put together by astronaut Andy Thomas and was acted out by actual NASA employees. According to Hobbyspace it's been known about for a while now.

If the issues acted out in the video are representative of NASA, it raises concerns about NASA's ability to complete large projects on time and on budget that are much more serious than just the usual problems of large organizations.

Specifically, my concern is this: suppose an organization with the management problems outlined in the video takes on a large project. This project is then divided into separate sub-projects with each sub-project corresponding to a subsystem that is specified with a list of requirements strictly based on a set of top level requirements that represent the goals of the system as a whole. The subsystems are then in turn broken down and more requirements are derived for them.

As a simple example, suppose there are 10 subsystems, each with its own project and management team. Each team is working hard to ensure that the subsystem meets its requirements, but no more than that. Because the people in the organization are dedicated and competent, 9 out of 10 projects result in a subsystem that meets the requirements, however one unlucky project team happens to get the subsystem with unknown problems, that couldn't have been predicted at the start of the project. They miss the requirements goals by about 30%. Now, because all the other teams were only trying to meet the requirements, not exceed them, and only managed to exceed them by a few percent at most, there are no easy ways to make up for the loss in performance or capability. It may be possible to ask some of the other teams to do a redesign, but the other teams, not wanting to make it seem that their initial design was flawed, all vouch for their own designs, based on the unarguable fact that their current design meets the requirements.

The only alternative is to push the problem up into the next higher level of integration, where problems with other sub-systems meeting their requirements are starting to accumulate and cause serious performance/cost/performance issues. Eventually, it is found that the entire system design is flawed as measured against the requirements, forcing either costly design changes or reducing the performance requirements to an obtainable level.

Short of deconstructing the entire organization, I can think of two possible mitigation measures that may be taken to protect against the above scenario. The first method is just to ensure that all performance requirements have generous margins. This involves giving the design team a set of performance requirements that are tighter than strictly necessary, with the option of loosening them later on, however it does somewhat corrupt the whole concept of having the requirements fully derived from the top level, and unless communication with the organization is very good, the individual teams may not have a clear idea of how critical the requirement actually is.

The second is to form 'tiger teams' that actively go out and search for bad designs, and correct them. I suppose this could be quite effective, and could also be used as a formidable 'stick' to threaten low level managers with if their team's design is suboptimal, given the ego bruising that would occur if one's own design were overhauled. However, I doubt that tiger teams are often used this way, although I could be wrong. And even if it were the case, a manager might just think it easier to weather the storm, when the tiger team members come knocking on his/her door, judging the risk of being 'caught' with a poor design preferable to a pre-emptive design change making it look like the team got it wrong from the start.

In any case this approach could have the unwanted, serious side effect of fostering a herd mentality amongst the staff, who learn never to stand out from the crowd for fear of being picked out by a bunch of domain experts hunting for a reason to justify their jobs. If done badly, a tiger team approach could be disastrous for employee morale.

Ultimately the best solution is 'organizational redundancy', by which I mean utilizing multiple organizations with differing management cultures and design methodologies, and allocating work strictly on the basis of past performance. Clark Lindsey commented in his blog entry about this video, saying "The only way I see NASA overcoming this problem is to move product development out to commercial companies and letting them compete to build the products". If this were the case, the US taxpayer could have a lot more confidence that their money is being used effectively, and the USA would have a much more vibrant and viable space program.

Unfortunately, given the political realities, an outsourced space program seems unlikely, in which case we can only hope that either the problems shown in the video are exaggerated, or that somehow NASA can reform itself. If the video is being used at management retreats, at least they have gotten to the stage of acknowledging the problem. The next stage is treatment, but I guess we won't know the results until the next big government mandated project. Personally I don't think the US taxpayer will want to wait that long.

Will Ares I and V be NASA's last launch vehicles?