Boeing CST-100 Starliner Testing Deficiencies Identified
The crew module of Boeing’s CST-100 Starliner spacecraft is lifted onto its service module on Oct. 16, 2019, inside the Commercial Crew and Cargo Processing Facility (C3PF) at Kennedy Space Center in Florida ahead of the company’s Orbital Flight Test to the International Space Station as part of NASA’s Commercial Crew Program. Photo: Boeing
Details are emerging about exactly what and how testing was performed prior to Boeing launching their Starliner test capsule on December 20, 2019, a flight which resulted in a mixed outcome instead of what was supposed to be a resounding and highly anticipated success.
Following the in-flight anomalies which caused the CST-100 Starliner to miss its intended rendezvous at the targeted destination, the International Space Station, a joint investigation team (JIT) was put together by Boeing and NASA to try and identify the root cause and contributing elements to the problems. From that, an unfortunate realization was made – that more than one problem occurred during the necessarily shortened flight, and that by not performing a full end to end integration test prior to launch, certain elements of the associated software were not tested adequately.
Coding in the critical software programs, which would normally guide the Starliner through its paces along the mission timeline, were found to have contained significant errors. With a greater understanding of what exactly happened during the flight, the JIT has released a preliminary report which outlines the initial findings, and how what they found could have had catastrophic consequences.
Among the concerns identified:
A: There was an error with the Mission Elapsed Timer (MET), which incorrectly polled time from the Atlas V booster nearly 11 hours prior to launch,
B: A software issue was found within the Service Module (SM) Disposal Sequence, which proceeded to incorrectly translate the SM disposal sequence into the SM Integrated Propulsion Controller (IPC), and
C: A deficiency was identified with the Intermittent Space-to-Ground (S/G) forward link, which became a critical issue when it impeded the Flight Control team’s ability to command and control the vehicle as the flight progressed.
The Boeing CST-100 Starliner lifted off from Cape Canaveral on December 20, 2019 at 6:36 am EDT (11:36 am UTC), atop the ULA ATLAS V N22 which is a crewed rated version of ULA’s workhorse rocket the ATLAS V.
The JIT, which met in the first half of January, identified the possible direct causes and preliminary corrective actions for the first two anomalies, which will hopefully fix these critical issues. The third element, the off nominal intermittent communications issues, are still under inquiry.
NASA has requested that the team perform a major in-depth analysis as to why these anomalies occurred in the first place, including but not limited to an analysis of whether these issues were indicative of faulty coding in the software processes or if there was failure in applying those processes prior to (and at critical intervals in) the mission timeline. The team is currently in the process of performing these additional points of testing and research, as well as continuing with the investigation of the intermittent communications issues which were identified.
After having reviewed these findings, as well as considering another 61 other identified issues that require correction, NASA is seeking for preventive processes to be put in place so that the same anomalies can not occur again in the future. Points where structured into four separate categories to help Boeing team manage and execute the upcoming work. They are, as taken from NASA Report:
- Perform code modifications: Boeing will review and correct the coding for the mission elapsed timer and service module disposal burn,
- Improve focused systems engineering: Boeing will strengthen its review process, including better peer and control board reviews, and improve its software process training,
- Improve software testing: Boeing will increase the fidelity in the testing of its software during all phases of flight. This includes improved end-to-end testing with the simulations, or emulators, similar enough to the actual flight system to adequately uncover issues, and
- Ensure product integrity: Boeing will check and recheck its software coding as hardware design changes are implemented into its system.
NASA has informed the Aerospace Safety Advisory Panel on the current status of the ongoing investigation. In arriving at this point in the investigation and with the technical points laid out above, the review team identified serious organizational issues which contributed to these anomalies. Boeing’s response has been that they plan to institutionalize improvements in their engineering board authority, improve their operational testing practices for both hardware and software, and standardize problem review and approval processes .
At the time of writing, it is still unclear whether NASA will require Boeing to provide a replacement Test flight of an un-crewed CST-100 Starliner capsule, to both ease the minds of worried officials and to instill greater confidence in Boeing, as they try and demonstrate that they can indeed get the job done as requested. Only time will tell.
Michael McCabe
A native of Lonedell, Missouri, Michael McCabe is a former Long Island firefighter and emergency medical technician. He is a non-active Florida EMT with 20 years of fire rescue experience. He is also a lifelong science fiction and space enthusiast. At the age of 10, he watched in his school classroom as the Space Shuttle Challenger exploded in 1986. In 2008, he moved to the Sunshine State and works as a private tour guide at Kennedy Space Center Visitor Complex for a private company based in Orlando. McCabe has been a fan of SpaceFlight Insider since our inception in 2013. He reached out to ask how he could assist our efforts to spread space flight awareness. Shortly thereafter, he was welcomed into our expanding team.
