Lights-Out Scenario Testing for the New Horizons Autonomous Operations Subsystem

New horizons is a NASA sponsored mission to explore Pluto and its largest moon Charon. The new horizons spacecraft, designed, built and operated by the Johns Hopkins University Applied Physics Laboratory (APL), was successfully launched in January 2006 and will perform its primary mission at Pluto in the summer of 2015. To support this mission, the spacecraft is equipped with onboard software that provides a rule based expert system for performing autonomous fault detection and recovery. This system has been updated nine times since launch and is continuously being tested to ascertain its performance in various spacecraft states. The test approach for the autonomous fault protection subsystem is to perform a combination of unit-level tests and full system scenario tests. For the scenario tests, we have developed a "lights out" test method and have been using it to reduce the time required to run each fault scenario test. This approach reduces the time it takes to develop a test, reduces the number of man hours required to run the test, and decouples the initial spacecraft state from the mechanisms used to inject faults. Decoupling the initial state from the fault injection allows for easy expansion in the number of initial state/fault combinations that can be tested. This significantly improves the test coverage of the scenario test suite. Using this approach, we will be able to run more tests and increase our working knowledge of the performance of the fault protection subsystem. This paper describes the evolution, benefits and cautions of the "lights out" scenario test process.