Criteria for the DoD 2400 bps vocoder selection

Use of vocoder competitions remains a prevalent method for selecting a candidate algorithm for low bit rate voice coding applications. Without proper safeguards in the evaluation process, the final selection may be perceived as biased or unfair. This paper describes the evaluation process that was used by DoD for the selection of a new 2400 bps standard. Specific attention was aimed at creating an evaluation environment such that each candidate received equitable treatment. This paper describes the structure of the evaluation team, the roles and responsibilities of its components, the information that was available for evaluation, and the equations and weights that were used to develop an overall figure of merit (FOM) for each candidate