Algorithms for filtering of market price data

The availability of market price data (MPD) is extensive and can be obtained through continuous electronic based real-time data feeds or through on-line links to historical data suppliers. Whatever the form of access and independent of the supplier, all MPD is subject to the possibility of embedded errors: whether due to transcription and input error, electronic transmission errors or simple transpositions such as evidenced when ask prices are less than the corresponding bid prices for a specific time point. The concern which arises with errors embedded in data are to do with the propagation which results when the unfiltered (or undetected) bad data point(s) is used in pricing, risk measures or any other quantitative data analysis. The authors are not strictly concerned with obvious data errors as outliers in time series. Indeed, most outlier detection, as is shown, is not challenging. Their interest is with demonstrating a set of data filtering and data cleaning analytic functions and their rational assembly in terms of generic computer algorithms which are not dependent on the market data type. They consider the attributes of: data frequency, data relationships and plausibility (e.g., bid is always less than ask), date and time sequences, negative rates, decimal point adjustment, bid price filtering, bid-ask spreads, ask price filtering and, of course, jumps and spike detection