Designing a big data processing platform for algorithm trading strategy evaluation

Algorithm trading techniques are adopted by institutional and individual investors with the expectation of making profit. Not only the algorithms are getting more complex, but also the data on which algorithms are running is becoming bigger, and various data are exploited to make more accurate prediction. Next generation of algorithm trading will be big data driven. This paper presents the big data processing platform for algorithm trading strategy evaluation, which will facilitate the back testing process so that algorithms could be put into production use with a high degree of confidence. Various data processing techniques should be integrated in a unified system. Some data accessing optimization techniques are also discussed.