High-Speed Classification of Financial Network Public Opinion Based on Hadoop

Financial network public opinion refers to those influential tendentious views and comments, spread through the network, on economic issues in real life. It reflects the economic focus in the corresponding period. To learn about the dynamics of financial market, processing and analysis of those opinions is needed. However, in the face of vast amounts of data, of which the analysis requires a great mount of storage and computations, standalone exposes the shortcoming of low storage capacity and slow processing speed problems. In this paper, the Hadoop distributed platform is applied to deal with this situation. Moreover, this article has presented the application of MapReduce programming model in key processes of public opinion text processing, including preprocessing, feature selection and vectorization of text. Experiments have showed that the Hadoop cluster well solved the disadvantages of low storage capacity and low-performance in computation.