Keyword extraction for web news documents based on LM-BP neural network

In view of the actual demand, the paper provides a new idea on keyword extraction for web news documents by adopting the improved LM algorithm based on BP artificial neural network. First, preprocess the web news documents which are of consistent HTML format. The preprocessed work includes noise filter, web content extraction, word segmentation, POS tagging, stop words removal, etc. Also, select effective features like TF, location of words based on the characteristics of news documents. Then the selected features will be considered in training and constructing the BP neural network. Finally, extract keywords with LM algorithm which has parameters adjustment and solves training too long and getting stuck in local minimum of BP so that improve network convergence speed and keyword classification performance. The results show that LM algorithm has better effect and convergence performance comparing with BP in the field of keyword extraction.