User Profile Discovery for Web Search

The web has not achieved its goal of providing easy access to online information. As its size is increasing the abandons of available info on the web cause the testing phenomenon of information overload to web users. The system implements an experiential process to approximate semantic likely-hood using page calculations and text fragments retrieved from a web search engine for two words. Specifically, we define various word co-occurrence measures using page counts and integrate those with lexical patterns extracted from text snippets. To identify the numerous semantic relations that exist between two given words, we propose a novel pattern extraction algorithm and a pattern clustering algorithm. The optimal combination of page counts-based co-occurrence measures and lexical pattern clusters is learned using support vector machines. The proposed method outperforms various baselines and previously proposed web-based semantic similarity measures on three benchmark data sets showing a high correlation with human ratings. Moreover, the proposed method significantly improves the accuracy in a community mining task.