Big data and the web discovering meaningful information from web data using data mining techniques

Summary form only given. With the explosive growth of data available on the World Wide Web (WWW), discovery and analysis of useful information from such source becomes a practical necessity. Big data is primarily concerned with the handling of massive scalable data and discovery of knowledge and aims to provide answers to questions that people do not know how to ask. It is not an automatic process but one that exhaustively explores very large volumes of data to determine otherwise hidden relationships. The process extracts high quality information that can be used to draw conclusions based on relationships or patterns within the data. Web data analysis began as a way for IT administrators to ensure adequate bandwidth and server capacity on their organizations website. Log file data can offer valuable insight into web site usage. It reflects actual usage in natural working condition, compared to the artificial setting of a usability lab. It represents the activity of many users, over potentially long period oftime, compared to a limited number of users for an hour or two each.