Beyond Traditional NLP: A Distributed Solution for Optimizing Chat Processing - Automatic Chat Assessment Using Tagged Latent Semantic Analysis

With the increasing popularity and evolution of Computer Supported Collaborative Learning systems, the need for developing a tool that automatically assesses instant messaging conversations has become imperative. The main reasons are the high volume of data and the increased amount of time spent for manually assessing conversations. We propose an automated analysis system based on Natural Language Processing (centered on Latent Semantic Analysis and Social Network Analysis) and optimize its runtime performance by means of distributed computing. Moreover, we provide a unique grading mechanism based on a multilayered architecture and induce an increase of speedup by deploying a Replicated Worker architecture. Load balancing and fault tolerance represent key aspects of this approach, besides the actual increase in performance.