Using Signals from Text to Identify Roles within a Group

This work investigates identifying social behaviors (adversarial behavior and influence) of participants in online discussion forums from how their language use in English, Arabic, and Chinese. We describe the challenges of annotating implicit information signaled by subtle queues and present two styles of annotation -- one using professional annotators and the other with Mechanical Turk. Our system, predicts confidence by identifying the most salient instances of the social behavior. We show that by selecting only the most salient instances of a behavior we are able to improve system precision. We also explore the impact of different types of features.