Visualization: detecting societal behaviors

An interesting area of study has been developing technologies to assist knowledge discovery with the human in the loop. It has been our experience that humans are adept at discovering patterns and anomalies, while automated systems are preferred for managing the found anomalies. It has also been our view that tools intrinsically limit the questions that can be asked, and therefore limit what can be understood about data. A particular niche we have investigated is the detection of societal behaviors, or ripples, in response to external events. The goal is to detect ripples as surrogates for the underlying event. A societal ripple is a change in personal activities or transactions. These ripples may be found in many data streams including retail transactions and communications. We have developed a data fusion, analysis and visualization prototype internally named SWIFT. This technology provides a data agnostic platform that can fuse multiple real-time data streams, provide connectivity to analysis capabilities and view current and historical animations of selected data dimensions. By applying this technology we can observe how society reacts to certain events and how behaviors differ between various events. By enabling interactive investigation of anomalies with a 100% sampling of data, we have discovered many hidden anomalies that are beneath the "radar screen"