Automatically classifying and interpreting polar datasets with Apache Tika

The Arctic and Antarctic are undergoing rapid change attributed to the Earth\´s changing climate. This change is captured via space and airborne remote sensing, in-situ measurement, and climate modeling. Those observations and simulations record data in myriad formats across a number of pertinent data archives funded by NSF, NASA, NOAA, and other federal agencies. Simply finding data may be hard, but we restrict our focus in this paper to the subject of what to do with the data (and metadata) once it is found - the "complexity" portion of the Big Data challenge. We present our current efforts for dealing with the complexity and heterogeneity of Arctic and Antarctic data - Apache Tika. Apache Tika is an open source framework for metadata exploration, automatic text mining, and information retrieval of 1200 of the most widely used data file formats and 20 rich metadata models to go along with those formats. Our current research efforts are targeted at expanding Apache Tika to parse, extract, and analyze common data formats used in Artie and Antarctic research making them more easily accessible, searchable, and retrievable by all major content management systems (Plone, Drupal, Alfresco, etc.). Furthermore, expanding Tika to handle common Polar data formats will also naturally invite the technology/open source community to deal with Polar use cases, helping to draw attention to and increase understanding of these remote regions.