Compression as Data Transformation

Summary form only given. Conventional compression techniques exploit general redundancy features in data to compress them. For example, Huffman or Lempel-Ziv techniques compresses data by statistical modeling or string matching while the Burrows-Wheeler Transform simply sorts data by context to improve compressibility. On the other hand, data can often be compressed better by exploiting their specific features. For example, columns or fields in a database table tend to be sparse, but not rows. Techniques have been developed to either group related table columns or compute dependency among them to transform data and enhance compressibility. The Vcodex data transformation platform provides a framework to develop and use such data transforms. That is, it treats compression techniques as invertible data transforms that can be composed together for specific tasks. In this way, data transformation remains general and can include techniques for encryption and others.