FCompress: a new technique for queriable compression of facts and datacubes

Decision support applications must analyze information from data warehouses efficiently. For this reason, huge data warehouses must have mechanisms to cope with massive amounts of data. Reducing and compressing fact tables, summary tables and data cubes is important for faster operation and smaller storage overhead. Traditional compression techniques are not useful in this context except for archiving, because they render the data unqueriable. Although data reduction techniques are useful for fast approximate answers to complex queries, their accuracy is not enough to replace the base data. We present FCompress, a new fact compression technique that effectively replaces the base data, compressing it while maintaining queriability. The approach is based on the premise that a very small and adjustable error is acceptable in many fact attributes. The technique is applicable to fact and summary tables and data cubes alike. It has been evaluated, showing that very small errors can be achieved for point reconstruction (typically below 2%) while the original fact table is reduced to about 35% to 60% of its size and the data cube is reduced to about 15% to 30% of the size. The error is even smaller for typical OLAP queries, usually less than 1%, depending on the degree of aggregation