Auditing a Database under Retention Restrictions

Auditing the changes to a database is critical for identifying malicious behavior, maintaining data quality, and improving system performance. But an accurate audit log is a historical record of the past that can also pose a serious threat to privacy. Policies which limit data retention conflict with the goal of accurate auditing, and data owners have to carefully balance the need for policy compliance with the goal of accurate auditing. In this paper, we provide a framework for auditing the changes to a database system while respecting data retention policies. Our framework includes a historical data model that supports flexible audit queries, along with a language for retention policies that hide individual attribute values or remove entire tuples from history. Under retention policies, the audit history is partially incomplete. We formalize the meaning of audit queries on the protected history, which can include imprecise results. We implement policy application and query answering efficiently in a standard relational system, and characterize (both theoretically and experimentally) the cases where accurate auditing can be achieved under retention restrictions.