Identification of Core, Semi-Core and Redundant Attributes of a Dataset

Data reduction is an essential step in pre-processing of a dataset and it is necessary for improving data quality and obtaining the relevant data from the dataset. Data reduction is performed by identifying and removing redundant attributes of the dataset. However, every non-redundant attribute does not have the same level of contribution to the decision (dependent variable). Therefore, the non-redundant attributes may be further divided into two sub-categories of core (attributes that totally contribute to the decision) and semi-core (attributes that partially contribute to the decision) attributes. In this paper, a methodology for separating core, semi-core, and redundant attributes is introduced and tested. The result shows that the proposed methodology has a high potential for use in any generalization process.