Hierarchical task-driven feature learning for tumor histology

Through learning small and large-scale image features, we can capture the local and architectural structure of tumor tissue from histology images. This is done by learning a hierarchy of dictionaries using sparse coding, where each level captures progressively larger scale and more abstract properties. By optimizing the dictionaries further using class labels, discriminating properties of classes that are not easily visually distinguishable to pathologists are captured. We explore this hierarchical and task-driven model in classifying malignant melanoma and the genetic subtype of breast tumors from histology images. We also show how interpreting our model through visualizations can provide insight to pathologists.