Measuring the Semantic Value of Folksonomies

Semantic metadata, which describes the meaning of documents, can be produced either manually or else semi-automatically using information extraction techniques. Manual techniques are expensive if they rely on skilled cataloguers, but a possible alternative is to make use of community produced annotations such as those collected in folksonomies. This paper reports on an experiment that we carried out to validate the assumption that folksonomies carry more semantic value than keywords extracted by machines. The experiment has been carried-out in two ways: automatically, by measuring the percentage of overlap between the folksonomy set and machine generated keywords set; and subjectively, by asking a human indexer to evaluate the quality of the generated keywords from both systems. The result of the experiment can be considered as evidence for the rich semantics of folksonomies, demonstrating that folksonomies used in the delicious bookmarking service can be used in the process of generating semantic metadata to annotate Web resources