Visual and Textual Feature Fusion for Automatic Customs Tariff Classification

The Harmonized Tariff Schedule for the classification of goods is a major determinant of customs duties and taxes. The basic HS Code is 6 digits long but can be extended according to the needs of the countries such as application of custom duties based on details of the product. Finding the correct, consistent, legally defensible HS Code is at the heart of Import Compliance. However finding the best code can be complicated, especially in the case of specialized products. In this paper, we propose an automatic HS code detection system based on visual properties of the product together with textual analysis of its labels/explanations. The proposed system first uses morphological parsing in order to extract roots of the words occurring in the textual phrases. Processed text information is further processed by the topic modeling module of the system to find the best matching HS Code definitions within the system. The result of the topic modeling is used to trigger visual search based on quantized local features. The proposed algorithm is evaluated using a database of 4494 Binding Tariffs published in 2014 by the European Union. The results show that accuracy rate above 80 % can be achieved for 4-digit HS Codes.