Machine vision for condition monitoring vegetation on railway embankments

National Railway Administrations in Northern Europe do not employ systematic procedures in monitoring the current state of vegetation to form the basis of maintenance decision making. Current day vegetation maintenance is largely based on human visual estimates. This paper investigates a machine vision (MV) approach to be able to automatically quantify the amount of vegetation on a given railway section. An investigation assessing the reliability of human estimates is also conducted along the same railway section.A machine vision algorithm was developed and implemented. Initially, the algorithm determines a region of interest (ROI), i.e. the desired monitored area in each collected image. This ROI is dependent on fixed objects in the image, namely the two rails. When the rails are found the algorithm will compute the ROI, which is predetermined by e.g. the railway administrator. After this, a perspective projection correction will be made, and the vegetation will be segmented. Cover is reported as a percentage of the total ROI for each image. Results: The machine vision algorithm is capable of processing 98% of the images. Failure in the remaining 2% of cases is attributed to the algorithms´ inability in find the rails within the image. Analysis of variance tests were conducted to compare the observers cover assessments in sample plots. Upon comparing the observers plot wise mean estimates with the machine vision output, results show that the human visual estimates do not correlate with the results reported by the machine vision output. As such, the result indicates that it is very hard to fit human estimates by regression with the machine vision result. Additionally the results show that humans are not in agreement with each other, and often are exaggerating the extent of vegetation cover compared to the machine vision output.The investigation shows that one should be very careful when trusting/interpreting human visual estimates. In conclusion, based on the resu- ts, the automated machine vision solution is proposed as complementing, or replacing, manual human inspections serving as a base for vegetation control decisions. Impact: By objectively measuring the quantity of vegetation, the maintenance planning and procurement can be effectively improved over time. A machine vision approach for condition monitoring of vegetation will enable condition based maintenance with prior consideration on issues mainly relevant to vegetation type, quantity and biodiversity.