Colour Constancy from Both Sides of the Shadow Edge

That image colours depend equally on the colour of surfaces and the colour of the prevailing light means that raw RGB pixel values are not correlates of object features and so, cannot be used directly in applications such as recognition and tracking. By estimating the colour of the light and then removing the colour bias due to illumination, image colours can be made to correlate with object reflectance. Yet, even the best algorithms can fail badly. The problem is made even harder in the many typical scenes where there are multiple lights, e.g. sun (yellow) and shadow (blue). We demonstrate that the second, harder case actually provides the basis of a novel solution strategy: an illumination edge in an image provides a powerful cue to determining the underlying material surface. This paper makes three important technical contributions. First, we take an existing analytic one-surface two-lights estimation algorithm (which assumes that lights lie on a line in chromaticity space) and re-engineer it as a voting algorithm which relaxes the prior assumption about where lights might lie. Second, we model illumination change using a 3 × 3 linear matrix as opposed to the more approximate diagonal model that is used in the prior art for this problem. Being able to more accurately account for illumination change provides a step change in the performance of our method. Lastly, we re-engineer a prior art shadow-edge generator to suit this problem.