On the Convexity of the Moment Mapping for Unitary Highest Weight Representations

To each continuous unitary representation of a Lie group G on a Hilbert space H we associate a moment map from the projective space of smooth vectors to the dual g* of the Lie algebra of G. For unitary highest weight representations we obtain a characterization of those highest weights for which the closure of the image of this map is convex, i.e., equal to the convex hull of the highest weight orbit. This result generalizes the corresponding result for representations of compact groups and holomorphic discrete series representations.