Abstract :
For 2⩽m⩽l/2, let G be a simply connected Lie group with g0=so(2m,2l−2m) as Lie algebra, let g=k⊕p be the complexification of the usual Cartan decomposition, let K be the analytic subgroup with Lie algebra k∩g0, and let U(g) be the universal enveloping algebra of g. This work examines the unitarity and K spectrum of representations in the “analytic continuation” of discrete series of G, relating these properties to orbits in the nilpotent radical of a certain parabolic subalgebra of g.
The roots with respect to the usual compact Cartan subalgebra are all ±ei±ej with 1⩽i2l−2, it is known that πs=πs′ and that πs′ is in the discrete series. Enright, Parthsarathy, Wallach, and Wolf showed for m⩽s⩽2l−2 that πs=πs′ and that πs′ is still unitary. The present paper shows that πs′ is unitary for 0⩽s⩽m−1 even though πs≠πs′, and it relates the K spectrum of the representations πs′ to the representation of LC on a suitable R(Y) with Y depending on s. Use of a branching formula of D. E. Littlewood allows one to obtain an explicit multiplicity formula for each K type in πs′; the variety Y is indispensable in the proof. The chief tools involved are an idea of B. Gross and Wallach, a geometric interpretation of Littlewoodʹs theorem, and some estimates of norms.
It is shown further that the natural invariant Hermitian form on πs′ does not make πs′ unitary for s<0 and that the K spectrum of πs′ in these cases is not related in the above way to the representation of LC on any R(Y).
A final section of the paper treats in similar fashion the simply connected Lie group with Lie algebra g0=so(2m,2l−2m+1), 2⩽m⩽l/2.
