Invariant theory for singular α-determinants

From the irreducible decompositionsʹ point of view, the structure of the cyclic GL n ( C ) -module generated by the α-determinant degenerates when α = ± 1 k ( 1 ⩽ k ⩽ n − 1 ) (see [S. Matsumoto, M. Wakayama, Alpha-determinant cyclic modules of gl n ( C ) , J. Lie Theory 16 (2006) 393–405]). In this paper, we show that − 1 k -determinant shares similar properties which the ordinary determinant possesses. From this fact, one can define a new (relative) invariant called a wreath determinant. Using ( GL m , GL n ) -duality in the sense of Howe, we obtain an expression of a wreath determinant by a certain linear combination of the corresponding ordinary minor determinants labeled by suitable rectangular shape tableaux. Also we study a wreath determinant analogue of the Vandermonde determinant, and then, investigate symmetric functions such as Schur functions in the framework of wreath determinants. Moreover, we examine coefficients which we call ( n , k ) -sign appeared at the linear expression of the wreath determinant in relation with a zonal spherical function of a Young subgroup of the symmetric group S n k .