Multiplicity-free actions and the geometry of nilpotent orbits

Kyo Nishiyama

Faculty of IHS, Kyoto Univeristy Sakyo, Kyoto 606-8501, Japan

               e-mail: kyo@math.h.kyoto-u.ac.jp

Compiled on November 12, 1999, Ver. 0.2 (Wed Oct 20 16:03:18 JST 1999)

Contents

1 Nilpotent orbits 3

2 Null cone 5

3 Harmonics on the null cone 7

4 Contraction of the null cone 10

5 Geometry of orbits 12

6 Reversed correspondence 14

Introduction

Let G be a reductive Lie group. Take a maximal compact subgroup K of G and denote their Lie algebras by 𝔤_{0} and 𝔨_{0} respectively. We get a Cartan decomposition 𝔤_{0} = 𝔨_{0} ⊕ 𝔰_{0}. Let 𝔤 be the complexification of 𝔤_{0}, and 𝔤 = 𝔨 ⊕ 𝔰 the complexified decomposition. The adjoint action restricted to K preserves the space 𝔰_{0}, hence K_{ℂ} acts on 𝔰, where K_{ℂ} denotes a complexification of K.

In this paper, we consider a series of small nilpotent K_{ℂ}-orbits in 𝔰 which are obtained from the dual pair (G, G′) = (O(p, q), Sp(2n, ℝ)) (cf. [4]).

We explain astonishing simple structures of these nilpotent orbits using generalized null cones (see §2 for the definition). For example, these orbits have a linear ordering with respect to the closure relation (Theorem 4.1); and K_{ℂ} acts on them in multiplicity-free manner. We clarify the K_{ℂ}-module structure of the regular function ring of the closure of these nilpotent orbits in detail (Corollary 5.3), and prove the normality (Corollary 5.2). All these results naturally comes from the analysis on the null cone 𝔑 (Theorems 2.1, 3.1 and 3.3) and the classical invariant theory. We also provide many interesting examples of multiplicity-free actions on conic algebraic varieties.

Let 𝔤′ = 𝔨′ ⊕𝔰′ be a complexified Cartan decomposition of 𝔤′ = Lie(G′)_{ℂ}. The key idea of the construction of the nilpotent orbits relies on the so-called unfolding maps φ_{1} and φ_{2} (see §1). We construct a double fibration of 𝔰 ⊂ 𝔤 and 𝔰′ ⊂ 𝔤′ by using φ_{1} and φ_{2}. In this way, we can consider the closure of a nilpotent K_{ℂ}-orbit \overline{O_{n}} as a geometric quotient of the null cone 𝔑 by K′_{ℂ}: \overline{O_{n}} ≃ 𝔑//K′_{ℂ}. On the other hand, the null cone itself is defined to be the inverse image of the origin of 𝔰′ via the map φ_{2}, so we can regard the nilpotent orbit O_{n} ⊂ 𝔰 as the “lift” of the trivial orbit in 𝔰′. In fact, we have \overline{O_{n}} = φ_{1}(𝔑) = φ_{1}(φ_{2}^{−1}(0)).

We also consider small nilpotent K′_{ℂ}-orbits in 𝔰′ and get similar results.

Let π be an irreducible admissible representation of G on a Hilbert space. Then the associated cycle of π is a finite union of nilpotent K_{ℂ}-orbits in 𝔰 together with multiplicities of the maximal orbits (see [8]). The geometry of the associated cycle (or, more simply, the associated variety) reflects various properties of the representation π. For example, the dimension of the associated variety is equal to the Gelfand-Kirillov dimension of π. Moreover, the Bernstein degree of π can be computed from the associated cycle ([6]).

We will use the results in this article to study the theta lift of the trivial representations for the dual pair (G, G′) = (O(p, q), Sp(2n, ℝ)) ([9]; see also [4]). The associated cycle of the theta lift of the trivial representation is the closure of an orbit obtained here without multiplicity. The details will appear in a forthcoming paper.

Let us explain each section briefly. Let (G, G′) = (O(p, q), Sp(2n, ℝ)) be a dual pair. We will denote by K′ a maximal compact subgroup of G′ and by 𝔤′ = 𝔨′⊕𝔰′ a complexified Cartan decomposition.

In §1, we define the two unfolding maps φ_{1} and φ_{2}. These maps relates the nilpotent K_{ℂ}-orbits in 𝔰 and 𝔰′ tightly. In §§2 and 3, we study the geometry of the null cone 𝔑 = φ_{2}^{−1}(0). In these sections, we assume that 2n < min(p, q), which means that the pair (G, G′) is in the stable range with G′ being the smaller member. We prove the null cone 𝔑 is an irreducible affine cone of complete intersection. Moreover, we construct an algebraic group which acts on 𝔑 in a multiplicity-free manner. In §§4 and 5, we consider the closure of a nilpotent orbit \overline{O_{n}} which is the image of the null cone via the unfolding map φ_{1}: \overline{O_{n}} = φ_{1}(𝔑). The orbit O_{n} is regarded as the theta lift of the trivial orbit in 𝔰′ for G′ = Sp(2n, ℝ). We prove its normality under the condition 2n + 1 < min(p, q), and K_{ℂ}-module structure of its regular function ring.

Finally, in §6, we assume that p+q < n and consider the nilpotent K′_{ℂ}-orbit O_{p,q} in 𝔰′. This orbit is the theta lift of the trivial orbit in 𝔰 for G = O(p, q). Therefore, we consider the “reversed” situation in a sense. In this section, we only state the results since the method is almost the same as in the former sections.

Acknowledgement. The author thanks Chen-bo Zhu and Jian-Shu Li for useful discussions. He owes a part of idea of using the double fibration of the unfolding maps to the discussion with Zhu.

1 Nilpotent orbits

Let us consider two Riemannian symmetric pairs

G/K = O(p, q)/O(p) × O(q) and G′/K′ = Sp(2n, ℝ)/U(n).

We denote 𝔨 = 𝔬(p)_{ℂ} ⊕ 𝔬(q)_{ℂ} and 𝔨′ = 𝔤𝔩(n, ℂ), the complexified Lie algebra of K and K′ respectively. Let 𝔤 = 𝔬(p + q, ℂ) = 𝔨 ⊕ 𝔰 and 𝔤′ = 𝔰𝔭(2n, ℂ) = 𝔨′ ⊕ 𝔰′ be Cartan decompositions. Namely, we realize

𝔤 = 𝔬(p + q, ℂ) = \{[ \begin{matrix}A & B \\ ^{t}B & D \end{matrix} ] | A ∈ Alt_{p}, D ∈ Alt_{q}, B ∈ M_{p,q}\} ,

𝔤′ = 𝔰𝔭(2n, ℂ) = \{[ \begin{matrix}A & B \\ C & −^{t}A \end{matrix} ] | A ∈ M_{n}, B, C ∈ Sym_{n}\} ,

where M_{p,q} = M_{p,q}(ℂ) denotes the space of all n × m-matrices, and Sym_{n} = Sym_{n}(ℂ) (respectively Alt_{n} = Alt_{n}(ℂ)) denotes the space of all symmetric (respectively alternating) matrices of size n. Note that 𝔰𝔭(2n, ℂ) here is not realized in the standard way (see [6, § 7.1], for example). Thus we can identify 𝔰 = M_{p,q} and 𝔰′ = Sym_{n} ⊕ Sym_{n}.

Let us consider the following three spaces:

W = M_{p,n} ⊕ M_{q,n}, 𝔰 = M_{p,q}, and 𝔰′ = Sym_{n} ⊕ Sym_{n}.

We define two maps among these spaces as follows. Take (A, B) ∈ M_{p,n} ⊕ M_{q,n} = W. Then we put

\{ \begin{matrix}φ_{1}(A, B) & = & A^{t}B ∈ M_{p,q} = 𝔰 and \\ φ_{2}(A, B) & = & (^{t}AA, ^{t}BB) ∈ Sym_{n} ⊕ Sym_{n} = 𝔰′. \end{matrix}

It can be readily checked that these maps are well-defined and homogeneous of degree two.

[figure: W = M_{p,n} ⊕ M_{q,n}; φ_{1}; φ_{2}; 𝔰 = M_{p,q}; Sym_{n} ⊕ Sym_{n} = 𝔰′]

Let us consider the following four algebraic groups over the field of complex numbers ℂ,

\begin{aligned}GL_{p} × GL_{q} & =: & M_{ℂ} \\ ∪ \\ O_{p} × O_{q} & =: & K_{ℂ} \end{aligned} and \begin{aligned}GL_{n} × GL_{n} & =: & M′_{ℂ} \\ ∪ \\ ΔGL_{n} & =: & K′_{ℂ} \end{aligned}

so that K_{ℂ} (respectively K′_{ℂ}) is a spherical subgroup of M_{ℂ} (respectively M′_{ℂ}) (in fact, these are symmetric pairs). We identify K_{ℂ} and K′_{ℂ} with the complexification of K and K′ respectively.

We define linear actions of these groups on W, 𝔰 and 𝔰′ as usual. To be more precise, let us write down explicit actions. Take an (A, B) ∈ W. Then an element g = ((g_{1}, h_{1}), (g_{2}, h_{2})) ∈ M_{ℂ} × M′_{ℂ} acts on (A, B) ∈ M_{p,n} ⊕ M_{q,n} as

g · (A, B) = (g_{1}A^{t}g_{2}, h_{1}Bh_{2}^{−1}).

For 𝔰, (g_{1}, h_{1}) ∈ M_{ℂ} acts on X ∈ M_{p,q} as

(g_{1}, h_{1}) · X = g_{1}X^{t}h_{1},

while M′_{ℂ} acts on 𝔰 trivially. For 𝔰′, (g_{2}, h_{2}) ∈ M′_{ℂ} acts on (C, D) ∈ Sym_{n} ⊕ Sym_{n} as

(g_{2}, h_{2}) · (C, D) = (g_{2}C^{t}g_{2}, ^{t}h_{2}^{−1}Ch_{2}^{−1}),

while M_{ℂ} acts on 𝔰′ trivially. As a subgroup, K_{ℂ} and K′_{ℂ} also act linearly on these spaces. It is easy to check that the map φ_{1} is M_{ℂ}×K′_{ℂ}-equivariant, and φ_{2} is K_{ℂ}×M′_{ℂ}-equivariant. Note that the action of K_{ℂ} on 𝔰 coincides with the adjoint action of K_{ℂ}. The same remark is applicable to the action of K′_{ℂ}.

The maps φ_{1} and φ_{2} are closely related to the nilpotent Ad K_{ℂ}-orbits in 𝔰.

Proposition 1.1 Let N_{𝔰} (respectively N_{𝔰′}) be the set of all nilpotent elements in 𝔰 (respectively 𝔰′). Then we have

φ_{1}(φ_{2}^{−1}(N_{𝔰′})) ⊂ N_{𝔰} and φ_{2}(φ_{1}^{−1}(N_{𝔰})) ⊂ N_{𝔰′}.

Proof. First we note the following easy observation.

Lemma 1.2 Let us consider an element

Y = [ \begin{matrix}0 & B \\ ^{t}B & 0 \end{matrix} ] ∈ 𝔰.

Then Y is nilpotent if and only if one of B^{t}B or ^{t}BB is nilpotent. Similarly, let us consider an element

X = [ \begin{matrix}0 & B \\ C & 0 \end{matrix} ] ∈ 𝔰′.

Then X is nilpotent if and only if one of BC or CB is nilpotent.

Take (A, B) ∈ φ_{2}^{−1}(X) ⊂ W. Then ^{t}AA^{t}BB and ^{t}BB^{t}AA are both nilpotent. Now, since φ_{2}(A, B) = A^{t}B, it is clear that

φ_{2}(A, B)^{t}φ_{2}(A, B) = A^{t}BB^{t}A

is nilpotent. This implies φ_{2}(A, B) is nilpotent.

The proof for Y is similar. Q.E.D.

2 Null cone

Now we assume that 2n < min(p, q). This assumption is essential in the following arguments.

Put 𝔑 = φ_{2}^{−1}(0) and call it a null cone. Since φ_{2} is a K_{ℂ} × M′_{ℂ}-equivariant map, 𝔑 is clearly K_{ℂ} × M′_{ℂ}-stable. Moreover, we have

Theorem 2.1 We assume min(p, q) > 2n.

(1) The null cone 𝔑 is the closure of a single K_{ℂ}^{∘} × M′_{ℂ}-orbit in W, where K_{ℂ}^{∘} denotes the connected component of the identity element. In particular, 𝔑 is an irreducible affine variety.

(2) 𝔑 is a variety of complete intersection, and dim 𝔑 = n(p+q)−n(n+1). In particular, the regular function ring ℂ[𝔑] is Cohen-Macaulay.

(3) If min(p, q) ≥ 2 + 2n holds, 𝔑 is a normal variety.

In the rest of this section, we shall prove this theorem.

First, we make a few remarks. Put W^{+} = M_{p,n} and W^{−} = M_{q,n}. Then W = W^{+}⊕W^{−} is a direct sum. We denote the restriction of φ_{2} on W^{±} by φ_{2}^{±}:

\begin{aligned}φ_{2}^{±} : & W^{±} & ⟶ & Sym_{n} \\ & ∪^{|} & & ∪^{|} \\ & Z & ⟼ & ^{t}ZZ \end{aligned} .

Then 𝔑^{±} = (φ_{2}^{±})^{−1}(0) is also called a null cone, and abviously 𝔑 is a direct product of these two null cones 𝔑^{±}. Therefore, our theorem above will follow from the results for 𝔑^{±} (for example, see [5, § 3.7.3] and the references there). However, we do not prove the theorem in such a way. Our proof here is direct, and it will clarify the structure of 𝔑 considerably.

Put

𝔑_{k,l} = {(A, B) ∈ 𝔑 | rank A = k, rank B = l}.

Then 𝔑_{k,l} is a locally closed set of 𝔑, and it is clearly stable under the action of K_{ℂ}×M′_{ℂ}.

Lemma 2.2 𝔑_{k,l} is an orbit of K_{ℂ}^{∘} × M′_{ℂ}, which is also stable under the action of K_{ℂ}. The dimension of 𝔑_{k,l} is give by

dim 𝔑_{k,l} = n(2n − (k + l)) + kp + lq − \frac{k(k + 1)}{2} − \frac{l(l + 1)}{2}.

Proof. Let (A, B) ∈ 𝔑_{k,l}. Then, since ^{t}AA = 0, the column vectors of the matrix A ∈ M_{p,n} generates an isotropic space in ℂ^{p} of dimension k < p/2. By the action of GL_{n} from the right, we can assume that the first k-columns of A are non-zero and linearly independent, and the rest is all zero. Let us fix a maximally isotropic space in ℂ^{p} for the symmetric bilinear form which defines O_{p}. Take an arbitrary basis for the space, and write it as {e_{1}, …, e_{[p/2]}}. Then it is easy to see that the action of SO_{p} brings the first k-columns of A to {e_{1}, …, e_{k}}. This means that an SO_{p} × GL_{n}-orbit through A has a representative Ξ_{k} = [ e_{1} e_{2} ⋯ e_{k} 0 ⋯ 0 ] ∈ M_{p,n}. Similarly, an SO_{q} ×GL_{n}-orbit through B has a representative Ξ′_{l} = [ e′_{1} e′_{2} ⋯ e′_{l} 0 ⋯ 0 ] ∈ M_{q,n}, where {e′_{1}, …, e′_{[q/2]}} is a fixed basis of a maximally totally isotropic space for O_{q}. Now it is clear that 𝔑_{k,l} is a single K_{ℂ}^{∘} × M′_{ℂ}-orbit thorough (Ξ_{k}, Ξ′_{l}).

Now the dimension of the orbit is easy to calculate. We omit it. Q.E.D.

From this lemma, we can immediately prove the following corollary, which is the statement (2) of Theorem 2.1.

Corollary 2.3 𝔑 is of complete intersection. In particular, it is Cohen-Macaulay.

Proof. Let {C_{ij} | 1 ≤ i ≤ j ≤ n} and {D_{ij} | 1 ≤ i ≤ j ≤ n} be a coordinate system of 𝔰′ = Sym_{n} ⊕ Sym_{n}, where we regard C_{ij} and D_{ij} as a linear function on Sym_{n} which extracts the (i, j)-element of a symmetric matrix. Let

φ_{2}^{∗} : ℂ[𝔰′] ⟶ ℂ[W]

be an algebra homomorphism induced by φ_{2}. Then φ_{2}^{∗}(C_{ij}) and φ_{2}^{∗}(D_{ij}) (1 ≤ i ≤ j ≤ n) define the null cone 𝔑. Note that we have an explicit description

φ_{2}^{∗}(C_{ij}) = ∑_{k=1}^{p} A_{ki}A_{kj} =: f_{ij},

φ_{2}^{∗}(D_{ij}) = ∑_{k=1}^{q} B_{ki}B_{kj} =: g_{ij},

where A_{ij} (respectively B_{ij}) is a coordinate system on M_{p,n} (respectively M_{q,n}). These {f_{ij}, g_{ij} | 1 ≤ i ≤ j ≤ n} are classical invariants of degree two for O_{p} and O_{q}, which generates all the invariants for K_{ℂ} vanishing at zero.

By Lemma 2.2, we know the codimension of the null cone in W, which is equal to n(n + 1). Since the number of defining equations of 𝔑 is also n(n + 1), it is a complete intersection. In general, complete intersection implies that its affine regluar function ring is Cohen-Macaulay (see [2, Proposition 18.13] for example). Q.E.D.

The following is also immediate from Lemma 2.2.

Corollary 2.4 We have the following decomposition of 𝔑 by the orbits of K_{ℂ} × M′_{ℂ} (or K_{ℂ}^{∘} × M′_{ℂ}):

𝔑 = ∐_{0≤k,l≤n} 𝔑_{k,l}.

Moreover, we have

\overline{𝔑_{k,l}} = ∐_{0≤i≤k, 0≤j≤l} 𝔑_{i,j}.

This corollary will prove the statement (1) of Theorem 2.1.

Let us prove (3) of Theorem 2.1, which is the most difficult part of the theorem. We use the following theorem quoted from [1, Theorem 2.2.11] which is attributed to Bernstein-Lunts, and Kunz.

Theorem 2.5 Let I be an indeal in ℂ[x_{1}, …, x_{n}] generated by {f_{1}, …, f_{p}}, and put X = V(I), the variety defined by I. Assume that

(1) ℂ[x_{1}, …, x_{n}]/I is Cohen-Macaulay.

(2) The differentials {df_{1}, …, df_{p}} are linearly independent at each point of a certain dense open subset X′ of X.

Then I is equal to the definig radical ideal I(X), hence ℂ[X] = ℂ[x_{1}, …, x_{n}]/I holds. Moreover, if X is irreducible and codim (X − X′) ≥ 2, then X is normal.

Let I = I^{K_{ℂ}} be an indeal of ℂ[W] generated by the invariants (without constant term) of K_{ℂ}. It is well-known that I^{K_{ℂ}} is generated by the second degree invariants {f_{ij} = φ_{2}^{∗}(C_{ij}) | 1 ≤ i ≤ j ≤ n} and {g_{ij} = φ_{2}^{∗}(D_{ij}) | 1 ≤ i ≤ j ≤ n}, where C_{ij} and D_{ij} are given in the proof of Corollary 2.3.

By Corollary 2.3, I satisfies the condition (1). Since the differentials {df_{ij}, dg_{ij}} are linearly independent at (Ξ_{n}, Ξ′_{n}) ∈ 𝔑_{n,n} and the K_{ℂ} × M′_{ℂ}-orbit 𝔑_{n,n} is open dense in 𝔑, the condition (2) also holds for X′ = 𝔑_{n,n}.

Note that 𝔑 = V(I^{K_{ℂ}}) is irreducible by (1) of Theorem 2.1. We must check the codimension of 𝔑 − 𝔑_{n,n}. By Lemma 2.2, we know

codim (𝔑 − 𝔑_{n,n}) = min\{codim \overline{𝔑_{k,l}} | 1 ≤ k, l ≤ n\}
= min\{codim \overline{𝔑_{n,n−1}}, codim \overline{𝔑_{n−1,n}}\}
= min{p − 2n, q − 2n}.

By the assumption min(p, q) ≥ 2 + 2n, we have codim (𝔑 − 𝔑_{n,n}) ≥ 2. Hence we conclude that X = 𝔑 is normal and I^{K_{ℂ}} is prime.

Now Theorem 2.1 is completely proved.

Remark 2.6 If min(p, q) > 2n, it is known that I^{K_{ℂ}} is prime. This is an immediate consequence from the fact that the same statement for 𝔑^{±} is valid (see [5, § 3.7.3]), and that the tensor product of integral domains over ℂ remains an integral domain.

Hereafter, we always assume that min(p, q) > 2n except §6.

3 Harmonics on the null cone

Let H(K_{ℂ}) be the space of K_{ℂ}-harmonic polynomials in ℂ[W]. Then ℂ[W] is a direct sum of harmonics and the ideal I^{K_{ℂ}} generated by invariants without constant term:

ℂ[W] = H(K_{ℂ}) ⊕ I^{K_{ℂ}}.

For the proof, see [1, Proposition 6.3.13]. From this, we immediately have Theorem 3.1 If min(p, q) > 2n, the ring of regular functions on the null cone 𝔑 is isomorphic to H(K_{ℂ}) as a K_{ℂ} × M′_{ℂ}-module:

ℂ[𝔑] ≃ H(K_{ℂ}).

Proof. By Remark 2.6 we have I(𝔑) = I^{K_{ℂ}}, so that ℂ[𝔑] ≃ ℂ[W]/I^{K_{ℂ}} ≃ H(K_{ℂ}). The last equality comes from the remark just before the theorem. Q.E.D.

We would like to clarify the explicit structure of H(K_{ℂ}). Let

P_{n} = {λ = (λ_{1}, …, λ_{n}) ∈ ℤ^{n} | λ_{1} ≥ λ_{2} ≥ ⋯ ≥ λ_{n} ≥ 0}

be the set of all partitions of length at most n. For λ ∈ P_{n}, we denote by τ_{λ} = τ_{λ}^{(n)} an irreducible finite dimensional representation of GL_{n} with highest weight λ. Consequently, τ_{λ} is a polynomial representation. Similarly, we denote by σ_{λ}^{(p)} an irreducible finite dimensional representation of SO_{p} with highest weight λ. Since λ ∈ P_{n} and 2n < p, an element h ∈ O_{p} − SO_{p} acts on σ_{λ}^{(p)} as an outer automorphism which fixes σ_{λ}^{(p)}. Therefore, the representation 1_{ℤ_{2}} ⊗ σ_{λ}^{(p)} of O_{p} = ℤ_{2} ⋉ SO_{p} is well-defined. We denote it by the same letter σ_{λ}^{(p)} by abuse of notation. Note that σ_{λ}^{(p)} is itself an irreducible representation of SO_{p}.

Lemma 3.2 We have

ℂ[𝔑^{+}] ≃ H^{+}(O_{p}) ≃ \begin{matrix}∑ & ⊕ \\ λ∈P_{n} \end{matrix} σ_{λ}^{(p)} ⊠ τ_{λ}^{∗}

as an O_{p} × GL_{n}-module, where H^{+}(O_{p}) denotes the space of O_{p}-harmonic polynomials in ℂ[W^{+}]. Similarly, we have

ℂ[𝔑^{−}] ≃ H^{−}(O_{q}) ≃ \begin{matrix}∑ & ⊕ \\ λ∈P_{n} \end{matrix} σ_{λ}^{(q)} ⊠ τ_{λ}

as an O_{q} × GL_{n}-module.

Proof. For 𝔑^{+} ⊂ W^{+}, the statement follows from [5]. Note that W^{−} is a contragredient of the usual action. Therefore we must take a contragredient in the expression of ℂ[𝔑^{−}], while (σ_{λ}^{(q)})^{∗} ≃ σ_{λ}^{(q)}. Q.E.D.

Theorem 3.3 We have a multiplicity free decomposition of the regular function ring of 𝔑,

ℂ[𝔑] ≃ \begin{matrix}∑ & ⊕ \\ λ,μ∈P_{n} \end{matrix}(σ_{λ}^{(p)} ⊠ σ_{μ}^{(q)}) ⊠ (τ_{λ}^{∗} ⊠ τ_{μ})

as a K_{ℂ} × M′_{ℂ} = (O_{p} × O_{q}) × (GL_{n} × GL_{n})-module. As a result, the action of K_{ℂ} × M′_{ℂ} on 𝔑 is multiplicity free.

Proof. Since ℂ[𝔑] ≃ ℂ[𝔑^{+}] ⊗ ℂ[𝔑^{−}], the satement follows from Lemma 3.2. Q.E.D.

From this theorem, we see that the null cone 𝔑 with the action of K_{ℂ} × M′_{ℂ} is an example of the variety X in Sato’s paper [7] with G_{1} = M′_{ℂ} ⊃ H = K′_{ℂ} and G_{2} = K_{ℂ}, where the boldface notation is adopted from Sato’s paper.

We use the notation from [6, § 8] and study the Poincare series and degrees of K′_{ℂ}- covariants. Recall that M′_{ℂ} = GL_{n} × GL_{n} ⊃ K′_{ℂ} = ΔGL_{n}. From Theorem 3.3, we know the ring of K′_{ℂ}-invariants is decomposed as follows

Γ(𝔑; 1_{K′_{ℂ}}) := ℂ[𝔑]^{K′_{ℂ}} ≃ \begin{matrix}∑ & ⊕ \\ λ∈P_{n} \end{matrix} σ_{λ}^{(p)} ⊠ σ_{λ}^{(q)} (as a K_{ℂ} = O_{p} × O_{q}-module).  (3.1)

Therefore, the Poincaré series of the invariants is given by

P (1_{K′_{ℂ}}; t) = ∑_{λ∈P_{n}} dim(σ_{λ}^{(p)} ⊠ σ_{λ}^{(q)}) t^{2|λ|},

where |λ| = ∑_{i=1}^{n} λ_{i} denotes the usual size function. Since the K′_{ℂ}-invariants are generated by degree two polynomials, we have

P (1_{K′_{ℂ}}; t) = \frac{Q(t)}{(1 − t^{2})^{d}} ,

where d is the transcendence degree of the quotient field of ℂ[𝔑]^{K′_{ℂ}} over ℂ (hence, it is a positive integer), and Q(t) is a polynomial in t which does not vanish at t = 1. We call Q(1) the degree of invariants, and denote Deg ℂ[𝔑]^{K′_{ℂ}} := Q(1). It is known that the degree is a positive integer. We shall return back to these formulas afterwards.

Next, let us examine covariants. Let τ_{ν} be an irreducible representation of GL_{n}. Note that ν need not be a partition but a dominant integral weight. Let us denote by Φ_{n}^{+} the set of dominant integral weights for GL_{n}. For λ, μ ∈ Φ_{n}^{+}, decompose τ_{λ} ⊗ τ_{μ} as a GL_{n}-module

τ_{λ} ⊗ τ_{μ} = ∑_{ν∈Φ_{n}^{+}} c_{λ μ}^{ν}τ_{ν},

where the multiplicity c_{λ μ}^{ν} is called the Littlewood-Richardson coefficient. Let us denote the highest weight of τ_{μ}^{∗} by μ^{∗}. Then we have a formula for τ_{ν}-covariants on 𝔑:

Γ(𝔑; τ_{ν}) ≃ \begin{matrix}∑ & ⊕ \\ λ,μ∈P_{n} \end{matrix} c_{λ^{∗} μ}^{ν}σ_{λ}^{(p)} ⊠ σ_{μ}^{(q)},

as a K_{ℂ} = O_{p} × O_{q}-module. Its Poincare series is given by

P (τ_{ν}; t) = ∑_{λ,μ∈P_{n}} c_{λ^{∗} μ}^{ν} dim(σ_{λ}^{(p)} ⊠ σ_{μ}^{(q)}) t^{|λ|+|μ|}.

Again, P (τ_{ν}; t) can be expressed as

P (τ_{ν}; t) = \frac{Q_{τ_{ν}}(t)}{(1 − t^{2})^{d}}

with the same d and a certain polynomial Q_{τ_{ν}}(t). We denote Deg Γ(𝔑; τ_{ν}) = Q_{τ_{ν}}(1) and call it the degree of covariants Γ(𝔑; τ_{ν}). The degree of covariants is a non-negative integer in general. Moreover, we have

Theorem 3.4 The degree of covariants is given by

Deg Γ(𝔑; τ_{ν}) = dim τ_{ν} Deg ℂ[𝔑]^{K′_{ℂ}}.

Proof. We can explicitly check the condition (1) – (3) in [6, § 8.2]. In [6, § 8], the variety X is assumed to be a vector space, however, all the arguments is valid in the case of the null cone. Now the statement follows from [6, Theorem 8.6]. Q.E.D.

4 Contraction of the null cone

In this section, we study the image of the null cone by the map φ_{1}. Namely, put

𝔛 = φ_{1}(φ_{2}^{−1}(0)) = {A^{t}B ∈ M_{p,q} | (A, B) ∈ W, and ^{t}AA = 0, ^{t}BB = 0}.

We call 𝔛 the contraction of the null cone 𝔑. Since φ_{1} is M_{ℂ} × K′_{ℂ}-equivariant and 𝔑 is K_{ℂ} × M′_{ℂ}-stable, 𝔛 is stable under the action of K_{ℂ}.

Theorem 4.1 The contraction 𝔛 of the null cone 𝔑 is the closure of a single K_{ℂ} = O_{p} × O_{q}-orbit. We denote the orbit by O_{n} so that 𝔛 = \overline{O_{n}}. The orbital decomposition of 𝔛 is given by

𝔛 = \overline{O_{n}} = ∐_{0≤k≤n} O_{k}.

Proof. Tentatively, we put

𝔛_{0} = {X ∈ M_{p,q} | X^{t}X = 0, ^{t}XX = 0, rank X ≤ n} ⊃ 𝔛.

We shall prove that 𝔛_{0} = 𝔛, hence we can conclude that 𝔛 is Zariski closed. It is convenient to work with the symmetric bilinear form induced by

J_{p} = [ \begin{matrix}0 & 1_{m} \\ 1_{m} & 0 \end{matrix} ] ,

where p = 2m is even, or

J_{p} = [ \begin{matrix}J_{2m} & 0 \\ 0 & 1 \end{matrix} ] ,

where p = 2m + 1 is odd. If we consider these symmetric bilinear forms, we shall denote O_{p}^{J}, 𝔛^{J}, etc., with the superscript J. Put

c_{p} = \frac{1}{\sqrt{2}} [ \begin{matrix}1_{m} & 1_{m} \\ \sqrt{−1} 1_{m} & −\sqrt{−1} 1_{m} \end{matrix} ] ,

where p = 2m is even, or

c_{p} = [ \begin{matrix}c_{2m} & 0 \\ 0 & 1 \end{matrix} ] ,

where p = 2m + 1 is odd. Then it is easy to verify that ^{t}c_{p}c_{p} = J_{p} and

O_{p}^{J} ∋ g ⟼ c_{p}gc_{p}^{−1} ∈ O_{p}

gives an isomorphism. Also we have

𝔛^{J} = {J_{p}A^{t}BJ_{q} | (A, B) ∈ 𝔑}, and
𝔛_{0}^{J} = {X ∈ M_{p,q}^{J} | XJ_{q} ^{t}X = 0, ^{t}XJ_{p}X = 0, rank X ≤ n},

where M_{p,q}^{J} = M_{p,q} as a vector space, but with the different action of orthogonal groups. Namely, the action of O_{p}^{J} × O_{q}^{J} on M_{p,q}^{J} is equivalent to that of O_{p} × O_{q} on M_{p,q} via the map

M_{p,q}^{J} ∋ X ⟼ c_{p}X^{t}c_{q} ∈ M_{p,q}.

Now we turn to the proof. First, note that the column vectors of X ∈ 𝔛_{0}^{J} generates a totally isotropic space in ℂ^{p}. Since k := rank X ≤ n, there exists a g ∈ O_{p}^{J} such that

gX = [ \begin{matrix}1_{k} & ∗ \\ 0 & 0 \end{matrix} ] .

The condition XJ_{q} ^{t}X = 0 tells as that the row vectors generates a totally isotropic space in ℂ^{q} as well. Since the row vectros of gX is already linearly independent, we can find an h ∈ O_{q}^{J} such that

gXh = [ \begin{matrix}1_{k} & 0 \\ 0 & 0 \end{matrix} ] =: Ξ_{k}^{J}.

This shows that 𝔛_{0}^{J} is a union of orbits through Ξ_{k}^{J} (0 ≤ k ≤ n). However, it is easy to see that Ξ_{k}^{J} ∈ 𝔛^{J} for all k. This means that 𝔛_{0} = 𝔛 and the theorem follows. We remark that O_{k} is a K_{ℂ}-orbit through Ξ_{k} = c_{p}^{−1}Ξ_{k}^{J} ^{t}c_{p}^{−1}. Q.E.D.

Corollary 4.2 The fiber of \overline{O_{k}} (0 ≤ k ≤ n) is given by

φ_{1}^{−1}(\overline{O_{k}}) = ∐_{0≤min(i,j)≤k} 𝔑_{i,j}.

Proof. Compare the rank, and we have the result. Q.E.D.

Lemma 4.3 For 0 ≤ k ≤ n, we have

dim \overline{O_{k}} = k((p + q) − (2k + 1)).

Proof. The Lie algebra of O_{p}^{J} is given by

𝔬_{p}^{J} = \{X = [ \begin{matrix}a & b \\ c & −^{t}a \end{matrix} ] | a ∈ 𝔤𝔩_{m}, b, c ∈ Alt_{m}\} ,

if p = 2m is even, and

𝔬_{p}^{J} = \{X = [ \begin{matrix}Y & b \\ −^{t}bJ_{2m} & 0 \end{matrix} ] | Y ∈ 𝔬_{2m}^{J}, b ∈ ℂ^{2m}\} ,

if p = 2m + 1 is odd.

For simplicity, we assume that both p and q are even. Put p = 2m and q = 2l. Then, an element (X, Y) ∈ 𝔬_{p}^{J} ⊕ 𝔬_{q}^{J} which kills Ξ_{k}^{J} is of the following form

(X, Y) = ([ \begin{matrix}A & B \\ C & −^{t}A \end{matrix} ] , [ \begin{matrix}D & E \\ F & −^{t}D \end{matrix} ]);

A = [ \begin{matrix}a & ∗ \\ 0 & ∗ \end{matrix} ] , C = [ \begin{matrix}0_{k} & 0 \\ 0 & ∗ \end{matrix} ] , D = [ \begin{matrix}−^{t}a & ∗ \\ 0 & ∗ \end{matrix} ] , F = [ \begin{matrix}0_{k} & 0 \\ 0 & ∗ \end{matrix} ] ,

where a ∈ M_{k,k} and B ∈ Alt_{m}, E ∈ Alt_{l} are arbitrary.

Now it is easy to count the dimension. Q.E.D.

5 Geometry of orbits

In this section, we invetigate the regular function ring of \overline{O_{n}} (0 ≤ 2n < min(p, q)) and prove the normality of \overline{O_{n}}.

Let φ_{1}^{∗} : ℂ[𝔰] ⟶ ℂ[𝔑] be an algebra morphism induced by the map φ_{1} : 𝔑 ⟶ 𝔰. Since the image φ_{1}(𝔑) is \overline{O_{n}}, it actually induces an injection

φ_{1}^{∗} : ℂ[\overline{O_{n}}] ⟶ ℂ[𝔑],  (5.1)

which is a graded morphism of degree two. Recall the action of K′_{ℂ} = ΔGL_{n} on 𝔑.

Theorem 5.1 The image of φ_{1}^{∗} in (5.1) coincides with the K′_{ℂ}-invariants in ℂ[𝔑], i.e.,

φ_{1}^{∗} : ℂ[\overline{O_{n}}] ⟶^{∼} ℂ[𝔑]^{K′_{ℂ}}.

Proof. Look at the original map φ_{1} : W ⟶ 𝔰. Then the induced map φ_{1}^{∗} : ℂ[𝔰] ⟶ ℂ[W] is given by

φ_{1}^{∗}(X_{ij}) = ∑_{k=1}^{n} A_{ik}B_{jk} =: F_{ij},

where {X_{ij} | 1 ≤ i ≤ p, 1 ≤ j ≤ q} is the set of coordinates on 𝔰 = M_{p,q}. By the classical invariant theory, it is known that {F_{ij} | 1 ≤ i ≤ p, 1 ≤ j ≤ q} generates the ring of K′_{ℂ}-invariants in ℂ[W], so that Image φ_{1}^{∗} = ℂ[W]^{K′_{ℂ}}.

Now consider the restriction φ_{1} : 𝔑 ⟶ 𝔰, which we denote by the same notation. By the above consideration, we know that the induced map gives actually an algebra morphism

φ_{1}^{∗} : ℂ[𝔰] ⟶ ℂ[W]^{K′_{ℂ}}/I(𝔑)^{K′_{ℂ}} ⊂ ℂ[𝔑] = ℂ[W]/I(𝔑).

This morphism is surjective by definition. Now, since ℂ[𝔰]/ Ker φ_{1}^{∗} = ℂ[\overline{O_{n}}] by Theorem 4.1, we conclude the theorem. Q.E.D.

Corollary 5.2 If 𝔑 is normal, \overline{O_{n}} is also a normal variety. In particular, if min(p, q) ≥ 2 + 2n, the closure \overline{O_{n}} is an irreducible normal affine variety.

Proof. Theorem 5.1 says that \overline{O_{n}} is a geometric quotient of 𝔑 by K′_{ℂ}. Since a geometric quotient of a normal variety is also normal, the corollary follows. Q.E.D.

Corollary 5.3 As a K_{ℂ} = O_{p} × O_{q}-module, we have

ℂ[\overline{O_{n}}] ≃ \begin{matrix}∑ & ⊕ \\ λ∈P_{n} \end{matrix} σ_{λ}^{(p)} ⊠ σ_{λ}^{(q)}.

Moreover, this is equal to the global section ℂ[O_{n}].

Proof. The first statement follows from Theorem 5.1 and (3.1). Since

dim O_{n} − dim O_{n−1} = p + q + 1 − 4n ≥ 2

by Lemma 4.3, we conclude ℂ[\overline{O_{n}}] = ℂ[O_{n}]. Q.E.D.

Corollary 5.4 We have

Deg ℂ[𝔑]^{K′_{ℂ}} = deg \overline{O_{n}},  and
trans.deg_{ℂ}ℂ[𝔑]^{K′_{ℂ}} = dim \overline{O_{n}} = n((p + q) − (2n + 1)),

where deg \overline{O_{n}} denotes the degree of a projective variety (note that \overline{O_{n}} is an affine cone).

Proof. These equations follow from the definition of ring theoretic degrees and geometric degrees. Q.E.D.

6 Reversed correspondence

Now we briefly summarize what we get if we lift the trivial orbit of 𝔰 = M_{p,q} to that of 𝔰′ = Sym_{n} ⊕ Sym_{n}.

Caution: we assume p + q < n in this section.

Let us define the null cone 𝔑′ by

𝔑′ = φ_{1}^{−1}(0) = {(A, B) ∈ W | A^{t}B = 0}.

Put

Ξ_{k} = [ \begin{matrix}1_{k} \\ & 0 \end{matrix} ] ∈ M_{p,n} ∈ W^{+}, Υ_{l} = [ \begin{matrix}0 \\ & 1_{l} \end{matrix} ] ∈ M_{q,n} ∈ W^{−},

and denote the M_{ℂ}×K′_{ℂ}-orbit through (Ξ_{k}, Υ_{l}) ∈ W by 𝔑′_{k,l}. Recall that M_{ℂ} = GL_{p}×GL_{q} and K′_{ℂ} = ΔGL_{n}.

Proposition 6.1 We assume p + q < n.

(1) The null cone 𝔑′ is the closure of a single M_{ℂ} × K′_{ℂ}-orbit 𝔑′_{p,q} in W. In particular, 𝔑′ is an irreducible affine variety.

(2) We have the following decomposition of 𝔑′ by the orbits of M_{ℂ} × K′_{ℂ}:

𝔑′ = ∐_{0≤k≤p,0≤l≤q} 𝔑′_{k,l}.

Moreover, we have

\overline{𝔑′_{k,l}} = ∐_{0≤i≤k,0≤j≤l} 𝔑′_{i,j}.

(3) 𝔑′ has the codimension pq in W. More generally, the dimension of 𝔑′_{k,l} is given by

dim 𝔑′_{k,l} = k(n + p − k) + l(n + q − l) − kl.

Theorem 6.2 We assume p + q < n. Then the null cone 𝔑′ is a variety of complete intersection, and dim 𝔑′ = n(p + q) − pq. In particular, the regular function ring ℂ[𝔑′] is Cohen-Macaulay. Moreover, 𝔑′ is an irreducible normal variety.

Theorem 6.3 If p+q < n, the ring of regular functions on the null cone 𝔑′ is isomorphic to H(K′_{ℂ}) as a M_{ℂ} × K′_{ℂ}-module:

ℂ[𝔑′] ≃ H(K′_{ℂ}).

Take two partitions λ = (λ_{1}, …, λ_{p}) ∈ P_{p} and μ = (μ_{1}, …, μ_{p}) ∈ P_{q}. We define a dominant integral weight λ ⊙ μ for GL_{n} by

λ ⊙ μ = (λ_{1}, …, λ_{p}, 0, …, 0, −μ_{q}, …, −μ_{1}) ∈ Φ_{n}^{+},

and call it a composit of λ and μ.

Theorem 6.4 We have a multiplicity free decomposition of the regular function ring of 𝔑′,

ℂ[𝔑′] ≃ \begin{matrix}∑ & ⊕ \\ λ∈P_{p},μ∈P_{q} \end{matrix} ((τ_{λ}^{(p)})^{∗} ⊠ τ_{μ}^{(q)}) ⊠ (τ_{λ⊙μ}^{(n)})^{∗} ≃ \begin{matrix}∑ & ⊕ \\ λ∈P_{p},μ∈P_{q} \end{matrix} ((τ_{λ}^{(p)})^{∗} ⊠ τ_{μ}^{(q)}) ⊠ τ_{μ⊙λ}^{(n)}

as a M_{ℂ} × K′_{ℂ} = (GL_{p} × GL_{q}) × ΔGL_{n}-module. As a result, the action of M_{ℂ} × K′_{ℂ} on 𝔑′ is multiplicity free.

We define a contraction of the null cone by

𝔜 = φ_{2}(𝔑′) = {(^{t}AA, ^{t}BB) ∈ 𝔰′ | (A, B) ∈ W, A^{t}B = 0}.

Theorem 6.5 The contraction 𝔜 of the null cone 𝔑′ is the closure of a single K′_{ℂ} = ΔGL_{n}-orbit through

([ \begin{matrix}1_{p} \\ & 0 \end{matrix} ] , [ \begin{matrix}0 \\ & 1_{q} \end{matrix} ]) ∈ 𝔰′ = Sym_{n} ⊕ Sym_{n}.

We denote the orbit by O_{p,q} so that 𝔜 = \overline{O_{p,q}}.

(1) The orbital decomposition of 𝔜 is given by

𝔜 = \overline{O_{p,q}} = ∐_{0≤k≤p,0≤l≤q} O_{k,l}.

(2) The fiber of \overline{O_{k,l}} (k ≤ p, l ≤ q) is given by

φ_{2}^{−1}(\overline{O_{k,l}}) = \overline{𝔑′_{k,l}}.

(3) For p + q < n, the dimension of the orbit is given by

dim \overline{O_{p,q}} = p (n − \frac{p − 1}{2}) + q (n − \frac{q − 1}{2}) − pq.

Theorem 6.6 The image of φ_{2}^{∗} : ℂ[\overline{O_{p,q}}] ⟶ ℂ[𝔑′] coincides with the K_{ℂ}-invariants in ℂ[𝔑′], i.e.,

φ_{2}^{∗} : ℂ[\overline{O_{p,q}}] ⟶^{∼} ℂ[𝔑′]^{K_{ℂ}},

where K_{ℂ} = O_{p} × O_{q}.

Corollary 6.7 If p + q < n, the closure \overline{O_{p,q}} is an irreducible normal affine variety.

Corollary 6.8 We assume that p + q < n. As a K′_{ℂ} ≃ GL_{n}-module, the regular function ring decompses as

ℂ[\overline{O_{p,q}}] ≃ \begin{matrix}∑ & ⊕ \\ λ∈P_{p},μ∈P_{q} \end{matrix}(τ_{2λ⊙2μ}^{(n)})^{∗}. ≃ \begin{matrix}∑ & ⊕ \\ λ∈P_{p},μ∈P_{q} \end{matrix} τ_{2μ⊙2λ}^{(n)}.

Moreover, this is equal to the global section ℂ[O_{p,q}].

References

[1] N. Chriss and V. Ginzburg, Representation theory and complex geometry, Birkhäuser Boston, Boston, MA, 1997 ISBN: 0-8176-3792-3.

[2] D. Eisenbud, Commutative Algebra with a View Toward Algebraic Geometry, GTM 150, Springer, New York, 1995.

[3] R. Howe, Remarks on classical invariant theory, Trans. Amer. Math. Soc. 313 (1989), no. 2, 539–570. Erratum to: “Remarks on classical invariant theory”, Trans. Amer. Math. Soc. 318 (1990), no. 2, 823.

[4] R. Howe, Transcending classical invariant theory. J. Amer. Math. Soc. 2 (1989), no. 3, 535–552.

[5] R. Howe, Perspectives on invariant theory: Schur duality, multiplicity-free actions and beyond, in The Schur lectures (1992) (Tel Aviv), 1–182, Bar-Ilan Univ., Ramat Gan, 1995.

[6] Kyo Nishiyama, Hiroyuki Ochiai, and Kenji Taniguchi, Bernstein degree and associated cycles of Harish-Chandra modules (Hermitian symmetric case), preprint, June 1999. Available at the following URL:

              http://w3rep.math.h.kyoto-u.ac.jp/papere.html#NOT

[7] F. Sato, On the stability of branching coefficients of rational representations of reductive groups, Comment. Math. Univ. St. Paul. 42 (1993), no. 2, 189–207.

[8] D. A. Vogan, Jr., Associated varieties and unipotent representations, in Harmonic analysis on reductive groups (Brunswick, ME, 1989), 315–388, Progr. Math. 101, Birkhäuser, Boston, Boston, MA, 1991.

[9] C.-B. Zhu and J.-S. Huang, On certain small representations of indefinite orthogonal groups, Represent. Theory 1 (1997), 190–206 (electronic).