## THE INDEX OF SUB-LAPLACIANS: BEYOND CONTACT MANIFOLDS

MAGNUS GOFFENG, BERNARD HELFFER

Abstract. In this paper we study the following question: do sub-Laplacian type operators have non-trivial index theory on Carnot manifolds in higher degree of nilpotency? The problem relates to characterizing the structure of the space of hypoelliptic sub-Laplacian type operators, and results going back to Rothschild-Stein and HelfferNourrigat. In two degrees of nilpotency, there is a rich index theory by work of van Erp-Baum on contact manifolds, that was later extended to polycontact manifolds by Goffeng-Kuzmin. We provide a plethora of examples in higher degree of nilpotency where the index theory is trivial.

## 1. Introduction

We consider a Carnot manifold X with degree of nilpotency r coming from an equiregular differential system. For further geometric context of Carnot manifolds defined from equiregular differential system, see [14, 30], and for further analytic context see [1, 8, 29]. Carnot manifolds are also known as filtered manifolds or equiregular sub-Riemannian manifolds. In other words, we have chosen a subbundle H ⊆ T X that induces a filtering

$$T \mathfrak { X } = T ^ { - r } \mathfrak { X } \supsetneq T ^ { - r + 1 } \mathfrak { X } \supsetneq \dots \supsetneq T ^ { - 2 } \mathfrak { X } \supsetneq T ^ { - 1 } \mathfrak { X } \supsetneq 0,$$

of sub-bundles such that T -1 X = H and T j -1 X = T j X + [ T -1 X,T j X ] for any j . The associated graded bundle t H X := ⊕ j T -j X /T -j +1 X carries a fibrewise Lie algebra structure induced from commutators of vector fields. While T -1 X = H is bracket generating, for any x ∈ X , the nilpotent Lie algebra t H X x is generated by T -1 x X and has nilpotency degree r . We write DO m H ( X ; E ) for the space of differential operators acting on a vector bundle E → X of Heisenberg order at most m relative to the filtering.

Among the differential operators in DO m H ( X ; E ), the H -elliptic operators form a distinguished class that define Fredholm operators on an appropriate scale of HeisenbergSobolev spaces. We recall the notion of H -ellipticity and the closely related Rockland condition two paragraphs below. By abstract deformation arguments [4, 31] one finds as rich index theory for H -elliptic operators in the Heisenberg calculus as one does in the ordinary pseudodifferential calculus - yet no concrete examples with non-trivial index theory are known in general. A natural quest is to look for classes of differential operators that are H -elliptic and carry non-trivial index theory. A further reaching goal is to find differential operators capturing the geometry of Carnot manifolds in the spirit that Dirac operators do for Riemannian geometry and classical elliptic operators [2, 3, 27].

In light of recent results [4, 13], we approach this line of questioning by studying operators D γ ∈ DO 2 H ( X ; E ) of the form

$$\cdots \cdot \cdot \cdot \\ D _ { \gamma } & = \sum _ { j = 1 } ^ { n } X _ { j } ^ { 2 } + \sum _ { l = 1 } ^ { m } \gamma _ { l } Y _ { l } + \mathcal { D O } _ { H } ^ { 1 } ( \mathfrak { X } ; E ), & ( 1. 2 ) \\ Y \text{ } \text{en} \, T ^ { - 1 } \mathfrak { Y } \text{ in all noints and } V. \quad V \text{ } \text{en} \, T ^ { - 2 } \mathfrak { Y } / T ^ { - 1 } \mathfrak { Y } \text{ in all noints}$$

where X ,. . . , X 1 n span T -1 X in all points and Y , . . . , Y 1 m span T -2 X /T -1 X in all points. We call such an operator a sub-Laplacian and note that D γ is uniquely determined modulo

Date : January 29, 2025.

DO 1 H ( X ; E ) from the classical principal symbol σ 2 ( ∑ n j =1 X 2 j ) and the restricted subprincipal symbol

The possible classical principal symbols σ 2 ( ∑ n j =1 X 2 j ) stand in a one-to-one correspondence with metrics on the sub-bundle H = T -1 X ⊆ T X .

$$\psi \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \, \text{$\psi$} \colon \\ \text{mbol} & \\ \gamma \colon = \sigma ^ { 1 } \left ( D _ { \gamma } - \sum _ { j = 1 } ^ { n } X _ { j } ^ { 2 } \right ) \Big | _ { ( T ^ { - 1 } \mathfrak { X } ) ^ { \perp } / ( T ^ { - 2 } \mathfrak { X } ) ^ { \perp } }. & & ( 1. 3 ) \\ \text{le classical principal symbols $\sigma^{2}(\sum_{j=1}^{n} X_{j}^{2})$ stand in a one-to-one correspon-} \\ \text{metrics on the sub-bundle $H = T^{-1} \mathfrak { X } \subseteq T\mathfrak { X }$}.$$

Let us briefly review the Heisenberg calculus [10, 13, 28, 38], see also [6, 34, 37] for the case of contact manifolds or more generally when T X /H has rank 1. Of particular importance is the Rockland condition and related notions of H -ellipticity that are later used to conclude analytic properties of sub-Laplacians. We write t H X := ⊕ j T j X /T j +1 X for the graded bundle associated with the filtration (1.1) and view t H X as a bundle of graded Lie algebras with fibrewise bracket induced from the Lie bracket of vector fields. We can fibrewise integrate the Lie algebra structure to a bundle T H X → X of simply connected, nilpotent Lie groups. By declaring a vector field with values in T -j X to have order ≤ j , we can for a natural number m define the space of differential operators DO m H ( X ; E ) of Heisenberg order m on a vector bundle E → X . By taking the principal part with respect to the Heisenberg order, we arrive at a symbol mapping

$$\sigma _ { H } ^ { m } \colon \mathcal { D O } _ { H } ^ { m } ( \mathfrak { X } ; E ) / \mathcal { D O } _ { H } ^ { m - 1 } ( \mathfrak { X } ; E ) \stackrel { \sim } { \to } C ^ { \infty } ( \mathfrak { X }, \mathcal { U } _ { m } ( \mathfrak { t } _ { H } \mathfrak { X } ) ), \\ \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \colon \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \ddots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots$$

where U m denotes the degree m part of the universal enveloping Lie algebra. A key feature is that the Lie algebraic machinery is compatible with the operators in the sense that σ m 1 + m 2 H ( D D 1 2 ) = σ m 1 H ( D 1 ) σ m 2 H ( D 2 ) when D 1 and D 2 are of order m 1 and m 2 , respectively. An important notion for a differential operator D ∈ DO m H ( X ; E ) is the Rockland condition, which is said to be satisfied if for any x ∈ X and any non-trivial, irreducible, unitary representation π of the simply connected Lie group T H X x integrating t H X x , the represented symbol π σ ( m H ( D ) x ) is injective on the space of smooth vectors C ∞ ( π ). For the operators D γ of interest in this paper, the Rockland condition was proven to be equivalent to maximal hypoellipticity in [36], with more general results found later [1, 10]. For the equiregular differential systems we consider, we can define H -ellipticity of a differential operator D ∈ DO m H ( X ; E ) as D admitting an inverse modulo lower order terms in the Heisenberg calculus, or equivalently that σ m H ( D ) admits an inverse in the Heisenberg symbol algebra [10, 13], that by [10] is equivalent to D and D ∗ satisfying the Rockland condition. The problem we will study is the following.

Problem 1. For a fixed metric on H = T -1 X , which first order polynomial sections

$$\gamma \colon ( T ^ { - 1 } \mathfrak { X } ) ^ { \perp } / ( T ^ { - 2 } \mathfrak { X } ) ^ { \perp } \to \text{End} ( E ), \\ \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \colon \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \,.$$

give rise to an H -elliptic operator D γ (as in (1.2) and (1.3) ) with non-trivial Fredholm index?

Problem 1 is completely understood in the case of contact manifolds in work by Baumvan Erp [4] that followed a substantial effort from Epstein-Melrose [12]. Similar results hold for polycontact manifolds [13]. Contactand polycontact manifolds are Carnot manifolds of nilpotency degree r = 2 with a strong non-degeneracy condition on the Levi bracket H × H → T X /H . For Carnot manifolds of higher nilpotency degree close to nothing is known about non-triviality of the index. The second part of Problem 1, that of computing the index, is by the methods of [4, 13] closely connected to the first part of Problem 1, that of characterizing H -ellipticity. Since H -ellipticity is equivalent to the Rockland condition, the problem of characterizing H -ellipticity reduces to pointwise properties. More precisely, given a graded nilpotent Lie algebra g = g -1 ⊕ g -2 ⊕···⊕ g -r , generated by g -1 , and N ∈ N &gt; 0 we consider the set

$$\mathcal { E } ( \mathfrak { g }, N ) \coloneqq \{ \gamma = ( \gamma _ { l } ) _ { l = 1 } ^ { m } \in M _ { N } ( \mathbb { C } ) ^ { m } \, \colon \text{the operator $(1.2)$ is hypoelliptic on $G$} \}.$$

Here m = dim( g -2 ) and we have implicitly chosen an inner product structure on g -1 that fixes the principal symbol σ 2 ( ∑ n j =1 X 2 j ) and ( Y l ) m l =1 in (1.2) as an orthonormal basis of g -2 . Also, N encodes the rank of the vector bundle E . The reader should beware that we can, and will, identify E ( g , N ) with a subset of g -2 ⊗ M N ( C ) as well as with a subset of linear polynomials g ∗ -2 → M N ( C ) . To ensure compatibility with (1.3) , this identification is made by defining

$$\gamma ( \xi ) \coloneqq i \xi ( \gamma ), \quad \xi \in \mathfrak { g } _ { - 2 } ^ { * }, \\ l e f i n e d \ f r o m \ c o n t r a c t i n a \ \sim \ \mathfrak { a }$$

where ξ γ ( ) ∈ M N ( C ) is defined from contracting γ ∈ g -2 ⊗ M N ( C ) with ξ ∈ g ∗ -2 in the first leg. By construction it is clear that a first order polynomial section γ : ( T -1 X ) ⊥ / T ( -2 X ) ⊥ → End( E ) on a compact Carnot manifold X defines an H -elliptic operator D γ (as in (1.2) and (1.3)) if and only if γ , x -γ ∗ x ∈ E ( t H X x , rk( E x )) for all x ∈ X . For contact, or more generally polycontact, manifolds E ( g , N ) consists precisely of those γ ∈ g -2 ⊗ M N ( C ) such that γ ξ ( ) has spectrum outside 2 N +rk( H / ) 2 for ξ ∈ g ∗ -2 being unit length in the inner product induced from that on g -1 and the Lie bracket.

Definition 2. Let g be a graded nilpotent Lie algebra such that g -1 has an inner product and generates g . For N ∈ N &gt; 0 we say that g has property E ∗ N if for any t ∈ (0 , 1] we have that

$$\ t { \mathcal { E } } ( \mathfrak { g }, N ) & \subseteq \mathcal { E } ( \mathfrak { g }, N ). \\ \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \t \t                                                                                                                                                                                                         }                                                                                                                                                                                                       
                                                                                                                                                                                                       .                                                                                                                                                                                                        -                                                                                                                                                                                                        +                                                                                                                                                                                                        < \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde                                                                                                                                                                                                        \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde } { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \that { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \t                                                                                                                                                                                                     
  \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \that \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde { \tilde	                                                                                                                                                                                                        |                                                                                                                                                                                                        )                                                                                                                                                                                                        "$$

The importance of property E ∗ N is found in Theorem 2.11 showing that if X is a compact Carnot manifold with t H X x having property E ∗ N for all x , then Problem 1 admits no solution γ whenever E has rank N .

Throughout the paper we prove property E ∗ N in various cases. In Corollary 3.2, we carefully apply Rothschild-Stein's work [35] to show that if g -2 has dimension different from 2 then property E ∗ 1 holds. In Proposition 3.3 we modify a work of Helffer [17] to show that if g has nilpotency degree &gt; 2 and g -2 is one-dimensional then property E ∗ N holds for all N . In Proposition 4.4 we show that for the nilpotent Lie algebra n (4) of strictly upper triangular 4 × 4 matrices, property E ∗ N holds for all N . Invoking Theorem 2.11 as discussed above we can deduce the next theorem.

Theorem 3. Let X denote a compact Carnot manifold defined from an equiregular differential system as above, and write r for its nilpotency degree. The operators D γ ∈ DO 2 H ( X ; E ) satisfy that if D γ is H -elliptic then

$$\text{index} ( D _ { \gamma } ) = 0,$$

under any of the following geometric assumptions on ( X , E ) :

- (1) E is a line bundle and for all x ∈ X , rank( T -2 X /T -1 X ) = 2 .

/negationslash

- (2) The nilpotency degree of X satisfies r &gt; 2 and T -2 X /T -1 X is a line bundle.
- (3) X is a regular parabolic geometry of type ( SL 4 ( R ) , P ) , where P is the minimal parabolic subgroup.

In fact, in all cases above it even holds that [ D γ ] = 0 in K 0 ( X ) whenever D γ is H -elliptic.

Here K 0 ( X ) denote the even K -homology of the topological space X , for details see [3, 24].

Remark 1.1 . Instead of an equiregular differential system, we can more generally consider a finitely generated submodule E ⊆ C ∞ ( X , T X ) that satisfies the H¨rmander condition, o i.e. E generates C ∞ ( X , T X ) as a Lie algebra. We have hope that the ideas underlying Theorem 3 extend also to this setting following [1]. A related example can be found in [32]. Indeed, when the system is not equiregular but satisfies the H¨rmander condition o of degree r , the Rockland condition that in the equiregular case is posed for all x ∈ X and all non-trivial, irreducible, unitary representations of T H X x can be replaced by a condition that over x ∈ X is posed for all non-trivial representations from a conical closed set Γ x in ̂ G p,r where G p,r is the free group whose Lie algebra g r,p is the maximal graded

algebra of nilpotency degree r with p generators of degree -1. This was conjectured by Helffer-Nourrigat in 1979 (see [23]) and recently proved in [1].

Theorem 3 shows that it is commonly occurring for Problem 1 to admit no solution, however it is unsatisfactory in the sense that all conditions rely on some structure being low dimensional. In the hope of providing a stronger result and understanding for which Carnot manifolds of higher nilpotency degree that Problem 1 admits a solution, we look to the work of Rothschild-Stein [35]. We write ‖ · ‖ W for the norm on g -2 ⊗ M N ( R ) defined from realizing g -2 ⊗ M N ( R ) as a quotient of so n ( M N ( R )), for n = dim( g -1 ). More details can be found in Subsection 2.1. This norm was studied in the literature [17, 35] for N = 1. We can write γ ∈ g -2 ⊗ M N ( C ) as γ = a + ib for a and b self-adjoint, and we say that γ has property α if for any unit vectors ξ ∈ g ∗ -2 and v ∈ Ker( ( ξ a )) we have that |〈 v, ξ ( b v ) 〉 C N | &lt; 1. Here ξ γ ( ) ∈ M N ( C ) is defined from ξ ∈ g ∗ -2 with γ ∈ g -2 ⊗ M N ( C ) in the first leg, so ξ γ ( ) = -iγ ( ξ ). Following the ideas in [35], we prove the following result in Subsection 3.3 that sandwiches E ( g , N ) between two sets that are starshaped with respect to 0 in the spirit of property E ∗ N .

Theorem 4. Let g be a graded nilpotent Lie algebra generated by g -1 . It holds that

$$\mathcal { E } ( \mathfrak { g }, N ) \supseteq \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m } \colon \| \text{Im} ( \gamma ) \| _ { W / \mathbb { R } \, R e ( \gamma ) ) } < 1 \}. \\ \colon & \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon } \, \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon = \colon$$

$$\begin{array} { c } \mathcal { E } ( \mathfrak { g }, N ) \subseteq \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m } \, \colon \, \gamma \, h a s \ p r o p e r t y \, \alpha \}. \\ \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \end{array} \,.$$

Moreover, if g has the property that the Kirillov form ω ξ ( X,Y ) := ξ ([ X,Y ]) is degenerate on g -1 for any ξ ∈ g ∗ -2 , then

The paper is organized as follows. In Section 2 we provide some preliminary material and a proof that property E ∗ N implies vanishing of the index of sub-Laplacians. We study instances when property E ∗ N holds in Section 3. We study several examples showcasing that Problem 1 often has no solutions in Section 4. Finally, in Section 5 we provide the final details in the proof of Rothschild-Stein's result [35, Theorem 2] covering a case that was overlooked.

Acknowledgements. The authors wish to thank the anonymous referees for their helpful suggestions. The first listed author was supported by the Swedish Research Council Grant VR 2018-0350.

## 2. Preliminaries

## 2.1. Some Lie algebra. We consider graded nilpotent Lie algebras

$$\dots \circ \dots \dots \dots \dots \dots \dots \\ \mathfrak { g } = \bigoplus _ { j = 1 } ^ { r } \mathfrak { g } _ { - j }, \\ \text{algebra.} \text{ such } L$$

We will study the H -ellipticity of the operators D γ on a Carnot manifolds by means of their Heisenberg principal symbol, which in other words mean that we fix a point x ∈ X and consider the translation invariant differential operator on the simply connected nilpotent Lie group G integrating g := t H X x that takes the form such that g -1 generates g as a Lie algebra. Such Lie algebras are also known as stratified Lie algebras. In particular, r coincides with the nilpotency degree of g and g -j = ∑ j -1 l =1 [ g -l , g -j + l ] for any j &gt; 1.

$$\dots \dots \circ \tau \cdot \dots \dots \dots \dots \cdot \cdot \cdot \\ D _ { \gamma } & = \sum _ { j = 1 } ^ { n } X _ { j } ^ { 2 } + \sum _ { l = 1 } ^ { m } \gamma _ { l } Y _ { l }, \\ s \, \text{of a subspace } \mathfrak { a } _ { - 1 } \subset \mathfrak { a } \text{ generation}$$

where X ,. . . , X 1 n is the basis of a subspace g -1 ⊆ g generating the nilpotent Lie algebra g and Y , . . . , Y 1 m is a basis for the subspace g -2 ⊆ g . We tacitly assume that g -1 is equipped with an inner product in which X ,. . . , X 1 n is an orthonormal basis and that Y , . . . , Y 1 m is an orthonormal basis for g -2 is equipped with the inner product induced from the

projection mapping ∧ 2 g -1 → g -2 that the Lie bracket defines. Here ( γ l ) m l =1 ⊆ M N ( C ) is some collection of matrices.

We remark that alternatively, we can also describe our operator in the form

$$D _ { \gamma } & = \sum _ { j = 1 } ^ { n } X _ { j } ^ { 2 } + \sum _ { k, l = 1 } ^ { m } \gamma _ { k, l } [ X _ { k }, X _ { l } ], & ( 2. 1 ) \\ \sim M _ { \dots } \hat { t } \hat { \Lambda } \times \cos \text{allation of matrix} \quad \text{with} \quad \text{$two$ for the algebraic}.$$

where ( γ k,l ) n k,l =1 ⊆ M N ( C ) is some collection of matrices. Write so n for the algebraic Lie algebra of anti-symmetric n × n -matrices, and so n ( M N ( C )) for the anti-symmetric matrices over M N ( C ). That is, ( γ k,l ) n k,l =1 ∈ so n ( M N ( C )) if and only if γ k,l = -γ l,k . We can assume that the collection ( γ k,l ) n k,l =1 has been choosen in so n ( M N ( C )). The collection ( γ k,l ) n k,l =1 ∈ so n ( M N ( C )) is uniquely determined from ( γ l ) m l =1 as above modulo the space

Note that S C ,N = M N ( C ) ⊗ C S C , 1 . We write S for the real part of S C , 1 .

$$\| _ {, l } ^ { n } \in \mathfrak { s o } _ { n } ( M _ { N } ( \mathbb { C } ) ) \text{ is uniquely determined from } ( \gamma _ { l } ) \| _ { l = 1 } ^ { m } \text{ as above modulo the sp} \\ \mathcal { S } _ { \mathbb { C }, N } \coloneqq \left \{ ( s _ { k, l } ) \| _ { k, l = 1 } ^ { n } \in \mathfrak { s o } _ { n } ( M _ { N } ( \mathbb { C } ) ) \colon \sum _ { l, k = 1 } ^ { n } s _ { l, k } [ X _ { l }, X _ { k } ] = 0 \right \}. \\ \text{te that } \mathcal { S } _ { \mathbb { C }, N } = M _ { N } ( \mathbb { C } ) \otimes _ { \mathbb { C } } \mathcal { S } _ { \mathbb { C }, 1 }. \text{ We write } \mathcal { S } \text{ for the real part of } \mathcal { S } _ { \mathbb { C }, 1 }. \\ \| \dots \| \dots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \dots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \dots \| \dots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdclots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \ddots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \cdots \| \end{array}$$

Proposition 2.1. There is a linear isomorphism

$$\delta \colon M _ { N } ( \mathbb { C } ) ^ { m } \to s \mathfrak { o } _ { n } ( M _ { N } ( \mathbb { C } ) ) / \mathcal { S } _ { \mathbb { C }, N },$$

defined from

$$\sum _ { k, l = 1 } ^ { n } \delta ( \gamma ) _ { k, l } [ X _ { k }, X _ { l } ] = \sum _ { l = 1 } ^ { m } \gamma _ { l } Y _ { l }. \\ \text{music community communication and hermitear}$$

The map δ respects entrywise complex conjugation and hermitean conjugates, where hermitean conjugate in so n ( M N ( C )) is entry-wise defined by (( γ k,l ) n k,l =1 ) ∗ := ( γ ∗ k,l ) n k,l =1 .

Proof. Since ( Y l ) m l =1 is a basis for g -2 , there exists a matrix ( γ k,l ) n k,l =1 ∈ so n ( M N ( C )) with ∑ m k,l =1 γ k,l [ X ,X k l ] = ∑ m l =1 γ Y l l and it is clear that δ γ ( ) := [( γ k,l ) n k,l =1 ] ∈ so n ( M N ( C )) / S C ,N is well defined. Again since ( Y l ) m l =1 is a basis for g -2 , the map δ is an isomorphism. It follows from the construction that δ respects entrywise complex conjugation and hermitean conjugates. /square

Remark 2.2 . The two step nilpotent Lie algebra g -1 ⊕ g -2 ≡ g / ⊕ r j =3 g -j is by Eberlein's construction [11] of the following form. We set V := g -1 which is an inner product space. We identify so ( V ) with the antisymmetric two-forms. There is an injective map ι : g -2 → so ( V ) defined from ι ( Y ) X := Ad ( ∗ X )( Y ) for Y ∈ g -2 and X ∈ V = g -1 , i.e. the 2-form ι ( Y ) is determined from ι ( Y )( X,X ′ ) = ( Y, X, X [ ′ ]). Write W := ι ( g -2 ). For ω ∈ V ∗ ∧ V ∗ = so ( V ) we write ω W ∈ W for the the orthogonal projection onto W . The space V W := V ⊕ W is a Lie algebra in the Lie bracket [( X,Y , ) ( X , Y ′ ′ )] := (0 , ( X ∧ X ′ ) W ) and the canonical linear isomorphism g / ⊕ r j =3 g -j ∼ - → V W is also a Lie algebra isomorphism. It is clear from a dimensional consideration that the composition

$$\mathfrak { g } _ { - 2 } \stackrel { \iota } { \rightarrow } \mathfrak { s o } ( V ) \rightarrow \mathfrak { s o } ( V ) / \mathcal { S },$$

is a linear isomorphism.

The isomorphism δ from Proposition 2.1 can be factorized as

$$M _ { N } ( \mathbb { C } ) ^ { m } = M _ { N } ( \mathbb { C } ) \otimes _ { \mathbb { R } } \mathfrak { g } _ { - 2 } & \xrightarrow { \text{id} \otimes \iota } M _ { N } ( \mathbb { C } ) \otimes _ { \mathbb { R } } \mathfrak { s o } ( V ) \to \\ & \to M _ { N } ( \mathbb { C } ) \otimes _ { \mathbb { R } } ( \mathfrak { s o } ( V ) / \mathcal { S } ) = \mathfrak { s o } _ { n } ( M _ { N } ( \mathbb { C } ) ) / \mathcal { S } _ { \mathbb { C }, N }. \\ \mathbb { R } ( \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \end{matrix}$$

The isomorphism δ from Proposition 2.1 is therefore isometric if we view g -2 ∼ = R m as the matrix normed space induced from the inclusion ι .

Remark 2.3 . Fixing our graded nilpotent Lie algebra g as above, and the basis for g -1 ⊕ g -2 , we can with any γ ∈ M N ( C ) m associate a first order polynomial on g ∗ -2 that we by an abuse of notation write

$$\mu \ w i v c \\ \gamma \colon \mathfrak { g } _ { - 2 } ^ { * } \to M _ { N } ( \mathbb { C } ), \quad \gamma ( \xi ) \coloneqq i \sum _ { l = 1 } ^ { m } \gamma _ { l } \xi _ { l }. \\ \text{comes from viewing $\mathfrak{g}_{-2}\otimes M_{N}(\mathbb { C } ) = M_{N}(\mathbb { C } ) ^ { m }$ as tl}$$

This identification comes from viewing g -2 ⊗ M N ( C ) = M N ( C ) m as the subspace of first order homogeneous polynomials on g ∗ -2 with values in M N ( C ). Our convention ensures that self-adjoint elements of g -2 ⊗ M N ( C ), with ∗ -operation ( ∑ l Y l ⊗ γ l ) ∗ = -∑ l Y l ⊗ γ l , correspond to first order homogeneous polynomials taking self-adjoint values.

## 2.2. A characterizing set for the Rockland condition.

Definition 2.4. Define the subset E ( g , N ) ⊆ M N ( C ) m to consist of matrices γ = ( γ l ) m l =1 such that the represented operator π D ( γ ) is injective in any non-trivial irreducible unitary representation π of G .

More generally, for a closed, conical subset Φ ⊆ ̂ G we write E Φ ( g , N ) ⊆ M N ( C ) m for the subset of matrices γ = ( γ l ) m l =1 such that the represented operator π D ( γ ) is injective for all π ∈ Φ \ { 1 } .

The extension of the Rockland condition to closed cones has been studied by Nourrigat [33] and Hebisch [15].

Remark 2.5 . Note that D γ is hypoelliptic if and only if γ ∈ E ( g , N ) and D γ is H-elliptic if and only if γ ∈ E ( g , N ) ∩ -E ( ( g , N ) ∗ ) where γ ∗ := ( γ ∗ l ) m l =1 is the total hermitean adjoint. We also note that by homogeneity, whenever Φ 0 ⊆ Φ \{ } 1 is a transversal for the dilation action on Φ \ { 1 } we have that E Φ ( g , N ) coincides with the set of all matrices γ = ( γ l ) m l =1 such that the represented operator π D ( γ ) is injective for all π ∈ Φ . 0

Proposition 2.6. Let g be a graded nilpotent Lie algebra such that g -1 generates g and is equipped with an inner product.

- (1) If Φ ⊆ Φ ′ then E Φ ′ ( g , N ) ⊆ E Φ ( g , N ) .
- (2) The subset

$$\mathcal { E } _ { \Phi } ( \mathfrak { g }, N$$

- (3) 0 ∈ E ( g , N ) .

Proof. The first item is obvious. The Rockland condition relative to Φ for a homogeneous element P of the enveloping algebra ensures maximal estimates for the operators π P ( ) with uniform constant with respect to π ∈ Φ \ { 1 } , see [33]. Now it follows from a continuity argument that E Φ ( g , N ) ⊆ M N ( C ) m is open, proving the second item. We have that 0 ∈ E ( g , N ) by H¨rmander's sum of squares theorem [25]. o /square is open for any closed, conical subset Φ ⊆ ̂ G \ { 1 } .

Proposition 2.7. Suppose that φ : g → g ′ is a surjective graded Lie algebra homomorphism which induces an isometry g -1 / (Ker φ ∩ g -1 ) → g ′ -1 . Set m := dim( g -2 ) and m ′ := dim( g ′ -2 ) . Then there is a mapping defined from

$$\phi _ { * } \colon M _ { N } ( \mathbb { C } )$$

$$\mathcal { U } ( \phi ) ( D _ { \gamma } ) = D _$$

$$\sum _ { l = 1 } ^ { m } ( \gamma _ { 1, l }$$

where U ( φ ) : U ( g ) →U ( g ′ ) is the functorial map between universal enveloping algebras. Moreover φ ∗ : E ( g , N ) → E ( g ′ , N ) is a well defined mapping such that φ ∗ ( γ 1 ) = φ ∗ ( γ 2 ) if and only if

The map φ ∗ : E ( g , N ) → E ( g ′ , N ) extends to a well defined surjection φ ∗ : E Φ ( g , N ) → E ( g ′ , N ) where Φ = ˆ ( φ G ̂ ′ ) \ { 1 } ⊆ ̂ G \ { 1 } and ˆ : φ ̂ G ′ → ̂ G denotes the induced map on the spectrum.

$$\begin{array} {$$

Proof. Since φ induces an isometry g -1 / (Ker φ ∩ g -1 ) → g ′ -1 , surjectivity of φ implies that where φ ∗ ( γ ) ∈ M N ( C ) m ′ is determined from ∑ l φ ∗ ( γ ) l Y ′ l = ∑ l γ φ Y l ( l ). Therefore φ ∗ : M N ( C ) m → M N ( C ) m ′ is well defined.

Remark 2.8 . It follows from the proof of Proposition 2.7 that when Φ = ˆ ( φ G ̂ ′ ) \ { 1 } then for γ ∈ M N ( C ) m , we have that γ ∈ E Φ ( g , N ) if and only if φ ∗ ( γ ) ∈ E ( g ′ , N ). Or in other words, for Φ = ˆ ( φ G ̂ ′ ) \ { 1 } we have an equality E Φ ( g , N ) = { γ ∈ M N ( C ) m : φ ∗ ( γ ) ∈ E ( g ′ , N ) } . (2.2)

If φ is surjective, the integrated map φ : G → G ′ is also surjective and induces an injective map ˆ : φ ̂ G ′ → ̂ G . Therefore, the represented operator π D ( γ ) is injective in any π = ˆ ( φ π ′ ) ∈ Φ if and only if π ′ ( D φ ∗ ( γ ) ) = π ◦ φ D ( γ ) is injective for any π ′ ∈ ̂ G ′ \ { 1 } . Therefore, φ ∗ ( γ ) ∈ E ( g ′ , N ) for γ ∈ E Φ ( g , N ). The defining equation ∑ l φ ∗ ( γ ) l Y ′ l = ∑ l γ φ Y l ( l ) and surjectivity of φ implies that φ ∗ : E Φ ( g , N ) →E ( g ′ , N ) is surjective. /square

Corollary 2.9. Let φ j : g → g ( j ) be a collection of graded, surjective Lie algebra homomorphisms onto Lie algebras such that ( g ( j )) -1 generate g ( j ) . Assume that

̂ G = ∪ j ̂ ̂ φ j ( G j ) . Then, equipping ( g ( j )) -1 with the inner product induced from φ j , we have the equality ( g , N ) = j γ M N ( C ) m : ( φ j ) ∗ γ ( g ( j ) , N ) .

Proof. By assumption, E ( g , N ) = ∩ E j Φ j ( g , N ) where Φ j = ̂ ̂ φ j ( G j ) \ { 1 } . The corollary is now immediate from the observation in Equation (2.2). /square

$$\mathcal { E } ( \mathfrak { g }, N ) = \cap _ { j } \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m } \, \colon ( \phi _ { j } ) _ { * } \gamma \in \mathcal { E } ( \mathfrak { g } ( j ), N ) \}. \\. \quad \, \quad \, \quad \, \quad \, \quad \, \quad \, \quad \, \widehat { \sim } \widehat { \sim } \cdot \cdot \cdot \cdot \, \dots \, \dots$$

## 2.3. Ramifications for the index of sub-Laplacians.

Proposition 2.10. Assume that X is a Carnot manifold of nilpotency degree &gt; 2 coming from an equiregular differential system. Then any sub-Laplacian D γ ∈ DO 2 H ( X ; E ) (as in Equation (1.2) ) on a vector bundle E of rank N is H -elliptic if and only if

$$\gamma \colon ( T ^ { - 1 } \mathfrak { X } ) ^ { \perp } / ( T ^ { - 2 } \mathfrak { X } ) ^ { \perp } & \to \text{End} ( E ), \\ \dots \, \iota \, \iota \, \iota \, \dots \quad \ast \sim \tau / \iota \quad \mathfrak { Y } \quad \tau \quad \tau \quad \tau \quad \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \, \dots \, \iota \end{array}$$

We note that by [36], the Rockland condition is equivalent to maximal hypoellipticity.

for any x ∈ X satisfies that γ , x -γ ∗ x ∈ E ( t H X x , N ) . In particular, if for any x ∈ X t , H X x has property E ∗ N then whenever D γ is H -elliptic then D tγ is H -elliptic for any t ∈ [0 , 1] .

Proof. It follows from the equivalence of the Rockland condition for D γ and D ∗ γ with H-ellipticity (see [10]) that D γ ∈ DO 2 H ( X ; E ) is H -elliptic if and only if γ , x -γ ∗ x ∈ E ( t H X x , N ) for any x ∈ X . By definition, if g has property E ∗ N then t E ( g , N ) ⊆ E ( g , N ) for any t ∈ (0 , 1] and by Proposition 2.6, item (3), 0 = 0 . E ( g , N ) ⊆ E ( g , N ). In particular, t E ( g , N ) ⊆ E ( g , N ) for any t ∈ [0 , 1] if g has property E ∗ N . The second conclusion follows. /square

Theorem 2.11. Assume that X is a compact Carnot manifold of nilpotency degree &gt; 2 coming from an equiregular differential system. If for any x ∈ X t , H X x has property E ∗ N , then any sub-Laplacian D γ ∈ DO 2 H ( X ; E ) (as in Equation (1.2) ) on a vector bundle E of rank N which is H -elliptic will have vanishing index. In particular, Problem 1 has a negative solution and we even have [ D γ ] = 0 in K 0 ( X ) .

Proof. If D γ is H -elliptic, Proposition 2.10 implies that the operator D tγ is H -elliptic for any t ∈ [0 , 1]. The path ( D tγ ) t ∈ [0 1] , is continuous so

$$\text{index} ( D _ { \gamma } ) = \text{index} ( D _ { 0 } ) = 0,$$

since D 0 is a relatively compact perturbation of a self-adjoint operator. The argument that [ D γ ] = 0 in K 0 ( X ) goes mutatis mutandis. /square

## 3. Sufficient conditions and necessary conditions

3.1. Results of Helffer and Rothschild-Stein. Both Helffer [17], see also [18], and Rothschild-Stein [35] provide sufficient conditions and necessary conditions for γ ∈ E ( g , 1). Recall the construction of the inclusion ι : g -2 → so g ( -1 ) from Remark 2.2. We present these results in a condensed form:

Theorem 3.1. Let g be a graded nilpotent Lie algebra of nilpotency degree r and generated by g -1 . Equip C m with the norm induced from C m ∼ = g -2 ⊗ R C ι -→ so g ( -1 ⊗ R C ) and the operator norm on so . Identify R m ⊆ C m the closed subspace closed under entrywise conjugation.

- (1) If dim( g -2 ) = 2 it holds that

$$\mathcal { E } ( \mathfrak { g }, 1 ) & = \{ \gamma \in \mathbb { C } ^ { m } \colon \| \text{Im} ( \gamma ) \| _ { \mathbb { R } ^ { m } / \mathbb { R } \text{Re} ( \gamma ) ) } < 1 \}. \\.. \. \. \ \ \cdots \.. \.. \.. \ \cdots \.. \ \cdots \.. \ \cdots \.. \.. \.. \.. \........................................................................................................................................................................................................$$

/negationslash

- (2) Assume that r &gt; 2 . We have that γ / ∈ E ( g , 1) if Re( γ ) = 0 and ‖ Im( γ ) ‖ C m ≥ 1 .

/negationslash

Part 1) of this theorem is found as [35, Theorem 1 and 2] and Part 2) of this theorem is stated as [17, Theoreme 5]. In [17, 18, 35], the results are stated in terms of the commutator description (2.1) of the operators D γ . Said results translate to the setting of Theorem 3.1 using the mapping δ of Proposition 2.1. We note that even though the statement of [35, Theorem 2] is correct, its proof contains a minor gap that is filled in Section 5 below. In [35, Theorem 2], the necessity of ‖ Im( γ ) ‖ R m / R Re( γ )) &lt; 1 is proved under the condition that ( g -1 ⊕ g -2 ) / ( R ∑ l Re( γ ) l Y l ) is not a Heisenberg algebra, which holds if dim( g -2 ) = 2.

/negationslash

Corollary 3.2. Any graded nilpotent Lie algebra g such that g -1 generates g and dim( g -2 ) = 2 has property E ∗ 1 .

Proof. We need to verify that for any t ∈ (0 , 1], we have an inclusion

$$\ t { \mathcal { E } } ( \mathfrak { g }, 1 ) & \subseteq \mathcal { E } ( \mathfrak { g }, 1 ). \\ \cdots \quad \cdots \quad \cdots$$

Using that R Re( γ ) = R Re( tγ ) for any t ∈ (0 , 1], Theorem 3.1 implies that

$$\max \in \mathbb { C } ( \sqrt { 1 } ) - \max \in \mathbb { C } ( \sqrt { 1 } ) \, \text{on any } \iota \in \mathbb { U }, \iota \, \text{in some} \, \iota \, \text{in prime} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{in any} \, \iota \, \text{\ } \leq \{ t \gamma \in \mathbb { C } ^ { m } \, \colon \| \, \text{Im} ( \gamma ) \| _ { \mathbb { R } ^ { m } / \mathbb { R } \, \text{Re} ( \gamma ) ) } < 1 \} = \mathcal { E } ( \mathfrak { g }, 1 ).$$

/square

For a real number p &gt; 0, write H p ( ) := ( -∞ -, p ] ∪ [ p, ∞ ⊆ ) C .

Proposition 3.3. Assume that g is nilpotent of step length &gt; 2 and that g -2 ≡ [ g -1 , g -1 ] has dimension 1 . Then it holds that

$$\mathcal { E } ( \mathfrak { g }, N ) = \{ \gamma \in M _ { N } ( \mathbb { C } ) \, \colon H ( 1 ) \cap \text{Spec} ( i \gamma ) = \emptyset \}. \\ \therefore \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdots$$

In particular, any nilpotent Lie algebra g of step length &gt; 2 and with g -2 one-dimensional has property E ∗ N for any N .

/negationslash

For N &gt; 1, we can up to an invertible matrix in C N , assume that γ ∈ M N ( C ) is in Jordan form. Without restriction, we can assume that γ is in fact one Jordan block

Proof. First, we consider N = 1. Since g -2 has dimension 1, R = R Re( γ ) as soon as Re( γ ) = 0 and in this case γ ∈ E ( g , 1) by part 1 of Theorem 3.1. If Re( γ ) = 0, Theorem 3.1 implies that γ ∈ E ( g , 1) if and only if ‖ Im( γ ) ‖ R &lt; 1. We conclude that E ( g , 1) = { γ ∈ C : iγ / H ∈ (1) } and the corollary follows for N = 1.

$$\gamma = \begin{pmatrix} \lambda & 1 & 0 & 0 & \cdots \\ 0 & \lambda & 1 & 0 & \cdots \\ \vdots & 0 & \ddots & \ddots & \vdots \\ 0 & & \lambda & 1 \\ 0 & & \lambda \end{pmatrix},$$

for some λ ∈ C . Considering the Rockland condition on the bottom right corner, we see from the case N = 1 that if γ ∈ E ( g , N ) then iλ / H ∈ (1). Conversely, a linear algebra argument and the Rockland condition shows that if iλ / H ∈ (1) then γ ∈ E ( g , N ). The corollary follows. /square

3.2. Related results using a quotient map. We say that a two-step nilpotent Lie group p = p -1 ⊕ p -2 = p -1 ⊕ z is polycontact if for any ξ ∈ z ∗ \ { 0 } the Kirillov form

$$\omega _ { \xi } ( X, Y ) \coloneqq \xi [ X, Y ],$$

is non-degenerate on p -1 = g / z . We write R [ -1] for the graded abelian Lie algebra R concentrated in degree -1. For an inner product space V , we write S V ( ) for its sphere.

Proposition 3.4. Assume that g is nilpotent and there is a graded Lie algebra quotient φ : g → ⊕ p R [ -1] where p is a step 2 polycontact Lie algebra. Let m := dim( g -2 ) . We set p := dim( p -1 ) / 2 , m p := dim( p -2 ) and write α for the smallest positive singular value of the quotient map of inner product spaces g -2 → p -2 . Then the constant p g := pα satisfies that

$$\mathcal { E } ( \mathfrak { g }, N ) \subseteq \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m } \, \colon H ( p _ { \mathfrak { g } } ^ { 0 } ) \cap S p e c ( \gamma ( \xi ) ) = \emptyset \quad \forall \xi \in S ( \text{im} ( \phi ^ { * } ) \cap \mathfrak { g } _ { - 2 } ^ { * } ) \}.$$

In fact,

$$\mathcal { E } ( \mathfrak { p } \oplus \mathbb { R } [ - 1 ], N ) & = \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m _ { \mathfrak { p } } } \colon H ( p ) \cap \text{Spec} ( \gamma ( \xi ) ) = \emptyset \quad \forall \xi \in S ( \mathfrak { p } _ { - 2 } ^ { * } ) \}. \\ \varkappa \quad \varkappa \quad \varkappa \quad \dots \quad \varkappa \quad. \quad \varkappa \quad. \quad \varkappa \quad. \quad \varkappa \quad. \quad \dots \quad. \quad. \quad. \quad.$$

↦

Proof. By Proposition 2.7, it suffices to compute E ( p ⊕ R [ -1] , N ). Indeed, linearity of ξ → γ ξ ( ) and scaling properties of the set-valued function

$$M _ { N } ( \mathbb { C } ) ^ { m } \ni \gamma \mapsto \cup _ { \xi \in S ( \lim ( \phi ^ { * } ) \cap _ { \mathfrak { g } ^ { * } _ { - 2 } } ^ { * } ) } \text{Spec} ( \gamma ( \xi ) ), \\ \dots \dots \dots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots$$

↦

imply that the constant p g := pα has the sought after property. Write X 0 for the relevant basis element of R [ -1] so

Let P denote the simply connected Lie group integrating p . The unitary, irreducible representations of P × R are parametrized by R ∗ × p ∗ / Ad( P ). Since P is polycontact, we have that p ∗ / Ad( P ) = p ∗ -1 /unionsq p ∗ -2 \ { 0 } , where p ∗ -1 induces the characters and p ∗ -2 \ { 0 } induces the flat orbit representations. For a non-trivial character π = ( ξ 0 , ξ 1 ) ∈ R ∗ × p ∗ -1 , π D ( γ ) = ξ 2 0 + | ξ 1 | 2 &gt; 0 and it remains to characterize the injectivity of π D ( γ ) for π = ( ξ 0 , ξ 2 ) ∈ R ∗ × p ∗ -2 \ { 0 } . A short computation shows that for such representations,

$$D _ { \gamma } & = X _ { 0 } ^ { 2 } + \sum _ { j = 1 } ^ { 2 p } X _ { j } ^ { 2 } + \sum _ { l = 1 } ^ { m } \gamma _ { l } Y _ { l }. \\ \text{plv connected Lie group integrating $\mathfrak{p}$}.$$

$$\cdot - \varepsilon \cdot \cdot \cdot \cdot \\ \pi ( D _ { \gamma } ) = \xi _ { 0 } ^ { 2 } + | \xi _ { 2 } | H + \sum _ { l = 1 } ^ { m } i \xi _ { 2, l } \gamma _ { l } = \xi _ { 0 } ^ { 2 } + | \xi _ { 2 } | H + \gamma ( \xi _ { 2 } ), \\ \varepsilon = ( \xi _ { 2 \ 1 }, \dots, \xi _ { 2 \ m } ) \text{ in the basis } Y _ { 1 }, \dots, Y _ { m } \text{ and } H \text{ denotes the harmonic } \alpha.$$

where ξ 2 = ( ξ 2 1 , , . . . , ξ 2 ,m ) in the basis Y , . . . , Y 1 m and H denotes the harmonic oscillator on R p . As above, we write γ ξ ( ) = i ∑ m l =1 ξ 2 ,l γ l . Using the auxiliary variable t = ξ 2 0 / ξ | 2 |

we conclude from the spectrum of the harmonic oscillator that π D ( γ ) is injective if and only if Spec( γ ξ / ξ ( 2 | 2 | )) does not intersect the set

$$\cup _ { t \geq 0, k \in \mathbb { N } } \{ ( t + k + p ), - ( t + k + p ) \} = H ( p ). \\ \Re \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdots$$

Therefore, π D ( γ ) is injective for all π ∈ R ∗ × p ∗ -2 \ { 0 } if and only if γ = ( γ , . . . , γ 1 l ) satisfies

$$\text{Spec} ( \gamma ( \xi ) ) \cap H ( p ) = \emptyset, \\ \text{one} \text{at} h$$

for all ξ ∈ p ∗ -2 \ { 0 } of unit length.

/square

Corollary 3.5. Let g be a graded nilpotent Lie algebra such that g -1 generates g . Assume that there are quotient maps φ j ( ) : g → p ( j ) ⊕ R [ -1] for polycontact step two nilpotent Lie algebras p ( j ) such that

$$S ( \mathfrak { g } _ { - 2 } ^ { * } ) = \cup _ { j } S ( \text{im} ( \phi ( j ) ^ { * } ) \cap \mathfrak { g } _ { - 2 } ^ { * } ). \\ \nu _ { 1 } \quad \sim \quad \nu _ { 2 } \quad \nu _ { 3 }.$$

Then there is a real number p g &gt; 0 such that

$$\mathcal { E } ( \mathfrak { g }, N ) & \subseteq \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m } \, \colon H ( p _ { \mathfrak { g } } ^ { 0 } ) \cap \text{Spec} ( \gamma ( \xi ) ) = \emptyset \quad \forall \xi \in S ( \mathfrak { g } ^ { * } _ { - 2 } ) \}. \\ \mathcal { \hat { \lambda } } \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot.$$

3.3. Paraphrasing Rotschild-Stein's theorem when N &gt; 1 . In this subsection we shall partially extend Rotschild-Stein's theorem (summarized in Theorem 3.1 above) to the matrix case. Equip M N ( C ) m with the norm induced from

$$M _ { N } ( \mathbb { C } ) ^ { m } & \cong \mathfrak { g } _ { - 2 } \otimes _ { \mathbb { R } } M _ { N } ( \mathbb { C } ) \stackrel { \iota } { \to } \mathfrak { s o } ( \mathfrak { g } _ { - 1 } \otimes _ { \mathbb { R } } M _ { N } ( \mathbb { C } ) ), \\ \dots \frac { 1 } { 2 } \dots \dots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots.$$

and the operator norm on so . Write W for the subspace of M N ( C ) m of elements invariant under entrywise hermitean conjugate.

Theorem 3.6. Let g be a graded nilpotent Lie algebra generated by g -1 . It holds that

$$\mathcal { E } ( \mathfrak { g }, N ) \supseteq \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m } \colon \| \text{Im} ( \gamma ) \| _ { W / \mathbb { R } \, R e ( \gamma ) ) } < 1 \}. \\ \cdot \hat { \lambda } \cdots \cdots \cdots \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \vec { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { \lambda } \hat { `}$$

Our argument follows that of [35, Theorem 1] closely.

Proof. Using the Rockland condition (or standard techniques) it suffices to show that if ‖ Im( γ ) ‖ W/ R Re( γ )) &lt; 1, there is a constant A &gt; 0 such that

For notational simplicity we write γ = a + ib where a ≡ Re( γ , b ) ≡ Im( γ ) ∈ M N ( C ) m are self-adjoint. By the same argument as in [35, Lemma 2.7], there is a t ∈ R such that ‖ b + ta ‖ W = ‖ ‖ b W/ a R .

$$\cdots \cdots \cdots & \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, \mathbb { C } ^ { N } ) } ^ { 2 } \leq A | \langle f, D _ { \gamma } f \rangle |, \quad \forall f \in C _ { c } ^ { \infty } ( G, \mathbb { C } ^ { N } ). & \quad \text{(3.1)} \\ \intertext { r notational simnifictv we write } \sim a + i h \, \text{where } a = R e ( \sim \lambda ) \ h = \text{Im} ( \sim \lambda ) \sim M x, \text{(} \mathbb { C } \rangle ^ { m } }$$

We can write

$$\langle$$

$$\text{We can write} \\ \langle f, D _ { \gamma } f \rangle = & \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, C ^ { N } ) } ^ { 2 } + \sum _ { l = 1 } ^ { m } \left [ \langle f, a _ { l } Y _ { l } f \rangle + i \langle f, b _ { l } Y _ { l } f \rangle \right ] = \\ = & \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, C ^ { N } ) } ^ { 2 } + ( 1 - i t ) \sum _ { l = 1 } ^ { m } \langle f, a _ { l } Y _ { l } f \rangle + i \sum _ { l = 1 } ^ { m } \langle f, ( b _ { l } + t a _ { l } ) Y _ { l } f \rangle \\ \text{Rearranging these terms and using the triangle equality give the estimate}$$

Rearranging these terms and using the triangle equality give the estimate

$$\begin{array} { c } \lambda \\ \text{Im} ( \langle f, D _ { \gamma } f \rangle ) = \sum _ { l = 1 } ^ { m } \langle f, a _ { l } Y _ { l } f \rangle, \end{array}$$

$$& \text{Rearranging these terms and using the triangle equality give the estimate} \\ & \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, C ^ { N } ) } ^ { 2 } \leq | \langle f, D _ { \gamma } f \rangle | + | 1 - i t | \left | \sum _ { l = 1 } ^ { m } \langle f, a _ { l } Y _ { l } f \rangle \right | + \left | \sum _ { l = 1 } ^ { m } \langle f, ( b _ { l } + t a _ { l } ) Y _ { l } f \rangle \right |. \ ( 3. 2 ) \\ & \quad \text{To estimate the right hand side, we make the following observations. We have that} \\ & \quad \cdots \, \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdclots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \cdots \, \colon$$

and therefore

Moreover, using Proposition 2.1 we write

$$\sum _ { l = 1 } ^ { m } \langle f, ( b _ { l } + t a _ { l } ) Y _ { l } f \rangle = \sum _ { k, l = 1 } ^ { n } \langle f, \delta ( b + t a ) _ { k, l } [ X _ { k }, X _ { l } ] f \rangle = \text{Tr} ( \delta ( b + t a ) \rho ),$$

$$\left | \sum _ { l = 1 } ^ { m } \langle f, a _ { l } Y _ { l } f \rangle \right | & \leq | \langle f, D _ { \gamma } f \rangle |. & ( 3. 3 ) \\ \text{position} 2. 1 \, \text{we write} \\ \cdot \, \tilde { \lambda } \, \vee \, \tilde { \nu } \, \cdot \, \sum _ { l = 1 } ^ { n } \, / \tilde { \lambda } \, \tilde { \lambda } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu } \, \cdot \, \tilde { \nu }$$

where

Following the same argument as in [35, Lemma 2.7], the trace norm of ρ satisfies the estimate

$$\rho = \int _ { G } ( [ X _ { k }, X _ { l } ] f ) ( g ) \otimes f ( g ) ^ { * } \mathrm d g \in \mathfrak s \mathfrak o _ { n } ( M _ { N } ( \mathbb { C } ) ). \\ \mathrm e \, \text{same argument as in } [ 3 5, \text{ Lemma} \, 2. 7 ], \text{ the trace norm of } \rho$$

$$\| \rho \| _ { \mathcal { L } ^ { 1 } } & \leq \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, \mathbb { C } ^ { N } ) } ^ { 2 }. \\ \text{per bound}$$

$$& \text{We therefore have the upper bound} \\ & \left | \sum _ { l = 1 } ^ { m } \langle f, ( b _ { l } + t a _ { l } ) Y _ { l } f \rangle \right | = & | \text{Tr} ( \delta ( b + t a ) \rho ) | \leq & ( 3. 4 ) \\ & \quad \leq \| \delta ( b + t a ) \| _ { M _ { n } ( M _ { N } ( \mathbb { C } ) ) } \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, \mathbb { C } ^ { N } ) } ^ { 2 } = \| b \| _ { W / \mathbb { R } a } \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, \mathbb { C } ^ { N } ) } ^ { 2 }.$$

We therefore have the upper bound

$$\leq & \| \delta ( b + t a ) \| _ { M _ { n } ( M _ { N } ( \mathbb { C } ) ) } \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, \mathbb { C } ^ { N } ) } ^ { 2 } = \| b \| _ { W / \mathbb { R } a } \sum _ { j = 1 } ^ { n } \| X _ { j } f \| _ { L ^ { 2 } ( G, \mathbb { C } ^ { N } ) } ^ { 2 }. \\ \intertext { r a n e o t h o a c t i m o t a ( 2 9 ) \, \min e t h o a c t i m o t a ( 2 9 ) \, \text{and} ( 2 9 ) \, \text{and} \, \text{and} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \,\text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or } \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \text{or} \, \end{$$

If we rearrange the estimate (3.2) using the estimates (3.3) and (3.4) we arrive at the desired estimate (3.1) for the constant

$$A = \frac { 1 + | 1 + i t | } { 1 - \| \text{Im} ( \gamma ) \| _ { W / \mathbb { R } \text{Re} ( \gamma ) ) } }.$$

/square

We now turn to partial converses of Theorem 3.6. Given a graded nilpotent Lie algebra g = ⊕ r j =1 g -j the truncated Lie algebra g -1 ⊕ g -2 of nilpotency degree 2 is defined by declaring g -2 central and the Lie bracket g -1 ∧ g -1 → g -2 defined from that on g . We note that if g -1 generates g , then g -1 generates the truncated Lie algebra g -1 ⊕ g -2 and the center of g -1 ⊕ g -2 is g -2 . In particular, a flat coadjoint orbit for g -1 ⊕ g -2 corresponds to a coadjoint equivalence class of ξ ∈ g ∗ -2 such that the Kirillov form ω ξ ( X,Y ) := ξ X, Y [ ] is non-degenerate on g -1 .

Theorem 3.7. Let g be a graded nilpotent Lie algebra generated by g -1 . If the Lie algebra g -1 ⊕ g -2 truncated to nilpotency degree 2 does not admit any flat coadjoint orbits, then

$$\mathcal { E } ( \mathfrak { g }, N ) \subseteq \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m } \colon \text{Spec} ( \gamma ( \xi ) ) \cap H ( 1 ) = \emptyset \quad \forall \xi \in S ( \mathfrak { g } _ { - 2 } ^ { * } ) \}. \\ \mathcal { - } \quad. \quad. \quad. \quad. \quad. \quad. \quad. \quad. \quad. \quad. \quad. \quad. \quad. \quad.$$

Proof. We prove that if γ ∈ M N ( C ) m satisfies that γ ξ ( ) has an eigenvalue ≤ -1 for some ξ ∈ S ( g ∗ -2 ) then D γ is not hypoelliptic. Indeed, if this is the case, we take such a ξ and the eigenvector v of γ ξ ( ) with eigenvalue ≤ -1. Since g -1 ⊕ g -2 admits no flat coadjoint orbits, ξ ∈ g ∗ -2 defines a degenerate Kirillov form on g -1 and the argument in [35, Proof of Theorem 2, Section 4] produces a non-trivial irreducible unitary representation ( π, H π ) of G and a vector H ∈ H π such that π D ( γ )( H ⊗ v ) = 0. In particular, D γ is not hypoelliptic. /square

Definition 3.8. Let γ ∈ g -2 ⊗ M N ( C ) be written as γ = a + ib for a and b self-adjoint. We say that γ has property α if for any unit vectors ξ ∈ g ∗ -2 and v ∈ Ker( ( ξ a )) we have that |〈 v, ξ ( b v ) 〉 C N | &lt; 1.

Theorem 3.9. Let g be a graded nilpotent Lie algebra generated by g -1 . If the Lie algebra g -1 ⊕ g -2 truncated to nilpotency degree 2 does not admit any flat coadjoint orbits, then

$$\mathcal { E } ( \mathfrak { g }, N ) \subseteq \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { m } \colon \gamma \, h a s \, p r o p e r t y \, \alpha \}. \\ \lambda \, \lambda \, \cdot \, \quad \omega \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdots$$

/negationslash

By Theorem 3.6 it suffices to show that if γ does not have property α , then D γ is not hypoelliptic. In fact, this can be seen immediately from Theorem 3.7 since if γ does not have property α then Spec( γ ξ ( )) ∩ H (1) = ∅ for some unit vector ξ ∈ g ∗ -2 . We will however take the chance to expand the argument and in parallel showcase the difference to the converse inclusion of Theorem 3.6. Our argument follows that of [35, Theorem 2] closely.

↦

↦

Since g -1 ⊕ g -2 admits no flat coadjoint orbits, ξ ∈ g ∗ -2 defines a degenerate Kirillov form on g -1 and the argument in [35, Proof of Theorem 2, Section 4] produces an irreducible unitary representation ( π, H π ) of G and a vector H ∈ H π such that π D ( ib )( H ⊗ v ) = 0. In particular, D ib is not hypoelliptic. If we in addition assume that γ does not have property α , we can assume that the vector state e ∈ M N ( C ) is of the form e = v 0 ⊗ v ∗ 0 where ξ a v ( ) 0 = 0. As such, the argument used in the proof of Theorem 3.7 coming from [35, Theorem 2] carries over and disproves hypoellipticity of D γ .

For notational simplicity we write γ = a + ib where a ≡ Re( γ , b ) ≡ Im( γ ) ∈ M N ( C ) m are self-adjoint. We note that ‖ ‖ b W/ a R is the maximal value of ρ → | Tr( ρb ) | where ρ ∈ M N ( C ) m ranges over the trace norm sphere ‖ ρ ‖ L 1 = 1 and satisfying Tr( ρa ) = 0. So by convexity of ρ → | Tr( ρb ) | the maximum is attained in an extremal point ξ ⊗ e for ξ ∈ g ∗ -2 of trace norm 1 and e ∈ M N ( C ) ∗ a vector state. We shall assume that Tr( ρb ) = -‖ ‖ b W/ a R ≤ -1. Since e is a vector state, the fact that Tr M N ( C ) m ( ρb ) = Tr M N ( eξ b ( )) ≤ -1 and the min-max principle implies that the self-adjoint ξ b ( ) ∈ M N ( C ) has an eigenvector v with eigenvalue ≤ -1.

Question 3.10 . In the statement of Theorem 3.6, can the property ‖ Im( γ ) ‖ W/ R Re( γ )) &lt; 1 be determined from pointwise spectral properties of the first degree polynomial γ ξ ( ) on g ∗ -2 similar to the condition in Theorem 3.7?

## 4. Examples

To describe the irreducible, unitary representations we use the Kirillov orbit method: ̂ G = g ∗ / Ad ∗ . In all examples we focus on describing the situation in a set of generic orbits Γ ⊆ ̂ G that will be an open, dense, Hausdorff subset. In all cases, hypoellipticy can be characterized from a 'good enough' invertibility argument in the generic orbits.

We first recall the notion of flat orbits. Writing z for the center of g , a ξ ∈ g ∗ belongs to a flat coadjoint orbit if the induced Kirillov form ω ξ ( X,Y ) := ξ ([ X,Y ]) is non-degenerate on g / z . If a flat orbit exists, we take the set of generic orbits Γ ⊆ ̂ G as the set corresponding to all flat orbits; if it is non-empty it is a Zariski-open, dense, Hausdorff subset that can be identified with the open subset

$$\Gamma & = \{ \xi \in \mathfrak { z } ^ { * } \colon P f ( \omega _ { \xi } ) \neq 0 \}. \\ \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \ dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots.$$

/negationslash

Here Pf denotes the Pfaffian, so the polynomial ξ → Pf( ω ξ ) takes a non-zero value at ξ if and only if ω ξ is non-degenerate.

↦

4.1. Step 2 with flat orbits. Consider the case when g is step 2, and admits a flat orbit. Consider the trivial vector bundle V := Γ × g / z over Γ and chose a metric thereon. The Kirillov form induces a symplectic form on V . The metric and the Kirillov form induces a canonical complex structure on V , which is adapted to a potentially different metric. We form the bosonic Fock space bundle

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00091v2-figure-0c924e02-fa5f-44f7-a6c5-e9b9b20f6845.jpg)

$$\begin{smallmatrix} \cdots \cdots \cdots \cdots \\ \mathfrak { F } _ { V } \coloneqq \bigoplus _ { k = 0 } ^ { \infty } V ^ { \otimes _ { c } ^ { s y m } k }. \end{smallmatrix}$$

This is a Hilbert space bundle over Γ. Its relevance comes from the fact that the complex structure and the Kirillov orbit method gives F V the structure of a bundle of representations over Γ. Using the metric, we define the complex linear symmetric operator g ω = | ω | = √ -ω 2 . We define the endomorphisms

$$s _ { k } \colon & V ^ { \otimes _ { C } ^ { s y m } k } \to V ^ { \otimes _ { C } ^ { s y m } k }, \\. &. &. &. &. &.$$

on the k :th symmetric tensor power, from

$$s _ { k } ( v _ { 1 } \otimes _ { \mathbb { C } } ^ { \text{sym} } \dots \otimes _ { \mathbb { C } } ^ { \text{sym} } v _ { k } ) \coloneqq \sum _ { j = 1 } ^ { k } v _ { 1 } \otimes _ { \mathbb { C } } ^ { \text{sym} } \dots \otimes _ { \mathbb { C } } ^ { \text{sym} } g _ { \omega } v _ { j } \otimes _ { \mathbb { C } } ^ { \text{sym} } \dots \otimes _ { \mathbb { C } } ^ { \text{sym} } v _ { k }. \\ \text{If we use a metric on $V$ making the complex structure defined from the Kirillov form}$$

If we use a metric on V making the complex structure defined from the Kirillov form adapted, then g ω is the identity operator and s k is k times the identity operator. We assume that g -2 = z in which case we can identify z with a subspace of so g ( -1 ). Recall the following theorem from [13].

Theorem 4.1. Let g be a step 2 nilpotent Lie group with flat orbits and g -2 = z of dimension m . Fix a metric on g -1 and an orthonormal basis X ,. . . , X 1 n . Then E ( g , N ) consists of those γ ∈ M N ( C ) m such that

↦

$$\gamma _ { k } \colon \Gamma \ni \xi \mapsto s _ { k } ( \xi ) + \frac { \text{Tr} ( | \omega _ { \xi } | ) } { 2 } + \gamma ( \xi ) \in \text{End} ( V ^ { \otimes _ { C } ^ { s y m } k } \otimes \mathbb { C } ^ { N } ),$$

satisfies that γ k ( ξ ) is invertible for all ξ ∈ Γ and

Remark 4.2 . In the special case that g is polycontact, i.e. Γ = z ∗ \ { 0 } , the homogeneity and compactness of the sphere in z ∗ ensure that E ( g , N ) consists of those γ ∈ so n ( M N ( C )) such that γ k is invertible for all ξ ∈ Γ.

$$& \text{satisfies that $\gamma_{k}(\xi)$ is invertible for all $\xi\in\Gamma$ and} \\ & \sup _ { k, \xi \in \Gamma } \left \| \left ( s _ { k } ( \xi ) + \frac { \text{Tr} ( | \omega _ { \xi } | ) } { 2 } \right ) \left ( s _ { k } ( \xi ) + \frac { \text{Tr} ( | \omega _ { \xi } | ) } { 2 } + \gamma ( \xi ) \right ) ^ { - 1 } \right \| _ { \text{End} ( V ^ { \circ } _ { C } s ^ { \text{sym} _ { k } } \otimes C ^ { N } ) } < \infty. \\ & \text{Remark $4.2.$ In the special case that $\mathfrak{g}$ is polycontact, i.e. $\Gamma = \mathfrak{s}^{*}\ \{0}, the homogeneity} \\ & \text{and compactness of the sphere in $\mathfrak{s}^{*}$ ensure that $\mathcal{E}(g, N)$ consists of those $\gamma \in \mathfrak{so}_{n}(M_{N}(\mathbb{C})}$)}$$

In general, Γ is noncompact (e.g. Γ = ( R × ) 2 for a product of two Heisenberg groups). Therefore the bound on the inverse is in general not superfluous.

4.2. The Engel-Lie algebra. Consider the four dimensional Lie algebra g spanned by X ,X ,X ,X 1 2 3 4 where the non-zero brackets are given by

$$[ X _ { 1 }, X _ { 2 } ] = X _ { 3 } \text{ \ and \ } [ X _ { 1 }, X _ { 3 } ] = X _ { 4 }.$$

This Lie algebra is called the Engel-Lie algebra as it relates to Engel structures on 4manifolds (whose existence is equivalent to parallellizability). The Lie algebra g has nilpotency degree 3 with g -1 = R X 1 + R X 2 , g -2 = R X 3 , and g -3 = z = R X 4 . Proposition 3.3 implies the following. Recall our notation H p ( ) = ( -∞ -, p ] ∪ [ p, ∞ ).

Proposition 4.3. The four dimensional, three step nilpotent Lie algebra g satisfies that

$$\mathcal { E } ( \mathfrak { g }, N ) = \{ \gamma \in M _ { N } ( \mathbb { C } ) \, \colon H ( 1 ) \cap \text{Spec} ( i \gamma ) = \emptyset \}. \\ \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \end{cases}$$

To get a better feeling, let us describe the situation in more detail. The representation space ̂ G of the Engel-Lie group was described in detail in [9, Example 1.3.10 and 2.2.2] and [26, Chapter 3.3]. By [26, page 80], are all subsets that are Hausdorff in their respective induced topologies. Here p and q refer to representation parameters coming from the polynomial invariants p = X 4 and q = 2 X X 2 4 -X 2 3 for the coadjoint action.

$$&, \text{Chapter } \vec { 3. 3 } \text{]. By } [ \vec { 2 6 }, \text{page } 8 0 ], \\ & \widehat { G } = \Gamma _ { 1 } \dot { \mathcal { U } } \Gamma _ { 2 } \dot { \mathcal { U } } _ { 3 }, \quad \text{where} \quad \begin{cases} \Gamma = \Gamma _ { 1 } = \mathbb { R } _ { p } ^ { \times } \times \mathbb { R } _ { q }, \\ \Gamma _ { 2 } = \{ 0 _ { p } ^ { + }, 0 _ { p } ^ { - } \} \times \mathbb { R } _ { q, < 0 }, \\ \Gamma _ { 3 } = \{ 0 _ { p q } \} \times \mathbb { R } ^ { 2 }, \end{cases} \\ & \text{subsets that are Hausdorff in their respective induced topologies.  Her } \text{representation parameters coming from the polynomial invariants } p \quad \\ & \quad \longrightarrow \quad.. \quad.. \quad..$$

We now describe π D ( γ ) where π ranges over ̂ G . Here D γ = X 2 1 + X 2 2 + γX . 3

For the characters ( ξ 1 , ξ 2 ) ∈ Γ , 3

which is invertible in all non-trivial characters. For π ± 0 ,q ∈ Γ 2 = { 0 + p , 0 -p } × R q,&lt; 0 , the functional dimension is 1 and the computation on [26, page 83] shows that

$$\pi _ { ( \xi _ { 1 }, \xi _ { 2 } ) } ( D _ { \gamma } ) = - \xi _ { 1 } ^ { 2 } - \xi _ { 2 } ^ { 2 }, \\ \dots \dots \cdots \cdots \cdots + \cdots.$$

$$\int a \min \min i a \min c o m p u v a \min \min \frac { 1 } { 2 }, \, \text{page} \, \cup _ { 1 } \, \text{now} \, \min a \\ \pi _ { 0, q } ^ { \pm } ( D _ { \gamma } ) = \partial _ { x } ^ { 2 } + 4 \pi ^ { 2 } q x ^ { 2 } \pm 2 \pi \sqrt { - q } \gamma = - 2 \pi \sqrt { - q } ( H \pm \gamma ), \\ \int \colon \, \min \min i a \min \min i a \min \min i b \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \max \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \sum \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \Min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \min \frac { 1 } { 2 } }$$

where H is unitarily equivalent to a harmonic oscillator with spectrum 2 N + 1. In the generic orbits π p,q ∈ Γ = Γ 1 = R × p × R q , the functional dimension is 1 and the computation on [26, page 83] shows that π p,q ( D γ ) is the anharmonic oscillator

$$\nu \nolimits \text{www.am.m. } \nu _ { p, q } ( \mathcal { U } _ { \gamma } ) & \text{is the ama} \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \ell \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Psi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \oplus \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \beta \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \P \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \PI \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \iota \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \xi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \implies \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \Pi \| \nolimits$$

There is a detailed analysis of invertibility of π p,q ( D γ ) in [17, 18] confirming the analysis above that invertibility is equivalent to iγ / H ∈ (1).

4.3. N (4) . Consider the six dimensional Lie algebra n (4) spanned by X ,X ,X ,Y ,Y ,Z 1 2 3 1 2 where the non-zero brackets are given by

$$[ X _ { 1 }, X _ { 2 } ] = Y _ { 1 }, [ X _ { 2 }, X _ { 3 } ] = Y _ { 2 } \text{ \ and \ } [ X _ { 1 }, Y _ { 2 } ] = [ Y _ { 1 }, X _ { 3 } ] = Z.$$

This is the Lie algebra of the group N (4) of all real upper unipotent 4 × 4-matrices. The group N (4) is the unipotent radical in the minimal parabolic subgroup of SL 4 ( R ). The Lie algebra n (4) has nilpotency degree 3 with n (4) -1 = R X 1 + R X 2 + R X 3 , n (4) -2 = R Y 1 + R Y 2 , and n (4) -3 = z = R Z .

Proposition 4.4. The six dimensional, three step nilpotent Lie algebra n (4) satisfies that

$$\mathcal { E } ( \mathfrak { n } ( 4 ), N ) \subseteq \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { 2 } \, \colon H ( 1 ) \cap \text{Spec} ( \gamma ( \xi ) ) = \emptyset, \, | \xi | = 1 \}, \\ \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \mathfrak { m } \, \text{ }$$

and n (4) has property E ∗ N for any N .

In this example, the necessary and sufficient condition in Theorem 3.1 does not stand in dichotomy as it did when dim g -2 = 1 in Proposition 3.3.

Proof. The representation space ̂ N (4) was described in detail in [9, Example 1.3.11 and 2.2.8]. By [9, Example 1.3.11],

$$\widehat { N ( 4 ) } = \Gamma _ { 1 } \dot { \cup } \Gamma _ { 2 } \dot { \cup } \Gamma _ { 3 } \dot { \cup } \Gamma _ { 4 }, \\ \dots \colon \colon \colon \colon \colon \colon \colon \colon \colon \colon \dots \colon \Gamma _ { 1 } \quad \mathbb { m } \times \dots \mathbb { m }$$

We want to characterize invertibility of π D ( γ ). Here where the space of generic orbits take the form Γ 1 = R × × R , and all form subsets that are Hausdorff in their respective induced topologies. The functional dimension of the representations in Γ 1 is 2, in Γ 2 and Γ 3 it is 1 and Γ 4 ∼ = R 3 consists of characters.

$$D _ { \gamma } = X _ { 1 } ^ { 2 } + X _ { 2 } ^ { 2 } + X _ { 3 } ^ { 2 } + \gamma _ { 1 } Y _ { 1 } + \gamma _ { 2 } Y _ { 2 }.$$

Trivially, π D ( γ ) is invertible for any non-trivial character π ∈ Γ 4 \ { 0 } . By the computations in [9, Example 2.2.8], all representations in Γ 2 ˙ Γ ∪ 3 are induced from quotients onto h 1 ⊕ R [ -1]. Therefore Proposition 3.4 implies that π D ( γ ) is injective for all π ∈ Γ 2 ˙ ∪ Γ 3 if and only if

$$\text{Spec} ( \xi _ { 1 } \gamma _ { 1 } + \xi _ { 2 } \gamma _ { 2 } ) \cap ( ( - i \infty, - i ] \cup [ i, i \infty ) ) = \emptyset, \\ \text{h to the } \epsilon ^ { 2 } \perp \epsilon ^ { 2 } - 1$$

for any ξ such that ξ 2 1 + ξ 2 2 = 1.

We see that

$$\mathcal { E } _ { \Gamma _ { 2 } \cup \Gamma _ { 3 } \cup \Gamma _ { 4 } \langle 1 \} } ( \mathfrak { n } ( 4 ), N ) = \{ \gamma \in M _ { N } ( \mathbb { C } ) ^ { 2 } \, \colon H ( 1 ) \cap \text{Spec} ( \gamma ( \xi ) ) = \emptyset, \, | \xi | = 1 \}.$$

We now turn our focus to the generic orbits Γ . 1 For an ( α, η ) ∈ R × × R ∼ = Γ , the 1 functional dimension is 2 and the computation [9, Example 2.2.8] gives us that

$$\pi _ { \alpha, \eta } ( D _ { \gamma } ) = - \Delta - 4 \pi ^ { 2 } ( \eta - \alpha x y ) ^ { 2 } - 2 \pi i \alpha ( \gamma _ { 2 } x - \gamma _ { 1 } y ). \quad \\ \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \end{array}$$

This operator was studied in [19]. We use the geometer's convention ∆ = -∂ 2 x -∂ 2 y . For notational simplicity, introduce the operator H α,η ( γ ) := π α,η ( D γ ). It is a self-adjoint operator with discrete spectrum on L 2 ( R 2 ) for ( α, η ) ∈ R × × R . In the new variables u = t 1 / 2 x and v = t 1 / 2 y ,

$$u = \tau ^ { \prime } \tilde { \ } x \text{ and } v = \tau ^ { \prime } \tilde { \ } y, \\ H _ { \alpha, \eta } ( t \gamma ) & = t ^ { - 1 } \left ( - \Delta _ { u, v } - 4 \pi ^ { 2 } ( \eta t ^ { 1 / 2 } - \alpha t ^ { - 1 / 2 } u v ) ^ { 2 } - 2 \pi i \alpha t ^ { - 1 / 2 } ( \gamma _ { 2 } u - \gamma _ { 1 } v \right ) = \\ & = t ^ { - 1 } H _ { t ^ { - 1 / 2 } \alpha, t ^ { 1 / 2 } \eta } ( \gamma ).$$

Combining this identity, with the computation of E Γ 2 ˙ Γ ∪ 3 ˙ Γ ∪ 4 \{ } 1 ( n (4) , N ) above and the fact that H α,η (0) is injective for any ( α, η ) ∈ R × × R (by H¨ ormander's sum of squares theorem) we see that if γ ∈ E ( n (4) , N ) then tγ ∈ E ( n (4) , N ) for any t ∈ [0 , 1]. /square

Question 4.5 . Can we characterize the subset E ( n (4) , N ) ⊆ M N ( C ) 2 more precisely? This subset consists precisely of the matrices γ = ( γ , γ 1 2 ) ∈ M N ( C ) 2 such that additionally to the requirement that H (1) ∩ Spec( γ ξ ( )) = ∅ , for | ξ | = 1, the operator

$$- \Delta & - 4 \pi ^ { 2 } ( \eta - \alpha x y ) ^ { 2 } - 2 \pi i \alpha ( \gamma _ { 2 } x - \gamma _ { 1 } y ) \\ \ell & \ell \ell \ell \end{pmatrix}$$

is invertible for any ( α, η ) ∈ R × × R .

Using the scaling invariance argument from the proof of Proposition 4.4 and a reflection in ( x, y ) and η , it follows that γ = ( γ , γ 1 2 ) ∈ E ( n (4) , N ) if and only if H (1) ∩ Spec( γ ξ ( )) = ∅ , for | ξ | = 1, and the operator

$$- \Delta - 4 \pi ^ { 2 } ( \eta - x y ) ^ { 2 } - 2 \pi i ( \gamma _ { 2 } x - \gamma _ { 1 } y ),$$

is invertible on L 2 ( R 2 ) for any η ∈ R .

4.4. ˜ h 2 m +1 . Let us consider another step 3 nilpotent Lie algebra. First, consider the Heisenberg Lie algebra h 2 m +1 of dimension 2 m +1 generated by X ,. . . , X 1 m , Y 1 , . . . , Y m , Z with the non-zero brackets given by

$$[ X _ { j }, Y _ { k } ] = \delta _ { j k } Z.$$

We have that h 2 m +1 is of step 2 with ( h 2 m +1 ) -1 = ∑ j R X j + R Y j and ( h 2 m +1 ) -2 = z = R Z . We now apply a construction of Mohsen [31], later expanded on in [13]. The Mohsen modification of h 2 m +1 is as a vector space defined by

$$\tilde { \mathfrak { h } } _ { 2 m + 1 } = \mathfrak { h } _ { 2 m + 1 } \oplus \left ( \mathfrak { h } _ { 2 m + 1 } \right ) ^ { * } \oplus \mathbb { R } Z _ { 0 }, \\ \tilde { \mathfrak { w } } \right ) \tilde { \mathfrak { w } } \right ) \tilde { \mathfrak { w } } \right ) \tilde { \mathfrak { w } } \right )$$

$$\vdash \circ \text{generic} \, \Lambda _ { 1 }, \dots, \Lambda _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \zeta, \\ \text{racket relations} \\ \begin{cases} [ \tilde { X } _ { j }, \tilde { Y } _ { k } ] = \delta _ { j k } \tilde { Z }, \\ [ \tilde { X } _ { j }, e _ { Z } ] = e _ { \tilde { Y } _ { j } }, \\ [ \tilde { Y } _ { j }, e _ { Z } ] = - e _ { \tilde { X } _ { j } }, \\ [ \tilde { Z }, e _ { Z } ] = Z _ { 0 }, \\ [ \tilde { X } _ { j }, e _ { X _ { k } } ] = [ \tilde { Y } _ { j }, e _ { Y _ { k } } ] = \delta _ { j, k } Z _ { 0 } \\ \end{cases} \\ \quad - 1 \, \text{is spanned by} \, \tilde { X } _ { 1 }, \dots, \tilde { X } _ { m }, \tilde { Y } _ { 1 }, \dots, \tilde { Y } _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \ielta _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m}, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dts, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dets, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \delta _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \ielta _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \ielta _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dts, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1}, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1}, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { m}, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { 1 }, \dots, \iota _ { m }, \iota _ { 1 }, \dots, \iota _ { 1 }, \dots, \iota _ { m }$$

and is generated by 4 m +3generators ˜ X ,. . . , X 1 ˜ m , Y ˜ 1 , . . . , Y ˜ m , Z ˜ , e X 1 , . . . , e X m , e Y 1 , . . . , e Y m , e Z , Z 0 subject to the bracket relations

We have that ( ˜ h 2 m +1 ) -1 is spanned by ˜ X ,. . . , X 1 ˜ m , Y ˜ 1 , . . . , Y ˜ m , e Z , ( ˜ h 2 m +1 ) -2 is spanned by ˜ Z,e X 1 , . . . , e X m , e Y 1 , . . . , e Y m and finally ( ˜ h 2 m +1 ) -3 = z is spanned by Z 0 .

The Mohsen modification always produces a nilpotent Lie algebra with one-dimensional center and flat orbits, so Γ = R × . An interesting feature of the Mohsen modification is

that a representation in a flat orbit /planckover2pi1 ∈ R × is an extension of the left regular representation. In light of this fact, we can readily describe π /planckover2pi1 ( D γ ) where /planckover2pi1 ranges over the flat orbits Γ = ∼ R × . Here

$$D _ { \gamma } = \sum _ { j = 1 } ^ { m } ( \tilde { X } _ { j } ^ { 2 } + \tilde { Y } _ { j } ^ { 2 } ) + e _ { Z } ^ { 2 } + \gamma _ { 0 } \tilde { Z } + \sum _ { l = 1 } ^ { m } ( \gamma _ { 2 l - 1 } e _ { X _ { l } } + \gamma _ { 2 l } e _ { Y _ { l } } ). \\ \text{convention in to that what in notation in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \,\text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in}\, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{\ } \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{In} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \, \text{in} \,$$

The representation in the flat orbit in particular has functional dimension 2 m +1, and we write the coordinates in that space by ( x, y, z ) ∈ R m × R m × R . By homogeneity, we compute that

$$\text{ute that} \\ \pi _ { h } ( D _ { \gamma } ) = | \hbar { | } \sum _ { j = 1 } ^ { m } & \left ( ( \partial _ { x _ { j } } - \frac { y } { 2 } \partial _ { z } ) ^ { 2 } + ( \partial _ { y _ { j } } + \frac { x } { 2 } \partial _ { z } ) ^ { 2 } \right ) - | \hbar { | } z ^ { 2 } + \\ & + \hbar \left ( \gamma _ { 0 } i \partial _ { z } - \frac { 1 } { 2 } \sum _ { l = 1 } ^ { m } ( \gamma _ { 2 l - 1 } x _ { l } + \gamma _ { 2 l } y _ { l } ) \right ). \\ \text{p to unitary equivalence, $\pi_{h}(D_{\gamma})$ coincides with the operator}$$

So, up to unitary equivalence, π /planckover2pi1 ( D γ ) coincides with the operator

$$\text{unary equivalence, $\pi_{h}(\mathcal{ }V_{\gamma})$ conicies with use operator} \\ | \hbar { | } ( - L _ { B } + \partial _ { B } ^ { 2 } ) + i \hbar { \left ( } \gamma _ { 0 } B - \frac { 1 } { 2 } \sum _ { l = 1 } ^ { m } ( \gamma _ { 2 l - 1 } x _ { l } + \gamma _ { 2 l } y _ { l } ) \right ), \\ \mathfrak { n } \, I ^ { 2 } ( \mathbb { R } ^ { m } \times \mathbb { R } ^ { m } \times \mathbb { R } _ { n } ) \, \text{ Here } I _ { n } \text{ denotes the Landon Hamiltonian at } r$$

acting on L 2 ( R m x × R m y × R B ). Here L B denotes the Landau hamiltonian at magnetic field strength B .

Proposition 4.6. The subset E ( ˜ h 2 m +1 , N ) ⊆ M N ( C ) 2 m +1 consists of the matrices γ = ( γ l ) 2 m l =0 ∈ M N ( C ) 2 m +1 such that the two operators

$$\nu ( \mathbb { U } ) ^ { \dots \cdot \cdot \cdot \cdot } & \text{ such una use two operaturos} \\ - L _ { B } + \partial _ { B } ^ { 2 } \pm i \left ( \gamma _ { 0 } B - \frac { 1 } { 2 } \sum _ { l = 1 } ^ { m } ( \gamma _ { 2 l - 1 } x _ { l } + \gamma _ { 2 l } y _ { l } ) \right ) \\ \mathbb { R } ^ { m } \times \mathbb { R } \, \mathfrak { m } \, \text{ are invertible}$$

on L 2 ( R m x × R m y × R B ) , are invertible.

Proof. The Lie algebra ˜ h 2 m +1 has flat coadjoint orbits and one-dimensional centre, so D γ is H -elliptic if and only if π /planckover2pi1 ( D 0 ) π /planckover2pi1 ( D γ ) -1 exists and is uniformly bounded in /planckover2pi1 ∈ Γ = z ∗ \ { 0 } , see [13, Theorem 22.20]. It is clear from homogeneity that π /planckover2pi1 ( D 0 ) π /planckover2pi1 ( D γ ) -1 exists and is uniformly bounded if and only π /planckover2pi1 ( D γ ) is invertible for /planckover2pi1 = ± 1 and the proposition follows. /square

## 5. A final detail in a proof of Rothschild-Stein

Theorem 2 in [35] (discussed above in Section 3) can nowadays be proved more simply by using the Rockland criterion that was not formulated at the time of [35] but is now a theorem, cf. [1, 21, 23, 28, 38]. Even if [35, Theorem 2] is correct as stated, the proof contains a minor oversight: in [35, equation (4.1) on page 258], they introduce 2 l ≤ n as the rank of a functional ρ ∗ while in the next paragraph, the authors construct a representation π λ while implicitly assuming that 2 l &lt; n . We discuss the proof here in a way where the case 2 l = n is also covered. In this remaining case 2 l = n , [35, Theorem 2] follows from applying the mean value theorem. As one can see from the statement at hand, it is a rank 2 statement and we consider the operator

$$P = \sum _ { j = 1 } ^ { n } X _ { j } ^ { 2 } + i \sum _ { l = 1 } ^ { m } b _ { l } Y _ { l } \\ \omega. \, \varepsilon \, \mathbb { D }$$

where [ g -1 , g -1 ] = g -2 , and b l ∈ R .

As observed by Helffer in [16] and Beals [5], one can use the results of Boutet-GrigisHelffer giving equivalent conditions to Rockland's criterion. The conditions of hypoellipticity can be expressed in the following way. For each η ∈ g ∗ -2 \ { 0 } , let r η denote the rank of the Kirillov form on g -1 × g -1 . The Kirillov form ω η is defined by

$$\omega _ { \eta } ( X, X ^ { \prime } ) = \eta ( [ X, X ^ { \prime } ] )$$

Our assumptions imply that

$$2 \leq r _ { \eta } \leq \dim \mathfrak { g } _ { - 1 } \,. \\. \, \text{cancel} \, \text{the case when}$$

By homogeneity, it is enough to consider the case when | η | = 1. In the notation of [35], r η = 2 l and n = dim g -1 .

Without entering into the details, we can with each η associate the value λ η ( ) &gt; 0 as the bottom of the spectrum of some Harmonic Oscillator H η . Write ( λ j ( η )) ∞ j =1 for the growing sequence of eigenvalues of the harmonic oscillator H η , so λ η ( ) = λ 1 ( η ). The criterion for hypoellipticity, which is a necessary and sufficient condition, reads

/negationslash

- · If r η &lt; dim g -1 , ∑ m l =1 b η l l ∈ [ λ η , ( ) + ∞ ) · If r η = dim g -1 , m l =1 b η l l ∈ { λ η , λ ( ) 2 ( η , λ ) 3 ( η , . . . ) } .

∑ Note also that λ η ( ) = λ ( -η ). We now rephrase these conditions under various assumptions. The first result is the following.

/negationslash

Proposition 5.1. If ∑ m l =1 b η l l &lt; λ η ( ) , for any | η | = 1 , the operator P is hypoelliptic. This corresponds to [35, Theorem 1']. To determine if the condition is necessary, we obtain the following two propositions from the discussion above.

Proposition 5.2. If for some η (with | η | = 1 ) we have ∑ m l =1 b η l l = λ η ( ) , the operator is not hypoelliptic.

Proposition 5.3. If for some η (with | η | = 1 ) we have ∑ m l =1 b η l l ≥ λ η ( ) , and r η &lt; dim g -1 , the operator is not hypoelliptic.

Since r η is even, this proposition in particular can be applied when dim g -1 is odd. This is the case for Example 4.3 with g = n (4). Finally, we have

Proposition 5.4. We assume dim g -2 ≥ 2 . If for some η (with | η | = 1 ) we have ∑ m l =1 b η l l ≥ λ η ( ) , and dim g -2 ≥ 2 , the operator is not hypoelliptic.

$$\mu - \alpha \\ \sum _ { l = 1 } ^ { m } b _ { l } \eta _ { l } ^ { \prime } = 0 \,.$$

Proof. Due to the previous propositions, it remains to treat the case when ∑ m l =1 b η l l &gt; λ η ( ) and r η = dim g -1 . Since dim g -2 ≥ 2, there exists an η ′ with | η ′ | = 1 such that

By the mean value theorem on the connected sphere of dimension dim g -2 -1, there exists an η ′′ such that | η ′′ | = 1 and ∑ m l =1 b η l ′′ l = λ η ( ′′ ). We can then apply Proposition 5.2. /square

It remains to consider the case when dim g -2 = 1. The case when r η &lt; dim g -1 is treated in Proposition 5.3 and the case when r η = dim g -1 corresponds to the Heisenberg algebra.

## References

- [1] I. Androulidakis, O. Mohsen, and R. Yuncken, A pseudodifferential calculus for maximally hypoelliptic operators and the Helffer-Nourrigat conjecture , arXiv:2201.12060.
- [2] M. F. Atiyah, and I. M. Singer, The index of elliptic operators. III. Ann. of Math. (2) 87 (1968), 546-604.
- [3] P. Baum, and R. G. Douglas, K -homology and index theory , Operator algebras and applications, Part I (Kingston, Ont., 1980), 1982, pp. 117-173.
- [4] P. F. Baum, and E. van Erp, K -homology and index theory on contact manifolds , Acta Math. 213 (2014), no. 1, 1-48.

- [5] R. Beals. Op´ erateurs invariants hypoelliptiques sur un groupe de Lie nilpotent. In: S´ eminaire Goulaouic-Schwartz 1976/1977: ´ quations aux d´riv´es partielles et analyse fonctionnelle, Exp. No. E e e 19. April 1977.
- [6] R. Beals, and P. Greiner, Calculus on Heisenberg manifolds , Annals of Mathematics Studies, 119. Princeton University Press, Princeton, NJ, 1988.
- [7] L. Boutet de Monvel, A. Grigis, and B. Helffer. Parametrixes d'op´ erateurs pseudo-diff´rentiels e ` a caract´ristiques multiples. (French) Journ´es: e e ´ equations aux D´ eriv´es Partielles de Rennes (1975), e pp. 93-121. Ast´risque, No. 34-35, Soc. Math. France, Paris, 1976. e
- [8] Y. Colin de Verdi`re, L Hillairet, and E. Tr´lat, e e Spectral asymptotics for sub-Riemannian Laplacians , arXiv:2212.02920.
- [9] L. J. Corwin, and F. P. Greenleaf, Representations of nilpotent Lie groups and their applications , Cambridge Studies Adv. Mathematics 18 . Cambridge University Press 1990.
- [10] S. Dave, and S. Haller, Graded hypoellipticity of BGG sequences , Ann. Global Anal. Geom. 62 (2022), no. 4, 721-789.
- [11] P. Eberlein, Geometry of 2-step nilpotent Lie groups , Modern dynamical systems and applications, 67-101, Cambridge Univ. Press, Cambridge, 2004.
- [12] C. L. Epstein, and R. B. Melrose, The Heisenberg algebra, index theory and homology , Preprint, 2000. Available online at http://www.math.mit.edu/ ∼ rbm/book.html.
- [13] M. Goffeng, and A. Kuzmin, Index theory of hypoelliptic operators on Carnot manifolds , arXiv:2203.04717.
- [14] M. Gromov, Carnot-Carath´odory e spaces seen from within , Sub-Riemannian geometry, 79-323, Progr. Math., 144, Birkh¨user, Basel, 1996. a
- [15] W. Hebisch, On operators satisfying the Rockland condition , Studia Math. 131 (1998), no. 1, 63-71.
- [16] B. Helffer, Hypoellipticit´ pour des op´rateurs diff´rentiels sur les groupes nilpotents. in Pseudodife e e ferential operators with applications lectures given at the Centro internazionale matematico estivo (C.I.M.E.) held in Bressanone (Bolzano), Italy, June 16-24, 1977.
- [17] B. Helffer, Sur l'hypoellipticit´ e des op´rateurs e de la forme ∑ Y 2 j + 1 2 ∑ c j,k [ Y , Y j k ], S´ eminaire de l'universit´ de Nantes, expos´ No 1, 1981-82 (d'apr`s Helffer-M´tivier-Nourrigat). e e e e
- [18] B. Helffer, Partial differential equations on nilpotent groups , Lie group representations, III (College Park, Md., 1982/1983), 210-253, Lecture Notes in Math., 1077, Springer, Berlin, 1984.
- [19] B. Helffer, On a conjecture of P. Jorgensen and W. Klink , Publ. Res. Inst. Math. Sci. 23 (1987), no. 6, 1007-1013.
- [20] B. Helffer and J. Nourrigat, Hypoellipticit´ e pour des groupes nilpotents de rang de nilpotence 3. Comm. Partial Differential Equations 3 (1978), no. 8, 643-743.
- [21] B. Helffer and J. Nourrigat, Caract´risation des op´rateurs hypoelliptiques homog`nes invariants ` e e e a gauche sur un groupe de Lie nilpotent gradu´. e Comm. Partial Differential Equations 4.8 (1979), pp. 899-958.
- [22] B. Helffer and J. Nourrigat, Th´ eor`mes e d'indice pour des op´rateurs e ` coefficients a polynomiaux. PDE Banach center publications, Vol. 10. Warsaw (1983), p. 203-212.
- [23] B. Helffer and J. Nourrigat, Hypoellipticit´ Maximale pour des Op´rateurs Polynˆmes de Champs e e o de Vecteurs. Progress in Mathematics, Birkh¨user, Vol. 58 (1985). a
- [24] N. Higson, and J. Roe, Analytic K -homology , Oxford Mathematical Monographs. Oxford Science Publications. Oxford University Press, Oxford, 2000.
- [25] L. H¨rmander, o Hypoelliptic second order differential equations , Acta Math. 119 (1967), 147-171.
- [26] A. A. Kirillov, Lectures on the orbit method , Graduate Studies in Mathematics, 64. American Mathematical Society, Providence, RI, 2004.
- [27] H. B. Jr. Lawson, H. Blaine, and M.-L. Michelsohn, Spin geometry , Princeton Mathematical Series, 38. Princeton University Press, Princeton, NJ, 1989.
- [28] A. Melin, Lie filtrations and pseudo-differential operators , Preprint, 1982.
- [29] G. M´ etivier, Fonction spectrale et valeurs propres d'une classe d'op´rateurs non elliptiques e , Comm. Partial Differential Equations 1 (1976), no. 5, 467-519.
- [30] J. Mitchell, On Carnot-Carath´ eodory metrics , J. Differential Geom. 21 (1985), no. 1, 35-45.

[31]

O. Mohsen,

Index theorem for inhomogeneous hypoelliptic differential operators

,

M¨ unster J. of Math.

15 (2022), 305-331.

- [32] O. Mohsen, On the index of maximally hypoelliptic differential operators , arXiv:2201.13049.
- [33] J. Nourrigat, In´galit´s e e L 2 et repr´sentations de groupes nilpotents e , J. Funct. Anal. 74 (1987), no. 2, 300-327.
- [34] R. S. Ponge, Heisenberg calculus and spectral theory of hypoelliptic operators on Heisenberg manifolds , Mem. Amer. Math. Soc. 194 (2008), no. 906, viii+ 134 pp.
- [35] L. P. Rothschild, and E. M. Stein, Hypoelliptic differential operators and nilpotent groups , Acta Math. 137 (1976), no. 3-4, 247-320.
- [36] L. P. Rothschild, A criterion for hypoellipticity of operators constructed from vector fields . Comm. Partial Differential Equations 4 (1979), no. 6, 645-699.

- [37] M. E. Taylor, Noncommutative microlocal analysis. I. Mem. Amer. Math. Soc. 52 (1984), no. 313, iv+182 pp.
- [38] E. van Erp, and R. Yuncken, A groupoid approach to pseudodifferential calculi , J. Reine Angew. Math. 756 (2019), 151-182.
- (M. Goffeng) Centre for Mathematical Sciences, Lund University, Box 118, 221 00 LUND, Sweden

Email address : magnus.goffeng@math.lth.se

- (B. Helffer) Laboratoire de Math´ ematiques Jean Leray, Nantes Universit´, 44 000 Nantes e FRANCE

Email address : bernard.helffer@univ-nantes.fr
Magnus Goffeng Bernard Helffer
The index of sub-laplacians: beyond contact manifolds
In this paper we study the following question: do sub-Laplacian type
operators have non-trivial index theory on Carnot manifolds in higher degree of
nilpotency? The problem relates to characterizing the structure of the space of
hypoelliptic sub-Laplacian type operators, and results going back to
Rothschild-Stein and Helffer-Nourrigat. In two degrees of nilpotency, there is
a rich index theory by work of van Erp-Baum on contact manifolds, that was
later extended to polycontact manifolds by Goffeng-Kuzmin. We provide a
plethora of examples in higher degree of nilpotency where the index theory is
trivial.