Abstract :
We study a partial differential operator H with analytic coefficients, which is of the form “sum of squares”. H is hypoelliptic on any open subset of R3, yet possesses the following properties: (1) H is not analytic hypoelliptic on any open subset of R3 that contains 0. (2) If u is any distribution defined near 0∈R3 with the property that Hu is analytic near 0, then u must be analytic near 0. (3) The point 0 lies on the projection of an infinite number of Treves curves (bicharacteristics).
These results are consistent with the Treves conjectures. However, it follows that the analog of Treves conjecture, in the sense of germs, is false.
As far as we know, H is the first example of a “sum of squares” operator which is not analytic hypoelliptic in the usual sense, yet is analytic hypoelliptic in the sense of germs.
