## Anais da Academia Brasileira de Ciências

##
*Print version* ISSN 0001-3765*On-line version* ISSN 1678-2690

### An. Acad. Bras. Ciênc. vol.74 no.2 Rio de Janeiro June 2002

#### http://dx.doi.org/10.1590/S0001-37652002000200002

**Maximum principles for hypersurfaces with vanishing curvature functions in an arbitrary Riemannian manifold **

**FRANCISCO X. FONTENELE ^{1} and SÉRGIO L. SILVA^{2}**

^{1}Universidade Federal Fluminense, Instituto de Matemática, Departamento de Geometria,

rua Mario Santos Braga s/no., 24020-140 Niterói, RJ, Brazil

^{2}Universidade Estadual do Rio de Janeiro-UERJ, Departamento de Estruturas Matemáticas-IME

20550-013 RJ, Brazil

* Manuscript received on October 30, 2001; accepted for publication on December 16, 2001;*

* presented by *MANFREDO DO CARMO

**ABSTRACT**

In this paper we generalize and extend to any Riemannian manifold maximum principles for Euclidean hypersurfaces with vanishing curvature functions obtained by Hounie-Leite.

** Key words:** maximum principle, hypersurface, rth mean curvature.

**1. INTRODUCTION **

In this paper we generalize and extend to any Riemannian manifold maximum principles for hypersurfaces of the Euclidean space with vanishing curvature function, obtained by Hounie-Leite (1995 and 1999). In order to state our results, we need to introduce some notations and consider some facts. Given an hypersurface *M*^{n} of a Riemannian manifold *N*^{n + 1}, denote by *k*_{1}(*p*),..., *k*_{n}(*p*) the principal curvatures of *M*^{n} at *p* with respect to a unitary vector that is normal to *M*^{n} at *p*. We always assume that *k*_{1}(*p*) £ *k*_{2}(*p*) £ ... £ *k*_{n}(*p*). The * rth mean curvature* *H _{r}* (

*p*) of

*M*

^{n}at

*p*is defined by

H (_{r}p) = s_{r}(k_{1}(p),..., k_{n}(p)), | (1) |

where s_{r} : ^{n}® is the rth elementary symmetric function. It is easy to see that s_{r} is positive on the positive cone ^{n} = {(*x*_{1},..., *x*_{n}) Î ^{n} : *x*_{i} > 0,"*i*}. Denote by G_{r} the connected component of {s_{r} > 0} that contains the vector (1,..., 1) Î ^{n}. It was proved in Gårding (1959) that G_{r} is an open convex cone and that

G_{1}É G_{2} É ... É G_{n}. | (2) |

Moreover on G_{r}, 1 £ *r* £ *n*, it holds that (see Caffarelli et al. 1985, Proposition 1.1)

> 0, 1 £ i £ n. | (3) |

As it was observed in Hounie-Leite (1995), the subset {s_{r} = 0} can be decomposed as the union of *r* continuous leaves *Z*_{1},..., *Z*_{r}, being *Z*_{1} the boundary ¶ G* _{r}* of the cone G

*. Furthermore each leaf*

_{r}*Z*

_{j}may be identified with the graph of a continuous function defined in the plane

*x*

_{1}+ ... +

*x*

_{n}= 0. Following Hounie-Leite(1995), we say that a point

*x*= (

*x*

_{1},...,

*x*

_{n}) Î

^{n}has rank

*r*if exactly

*r*components of

*x*do not vanish.

As in Fontenele-Silva (2001), given *p* Î *M*^{n} and a unitary vector *h*_{o} that is normal to *M*^{n} at *p*, we can parameterize a neighborhood of *M*^{n} containing *p* and contained in a normal ball of *N*^{n + 1} as

j(x) = exp_{p}(x + m(x)h_{o}), | (4) |

where the vector *x* varies in a neighborhood *W* of zero in *T _{p}M* and m :

*W*® satisfies m (0) = 0 and m(0) = 0, being the gradient operator in the Euclidean space

*T*. Choosing a local orientation

_{p}M*h*:

*W*®

*M*of

*M*

^{n}with

*h*(0) =

*h*

*, we denote by*

_{o}*H*(

_{r}*x*) the rth mean curvature of

*M*

^{n}at j(

*x*) with respect to

*h*(

*x*).

Given hypersurfaces *M* and *M'* of *N ^{n }*

^{+ 1}that are tangent at

*p*and a unitary vector

*h*

_{o}that is normal to

*M*at

*p*, we parameterize

*M*and

*M'*as in (4), obtaining correspondent functions m :

*W*® and m' :

*W*® , defined in a neighborhood

*W*of zero in

*T*=

_{p}M*T*. As in Fontenele-Silva (2001), we say that

_{p}M'*M*remains above

*M'*in a neighborhood of

*p*with respect to

*h*

*if m(*

_{o}*x*) ³ m'(

*x*) for all

*x*in a neighborhood of zero. We say that

*M*remains on one side of

*M'*in a neighborhood of

*p*if either

*M*is above

*M'*or

*M'*is above

*M*in a neighborhood of

*p*. Finally, denote by (

*p*) = (

*k*

_{1}(

*p*),...,

*k*

_{n}(

*p*)) and by (

*p*) = (

*k'*

_{1}(

*p*),...,

*k'*

_{n}(

*p*)) the principal curvature vectors at

*p*of respectively

*M*and

*M'*.

We can now state our results:

THEOREM 1.a. * Let M and M' be hypersurfaces of N^{n }*

^{+ 1}

*that are tangent at*(

*p*, with normal vectors pointing in the same direction. Suppose that*M*remains on one side of*M'*and that H_{r}*x*)

*= H'*(

_{r}*x*)

*in a neighborhood of zero in*£

*T*_{p}*M*, for some*r*, 1*³*

*r*<*n*. If*r**2, suppose further that (*

*p*) and (*p*) belong to same leaf of_{}{s

*= 0}*

_{r}*and the rank of either (*

*p*) or (*p*) is at least*r*. Then,*M*and*M'*must coincide in a neighborhood of*p*. THEOREM 1.b. * Let M and M' be hypersurfaces of N^{n }*

^{+ 1}

*with boundaries*¶

*¶*

*M*and*¶*

*M'*, respectively, and assume that*M*and*M'*, as well as*¶*

*M*and*Î*

*M'*, are tangent at*p**¶*

*M*Ç¶

*M', with normal vectors pointing in the same direction. Suppose that*(

*M*remains on one side of*M'*and that H_{r}*x*)

*= H'*(

_{r}*x*)

*in a neighborhood of zero in*£

*T*_{p}*M*, for some*r*, 1*³*

*r*<*n*. If*r**2, suppose further that (*

*p*) and (*p*) belong to same leaf of_{}{s

*= 0}*

_{r}*and the rank of either (*

*p*) or (*p*) is at least*r*. Then,*M*and*M'*must coincide in a neighborhood of*p*.As a consequence of Theorems 1.a and 1.b, we obtain the following corollaries, that extend Theorem 0.1 in Hounie-Leite (1995) to any Riemannian manifold.

COROLLARY 1.a. * Let M and M' be hypersurfaces of N^{n }*

^{+ 1}

*that are tangent at*£

*p*, with normal vectors pointing in the same direction and with both having*r*-mean curvature equal to zero for some*r*, 1*³*

*r*<*n*. For*r**2, suppose further that (*{s

*p*) and (*p*) belong to same leaf of*= 0}*

_{r}*and the rank of either (*

*p*) or (*p*) is at least*r*. Under these conditions, if*M*remains on one side of*M'*, then*M*and*M'*must coincide in a neighborhood of*p*. COROLLARY 1.b. * Let M and M' be hypersurfaces of N ^{n }*

^{+ 1}

*with boundaries*¶

*M and*¶

*M', respectively, so that M and M', as well as*¶

*M and*¶

*M', are tangent at p*Î ¶

*M*Ç

*M', with normal vectors pointing in the same direction. Assume that M and M' have r-mean curvature equal to zero for some r, 1*£

*r*<

*n. For r*³

*2, suppose further that (p) and (p) belong to same leaf of*{s

_{r}= 0}

*and the rank of either (p) or (p) is at least r. Under these conditions, if M remains on one side of M', then M and M' must coincide in a neighborhood of p.*

The extension of Theorem 1.3 in Hounie-Leite (1999) is given in the following theorems.

THEOREM 2.a. * Let M and M' be hypersurfaces of N ^{n }*

^{+ 1}

*that are tangent at*³

*p*, with normal vectors pointing in the same direction. Suppose that*M*remains above*M'*and that H'_{r}*0*³

*H*³

_{r}, for some*r*, 2 £*r*<*n*. Suppose further that H'_{j}(p)*0, 1*£

*j*£

*r –*1

*, and either H*

_{r}_{+1}(p) ¹ 0

*or H'*

_{r}_{+1}(p) ¹ 0

*.*Then

*,*and

*M**must coincide in a neighborhood of*

*M'*

*p*. THEOREM 2.b. * Let M and M' be hypersurfaces of N^{n }*

^{+ 1}

*with boundaries*¶

*¶*

*M*and*¶*

*M'*, respectively, and assume that*M*and*M'*, as well as*¶*

*M*and*Î ¶*

*M'*, are tangent at*p**Ç ¶*

*M**³*

*M'*with normal vectors pointing in the same direction. Suppose that*M*remains above*M'*and that H'_{r}*0*³

*H*£

_{r}, for some*r*, 2*³*

*r*<*n*. Suppose further that H'_{j}(p)*0, 1*£

*j*£

*r –*1

*, and either H*

_{r}_{+1}(p) ¹ 0

*or H'*

_{r}_{+1}(p) ¹ 0. Then

*M*and

*M'*must coincide in a neighborhood of

*p*.

It will be clear from the proofs that in Theorems 2.a and 2.b we only need to require *H' _{r}* (

*x*) ³

*H*(

_{r}*x*), in a neighborhood of zero in

*T*, and

_{p}M*H'*(

_{r}*p*) ³ 0 ³

*H*(

_{r}*p*) instead of

*H'*³ 0 ³

_{r}*H*everywhere. For

_{r }*r*= 1, it must be observed that, in Theorems 2.a and 2.b, we can assume only that

*H'*

_{r }_{(x)}³

*H*

_{r }_{(x)}

*and that*

_{ }*M*remains above

*M'*in a neighborhood of zero in

*T*(see Theorems 1.1 and 1.2 in Fontenele-Silva (2001)).

_{p}M

**2. PRELIMINARIES **

In this section we will present the necessary material for our proofs.

Following Hounie-Leite (1995), we say that *x* Î ^{n} is an elliptic root of s* _{r}* if s

*(*

_{r}*x*) = 0 and either (

*x*) > 0,

*j*= 1,...,

*n*, or (

*x*) < 0,

*j*= 1,...,

*n*. It is easy to see that any root of s

_{1}= 0 is elliptic. For 2 £

*r*<

*n*, we have the following criterion of ellipticity (see Corollary 2.3 in Hounie-Leite (1995) and Lemma 1.1 in Hounie-Leite (1999)):

LEMMA 1. * Let x *Î

*s*

^{n}and assume that*(*

_{r}*x*)

*= 0*

*for some 2*£

*<*

*r*

*n*. Then, the following conditions are equivalent (i) *x* * is elliptic.*

(ii) the rank of *x* is at least *r*.

(iii) s_{r}_{+1}(*x*) ¹ 0

For the proofs of our results, we will also need of the following lemmas:

LEMMA 2. * If y, w belong to a leaf Z_{j} of *s

*0*

_{r}=*,*

*w*-*y*belongs to the closure of^{n}and either*y*or*w*is an elliptic root, then*y*=*w*. LEMMA 3. * For *1 £ * r *£

*Î*

*n*, if*s*

*satisfies*^{n}*(*

_{j}*x*)

*³*

*0, 1*£

*£*

*j**Î*

*r*, then*x**.*

Lemma 2 is a particular case of Lemma 1.3 in Hounie-Leite (1995) and Lemma 3 follows from the proof of Lemma 1.2 in Hounie-Leite (1999).

For *d* = (*n*(*n* + 1)/2) + 2*n* + 1, write an arbitrary point *p* at ^{d} as

*p* = (*r*_{11},..., *r*_{1n}, *r*_{22},...*r*_{2n},..., *r*_{(n - 1)n}, *r _{nn}*,

*r*

_{1},...,

*r*,

_{n}*z*,

*x*

_{1},...,

*x*)

_{n}or, in short, as *p* = (*r _{ij}*,

*r*,

_{i}*z*,

*x*) with 1 £

*i*£

*j*£

*n*, and

*x*= (

*x*

_{1},...,

*x*

_{n}). A

*C*

^{1}-function F : G ®

*defined in an open set G of*

^{d}is said to be elliptic in

*p*Î G if

(p)x_{i}x_{j} > 0 for all nonzero (x_{1},x_{2},...,x_{n}) Î ^{n}. | (5) |

We say that F is elliptic in G if F is elliptic in *p* for all *p* Î G. Given a function *f* : *U* ® * of class **C*^{2}, defined in an open set *U* Ì ^{n}, and *x* Î *U*, we associate a point L(*f* )(*x*) in ^{d} setting

L(f )(x) = (f_{ij}(x), f_{i}(x), f (x), x), | (6) |

where *f*_{ij}(*x*) and *f*_{i}(*x*) stand for (*x*) and (*x*), respectively. Saying that the function F is elliptic with respect to *f* means that L(*f* )(*x*) belongs to G and F is elliptic in L(*f* )(*x*) for all *x* Î *U*. For elliptic functions it holds the following maximum principle(see Alexandrov 1962):

MAXIMUM PRINCIPLE. * Let f, g *:

*U*®

*be*F : G Ì

*C*^{2}-functions defined in an open set*U*of^{n}and let*F*

^{d}be a function of class*C*^{1}. Suppose that*is elliptic with respect to the functions*(1 -

*t*)

*Î [0, 1]*

*f*+*tg*,*t**. Assume also that*

F(L(f )(x)) ³ F(L(g)(x)) ,"x Î U, | (7) |

* and that f *£

*<*

*g*on*U*. Then,*f**Î*

*g*on*U*unless*f*and*g*coincide in a neighborhood of any point*x*_{o}

*U*such that*f*(*x*_{o}) =*g*(*x*_{o}). Consider now a hypersurface *M ^{n}* Ì

*N*

^{n}^{ + 1}, a point

*p*Î

*M*and a unitary vector

*h*

_{o}that is normal to

*M*at

^{n}*p*. Fix an orthonormal basis

*e*

_{1},...,

*e*

_{n}in

*T*and introduce coordinates in

_{p}M*T*by setting

_{p}M*x*=

*x*

_{i}

*e*

_{i}for all

*x*Î

*T*. Parameterize a neighborhood of

_{p}M*p*in

*M*as in (4), obtaining a function m :

*W*Ì

*T*®

_{p}M*. Recall that m(0) = 0 and (0) = 0, for all*

*i*, 1 £

*i*£

*n*. Choose a local orientation h :

*W*®

*M*of

*M*with

^{n}*h*(0) = h

_{0}and denote by

*A*

_{h}

_{(x)}the second fundamental form of

*M*in the direction

^{n}*h*(

*x*). Denote by j(

*x*) the vector (

*x*) and by

*A*(

*x*) = (

*a*

_{ij}(

*x*)) the matrix of

*A*

_{h}

_{(x)}in the basis j

_{i}(

*x*). In Fontenele-Silva (2001), it is proved the existence of an

*n*´

*n*-matrix valued function

*Ã*defined in an open set

^{(n(n + 1)/2) + n}´ Ì

^{d}, being an open set of

^{n + 1}, containing the origin of

^{d}such that

Ã(L(m)(x)) = A(x) , x Î W. | (8) |

Moreover, we have *Ã*(*r _{ij}*,

*r*,

_{i}*z*,

*x*) diagonalizable for all (

*r*,

_{ij}*r*,

_{i}*z*,

*x*) Î

^{(n(n + 1)/2) + n}´ . Consider the function F

*:*

_{r}^{(n(n + 1)/2) + n}´ ®

*defined by*

F = s_{r}_{r}_{o}loÃ, | (9) |

where l(*Ã*(*w*)) = (l_{1}(*Ã*(*w*)),...,l_{n}(*Ã*(*w*)) for all *w* Î ^{(n(n + 1)/2) + n} ´ . Here l_{1}(*Ã*(*w*)) £ ^{ ... } £ l_{n}(*Ã*(*w*)) are the eigenvalues of *Ã*(*w*). It follows from (1), (8) and (9) that

H(_{r}x) = F(L(m)(_{r}x)) , x Î W. | (10) |

The proof of Proposition 3.4 in Fontenele-Silva (2001) gives

(r, 0, 0, 0) x_{ij}x_{k} = (_{l}Ã((r, 0, 0, 0))) x_{ij}x_{k}, _{l} | (11) |

for all (*r _{ij}*, 0, 0, 0) Î

^{d}.

We also make use of the following lemma

LEMMA 4. * If A_{o} *Î

*l*

^{n}() is symmetric and (*(*>

*A*_{o}))*0 (*<

*0) for all 1*£

*£*

*i*

*n*, then (A_{o}) xx_{i} > 0 ( < 0) , " x = (x_{l},...,x_{1}) ¹ 0. _{n} | (12) |

The proof of Lemma 4 follows from the proof of Lemma 3.3 in Fontenele-Silva (2001).

** 3. PROOFS OF OUR RESULTS **

We will prove only Theorems 1.a and 2.a, since the proofs of Theorems 1.b and 2.b are analogous.

PROOF OF THEOREM 1.a. If *r* = 1, the theorem follows from Theorem 1.1 in Fontenele-Silva (2001). Thus, we assume that 2 £ *r* *<* *n*. The assumption *H _{r}*(

*x*) =

*H'*(

_{r}*x*) in a neighborhood

*W*of zero in

*T*and (10) imply that

_{p}M F(L(m)(_{r}x)) = F(L(m')(_{r}x)) , x Î W. | (13) |

On the other hand, (*p*) and (*p*) are both roots of s* _{r}* = 0 and one of them is elliptic by our hypothesis and Lemma 1. The fact that

*M*remains on one side of

*M'*implies that either (

*p*) - (

*p*) or (

*p*) - (

*p*) belongs to . Since (

*p*) and (

*p*) belong to same leaf of {s

*= 0} by assumption, it follows from Lemma 2 that*

_{r} (p) = (p). | (14) |

For each *t* Î [0, 1], if we consider the hypersurface *M _{t}* parameterized by

j(x) = exp_{p}(x + ((1 - t)m + tm')(x)h_{o}) , x Î W, | (15) |

we have that *M _{t}* is tangent to both

*M*and

*M'*in

*p*and that

*M*

_{t}is between

*M*and

*M'*in a neighborhood of

*p*. Using (14), we conclude that the principal curvature vector of

*M*at

_{t}*p*is equal to (

*p*) = (

*p*), for all

*t*Î [0, 1]. This implies, by (8), that

l o ((1 - Ãt)L(m)(0) + tL(m')(0)) = (p) = (p) , | (16) |

for all *t* Î [0, 1]. Since (*p*) = (*p*) is elliptic, it follows from (11) and Lemma 4 that either F* _{r}* or -F

*is elliptic along the line segment (1 -*

_{r}*t*)L(m)(0) +

*t*L(m')(0) Ì

^{(n(n + 1)/2) + n}´ Ì

^{d}. Since ellipticity is an open condition, restricting

*W*if necessary, we conclude by continuity and by the compactness of [0,1] that either F

*or -F*

_{r}*is elliptic in (1 -*

_{r}*t*)L(m)(

*x*) +

*t*L(m')(

*x*), for all

*t*Î [0, 1] and

*x*Î

*W*. Consequently either F

*or -F*

_{r}*is elliptic with respect to the functions (1 -*

_{r}*t*)m +

*t*m',

*t*Î [0, 1]. So, by (13), we can apply the maximum principle to conclude that m and m' coincide in a neighborhood of zero. Therefore,

*M*and

*M'*coincide in a neighborhood of

*p*.

PROOF OF THEOREM 2.a. By our assumptions it holds that *H' _{r}* (

*x*) ³

*H*(

_{r}*x*)

*for*Î

*x**W*. This and (10) imply that

F(L(m)(_{r}x)) - F(L(m)(_{r}x)) ³ 0 , x Î W. | (17) |

Since *M* remains above *M'*, we have (*p*) - (*p*) Î ^{n}. It follows from our assumptions and Lemma 3 that (*p*) Î . We claim that (*p*) Î ¶G* _{r}* . Otherwise, by Lemma 4.1 in Fontenele-Silva (2001), we would have that (

*p*) Î G

*, which is a contradiction since*

_{r}*H*(

_{r}*p*) £ 0. So (

*p*) Î

*Z*

_{1}= ¶G

*. We can use Lemma 4.1 in Fontenele-Silva (2001) to conclude that (*

_{r}*p*) Î

*Z*

_{1}= ¶G

*. As in the proof of Theorem 1.a, we can use Lemmas 1 and 2 to obtain that (*

_{r}*p*) = (

*p*). Since > 0 on G

*, 1 £*

_{r}*i*£

*n*, (

*p*) = (

*p*) is an elliptic root of s

*= 0 and (*

_{r}*p*) = (

*p*) Î¶G

*, we deduce that*

_{r} ((p)) > 0 , "i = 1,..., n. | (18) |

Now, proceeding as in the proof of Theorem 1.a, we conclude that is elliptic with respect to the functions (1 - *t*)m + *t*m', *t* Î [0, 1]. It follows from (17) and the maximum principle that m and m' coincide in a neighborhood of zero. Therefore, *M* and *M'* coincide in a neighborhood of *p*.

**ACKNOWLEDGMENTS **

We would like to thank M.P. do Carmo for suggesting us to publish this work.

** RESUMO **

Neste trabalho nós generalizamos e estendemos para uma variedade Riemanniana arbitrária princípios do máximo para hipersuperfícies com *r*-ésima curvatura média zero no espaço Euclidiano, obtidos por Hounie-Leite.

** Palavras-chave:** princípio do máximo, hipersuperfície, *r*-ésima curvatura média.

** REFERENCES **

ALEXANDROV AD. 1962. Uniqueness theorems for surfaces in the large I. Amer Math Soc Transl, Ser 2, 21: 341-354. [ Links ]

CAFFARELLI L, NIRENBERG L AND SPRUCK J. 1985. The Dirichlet problem for nonlinear second order elliptic equations III: Functions of the eigenvalues of the hessian. Acta Math 155: 261-301. [ Links ]

FONTENELE F AND SILVA SL. 2001. A tangency principle and applications. Illinois J Math 45: 213-228. [ Links ]

GåRDING L. 1959. An inequality for hyperbolic polynomials. J Math Mech 8: 957-965. [ Links ]

HOUNIE J AND LEITE ML. 1995. The maximum principle for hypersurfaces with vanishing curvature functions. J Differ Geom 41: 247-258. [ Links ]

HOUNIE J AND LEITE ML. 1999. Two-ended hypersurfaces with zero scalar curvature. Indiana Univ Math J 48: 817-882. [ Links ]

Correspondence to: Francisco X. Fontenele

E-mail: fontenele@mat.uff.br