X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C795FD.4B1B9D78@onstor-exch02.onstor.net>; Mon, 14 May 2007 00:55:52 -0700
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C795FD.4B1B9D78"
References: <BB375AF679D4A34E9CA8DFA650E2B04E028FB43E@onstor-exch02.onstor.net><BB375AF679D4A34E9CA8DFA650E2B04E0A91F0@onstor-exch02.onstor.net> <20070514004707.4f2e22da@ripper.onstor.net>
Content-class: urn:content-classes:message
Subject: RE: ssh configuration (Defect 18513)
Date: Mon, 14 May 2007 00:51:43 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E028FB443@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: ssh configuration (Defect 18513)
Thread-Index: AceV/BMaMWqJvdKlTQW3ZDYeAODqCQAAKOSO
From: "Mike Lee" <mike.lee@onstor.com>
To: "Andy Sharp" <andy.sharp@onstor.com>,
	"Larry Scheer" <larry.scheer@onstor.com>
Cc: "Brian DeForest" <brian.deforest@onstor.com>,
	"Rendell Fong" <rendell.fong@onstor.com>,
	"Sandrine Boulanger" <sandrine.boulanger@onstor.com>,
	"Tim Gardner" <tim.gardner@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C795FD.4B1B9D78
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Andy:
=20
I'll read over the QA script more carefully tomorrow, but I don't recall =
it kicking off nfxsh.
I think all it does is start an ssh session, in which a process ID =
search is done, followed by a kill.
However, I could be wrong...
=20
Got a bad headache at the moment, so will have to call it a night now.
=20
Thanks.
=20
-Mike

________________________________

From: Andy Sharp
Sent: Mon 5/14/2007 12:47 AM
To: Larry Scheer
Cc: Mike Lee; Brian DeForest; Rendell Fong; Sandrine Boulanger; Tim =
Gardner
Subject: Re: ssh configuration (Defect 18513)



Something just doesn't add up here.  40KB per ssh login?  I guarantee
it's a lot more than that.  But Larry's point still persists, even if
it's 10 times that, it doesn't make sense that there are 1000 ssh
sessions going.  Are there?  The amount of memory used by each instance
of nfxsh is going to be in the megabytes per, and that's before you run
any commands.  Even if you add that to the ssh usage, it still sounds
wonky.

Something is amiss or missing with this analysis.

Cheers,

a


On Sun, 13 May 2007 21:36:12 -0700 "Larry Scheer"
<larry.scheer@onstor.com> wrote:

> How many concurrent SSH connections were there?
> At 40Kbytes of memory each session, you would need 6554 sessions
> running to exhaust 256Mbytes of memory. Are you saying the real
> problem is a runaway process spawning SSH connections?
>
> When do we ever have dozens of SSH processes running running on the
> SSC? I can't imagine hundreds much less thousands of SSH processes.
> What are seeing that I am missing here?
>
> Larry
>
> -----Original Message-----
> From: Mike Lee
> Sent: Sun 5/13/2007 8:29 PM
> To: Andy Sharp; Larry Scheer
> Cc: Brian DeForest; Rendell Fong; Sandrine Boulanger; Tim Gardner
> Subject: ssh configuration (Defect 18513)
>=20
> Gentlemen:
>
> Concerning that BSD panic due to kernel memory exhaustion, Rendell
> figured out that it was due to too many concurrent ssh connections to
> our filer, where each connection ate up 40K of memory.=20
>
> As such, I think we need to configure our ssh daemon to limit the
> maximum number of concurrent connections.  I searched a bit online
> and the only thing I found was the MaxStartups setting, but it is for
> "concurrent unauthenticated connections".=20
>
> Do you know of a way to limit number of connections, authenticated or
> unauthenticated?
>
> Thanks!
>
> -Mike
>
>
> MaxStartups
> Specifies the maximum number of concurrent unauthenticated
> connections to the sshd daemon. Additional connections will be
> dropped until authentication succeeds or the LoginGraceTime expires
> for a connection. The default is 10. Alternatively, random early drop
> can be enabled by specifying the three colon separated values
> ``start:rate:full'' (e.g., "10:30:60"). sshd will refuse connection
> attempts with a probability of ``rate/100'' (30%) if there are
> currently ``start'' (10) unauthenticated connections. The probability
> increases linearly and all connection attempts are refused if the
> number of unauthenticated connections reaches ``full'' (60).
>



------_=_NextPart_001_01C795FD.4B1B9D78
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<HTML dir=3Dltr><HEAD><TITLE>Re: ssh configuration (Defect 18513)</TITLE>=0A=
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Dunicode">=0A=
<META content=3D"MSHTML 6.00.2800.1589" name=3DGENERATOR></HEAD>=0A=
<BODY>=0A=
<DIV id=3DidOWAReplyText70864 dir=3Dltr>=0A=
<DIV dir=3Dltr><FONT face=3DArial color=3D#000000 =
size=3D2>Andy:</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2>I'll read over the QA script =
more carefully tomorrow, but I don't recall it kicking off =
nfxsh.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2>I think all it does is start =
an ssh session, in which a process ID search is done, followed by a =
kill.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2>However, I could be =
wrong...</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2>Got a bad headache at the =
moment, so will have to call it a night now.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2>Thanks.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2>-Mike</FONT></DIV></DIV>=0A=
<DIV dir=3Dltr><BR>=0A=
<HR tabIndex=3D-1>=0A=
<FONT face=3DTahoma size=3D2><B>From:</B> Andy Sharp<BR><B>Sent:</B> Mon =
5/14/2007 12:47 AM<BR><B>To:</B> Larry Scheer<BR><B>Cc:</B> Mike Lee; =
Brian DeForest; Rendell Fong; Sandrine Boulanger; Tim =
Gardner<BR><B>Subject:</B> Re: ssh configuration (Defect =
18513)<BR></FONT><BR></DIV>=0A=
<DIV>=0A=
<P><FONT size=3D2>Something just doesn't add up here.&nbsp; 40KB per ssh =
login?&nbsp; I guarantee<BR>it's a lot more than that.&nbsp; But Larry's =
point still persists, even if<BR>it's 10 times that, it doesn't make =
sense that there are 1000 ssh<BR>sessions going.&nbsp; Are there?&nbsp; =
The amount of memory used by each instance<BR>of nfxsh is going to be in =
the megabytes per, and that's before you run<BR>any commands.&nbsp; Even =
if you add that to the ssh usage, it still =
sounds<BR>wonky.<BR><BR>Something is amiss or missing with this =
analysis.<BR><BR>Cheers,<BR><BR>a<BR><BR><BR>On Sun, 13 May 2007 =
21:36:12 -0700 "Larry Scheer"<BR>&lt;larry.scheer@onstor.com&gt; =
wrote:<BR><BR>&gt; How many concurrent SSH connections were =
there?<BR>&gt; At 40Kbytes of memory each session, you would need 6554 =
sessions<BR>&gt; running to exhaust 256Mbytes of memory. Are you saying =
the real<BR>&gt; problem is a runaway process spawning SSH =
connections?<BR>&gt;<BR>&gt; When do we ever have dozens of SSH =
processes running running on the<BR>&gt; SSC? I can't imagine hundreds =
much less thousands of SSH processes.<BR>&gt; What are seeing that I am =
missing here?<BR>&gt;<BR>&gt; Larry<BR>&gt;<BR>&gt; -----Original =
Message-----<BR>&gt; From: Mike Lee<BR>&gt; Sent: Sun 5/13/2007 8:29 =
PM<BR>&gt; To: Andy Sharp; Larry Scheer<BR>&gt; Cc: Brian DeForest; =
Rendell Fong; Sandrine Boulanger; Tim Gardner<BR>&gt; Subject: ssh =
configuration (Defect 18513)<BR>&gt;&nbsp;<BR>&gt; =
Gentlemen:<BR>&gt;<BR>&gt; Concerning that BSD panic due to kernel =
memory exhaustion, Rendell<BR>&gt; figured out that it was due to too =
many concurrent ssh connections to<BR>&gt; our filer, where each =
connection ate up 40K of memory.&nbsp;<BR>&gt;<BR>&gt; As such, I think =
we need to configure our ssh daemon to limit the<BR>&gt; maximum number =
of concurrent connections.&nbsp; I searched a bit online<BR>&gt; and the =
only thing I found was the MaxStartups setting, but it is for<BR>&gt; =
"concurrent unauthenticated connections".&nbsp;<BR>&gt;<BR>&gt; Do you =
know of a way to limit number of connections, authenticated or<BR>&gt; =
unauthenticated?<BR>&gt;<BR>&gt; Thanks!<BR>&gt;<BR>&gt; =
-Mike<BR>&gt;<BR>&gt;<BR>&gt; MaxStartups<BR>&gt; Specifies the maximum =
number of concurrent unauthenticated<BR>&gt; connections to the sshd =
daemon. Additional connections will be<BR>&gt; dropped until =
authentication succeeds or the LoginGraceTime expires<BR>&gt; for a =
connection. The default is 10. Alternatively, random early drop<BR>&gt; =
can be enabled by specifying the three colon separated values<BR>&gt; =
``start:rate:full'' (e.g., "10:30:60"). sshd will refuse =
connection<BR>&gt; attempts with a probability of ``rate/100'' (30%) if =
there are<BR>&gt; currently ``start'' (10) unauthenticated connections. =
The probability<BR>&gt; increases linearly and all connection attempts =
are refused if the<BR>&gt; number of unauthenticated connections reaches =
``full'' (60).<BR>&gt;<BR></FONT></P></DIV></BODY></HTML>
------_=_NextPart_001_01C795FD.4B1B9D78--
