X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C782D1.80491CAF@onstor-exch02.onstor.net>; Thu, 19 Apr 2007 15:24:31 -0700
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C782D1.80491CAF"
Content-class: urn:content-classes:message
Subject: RE: Block size for NFS
Date: Thu, 19 Apr 2007 15:24:31 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E0351BFB2@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E0351BF06@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: Block size for NFS
Thread-Index: AceBic2Vf4Hru1jRQ8+esydCBD+yAgArNxRwACMKPcAAAIuP4AADA0Ag
From: "Eric Barrett" <eric.barrett@onstor.com>
To: "Maxim Kozlovsky" <maxim.kozlovsky@onstor.com>,
	"Rich LaReau" <rich.lareau@onstor.com>,
	"dl-cstech" <dl-cstech@onstor.com>
Cc: "Frontline Support" <support@cc.onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C782D1.80491CAF
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Also look for a crappy network.  If they are using NFS over UDP, then
reducing the rsize/wsize will lessen the impact of packet loss,
significantly moreso than NFS over TCP (which is more tolerant to begin
with).
=20


________________________________

From: Maxim Kozlovsky=20
Sent: Thursday, April 19, 2007 2:07 PM
To: Rich LaReau; dl-cstech
Cc: Frontline Support
Subject: RE: Block size for NFS



It does not make sense to specify the read or write size to the value
less than the server can support. This value is the maximum read or
write that can be used. The client will send smaller reads and writes if
it wishes to do so. Rsize and wsize settings should not depend on
whether the workload is streaming or not.=20

=20

Setting the block size to 2k must have slowed down the things so the
"hang" does not happen anymore or is not noticeable (because it is slow
enough already).=20

________________________________

From: Rich LaReau=20
Sent: Thursday, April 19, 2007 1:53 PM
To: dl-cstech
Cc: Frontline Support
Subject: Block size for NFS

=20

=20

We had a recent case, 4944, where a customer was getting periodic
"hangs" on one of his vsvrs.  Normally traffic would run fine, then have
a couple minutes of extreme slow-down (never to zero and with no
crashes.)  It turned out that particular vsvr was hosting NFS Squid
server data (http cache) which contains zillions of tiny files.  They
fixed it by changing the nfs rw mount options to 2k, justified by more
"closely matching" the application's data.

=20

We recommend for "streaming" applications to set these mount options to
32k:

=20

http://wiki/wiki/Best_practices_for_NFS

=20

Would a Squid server application be considered "streaming?"  Could
somebody describe briefly why we recommend 32k (this seems to be the
standard recommendation for nfs mounts) and how setting that to 2k
affected the system?  I imagine that it would throttle down the overall
throughput, which is now fixing or masking whatever was getting
temporarily stuck.

=20

Thanks,

Rich

=20

=20

=20

-----Original Message-----
From: Raghu Murthy=20
Sent: Wednesday, April 18, 2007 8:58 PM
To: Rich LaReau
Subject: FW: [Silverpop Systems Inc] 4944: "df -h hangs for a period of
time then goes through" - New case comment.

=20

=20

=20

May be useful for your performance doc

--------------------

We have identified that our 3 response master servers, that write lots
of small files, is contributing to the problem. We have another
production environment that does more volume of files and quantity of
space and it's mounts are on onstor01, and does not have this problem.
We have adjusted the NFS parameters to 2k rw buffer size on the problem
servers.

----------------

=20

-----Original Message-----

From: dl-Customer Service Cases=20

Sent: Wednesday, April 18, 2007 12:18 AM

To: dl-Customer Service Cases

Subject: [Silverpop Systems Inc] 4944: "df -h hangs for a period of time
then goes through" - New case comment.

=20

=20

A new comment has been added to case 4944 by Bob Kelleher.

https://ssl.salesforce.com/50000000000mBx1AAE

=20

---------------- Comment: ----------------

After changing the r/w buff size on the clients, all has been quite
well.  We are investigating why this single VS has 3 to 4 time more
traffic than the other 4 combined, running on the other gateway.

=20

=20

=20

You can close this ticket, thanks for the buff size settings
recommendation.

=20

=20

=20

rk

=20


------_=_NextPart_001_01C782D1.80491CAF
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns=3D"http://www.w3.org/TR/REC-html40" xmlns:v =3D=20
"urn:schemas-microsoft-com:vml" xmlns:o =3D=20
"urn:schemas-microsoft-com:office:office" xmlns:w =3D=20
"urn:schemas-microsoft-com:office:word"><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii">
<META content=3D"MSHTML 6.00.2900.3059" name=3DGENERATOR><!--[if !mso]>
<STYLE>v\:* {
	BEHAVIOR: url(#default#VML)
}
o\:* {
	BEHAVIOR: url(#default#VML)
}
w\:* {
	BEHAVIOR: url(#default#VML)
}
.shape {
	BEHAVIOR: url(#default#VML)
}
</STYLE>
<![endif]-->
<STYLE>@font-face {
	font-family: Tahoma;
}
@page Section1 {size: 8.5in 11.0in; margin: 1.0in 77.95pt 1.0in 77.95pt; =
}
P.MsoNormal {
	FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
LI.MsoNormal {
	FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
DIV.MsoNormal {
	FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
A:link {
	COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
	COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
	COLOR: purple; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
	COLOR: purple; TEXT-DECORATION: underline
}
P.MsoPlainText {
	FONT-SIZE: 10pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Courier New"
}
LI.MsoPlainText {
	FONT-SIZE: 10pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Courier New"
}
DIV.MsoPlainText {
	FONT-SIZE: 10pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Courier New"
}
SPAN.EmailStyle18 {
	COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal-reply
}
DIV.Section1 {
	page: Section1
}
</STYLE>
<!--[if gte mso 9]><xml>
 <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
 <o:shapelayout v:ext=3D"edit">
  <o:idmap v:ext=3D"edit" data=3D"1" />
 </o:shapelayout></xml><![endif]--></HEAD>
<BODY lang=3DEN-US vLink=3Dpurple link=3Dblue>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D971192122-19042007><FONT =
face=3DArial=20
size=3D2>Also look for a crappy network.&nbsp; </FONT></SPAN><SPAN=20
class=3D971192122-19042007><FONT face=3DArial size=3D2>If they are using =
NFS over UDP,=20
then reducing the rsize/wsize will lessen the impact of packet loss,=20
significantly moreso than NFS over TCP (which is more tolerant to begin=20
with).</FONT></SPAN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D971192122-19042007><FONT =
face=3DArial=20
size=3D2></FONT></SPAN>&nbsp;</DIV><FONT face=3DArial =
size=3D2></FONT><FONT face=3DArial=20
size=3D2></FONT><FONT face=3DArial size=3D2></FONT><BR>
<DIV class=3DOutlookMessageHeader lang=3Den-us dir=3Dltr align=3Dleft>
<HR tabIndex=3D-1>
<FONT face=3DTahoma size=3D2><B>From:</B> Maxim Kozlovsky =
<BR><B>Sent:</B> Thursday,=20
April 19, 2007 2:07 PM<BR><B>To:</B> Rich LaReau; =
dl-cstech<BR><B>Cc:</B>=20
Frontline Support<BR><B>Subject:</B> RE: Block size for =
NFS<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=3DSection1>
<P class=3DMsoNormal><FONT face=3DArial color=3Dnavy size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">It does not =
make sense=20
to specify the read or write size to the value less than the server can =
support.=20
This value is the <I><SPAN style=3D"FONT-STYLE: =
italic">maximum</SPAN></I> read or=20
write that can be used. The client will send smaller reads and writes if =
it=20
wishes to do so. Rsize and wsize settings should not depend on whether =
the=20
workload is streaming or not. <o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial color=3Dnavy size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: =
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial color=3Dnavy size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Setting the =
block size=20
to 2k must have slowed down the things so the &#8220;hang&#8221; does =
not happen anymore or=20
is not noticeable (because it is slow enough already).=20
<o:p></o:p></SPAN></FONT></P>
<DIV>
<DIV class=3DMsoNormal style=3D"TEXT-ALIGN: center" align=3Dcenter><FONT =

face=3D"Times New Roman" size=3D3><SPAN style=3D"FONT-SIZE: 12pt">
<HR tabIndex=3D-1 align=3Dcenter width=3D"100%" SIZE=3D2>
</SPAN></FONT></DIV>
<P class=3DMsoNormal><B><FONT face=3DTahoma size=3D2><SPAN=20
style=3D"FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: =
Tahoma">From:</SPAN></FONT></B><FONT=20
face=3DTahoma size=3D2><SPAN style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Tahoma"> Rich=20
LaReau <BR><B><SPAN style=3D"FONT-WEIGHT: bold">Sent:</SPAN></B> =
Thursday, April=20
19, 2007 1:53 PM<BR><B><SPAN style=3D"FONT-WEIGHT: bold">To:</SPAN></B>=20
dl-cstech<BR><B><SPAN style=3D"FONT-WEIGHT: bold">Cc:</SPAN></B> =
Frontline=20
Support<BR><B><SPAN style=3D"FONT-WEIGHT: bold">Subject:</SPAN></B> =
Block size for=20
NFS</SPAN></FONT><o:p></o:p></P></DIV>
<P class=3DMsoNormal><FONT face=3D"Times New Roman" size=3D3><SPAN=20
style=3D"FONT-SIZE: 12pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: Arial">We had a recent case, =
4944, where a=20
customer was getting periodic "hangs" on one of his vsvrs.&nbsp; =
Normally=20
traffic would run fine, then have a couple minutes of extreme slow-down =
(never=20
to zero and with no crashes.)&nbsp; It turned out that particular vsvr =
was=20
hosting NFS Squid server data (http cache) which contains zillions of =
tiny=20
files.&nbsp; They fixed it by changing the nfs rw mount options to 2k, =
justified=20
by more "closely matching" the application's =
data.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: Arial">We recommend for =
"streaming"=20
applications to set these mount options to =
32k:<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial">http://wiki/wiki/Best_practices_for_NFS<o:p></o:p></SPAN></FONT></=
P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: Arial">Would a Squid server =
application be=20
considered "streaming?"&nbsp; Could somebody describe briefly why we =
recommend=20
32k (this seems to be the standard recommendation for nfs mounts) and =
how=20
setting that to 2k affected the system?&nbsp; I imagine that it would =
throttle=20
down the overall throughput, which is now fixing or masking whatever was =
getting=20
temporarily stuck.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial">Thanks,<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial">Rich<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">-----Original Message-----<BR>From: Raghu =
Murthy=20
<BR>Sent: Wednesday, April 18, 2007 8:58 PM<BR>To: Rich =
LaReau<BR>Subject: FW:=20
[Silverpop Systems Inc] 4944: "df -h hangs for a period of time then =
goes=20
through" - New case comment.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">May be useful for your performance=20
doc<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: =
10pt">--------------------<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">We have identified that our 3 response master =
servers,=20
that write lots of small files, is contributing to the problem. We have =
another=20
production environment that does more volume of files and quantity of =
space and=20
it's mounts are on onstor01, and does not have this problem. We have =
adjusted=20
the NFS parameters to 2k rw buffer size on the problem=20
servers.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">----------------<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">-----Original =
Message-----<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">From: dl-Customer Service Cases=20
<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">Sent: Wednesday, April 18, 2007 12:18=20
AM<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">To: dl-Customer Service=20
Cases<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">Subject: [Silverpop Systems Inc] 4944: "df -h =
hangs for=20
a period of time then goes through" - New case=20
comment.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">A new comment has been added to case 4944 by =
Bob=20
Kelleher.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: =
10pt">https://ssl.salesforce.com/50000000000mBx1AAE<o:p></o:p></SPAN></FO=
NT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">---------------- Comment:=20
----------------<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">After changing the r/w buff size on the =
clients, all has=20
been quite well.&nbsp; We are investigating why this single VS has 3 to =
4 time=20
more traffic than the other 4 combined, running on the other=20
gateway.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">You can close this ticket, thanks for the buff =
size=20
settings recommendation.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt">rk<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoPlainText><FONT face=3D"Courier New" size=3D2><SPAN=20
style=3D"FONT-SIZE: =
10pt"><o:p>&nbsp;</o:p></SPAN></FONT></P></DIV></BODY></HTML>

------_=_NextPart_001_01C782D1.80491CAF--
