X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C713E1.7FE19627@onstor-exch02.onstor.net>; Wed, 29 Nov 2006 10:09:23 -0800
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C713E1.7FE19627"
Content-class: urn:content-classes:message
Subject: RE: quick Clio update
Date: Wed, 29 Nov 2006 10:09:22 -0800
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E018995FB@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E018995D6@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: quick Clio update
thread-index: AccTYoNq8hbp8v5rSpWC2ZzQHHvkQAABB+7yAAahHsAAAY7zIAAVAUtgAADkKNAAAEeecA==
From: "Jonathan Goldick" <jonathan.goldick@onstor.com>
To: "Jobi Ariyamannil" <jobi.ariyamannil@onstor.com>,
	"Vikas Saini" <vikas.saini@onstor.com>,
	"Paul Hammer" <paul.hammer@onstor.com>,
	"Sandrine Boulanger" <sandrine.boulanger@onstor.com>,
	"dl-Clio" <dl-Clio@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C713E1.7FE19627
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

The cache hit rates are very high for all types except inodes and
quotas.

=20

inodeBuf 99.4%=20

8knondir  99.1%

1knondir 100%

8kdir  99.6%

1kdir 99.99%

Inodes 90% ***********

Quotas 0%  ***********

=20

=20

I don't see why you would increase 8knondir or 1kdir.  We are getting
killed on quota entries.

________________________________

From: Jobi Ariyamannil=20
Sent: Wednesday, November 29, 2006 9:54 AM
To: Vikas Saini; Paul Hammer; Sandrine Boulanger; dl-Clio
Subject: RE: quick Clio update

=20

Please increase HWM for 8knondir.
Also increasing HWM for 1kdir, inodes and quotas may improve overall eek
run time.

=20

Are there any other workloads on the filer ?

=20

Also is it possible to test p4 change 22002 ?

=20

Regards,

Jobi

=20

________________________________

From: Vikas Saini=20
Sent: Wednesday, November 29, 2006 9:29 AM
To: Paul Hammer; Jobi Ariyamannil; Sandrine Boulanger; dl-Clio
Subject: RE: quick Clio update

=20

This is with OPT build. Here are the cache stats

=20

# fscmd cachestats

=20

Cachestats for all volumes:

=20

Type            Max        Used       Dirty        Hits      Misses 1/00
COW        LWM        HWM

------------------------------------------------------------------------
-----------------------------------

inodeBuf      85630         144           0   203178455     1299676    6
35      42930     107367

8knondir      35563       35563           0   392994894     3411709    8
98      14310      35563

1knondir     182783           1           0       36723           0    0
0     182783     224309

8kdir          7372          22           0      177523         773    4
255       2862       8589

1kdir        418752           4           0    21414637        1513    0
36     208895     418752

user         185343           0           0           0           0    0
0     185343     322103

inodes      1836266         399           1   737060853    86360953  117
742737    1836266

quotas         1073           4           0         106      836856
7894867         15       1073       1073

=20

I/O         Curr        Max           Meta        User         Log

Type        Pending     Pending       Total       Total       Total

-------------------------------------------------------------------

read              0         268    33121198          28         425

write             0         257       69761          31      322871

#

=20

=20

=20

May be it a good idea to have an eek which skips Quota records and
security blobs or any other thing which can take a lot of time .

=20

Thanks

Vikas

=20

=20

________________________________

From: Paul Hammer=20
Sent: Tuesday, November 28, 2006 11:30 PM
To: Jobi Ariyamannil; Sandrine Boulanger; Vikas Saini; dl-Clio
Subject: RE: quick Clio update

=20

Cool, all good inputs, thanks.=20

=20

Believe these eek runs were started before we moved to opt builds but
not sure. Vikas?=20

=20

Not sure we can chalk up all of the time to Quotas though, hence we were
trying to make sure the volumes were clean independent of the
configurations so that we could continue to test post all the eek/fs
corrections that are now in the release.

=20

-Paul.

=20

=20

=20

________________________________

From: Jobi Ariyamannil
Sent: Tue 11/28/2006 10:47 PM
To: Sandrine Boulanger; Vikas Saini; dl-Clio
Subject: RE: quick Clio update

EEK is very slow processing quota records and QA have been reporting EEK
running for long time when filesystems have large number of quota
records.
On the other hand, I have not seen any customer reporting the same issue
so far.

Are you running EEK with debug or opt build ? We should run EEK on such
volumes with OPT builds only which may be faster than debug builds.
Also please check the output of "fscmd cachestats" to see whether we are
bottlenecking on the caches.  If so, we can tune that and make eek
finish faster.

Also I checked in a change (p4 change 22002) today which will add read
ahead while eek processing quota records.  It would be good to check
whether that improves the situation or not.

In any case, it is not good to hold the QA resources for 8 days (and
more) just to verify quota records which from past experiences never
reported any corruptions.  We can even provide an eek which skip that
phase for QA testing.  (Also all the snapshots can be removed offline by
passing "-c" option to eek).

Please let me any comments on this.

Regards,
Jobi

-----Original Message-----
From: Sandrine Boulanger
Sent: Tuesday, November 28, 2006 7:30 PM
To: Vikas Saini; dl-Clio
Subject: RE: quick Clio update

Cheetah soak has been running some traffic since last wednesday on the
filer that is not running eek. We'll send out a report tomorrow.


-----Original Message-----
From: Vikas Saini
Sent: Tue 11/28/2006 7:00 PM
To: dl-Clio
Subject: quick Clio update

Defects

MF in Dev 26

MF in QA for Clio       38
NMF in QA for Clio      13
All in QA for Clio      51
All in QA               134

We have run eek on all the filers which are being used for Clio testing.
Volumes which were part of Bobcat Soak and Cheetah Soak. We are still
running eek on them. Eek is taking a lot of time on those volumes. Eek
on a bobcat soak volume is running since last 8 days and has not
completed yet. Below is the output for that eek. We have a similar
volume on cheetah soak on which eek in running since last 6 days.


# fscmd eek eng49-3 lvolOnline -r -f
running eek in repair mode
Force mode: corrupt snapshots will be automatically removed.
volId 0x17a000000d0 is not online, state is 1
eek: trying to bring the logical volume online
Done.
Mon Nov 20 09:22:21 2006 =3D=3D eng49-3 =3D=3D verifying meta inodes =
=3D=3D
eng49-3: Inode (I#1) file invalid numblocks 3036141 in superblock,
expected 3036142
Mon Nov 20 09:22:35 2006 =3D=3D eng49-3 =3D=3D mark and compare =
reference counts
=3D=3D
Mon Nov 20 09:23:08 2006 =3D=3D eng49-3 =3D=3D mark used blocks =3D=3D
eng49-3: Inode (I#1) file invalid numblocks 3036141 in superblock,
expected 3036142
processed 194314240 of 194314240 inodes

Mon Nov 20 10:20:38 2006 =3D=3D eng49-3 =3D=3D compare used blocks =
=3D=3D
processed 105574336 of 105574336 blocks

Mon Nov 20 11:31:12 2006 =3D=3D eng49-3 =3D=3D processing quota trees =
=3D=3D
processed 1 quota tree records
Mon Nov 20 11:31:12 2006 =3D=3D eng49-3 =3D=3D mark used inodes =3D=3D
proceprocessed 194314240 of 194314240 inodes, 16441522 of 16441522
directories

Tue Nov 21 01:42:55 2006 =3D=3D eng49-3 =3D=3D detecting unused quota =
trees =3D=3D
Tue Nov 21 01:42:55 2006 =3D=3D eng49-3 =3D=3D processing user quotas =
=3D=3D
processed 1220544 user quota records
processed 2466048 user quota records
processed 2671032 user quota records


On testing side from schedule point of view, we are couple of days
behind but will catch up this week.

Soak env is not fully operational yet. Bobcat soak and cheetah soak are
running eek and super soak is still in setup process.

More detailed Clio update will be sent in a day or two.

Thanks
Vikas


------_=_NextPart_001_01C713E1.7FE19627
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:st1=3D"urn:schemas-microsoft-com:office:smarttags" =
xmlns=3D"http://www.w3.org/TR/REC-html40">

<head>
<meta http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<title>RE: quick Clio update</title>
<o:SmartTagType =
namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"
 name=3D"place"/>
<o:SmartTagType =
namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"
 name=3D"PersonName"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
 /* Font Definitions */
 @font-face
	{font-family:"MS Mincho";
	panose-1:2 2 6 9 4 2 5 8 3 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
	{font-family:"\@MS Mincho";
	panose-1:2 2 6 9 4 2 5 8 3 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:navy;
	text-decoration:underline;}
p
	{mso-margin-top-alt:auto;
	margin-right:0in;
	mso-margin-bottom-alt:auto;
	margin-left:0in;
	font-size:12.0pt;
	font-family:"Times New Roman";}
span.EmailStyle18
	{mso-style-type:personal;
	font-family:Arial;
	color:navy;}
span.EmailStyle19
	{mso-style-type:personal;
	font-family:Arial;
	color:navy;}
span.EmailStyle20
	{mso-style-type:personal-reply;
	font-family:Arial;
	color:navy;}
@page Section1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
 <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
 <o:shapelayout v:ext=3D"edit">
  <o:idmap v:ext=3D"edit" data=3D"1" />
 </o:shapelayout></xml><![endif]-->
</head>

<body lang=3DEN-US link=3Dblue vlink=3Dnavy>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>The cache hit rates are very high =
for all
types except inodes and quotas.<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>inodeBuf 99.4% =
<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>8knondir =
&nbsp;99.1%<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>1knondir =
100%<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>8kdir&nbsp; =
99.6%<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>1kdir =
99.99%<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Inodes 90% =
***********<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Quotas 0%&nbsp; =
***********<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>I don&#8217;t see why you would =
increase
8knondir or 1kdir. &nbsp;We are getting killed on quota =
entries.<o:p></o:p></span></font></p>

<div>

<div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><font =
size=3D3
face=3D"Times New Roman"><span style=3D'font-size:12.0pt'>

<hr size=3D2 width=3D"100%" align=3Dcenter tabindex=3D-1>

</span></font></div>

<p class=3DMsoNormal><b><font size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font =
size=3D2
face=3DTahoma><span style=3D'font-size:10.0pt;font-family:Tahoma'> Jobi =
Ariyamannil
<br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> Wednesday, November =
29, 2006
9:54 AM<br>
<b><span style=3D'font-weight:bold'>To:</span></b> Vikas Saini; Paul =
Hammer;
Sandrine Boulanger; dl-Clio<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> RE: quick Clio =
update</span></font><o:p></o:p></p>

</div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Please increase HWM for =
8knondir.<br>
Also increasing HWM for 1kdir, inodes and quotas may improve overall eek =
run
time.<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Are there any other workloads on =
the filer
?<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Also is it possible to test p4 =
change
22002 ?<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Regards,<o:p></o:p></span></font></p=
>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Jobi<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<div>

<div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><font =
size=3D3
face=3D"Times New Roman"><span style=3D'font-size:12.0pt'>

<hr size=3D2 width=3D"100%" align=3Dcenter tabindex=3D-1>

</span></font></div>

<p class=3DMsoNormal><b><font size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font =
size=3D2
face=3DTahoma><span style=3D'font-size:10.0pt;font-family:Tahoma'> Vikas =
Saini <br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> Wednesday, November =
29, 2006
9:29 AM<br>
<b><span style=3D'font-weight:bold'>To:</span></b> <st1:PersonName =
w:st=3D"on">Paul
 Hammer</st1:PersonName>; Jobi Ariyamannil; Sandrine Boulanger; =
dl-Clio<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> RE: quick Clio =
update</span></font><o:p></o:p></p>

</div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>This is with OPT build. Here are =
the cache
stats<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'># fscmd =
cachestats<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Cachestats for all =
volumes:<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Type&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Max&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Used&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Dirty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Hits&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Misses =
1/00&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
COW&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
LWM&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
HWM<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>------------------------------------=
-----------------------------------------------------------------------<o=
:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>inodeBuf&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;
85630&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
144&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
0&nbsp;&nbsp;
203178455&nbsp;&nbsp;&nbsp;&nbsp; 1299676&nbsp;&nbsp;&nbsp; =
6&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
35&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 42930&nbsp;&nbsp;&nbsp;&nbsp; =
107367<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>8knondir&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;
35563&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
35563&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
0&nbsp;&nbsp;
392994894&nbsp;&nbsp;&nbsp;&nbsp; 3411709&nbsp;&nbsp;&nbsp;
8&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
98&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 14310&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
35563<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>1knondir&nbsp;&nbsp;&nbsp;&nbsp;
182783&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
1&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=

36723&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp; =
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp;&nbsp; 182783&nbsp;&nbsp;&nbsp;&nbsp; =
224309<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>8kdir&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;
7372&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
22&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
177523&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
773&nbsp;&nbsp;&nbsp; 4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
255&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
2862&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
8589<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>1kdir&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;
418752&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp; 21414637&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
1513&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;36&nbsp;&nbsp;&nbsp;&nbsp; 208895&nbsp;&nbsp;&nbsp;&nbsp; =
418752<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;
185343&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp; =
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp;&nbsp; 185343&nbsp;&nbsp;&nbsp;&nbsp; =
322103<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>inodes&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=

1836266&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
399&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
1&nbsp;&nbsp;
737060853&nbsp;&nbsp;&nbsp; 86360953&nbsp; =
117&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;
742737&nbsp;&nbsp;&nbsp; 1836266<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>quotas&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;
1073&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
106&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
836856 7894867&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
15&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
1073&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
1073<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>I/O&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;
Curr&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Max&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
<st1:place
w:st=3D"on">Meta</st1:place>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
User&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
Log<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Type&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;
Pending&nbsp;&nbsp;&nbsp;&nbsp; =
Pending&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Total&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Total&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
Total<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>------------------------------------=
-------------------------------<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>read&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 268&nbsp;&nbsp;&nbsp;
33121198&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
28&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
425<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>write&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
257&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
69761&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
31&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 322871<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>#<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>May be it a good idea to have an =
eek which
skips Quota records and security blobs or any other thing which can take =
a lot
of time .<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Thanks<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Vikas<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<div>

<div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><font =
size=3D3
face=3D"Times New Roman"><span style=3D'font-size:12.0pt'>

<hr size=3D2 width=3D"100%" align=3Dcenter tabindex=3D-1>

</span></font></div>

<p class=3DMsoNormal><b><font size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font =
size=3D2
face=3DTahoma><span style=3D'font-size:10.0pt;font-family:Tahoma'> =
<st1:PersonName
w:st=3D"on">Paul Hammer</st1:PersonName> <br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> Tuesday, November =
28, 2006
11:30 PM<br>
<b><span style=3D'font-weight:bold'>To:</span></b> Jobi Ariyamannil; =
Sandrine
Boulanger; Vikas Saini; dl-Clio<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> RE: quick Clio =
update</span></font><o:p></o:p></p>

</div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

<div id=3DidOWAReplyText27844>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>Cool, all good inputs, thanks. =
</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>Believe these eek runs were =
started
before we moved to opt builds but not sure. Vikas? =
</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>Not sure we can chalk up all of =
the time
to Quotas though, hence we were trying to make sure the volumes were =
clean
independent of the configurations so that we could continue to test post =
all
the eek/fs corrections that are now in the =
release.</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>-Paul.</span></font><o:p></o:p></p>=


</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

<div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><font =
size=3D3
face=3D"Times New Roman"><span style=3D'font-size:12.0pt'>

<hr size=3D2 width=3D"100%" align=3Dcenter tabIndex=3D-1>

</span></font></div>

<p class=3DMsoNormal style=3D'margin-bottom:12.0pt'><b><font size=3D2 =
face=3DTahoma><span
style=3D'font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</spa=
n></font></b><font
size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;font-family:Tahoma'> Jobi
Ariyamannil<br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> Tue 11/28/2006 =
10:47 PM<br>
<b><span style=3D'font-weight:bold'>To:</span></b> Sandrine Boulanger; =
Vikas
Saini; dl-Clio<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> RE: quick Clio =
update</span></font><o:p></o:p></p>

</div>

<div>

<p style=3D'margin-bottom:12.0pt'><font size=3D2 face=3D"Times New =
Roman"><span
style=3D'font-size:10.0pt'>EEK is very slow processing quota records and =
QA have
been reporting EEK running for long time when filesystems have large =
number of
quota records.<br>
On the other hand, I have not seen any customer reporting the same issue =
so
far.<br>
<br>
Are you running EEK with debug or opt build ? We should run EEK on such =
volumes
with OPT builds only which may be faster than debug builds.<br>
Also please check the output of &quot;fscmd cachestats&quot; to see =
whether we
are bottlenecking on the caches.&nbsp; If so, we can tune that and make =
eek
finish faster.<br>
<br>
Also I checked in a change (p4 change 22002) today which will add read =
ahead
while eek processing quota records.&nbsp; It would be good to check =
whether
that improves the situation or not.<br>
<br>
In any case, it is not good to hold the QA resources for 8 days (and =
more) just
to verify quota records which from past experiences never reported any
corruptions.&nbsp; We can even provide an eek which skip that phase for =
QA
testing.&nbsp; (Also all the snapshots can be removed offline by passing =
&quot;-c&quot;
option to eek).<br>
<br>
Please let me any comments on this.<br>
<br>
Regards,<br>
Jobi<br>
<br>
-----Original Message-----<br>
From: Sandrine Boulanger<br>
Sent: Tuesday, November 28, 2006 7:30 PM<br>
To: Vikas Saini; dl-Clio<br>
Subject: RE: quick Clio update<br>
<br>
Cheetah soak has been running some traffic since last wednesday on the =
filer
that is not running eek. We'll send out a report tomorrow.<br>
<br>
<br>
-----Original Message-----<br>
From: Vikas Saini<br>
Sent: Tue 11/28/2006 7:00 PM<br>
To: dl-Clio<br>
Subject: quick Clio update<br>
<br>
Defects<br>
<br>
MF in Dev 26<br>
<br>
MF in QA for Clio&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 38<br>
NMF in QA for Clio&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 13<br>
All in QA for Clio&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 51<br>
All in QA&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 134<br>
<br>
We have run eek on all the filers which are being used for Clio testing.
Volumes which were part of Bobcat Soak and Cheetah Soak. We are still =
running
eek on them. Eek is taking a lot of time on those volumes. Eek on a =
bobcat soak
volume is running since last 8 days and has not completed yet. Below is =
the
output for that eek. We have a similar volume on cheetah soak on which =
eek in
running since last 6 days.<br>
<br>
<br>
# fscmd eek eng49-3 lvolOnline -r -f<br>
running eek in repair mode<br>
Force mode: corrupt snapshots will be automatically removed.<br>
volId 0x17a000000d0 is not online, state is 1<br>
eek: trying to bring the logical volume online<br>
Done.<br>
Mon Nov 20 09:22:21 2006 =3D=3D eng49-3 =3D=3D verifying meta inodes =
=3D=3D<br>
eng49-3: Inode (I#1) file invalid numblocks 3036141 in superblock, =
expected
3036142<br>
Mon Nov 20 09:22:35 2006 =3D=3D eng49-3 =3D=3D mark and compare =
reference counts =3D=3D<br>
Mon Nov 20 09:23:08 2006 =3D=3D eng49-3 =3D=3D mark used blocks =
=3D=3D<br>
eng49-3: Inode (I#1) file invalid numblocks 3036141 in superblock, =
expected
3036142<br>
processed 194314240 of 194314240 inodes<br>
<br>
Mon Nov 20 10:20:38 2006 =3D=3D eng49-3 =3D=3D compare used blocks =
=3D=3D<br>
processed 105574336 of 105574336 blocks<br>
<br>
Mon Nov 20 11:31:12 2006 =3D=3D eng49-3 =3D=3D processing quota trees =
=3D=3D<br>
processed 1 quota tree records<br>
Mon Nov 20 11:31:12 2006 =3D=3D eng49-3 =3D=3D mark used inodes =
=3D=3D<br>
proceprocessed 194314240 of 194314240 inodes, 16441522 of 16441522 =
directories<br>
<br>
Tue Nov 21 01:42:55 2006 =3D=3D eng49-3 =3D=3D detecting unused quota =
trees =3D=3D<br>
Tue Nov 21 01:42:55 2006 =3D=3D eng49-3 =3D=3D processing user quotas =
=3D=3D<br>
processed 1220544 user quota records<br>
processed 2466048 user quota records<br>
processed 2671032 user quota records<br>
<br>
<br>
On testing side from schedule point of view, we are couple of days =
behind but
will catch up this week.<br>
<br>
Soak env is not fully operational yet. Bobcat soak and cheetah soak are =
running
eek and super soak is still in setup process.<br>
<br>
More detailed Clio update will be sent in a day or two.<br>
<br>
Thanks<br>
Vikas</span></font><o:p></o:p></p>

</div>

</div>

</body>

</html>

------_=_NextPart_001_01C713E1.7FE19627--
