X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C7A924.1E36FB4B@onstor-exch02.onstor.net>; Thu, 7 Jun 2007 09:51:39 -0700
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C7A924.1E36FB4B"
Content-class: urn:content-classes:message
Subject: RE: DeLorean Submittal 22 Ready for Acceptance Testing - Was: RE: Please include this in sub #22
Date: Thu, 7 Jun 2007 09:51:38 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E040F089F@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E02F149ED@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: DeLorean Submittal 22 Ready for Acceptance Testing - Was: RE: Please include this in sub #22
Thread-Index: Acen5nuoEYeuTUAkRBCqKqEjQ2vuAAADz/5wABK8ZnQABh3D0AABv9UqAABRq9AAADdViQAAF/LQAAAi/FYAAyY1zAAARo9AAADlqJAAA8VE0AAeuyCcAAhEdLAAAEMspwAAeTsQ
From: "Jobi Ariyamannil" <jobi.ariyamannil@onstor.com>
To: "Raj Kumar" <raj.kumar@onstor.com>,
	"Ken Renshaw" <ken.renshaw@onstor.com>,
	"Brian DeForest" <brian.deforest@onstor.com>
Cc: "Sandrine Boulanger" <sandrine.boulanger@onstor.com>,
	"Paul Hammer" <paul.hammer@onstor.com>,
	"Andy Sharp" <andy.sharp@onstor.com>,
	"Tim Gardner" <tim.gardner@onstor.com>,
	"Jay Michlin" <jay.michlin@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C7A924.1E36FB4B
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Your tests always fill up the initial portion of the filesystem and when
you grow the filesystem over time, all the free space accumulate at the
end of the filesystem.

With our default allocation policy, when you create a new file, we try
to allocate somewhere in the first half of the filesystem and end up
looking up the refcounts from that point to end of filesystem (which is
very huge in your case for the 33 TB filesystem).  On top of that, when
you add other tests on the filer, all of them race for resources and
there will be a lot of cache misses.

=20

If you are absolutely sure the performance dropped after upgrade (which
I am not sure since you were running into watch dog timeouts frequently
and was not making much progress at all), you may downgrade the filer to
submittal 21 and retry.  I am also not advocating to take change for
19400 in delorean (we can back out that).

=20

I think there are many scaling issues with large filesystems and the way
you doing these tests.

=20

Regards,

Jobi

=20

________________________________

From: Raj Kumar=20
Sent: Thursday, June 07, 2007 9:27 AM
To: Jobi Ariyamannil; Ken Renshaw; Brian DeForest
Cc: Sandrine Boulanger; Paul Hammer; Andy Sharp
Subject: RE: DeLorean Submittal 22 Ready for Acceptance Testing - Was:
RE: Please include this in sub #22

=20

We got 35 bm/s - 90 mb/s before the upgrade.

=20

The volume is not nearly full.

=20

sourcevol     Mounted     Standard  38287455.06   No Limit
33627270.76   4660184.30    87.83%

=20

Not trying to doany performance benchmarking, just noticed that the
performance dropped very significantly.=20

=20

Just ran into a FP crash, will update the defect.

=20

Thu Jun  7 08:36:50 2007 (721149 usec) 243: FS: sourcevol
0x22700000098 - daemon 11 - spa - buddy
 recal summary thread: exiting


Exception Cause =3D Watchdog Timeout/NMI
ERREPC:   0xffffffff830c4dc4
RA:       0xffffffff830c3520
SR:       0x20086fa1
NMI : Watchdog Timeout NMI
Image Version : NFP_FP : EverON-3.0.0.0BC : Wed Jun  6 15:35:45 2007
PROM Version  : PROM_SIBYTE_BC : prom-2.0.0 : Mon May 16 15:58:00 2005
Boot Time  : Thu Jun  7 08:06:32 GMT 2007
Crash Time : Thu Jun  7 08:36:54 GMT 2007
core_dump_open 0 1 bmc12500_install_allocators: install CRASH allocators
bmc12500Eth_initModule w/AUTO TX Retrans enabled
bcm12500Eth_initModule SEQ RRX Retrans enabled
isModel2280: Model 2280 FIFO mode enabled
OK
writing seg 0 off 0x0 sz 0x3000
writing seg 30 off 0x3000 sz 0x2fdc000 va 0xffffffff83000000 from
0xffffffff83000000
writing seg 31 off 0x2fdf000 sz 0xa0000000 va 0x1000000000 from
0x1000000000
writing seg 32 off 0xa2fdf000 sz 0x58000000 va 0x4000000000 from
0x4000000000
writing seg 33 off 0xfafdf000 sz 0xc8000000 va 0x2000000000 from
0x2000000000
writing seg 34 off 0x1c2fdf000 sz 0x10000 va 0x60000000 from 0x60000000
writing seg 0 sz 0x3000 total number of bytes in dump file 0x1c2fef000
core_dump_close=20
Core written successfully
Autoreboot "off", stopping in debug mode
DEBUG-FP1:1 >=20

=20

________________________________

From: Jobi Ariyamannil
Sent: Thu 6/7/2007 9:23 AM
To: Raj Kumar; Ken Renshaw; Brian DeForest
Cc: Sandrine Boulanger; Paul Hammer; Andy Sharp
Subject: RE: DeLorean Submittal 22 Ready for Acceptance Testing - Was:
RE: Please include this in sub #22

What was the performance before the upgrade ?  When you operate on a
large filesystem which is always nearly full, the performance will be
poor.  Whenever we need to find some free space to allocate in the
filesystem, we need to read a lot of stuff (refcounts and initialize
buddy pages) from disk.  Also I understand you overloaded the testing
with snapshots and quotas.  Also many other volumes are serving data and
participating in various tests from the same filer causing contention on
various resources.  Overall, does not look like a good test bed for
performance measurements.  Defect 19400 just fixed a watchdog timeout
and should not have any impact on performance.

=20

Regards,

Jobi

=20

________________________________

From: Raj Kumar=20
Sent: Thursday, June 07, 2007 5:45 AM
To: Ken Renshaw; Brian DeForest
Cc: Sandrine Boulanger; Paul Hammer; Andy Sharp; Jobi Ariyamannil
Subject: RE: DeLorean Submittal 22 Ready for Acceptance Testing - Was:
RE: Please include this in sub #22

=20

AT Results:

=20

1. Upgrade - Successful

=20

2. 40 TB volume to check PR 19400 - After upgrading the performance is
very very poor. Just getting ~200 bytes per sec. PR# 19465: 40TB volume:
Very very low ops after upgrading to sub 22, elog full of "buddy regions
are full" messages


3. Standalone upgrade: The tool is not able to get the image from the
ftp server. Not sure whether this is because of the invalid response to
the ftp server or because it didnt set the target partition as writable.
PR# 19467: CW-install : Not able to download the image from the ftp
servr

                                                            =20

=20

# ./cw_install.sh =20

Welcome to the ONStor Standalone Upgrade System

This program will upgrade your secondary flash card to EverON
version 3.0.0.0 and boot your system to the new software.

Would you like to run interactive or unattended?  Answer 'yes'
to run in interactive mode.  You will be prompted prior to any
reboots of the system. Otherwise you will only be prompted for
the FTP URL of the distribution file.  The default should work
for most cases.

Would you like to run in interactive mode? [No] yes
I'll take yes as a 'yes' and run in interactive mode.
Formatting secondary Compact Flash Card
CF card is 488 MiB
Downloading the EverON software distribution.
Please enter the ftp URL of the distribution:
[ftp://ftp:anon@support.onstor.com/outbound/software/EverON/3.0.0.0/22xx
/EverON-3.0.0.0BC.tar.gz] ftp://f <ftp://f/>=20
tp:upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0BC.tar.gz
This will take at least 138 seconds to download.
Thank you for your patience.

Downloading external system software distribution.
hookup: 228: Onstor src ip =3D 0x0
Connected to 10.2.0.2.
220 stargate.onstor.lab FTP server (Version wu-2.6.1-16) ready.
331 Guest login ok, send your complete e-mail address as password.
230-The response 'upgrade' is not valid
230-Next time please use your e-mail address as your password
230-        for example: joe@eng71.onstor.lab
230 Guest login ok, access restrictions apply.
200 Type set to I.
250 CWD command successful.
550 home/upgrade: No such file or directory.
Retrieving home/upgrade/EverON-3.0.0.0BC.tar.gz
local: EverON-3.0.0.0BC.tar.gz remote: EverON-3.0.0.0BC.tar.gz
ftp: local: EverON-3.0.0.0BC.tar.gz: Read-only file system
221-You have transferred 0 bytes in 0 files.
221-Total traffic for this session was 558 bytes in 0 transfers.
221 Thank you for using the FTP service on stargate.onstor.lab.
Ftp retrieval of distribution file failed: 1
Please try again.
         =20

=20

________________________________

From: Ken Renshaw
Sent: Wed 6/6/2007 8:52 PM
To: Brian DeForest; Raj Kumar
Cc: Sandrine Boulanger; Ken Renshaw; Paul Hammer; Andy Sharp
Subject: DeLorean Submittal 22 Ready for Acceptance Testing - Was: RE:
Please include this in sub #22

Here is submittal 22 for DeLorean 3.0.0.0 including the cw_install
standalone install script that facilitates one-step upgrades from 1.3.x
and 2.x to 3.0.

The changes since sub21 are these three:

Change 24169 by andys@ripper on 2007/06/04 09:59:34

      Nice down "tar -zx" process to increase responsiveness of
managment processor
      during upgrades, hopefully putting an end to silent reboots during
upgrades.

      TED00018378 - "Cheetah Upgrade of 2nd flash from
1.3.3.15->Delorean#11: Silent reboot"

Change 24171 by andys@ripper on 2007/06/04 13:56:21

      First checkin of standalone Install script and related changes.
      If we've all been living clean lives, the last checkin.

      TED00018276 - "Upgrade from 2221 to 2.3 on secondary flash fails
to copy files"

Change 24188 by jobia@jobi:jobi on 2007/06/05 12:22:27

         Fix a watchdog timeout in fs_buddy_bnum_in_free_region().

         Defect TED00019400.

EverON images ( prior sub21 images have been renamed with a '-sub21'
suffix ):

Cheetah optimized:

ftp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0.tar.gz

Bobcat optimized:

ftp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0BC.tar.gz

Cheetah debug:

ftp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0DBG.tar.gz

Bobcat debug:

ftp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0BCDBG.tar.gz

IMPORTANT NOTE: To test the cw_install.sh script you must use the
version corresponding to
the EverON variant you are using., i.e. cheetah debug, bobcat opt, etc.
At the moment, the
cw_install.sh script is built into the tarball it's supposed to support,
so there
must be workflows designed and tested to extract the file, etc. for the
customer. Right now their are
four versions of the file cw_install.sh that have different contents and
must be managed separately.

In lieu of that, internally I am posting all versions of the script on
both the FTP server and under
Build-Trees for build distribution.

Bobcat optimized:

ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/bc-opt/cw_install.sh
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree/Build/bc/opt/Release/cw
_install.sh

Cheetah optimized:

ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/ch-opt/cw_install.sh
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree/Build/ch/opt/Release/cw
_install.sh

Bobcat debug:

ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/bc-dbg/cw_install.sh
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree/Build/bc/dbg/Release/cw
_install.sh

Cheetah debug:

ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/ch-dbg/cw_install.sh
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree/Build/ch-dbg/Release/cw
_install.sh

Source tree:

/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree

Perforce sync info:

//depot/FB-DELOREAN/...@24204 modulo one nfx-tree/Makefile change for
the 'make release' target support for cw_install

Thanks,

-Ken





________________________________

From: Brian DeForest
Sent: Wednesday, June 06, 2007 1:00 PM
To: Ken Renshaw; Raj Kumar
Cc: Sandrine Boulanger
Subject: RE: Please include this in sub #22



We need 2 things for sub22:



1.      the P4 changelist #24171 includes a changed openbsd file.
Since it doesn't need to be compiled (/etc/rc) I suggest simply patching
this into the latest BSD build rather than do a full BSD build, if Ken
agrees after reviewing the changelist.
2.      merge the fix for CQ #19400 from Zonda to Delorean & have Jobi
review the merge



________________________________

From: Ken Renshaw
Sent: Wednesday, June 06, 2007 12:30 PM
To: Raj Kumar; Brian DeForest
Cc: Sandrine Boulanger
Subject: RE: Please include this in sub #22



I need to finish a build for Ed first, but I can build sub22 later this
afternoon.



-Ken



________________________________

From: Raj Kumar
Sent: Wednesday, June 06, 2007 12:21 PM
To: Raj Kumar; Brian DeForest
Cc: Sandrine Boulanger; Ken Renshaw
Subject: RE: Please include this in sub #22



When should we start the build?



________________________________

From: Raj Kumar
Sent: Wed 6/6/2007 10:50 AM
To: Brian DeForest
Cc: Sandrine Boulanger
Subject: RE: Please include this in sub #22

yeah thats my suggestion.



________________________________

From: Brian DeForest
Sent: Wed 6/6/2007 10:48 AM
To: Raj Kumar
Cc: Sandrine Boulanger
Subject: RE: Please include this in sub #22

I'll confirm w/ Andy that this is ready.    So that would be 2 builds,
sub22 this week and sub23 next week w/ more MF fixes.



________________________________

From: Raj Kumar
Sent: Wednesday, June 06, 2007 10:44 AM
To: Brian DeForest
Cc: Sandrine Boulanger
Subject: RE: Please include this in sub #22



I like to complete 40 TB by 6/15. So we need a build ASAP since it takes
time to create data.



Also Paul has asked us to start with CW-install tests, so we need to
build.



________________________________

From: Brian DeForest
Sent: Wed 6/6/2007 10:39 AM
To: Raj Kumar
Cc: Sandrine Boulanger
Subject: RE: Please include this in sub #22

But is the 40TB testing happening this week or could this wait for the
next build, likely early next week?



I'd prefer to do one more build, if possible, but we may need 2 if you
need this fix sooner.



________________________________

From: Raj Kumar
Sent: Wednesday, June 06, 2007 10:29 AM
To: Brian DeForest; Jobi Ariyamannil
Cc: May Ma; Sandrine Boulanger
Subject: RE: Please include this in sub #22



I think completing the 40TB test should give us enough test coverage for
this change. We should take this



________________________________

From: Brian DeForest
Sent: Wed 6/6/2007 9:40 AM
To: Raj Kumar; Jobi Ariyamannil
Cc: May Ma; Sandrine Boulanger
Subject: RE: Please include this in sub #22

I had been considering this one, and Sandrine and I had discussed it as
a possibility.   On the other hand, I also want to stabilize the code
base and minimize the checkins.



What is the test impact?



________________________________

From: Raj Kumar
Sent: Wednesday, June 06, 2007 6:44 AM
To: Jobi Ariyamannil
Cc: May Ma; Sandrine Boulanger; Brian DeForest; Ken Renshaw
Subject: Please include this in sub #22



Hi,



Like to pull this into sub #22.



Thanks.



________________________________

From: Jobi Ariyamannil
Sent: Tue 6/5/2007 9:51 PM
To: Raj Kumar
Cc: May Ma; Sandrine Boulanger
Subject: RE: Defect TED00019400 FP watchdog timeout on almost full
filesystem (33TB) in fs_buddy_bnum_in_free_region()

Is your filesystem becoming almost full frequently?

If your intention is to populate 40TB, you may temporarily disable the
snapshots.

Also if this is an absolute showstopper for you, we may port the fix to
delorean.

Regards,
Jobi

-----Original Message-----
From: raj.kumar@onstor.com [mailto:raj.kumar@onstor.com]
Sent: Tuesday, June 05, 2007 7:58 PM
To: Jobi Ariyamannil; Jobi Ariyamannil; May Ma
Cc: May Ma; Sandrine Boulanger
Subject: Defect TED00019400 FP watchdog timeout on almost full
filesystem (33TB) in fs_buddy_bnum_in_free_region()

Headline: FP watchdog timeout on almost full filesystem (33TB)  in
fs_buddy_bnum_in_free_region()
id: TED00019400
Note_Entry: Jobi,

ENG7 (40 TB setup) is continously crashing due to FS issue. Here is the
trace, is this trace same issue as this defect?

#0  0x8304ae50 in fs_buddy_bnum_in_free_region (context=3D0x1ae3e000,
    bnum=3D4266274790, len=3D1, isFull=3D0xffffffff85621118) at
fs-buddy.c:2235
#1  0x8304a550 in fs_buddyFindExtent (context=3D0x1ae3e000, len=3D1, =
target=3D
      {block =3D 4266274790}, isFirstFileExtent=3D0, noSnap=3Dfalse)
    at fs-inlines.c:99
#2  0x83038744 in fs_allocateBuddyExtent (context=3D0x101ae3e000,
requested=3D1,
    target=3D{block =3D 4266274790}, anySize=3D0, noSnap=3D0,
isFirstFileExtent=3D0)
    at fs-alloc.c:856
#3  0x83035d94 in fs_allocateExtent (context=3D0x101ae3e000,
inode=3D0x1025e442d0,
    page=3D14069157, len32=3D1, anySize=3D0, contigAlloc=3D0) at =
fs-alloc.c:1280
#4  0x83057998 in fs_bufAllocDoBufs (context=3D0x101ae3e000,
inode=3D0x1025e442d0,
    numInBufs=3D1, bufs=3D0xffffffff856217d8, commitInode=3Dtrue,
contigAlloc=3Dfalse)
    at fs-buf.c:6870
#5  0x830e40a0 in fs_writeThread (context=3D0x101ae3e000) at =
fs-nfs.c:4856
#6  0x83067fa4 in fs_threadBegin (handle=3D10223717, arg=3D0x101ae3e000)
    at fs-context.c:2781
#7  0x8301d2a8 in esmthread_start (handle=3D448921600, nextState=3D0, =
event=3D
          {event =3D 0, p1 =3D {u =3D {dw =3D 18446744071612824480, w =
=3D
2198240160, p =3D
0xffffffff83067ba0}}, p2 =3D {u =3D {dw =3D 18446744072094767276, w =3D
2680182956, p =3D
0xffffffff9fc058ac}}}, userInfo=3D0x101ae3e000) at thread-api.c:311
#8  0x8301a5c4 in esm_threadMain (arg=3D0x1000a33f20) at esm-nfx.c:224
#9  0x8301a568 in esm_threadMain (arg=3D0x1000a33f20) at esm-nfx.c:195
State: Assigned
history: 33706857       May 31 2007  3:31PM     maym    Submit  no_value
Opened
33706861        May 31 2007  3:51PM     jobia   Need_more_info  Opened
More_Info
33706874        May 31 2007  5:10PM     jobia   Back_to_triage
More_Info       Re-opened
33706892        May 31 2007 10:38PM     jobia   Modify  Re-opened
Re-opened
33707130        Jun  4 2007  2:13PM     jobia   Modify  Re-opened
Re-opened
33707215        Jun  4 2007  9:37PM     sandrineb       Review
Re-opened       Reviewed
33707284        Jun  5 2007  1:20PM     jobia   Assign  Reviewed
Assigned
33707347        Jun  5 2007  7:48PM     sandrineb       Modify  Assigned
Assigned
33707348        06/05/2007 19:57:53 PM  rajk    Modify  Assigned
Assigned


------_=_NextPart_001_01C7A924.1E36FB4B
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns=3D"http://www.w3.org/TR/REC-html40">

<head>
<meta http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<title>DeLorean Submittal 22 Ready for Acceptance Testing - Was: RE: =
Please
include this in sub #22</title>
<style>
<!--
font-face=0A=
	{font-family:Tahoma;}

 /* Font Definitions */
 @font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:blue;
	text-decoration:underline;}
p
	{mso-margin-top-alt:auto;
	margin-right:0in;
	mso-margin-bottom-alt:auto;
	margin-left:0in;
	font-size:12.0pt;
	font-family:"Times New Roman";}
span.emailstyle19
	{font-family:Arial;
	color:navy;}
span.EmailStyle20
	{mso-style-type:personal-reply;
	font-family:Arial;
	color:navy;}
@page Section1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
 <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
 <o:shapelayout v:ext=3D"edit">
  <o:idmap v:ext=3D"edit" data=3D"1" />
 </o:shapelayout></xml><![endif]-->
</head>

<body lang=3DEN-US link=3Dblue vlink=3Dblue>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Your tests always fill up the =
initial
portion of the filesystem and when you grow the filesystem over time, =
all the
free space accumulate at the end of the =
filesystem.<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>With our default allocation policy, =
when
you create a new file, we try to allocate somewhere in the first half of =
the
filesystem and end up looking up the refcounts from that point to end of
filesystem (which is very huge in your case for the 33 TB =
filesystem).&nbsp; On
top of that, when you add other tests on the filer, all of them race for
resources and there will be a lot of cache =
misses.<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>If you are absolutely sure the =
performance
dropped after upgrade (which I am not sure since you were running into =
watch
dog timeouts frequently and was not making much progress at all), you =
may
downgrade the filer to submittal 21 and retry.&nbsp; I am also not =
advocating
to take change for 19400 in delorean (we can back out =
that).<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>I think there are many scaling =
issues with
large filesystems and the way you doing these =
tests.<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Regards,<o:p></o:p></span></font></p=
>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Jobi<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<div>

<div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><font =
size=3D3
face=3D"Times New Roman"><span style=3D'font-size:12.0pt'>

<hr size=3D2 width=3D"100%" align=3Dcenter tabindex=3D-1>

</span></font></div>

<p class=3DMsoNormal><b><font size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font =
size=3D2
face=3DTahoma><span style=3D'font-size:10.0pt;font-family:Tahoma'> Raj =
Kumar <br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> Thursday, June 07, =
2007 9:27
AM<br>
<b><span style=3D'font-weight:bold'>To:</span></b> Jobi Ariyamannil; Ken =
Renshaw;
Brian DeForest<br>
<b><span style=3D'font-weight:bold'>Cc:</span></b> Sandrine Boulanger; =
Paul
Hammer; Andy Sharp<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> RE: DeLorean =
Submittal 22
Ready for Acceptance Testing - Was: RE: Please include this in sub =
#22</span></font><o:p></o:p></p>

</div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

<div id=3DidOWAReplyText66884>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>We got 35 bm/s - 90 mb/s before =
the
upgrade.</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'>The volume is not nearly =
full.</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>sourcevol&nbsp;&nbsp;&nbsp;&nbsp; =
Mounted&nbsp;&nbsp;&nbsp;&nbsp;
Standard&nbsp; 38287455.06&nbsp;&nbsp; No
Limit&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 33627270.76&nbsp;&nbsp; =
4660184.30&nbsp;&nbsp;&nbsp;
87.83%<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>Not trying to doany performance benchmarking, just noticed that =
the
performance dropped very significantly. <o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>Just ran into a FP crash, will update the =
defect.<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>Thu Jun&nbsp; 7 08:36:50 2007 (721149 usec) 243: FS:
sourcevol&nbsp;&nbsp;&nbsp; 0x22700000098 - daemon 11 - spa - buddy<br>
&nbsp;recal summary thread: exiting<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'><br>
Exception Cause =3D Watchdog Timeout/NMI<br>
ERREPC:&nbsp;&nbsp; 0xffffffff830c4dc4<br>
RA:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0xffffffff830c3520<br>
SR:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0x20086fa1<br>
NMI : Watchdog Timeout NMI<br>
Image Version : NFP_FP : EverON-3.0.0.0BC : Wed Jun&nbsp; 6 15:35:45 =
2007<br>
PROM Version&nbsp; : PROM_SIBYTE_BC : prom-2.0.0 : Mon May 16 15:58:00 =
2005<br>
Boot Time&nbsp; : Thu Jun&nbsp; 7 08:06:32 GMT 2007<br>
Crash Time : Thu Jun&nbsp; 7 08:36:54 GMT 2007<br>
core_dump_open 0 1 bmc12500_install_allocators: install CRASH =
allocators<br>
bmc12500Eth_initModule w/AUTO TX Retrans enabled<br>
bcm12500Eth_initModule SEQ RRX Retrans enabled<br>
isModel2280: Model 2280 FIFO mode enabled<br>
OK<br>
writing seg 0 off 0x0 sz 0x3000<br>
writing seg 30 off 0x3000 sz 0x2fdc000 va 0xffffffff83000000 from
0xffffffff83000000<br>
writing seg 31 off 0x2fdf000 sz 0xa0000000 va 0x1000000000 from =
0x1000000000<br>
writing seg 32 off 0xa2fdf000 sz 0x58000000 va 0x4000000000 from =
0x4000000000<br>
writing seg 33 off 0xfafdf000 sz 0xc8000000 va 0x2000000000 from =
0x2000000000<br>
writing seg 34 off 0x1c2fdf000 sz 0x10000 va 0x60000000 from =
0x60000000<br>
writing seg 0 sz 0x3000 total number of bytes in dump file =
0x1c2fef000<br>
core_dump_close <br>
Core written successfully<br>
Autoreboot &quot;off&quot;, stopping in debug mode<br>
DEBUG-FP1:1 &gt; <o:p></o:p></span></font></p>

</div>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

<div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><font =
size=3D3
face=3D"Times New Roman"><span style=3D'font-size:12.0pt'>

<hr size=3D2 width=3D"100%" align=3Dcenter tabIndex=3D-1>

</span></font></div>

<p class=3DMsoNormal style=3D'margin-bottom:12.0pt'><b><font size=3D2 =
face=3DTahoma><span
style=3D'font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</spa=
n></font></b><font
size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;font-family:Tahoma'> Jobi
Ariyamannil<br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> Thu 6/7/2007 9:23 =
AM<br>
<b><span style=3D'font-weight:bold'>To:</span></b> Raj Kumar; Ken =
Renshaw; Brian
DeForest<br>
<b><span style=3D'font-weight:bold'>Cc:</span></b> Sandrine Boulanger; =
Paul
Hammer; Andy Sharp<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> RE: DeLorean =
Submittal 22
Ready for Acceptance Testing - Was: RE: Please include this in sub =
#22</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>What was the performance before the
upgrade ?&nbsp; When you operate on a large filesystem which is always =
nearly
full, the performance will be poor.&nbsp; Whenever we need to find some =
free
space to allocate in the filesystem, we need to read a lot of stuff =
(refcounts
and initialize buddy pages) from disk.&nbsp; Also I understand you =
overloaded
the testing with snapshots and quotas.&nbsp; Also many other volumes are
serving data and participating in various tests from the same filer =
causing
contention on various resources.&nbsp; Overall, does not look like a =
good test
bed for performance measurements.&nbsp; Defect 19400 just fixed a =
watchdog
timeout and should not have any impact on =
performance.</span></font><o:p></o:p></p>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Regards,</span></font><o:p></o:p></p=
>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Jobi</span></font><o:p></o:p></p>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

<div>

<div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><font =
size=3D3
face=3D"Times New Roman"><span style=3D'font-size:12.0pt'>

<hr size=3D2 width=3D"100%" align=3Dcenter tabIndex=3D-1>

</span></font></div>

<p class=3DMsoNormal><b><font size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font =
size=3D2
face=3DTahoma><span style=3D'font-size:10.0pt;font-family:Tahoma'> Raj =
Kumar <br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> Thursday, June 07, =
2007 5:45
AM<br>
<b><span style=3D'font-weight:bold'>To:</span></b> Ken Renshaw; Brian =
DeForest<br>
<b><span style=3D'font-weight:bold'>Cc:</span></b> Sandrine Boulanger; =
Paul
Hammer; Andy Sharp; Jobi Ariyamannil<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> RE: DeLorean =
Submittal 22
Ready for Acceptance Testing - Was: RE: Please include this in sub =
#22</span></font><o:p></o:p></p>

</div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

<div id=3DidOWAReplyText76354>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>AT =
Results:</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>1. Upgrade - Successful<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>2. 40 TB volume to check PR 19400 - After upgrading the =
performance is
very very poor. Just getting ~200 bytes per sec. <strong><b><font
face=3D"Times New Roman">PR# 19465: =
</font></b></strong></span></font><font
size=3D2><span style=3D'font-size:10.0pt'>40TB volume: Very very low ops =
after
upgrading to sub 22, elog full of &quot;buddy regions are full&quot; =
messages</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'><br>
3. Standalone upgrade: The tool is not able to get the image from the =
ftp
server. Not sure whether this is because of the invalid response to the =
ftp
server or because it didnt set the target partition as writable. =
<strong><b><font
face=3D"Times New Roman">PR# 19467: =
</font></b></strong></span></font><font
size=3D2><span style=3D'font-size:10.0pt'>CW-install : Not able to =
download the
image from the ftp servr</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'># =
./cw_install.sh&nbsp;&nbsp;</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>Welcome to the ONStor Standalone =
Upgrade
System</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>This program will upgrade your =
secondary
flash card to EverON<br>
version 3.0.0.0 and boot your system to the new =
software.</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>Would you like to run interactive =
or
unattended?&nbsp; Answer 'yes'<br>
to run in interactive mode.&nbsp; You will be prompted prior to any<br>
reboots of the system. Otherwise you will only be prompted for<br>
the FTP URL of the distribution file.&nbsp; The default should work<br>
for most cases.</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>Would you like to run in =
interactive
mode? [No] yes<br>
I'll take yes as a 'yes' and run in interactive mode.<br>
Formatting secondary Compact Flash Card<br>
CF card is 488 MiB<br>
Downloading the EverON software distribution.<br>
Please enter the ftp URL of the distribution:<br>
[ftp://ftp:anon@support.onstor.com/outbound/software/EverON/3.0.0.0/22xx/=
EverON-3.0.0.0BC.tar.gz]
<a href=3D"ftp://f/">ftp://f</a><br>
tp:upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0BC.tar.gz<br>
This will take at least 138 seconds to download.<br>
Thank you for your patience.</span></font><o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:black'>Downloading external system =
software
distribution.<br>
hookup: 228: Onstor src ip =3D 0x0<br>
Connected to 10.2.0.2.<br>
220 stargate.onstor.lab FTP server (Version wu-2.6.1-16) ready.<br>
331 Guest login ok, send your complete e-mail address as password.<br>
230-<strong><b><font face=3DArial><span style=3D'font-family:Arial'>The =
response
'upgrade' is not valid</span></font></b></strong><b><span =
style=3D'font-weight:
bold'><br>
</span></b>230-Next time please use your e-mail address as your =
password<br>
230-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; for example: <a
href=3D"mailto:joe@eng71.onstor.lab">joe@eng71.onstor.lab</a><br>
230 Guest login ok, access restrictions apply.<br>
200 Type set to I.<br>
250 CWD command successful.<br>
550 <strong><b><font face=3DArial><span =
style=3D'font-family:Arial'>home/upgrade:
No such file or directory.</span></font></b></strong><b><span =
style=3D'font-weight:
bold'><br>
</span></b>Retrieving home/upgrade/EverON-3.0.0.0BC.tar.gz<br>
local: EverON-3.0.0.0BC.tar.gz remote: EverON-3.0.0.0BC.tar.gz<br>
<strong><b><font face=3DArial><span style=3D'font-family:Arial'>ftp: =
local:
EverON-3.0.0.0BC.tar.gz: Read-only file =
system</span></font></b></strong><br>
221-You have transferred 0 bytes in 0 files.<br>
221-Total traffic for this session was 558 bytes in 0 transfers.<br>
221 Thank you for using the FTP service on stargate.onstor.lab.<br>
Ftp retrieval of distribution file failed: 1<br>
Please try again.<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></font=
><o:p></o:p></p>

</div>

</div>

<div>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:
12.0pt'>&nbsp;<o:p></o:p></span></font></p>

<div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><font =
size=3D3
face=3D"Times New Roman"><span style=3D'font-size:12.0pt'>

<hr size=3D2 width=3D"100%" align=3Dcenter tabIndex=3D-1>

</span></font></div>

<p class=3DMsoNormal style=3D'margin-bottom:12.0pt'><b><font size=3D2 =
face=3DTahoma><span
style=3D'font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</spa=
n></font></b><font
size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;font-family:Tahoma'> Ken
Renshaw<br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> Wed 6/6/2007 8:52 =
PM<br>
<b><span style=3D'font-weight:bold'>To:</span></b> Brian DeForest; Raj =
Kumar<br>
<b><span style=3D'font-weight:bold'>Cc:</span></b> Sandrine Boulanger; =
Ken
Renshaw; Paul Hammer; Andy Sharp<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> DeLorean =
Submittal 22
Ready for Acceptance Testing - Was: RE: Please include this in sub =
#22</span></font><o:p></o:p></p>

</div>

<div>

<p style=3D'margin-bottom:12.0pt'><font size=3D2 face=3D"Times New =
Roman"><span
style=3D'font-size:10.0pt'>Here is submittal 22 for DeLorean 3.0.0.0 =
including
the cw_install standalone install script that facilitates one-step =
upgrades
from 1.3.x and 2.x to 3.0.<br>
<br>
The changes since sub21 are these three:<br>
<br>
Change 24169 by andys@ripper on 2007/06/04 09:59:34<br>
<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Nice down &quot;tar -zx&quot; process to
increase responsiveness of managment processor<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; during upgrades, hopefully putting an end =
to
silent reboots during upgrades.<br>
<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; TED00018378 - &quot;Cheetah Upgrade of =
2nd flash
from 1.3.3.15-&gt;Delorean#11: Silent reboot&quot;<br>
<br>
Change 24171 by andys@ripper on 2007/06/04 13:56:21<br>
<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; First checkin of standalone Install =
script and
related changes.<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; If we've all been living clean lives, the =
last
checkin.<br>
<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; TED00018276 - &quot;Upgrade from 2221 to =
2.3 on
secondary flash fails to copy files&quot;<br>
<br>
Change 24188 by jobia@jobi:jobi on 2007/06/05 12:22:27<br>
<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Fix a watchdog timeout =
in
fs_buddy_bnum_in_free_region().<br>
<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Defect TED00019400.<br>
<br>
EverON images ( prior sub21 images have been renamed with a '-sub21' =
suffix ):<br>
<br>
Cheetah optimized:<br>
<br>
<a =
href=3D"ftp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0.tar.gz">ftp://=
upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0.tar.gz</a><br>
<br>
Bobcat optimized:<br>
<br>
<a =
href=3D"ftp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0BC.tar.gz">ftp:=
//upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0BC.tar.gz</a><br>
<br>
Cheetah debug:<br>
<br>
<a =
href=3D"ftp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0DBG.tar.gz">ftp=
://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0DBG.tar.gz</a><br>
<br>
Bobcat debug:<br>
<br>
<a =
href=3D"ftp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0BCDBG.tar.gz">f=
tp://upgrade@10.2.0.2/home/upgrade/EverON-3.0.0.0BCDBG.tar.gz</a><br>
<br>
IMPORTANT NOTE: To test the cw_install.sh script you must use the =
version
corresponding to<br>
the EverON variant you are using., i.e. cheetah debug, bobcat opt, etc. =
At the
moment, the<br>
cw_install.sh script is built into the tarball it's supposed to support, =
so
there<br>
must be workflows designed and tested to extract the file, etc. for the
customer. Right now their are<br>
four versions of the file cw_install.sh that have different contents and =
must
be managed separately.<br>
<br>
In lieu of that, internally I am posting all versions of the script on =
both the
FTP server and under<br>
Build-Trees for build distribution.<br>
<br>
Bobcat optimized:<br>
<br>
<a
href=3D"ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/bc-opt/cw_inst=
all.sh">ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/bc-opt/cw_inst=
all.sh</a><br>
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree/Build/bc/opt/Release/cw_=
install.sh<br>
<br>
Cheetah optimized:<br>
<br>
<a
href=3D"ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/ch-opt/cw_inst=
all.sh">ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/ch-opt/cw_inst=
all.sh</a><br>
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree/Build/ch/opt/Release/cw_=
install.sh<br>
<br>
Bobcat debug:<br>
<br>
<a
href=3D"ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/bc-dbg/cw_inst=
all.sh">ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/bc-dbg/cw_inst=
all.sh</a><br>
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree/Build/bc/dbg/Release/cw_=
install.sh<br>
<br>
Cheetah debug:<br>
<br>
<a
href=3D"ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/ch-dbg/cw_inst=
all.sh">ftp://upgrade@10.2.0.2/home/upgrade/DeLorean/sub22/ch-dbg/cw_inst=
all.sh</a><br>
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree/Build/ch-dbg/Release/cw_=
install.sh<br>
<br>
Source tree:<br>
<br>
/n/Build-Trees/R3.0.0.0/R3.0.0.0-060607/nfx-tree<br>
<br>
Perforce sync info:<br>
<br>
//depot/FB-DELOREAN/...@24204 modulo one nfx-tree/Makefile change for =
the 'make
release' target support for cw_install<br>
<br>
Thanks,<br>
<br>
-Ken<br>
<br>
<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Brian DeForest<br>
Sent: Wednesday, June 06, 2007 1:00 PM<br>
To: Ken Renshaw; Raj Kumar<br>
Cc: Sandrine Boulanger<br>
Subject: RE: Please include this in sub #22<br>
<br>
<br>
<br>
We need 2 things for sub22:<br>
<br>
<br>
<br>
1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; the P4 changelist #24171 includes a =
changed
openbsd file.&nbsp;&nbsp; Since it doesn't need to be compiled (/etc/rc) =
I
suggest simply patching this into the latest BSD build rather than do a =
full
BSD build, if Ken agrees after reviewing the changelist.<br>
2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; merge the fix for CQ #19400 from Zonda =
to
Delorean &amp; have Jobi review the merge<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Ken Renshaw<br>
Sent: Wednesday, June 06, 2007 12:30 PM<br>
To: Raj Kumar; Brian DeForest<br>
Cc: Sandrine Boulanger<br>
Subject: RE: Please include this in sub #22<br>
<br>
<br>
<br>
I need to finish a build for Ed first, but I can build sub22 later this =
afternoon.<br>
<br>
<br>
<br>
-Ken<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Raj Kumar<br>
Sent: Wednesday, June 06, 2007 12:21 PM<br>
To: Raj Kumar; Brian DeForest<br>
Cc: Sandrine Boulanger; Ken Renshaw<br>
Subject: RE: Please include this in sub #22<br>
<br>
<br>
<br>
When should we start the build?<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Raj Kumar<br>
Sent: Wed 6/6/2007 10:50 AM<br>
To: Brian DeForest<br>
Cc: Sandrine Boulanger<br>
Subject: RE: Please include this in sub #22<br>
<br>
yeah thats my suggestion.<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Brian DeForest<br>
Sent: Wed 6/6/2007 10:48 AM<br>
To: Raj Kumar<br>
Cc: Sandrine Boulanger<br>
Subject: RE: Please include this in sub #22<br>
<br>
I'll confirm w/ Andy that this is ready.&nbsp;&nbsp;&nbsp; So that would =
be 2
builds, sub22 this week and sub23 next week w/ more MF fixes.<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Raj Kumar<br>
Sent: Wednesday, June 06, 2007 10:44 AM<br>
To: Brian DeForest<br>
Cc: Sandrine Boulanger<br>
Subject: RE: Please include this in sub #22<br>
<br>
<br>
<br>
I like to complete 40 TB by 6/15. So we need a build ASAP since it takes =
time
to create data.<br>
<br>
<br>
<br>
Also Paul has asked us to start with CW-install tests, so we need to =
build.<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Brian DeForest<br>
Sent: Wed 6/6/2007 10:39 AM<br>
To: Raj Kumar<br>
Cc: Sandrine Boulanger<br>
Subject: RE: Please include this in sub #22<br>
<br>
But is the 40TB testing happening this week or could this wait for the =
next
build, likely early next week?<br>
<br>
<br>
<br>
I'd prefer to do one more build, if possible, but we may need 2 if you =
need
this fix sooner.<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Raj Kumar<br>
Sent: Wednesday, June 06, 2007 10:29 AM<br>
To: Brian DeForest; Jobi Ariyamannil<br>
Cc: May Ma; Sandrine Boulanger<br>
Subject: RE: Please include this in sub #22<br>
<br>
<br>
<br>
I think completing the 40TB test should give us enough test coverage for =
this
change. We should take this<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Brian DeForest<br>
Sent: Wed 6/6/2007 9:40 AM<br>
To: Raj Kumar; Jobi Ariyamannil<br>
Cc: May Ma; Sandrine Boulanger<br>
Subject: RE: Please include this in sub #22<br>
<br>
I had been considering this one, and Sandrine and I had discussed it as =
a
possibility.&nbsp;&nbsp; On the other hand, I also want to stabilize the =
code
base and minimize the checkins.<br>
<br>
<br>
<br>
What is the test impact?<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Raj Kumar<br>
Sent: Wednesday, June 06, 2007 6:44 AM<br>
To: Jobi Ariyamannil<br>
Cc: May Ma; Sandrine Boulanger; Brian DeForest; Ken Renshaw<br>
Subject: Please include this in sub #22<br>
<br>
<br>
<br>
Hi,<br>
<br>
<br>
<br>
Like to pull this into sub #22.<br>
<br>
<br>
<br>
Thanks.<br>
<br>
<br>
<br>
________________________________<br>
<br>
From: Jobi Ariyamannil<br>
Sent: Tue 6/5/2007 9:51 PM<br>
To: Raj Kumar<br>
Cc: May Ma; Sandrine Boulanger<br>
Subject: RE: Defect TED00019400 FP watchdog timeout on almost full =
filesystem
(33TB) in fs_buddy_bnum_in_free_region()<br>
<br>
Is your filesystem becoming almost full frequently?<br>
<br>
If your intention is to populate 40TB, you may temporarily disable the
snapshots.<br>
<br>
Also if this is an absolute showstopper for you, we may port the fix to
delorean.<br>
<br>
Regards,<br>
Jobi<br>
<br>
-----Original Message-----<br>
From: raj.kumar@onstor.com [<a =
href=3D"mailto:raj.kumar@onstor.com">mailto:raj.kumar@onstor.com</a>]<br>=

Sent: Tuesday, June 05, 2007 7:58 PM<br>
To: Jobi Ariyamannil; Jobi Ariyamannil; May Ma<br>
Cc: May Ma; Sandrine Boulanger<br>
Subject: Defect TED00019400 FP watchdog timeout on almost full =
filesystem
(33TB) in fs_buddy_bnum_in_free_region()<br>
<br>
Headline: FP watchdog timeout on almost full filesystem (33TB)&nbsp; in
fs_buddy_bnum_in_free_region()<br>
id: TED00019400<br>
Note_Entry: Jobi,<br>
<br>
ENG7 (40 TB setup) is continously crashing due to FS issue. Here is the =
trace,
is this trace same issue as this defect?<br>
<br>
#0&nbsp; 0x8304ae50 in fs_buddy_bnum_in_free_region =
(context=3D0x1ae3e000,<br>
&nbsp;&nbsp;&nbsp; bnum=3D4266274790, len=3D1, =
isFull=3D0xffffffff85621118) at
fs-buddy.c:2235<br>
#1&nbsp; 0x8304a550 in fs_buddyFindExtent (context=3D0x1ae3e000, =
len=3D1, target=3D<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {block =3D 4266274790}, =
isFirstFileExtent=3D0,
noSnap=3Dfalse)<br>
&nbsp;&nbsp;&nbsp; at fs-inlines.c:99<br>
#2&nbsp; 0x83038744 in fs_allocateBuddyExtent (context=3D0x101ae3e000,
requested=3D1,<br>
&nbsp;&nbsp;&nbsp; target=3D{block =3D 4266274790}, anySize=3D0, =
noSnap=3D0,
isFirstFileExtent=3D0)<br>
&nbsp;&nbsp;&nbsp; at fs-alloc.c:856<br>
#3&nbsp; 0x83035d94 in fs_allocateExtent (context=3D0x101ae3e000,
inode=3D0x1025e442d0,<br>
&nbsp;&nbsp;&nbsp; page=3D14069157, len32=3D1, anySize=3D0, =
contigAlloc=3D0) at
fs-alloc.c:1280<br>
#4&nbsp; 0x83057998 in fs_bufAllocDoBufs (context=3D0x101ae3e000,
inode=3D0x1025e442d0,<br>
&nbsp;&nbsp;&nbsp; numInBufs=3D1, bufs=3D0xffffffff856217d8, =
commitInode=3Dtrue,
contigAlloc=3Dfalse)<br>
&nbsp;&nbsp;&nbsp; at fs-buf.c:6870<br>
#5&nbsp; 0x830e40a0 in fs_writeThread (context=3D0x101ae3e000) at =
fs-nfs.c:4856<br>
#6&nbsp; 0x83067fa4 in fs_threadBegin (handle=3D10223717, =
arg=3D0x101ae3e000)<br>
&nbsp;&nbsp;&nbsp; at fs-context.c:2781<br>
#7&nbsp; 0x8301d2a8 in esmthread_start (handle=3D448921600, =
nextState=3D0, event=3D<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {event =3D 0, p1 =
=3D {u =3D
{dw =3D 18446744071612824480, w =3D 2198240160, p
=3D&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0xffffffff83067ba0}}, p2 =3D {u =3D {dw =3D 18446744072094767276, w =3D =
2680182956, p
=3D&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
0xffffffff9fc058ac}}}, userInfo=3D0x101ae3e000) at thread-api.c:311<br>
#8&nbsp; 0x8301a5c4 in esm_threadMain (arg=3D0x1000a33f20) at =
esm-nfx.c:224<br>
#9&nbsp; 0x8301a568 in esm_threadMain (arg=3D0x1000a33f20) at =
esm-nfx.c:195<br>
State: Assigned<br>
history: 33706857&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; May 31 2007&nbsp;
3:31PM&nbsp;&nbsp;&nbsp;&nbsp; maym&nbsp;&nbsp;&nbsp; Submit&nbsp;
no_value&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Opened<br>
33706861&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; May 31 2007&nbsp;
3:51PM&nbsp;&nbsp;&nbsp;&nbsp; jobia&nbsp;&nbsp; Need_more_info&nbsp;
Opened&nbsp; More_Info<br>
33706874&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; May 31 2007&nbsp;
5:10PM&nbsp;&nbsp;&nbsp;&nbsp; jobia&nbsp;&nbsp; Back_to_triage&nbsp;
More_Info&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Re-opened<br>
33706892&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; May 31 2007
10:38PM&nbsp;&nbsp;&nbsp;&nbsp; jobia&nbsp;&nbsp; Modify&nbsp; =
Re-opened&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Re-opened<br>
33707130&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Jun&nbsp; 4 =
2007&nbsp;
2:13PM&nbsp;&nbsp;&nbsp;&nbsp; jobia&nbsp;&nbsp; Modify&nbsp;
Re-opened&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Re-opened<br>
33707215&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Jun&nbsp; 4 =
2007&nbsp;
9:37PM&nbsp;&nbsp;&nbsp;&nbsp; =
sandrineb&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Review&nbsp; Re-opened&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Reviewed<br>
33707284&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Jun&nbsp; 5 =
2007&nbsp;
1:20PM&nbsp;&nbsp;&nbsp;&nbsp; jobia&nbsp;&nbsp; Assign&nbsp; =
Reviewed&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Assigned<br>
33707347&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Jun&nbsp; 5 =
2007&nbsp;
7:48PM&nbsp;&nbsp;&nbsp;&nbsp; =
sandrineb&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Modify&nbsp; Assigned&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
Assigned<br>
33707348&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 06/05/2007 19:57:53 =
PM&nbsp;
rajk&nbsp;&nbsp;&nbsp; Modify&nbsp;
Assigned&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
Assigned</span></font><o:p></o:p></p>

</div>

</div>

</div>

</body>

</html>

------_=_NextPart_001_01C7A924.1E36FB4B--
