X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C86372.B6C5C5A5@onstor-exch02.onstor.net>; Wed, 30 Jan 2008 12:02:52 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-class: urn:content-classes:message
Subject: RE: "system upgrade" vs. script for Cougar
Date: Wed, 30 Jan 2008 12:02:52 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E056C9322@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E08057FE7@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: "system upgrade" vs. script for Cougar
Thread-Index: Ache8fxYV4sy+k48R/6Gwp/9aDzS6wAAPmUAACmuf3AA8kZs0AADy7yw
From: "Larry Scheer" <larry.scheer@onstor.com>
To: "Rich LaReau" <rich.lareau@onstor.com>,
	"Tim Gardner" <tim.gardner@onstor.com>,
	"Andy Sharp" <andy.sharp@onstor.com>
Cc: "Andrew LeFebvre" <andrew.lefebvre@onstor.com>

Rich,
   I reviewed TED 21999. It sounds like you are requesting the default
behavior of system upgrade to be that it always initializes the standby
flash. What if this default behavior changed the installation time from
5 minutes to 15 to 20 minutes? Would you still want that to be the
default behavior of system upgrade?

-----Original Message-----
From: Rich LaReau=20
Sent: Wednesday, January 30, 2008 9:14 AM
To: Larry Scheer; Tim Gardner; Andy Sharp
Cc: Andrew LeFebvre
Subject: FW: "system upgrade" vs. script for Cougar

=20
Hi guys,

At yesterday's Cougar meeting Tim said that he (?) is working on a
design doc for "system upgrade".  I just wanted to ask if there was
anything left for me to do here.  I filed an ECR which Larry was going
to review.  Here was the request:


TED 21999
Would like to get a few changes made to "system upgrade" behavior
(unclear if these should be separate ECR's)

1)  Combine "system copy init" to default "system upgrade".  We want
default behavior to be to initialize secondary flash and then run the
upgrade, all in one step.  (Include a flag which will disable the
initialization)

2)  Make secondary flash upgrade default.  Include flag which upgrades
primary flash.  (Obviously, the "init" step above will never run under
this option.)

3)  "system upgrade" should be able to recognize an initialized flash,
and skip the step where it scans for which files it needs to download.
The flash is empty, everything needs to be downloaded, so we should skip
this step to save some time.


Was there anything else?
Rich



-----Original Message-----
From: Rich LaReau=20
Sent: Friday, January 25, 2008 1:34 PM
To: Eric Barrett; Andy Sharp; Dennis Arellano
Cc: Sandrine Boulanger; Larry Scheer; Vikas Saini; Sudheesh Nair; Tim
Gardner; Andrew LeFebvre; Caeli Collins; Paul Hammer; Brian Stark
Subject: RE: "system upgrade" vs. script for Cougar


I filed an ECR to capture the requests for the system upgrade.  Please
have a look and let me know if this will meet our needs, and if it needs
more information, etc.

TED00021999=20

Thanks for all the help and valuable input!
Rich


-----Original Message-----
From: Eric Barrett
Sent: Thursday, January 24, 2008 5:39 PM
To: Andy Sharp; Dennis Arellano
Cc: Sandrine Boulanger; Rich LaReau; Larry Scheer; Vikas Saini; Sudheesh
Nair; Tim Gardner; Andrew LeFebvre; Caeli Collins; Paul Hammer; Brian
Stark
Subject: RE: "system upgrade" vs. script for Cougar

That kind of exception makes a lot of sense for internal use, but not
much for customers.  Why not also make that an 'exception' flag, akin to
what Tim suggested about making the secondary the default target, and
the primary usable via -p?
=20

-----Original Message-----
From: Andy Sharp
Sent: Thursday, January 24, 2008 5:31 PM
To: Dennis Arellano
Cc: Sandrine Boulanger; Rich LaReau; Larry Scheer; Eric Barrett; Vikas
Saini; Sudheesh Nair; Tim Gardner; Andrew LeFebvre; Caeli Collins; Paul
Hammer; Brian Stark
Subject: Re: "system upgrade" vs. script for Cougar

I like all of what I'm hearing, except for the bug part, of course.
Nice bit of teamwork.

As for Tim's question, I can picture instances where there are
mods/customizations, bashrc files, etc., that I wouldn't want wiped out
just to do an upgrade, so I think keeping it a two stepper is OK.  Keep
in mind that on Linux, the 'init' step takes 30s or less, so the impact
is lower and the impatience factor is also lower.  In the near future.
~:^)

Cheers,

a

On Thu, 24 Jan 2008 17:17:31 -0800 "Dennis Arellano"
<dennis.arellano@onstor.com> wrote:

> Sounds like we are close to closure.=20
>=20
> Rich, when you have all the details worked out, we need to sit down=20
> and write up the new Upgrade Procedure.
>=20
> Thanks, Dennis
>=20
> -----Original Message-----
> From: Sandrine Boulanger
> Sent: Thursday, January 24, 2008 4:53 PM
> To: Rich LaReau; Larry Scheer; Andy Sharp; Eric Barrett
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew=20
> LeFebvre; Caeli Collins; Paul Hammer; Brian Stark
> Subject: RE: "system upgrade" vs. script for Cougar
>=20
> There a bug that ins some cases, after a copy init, a system upgrade=20
> -s does not work with the error "not compatible with this system" or=20
> similar. I ran into it this morning. I had to manually mount / from=20
> secondary flash and copy /version from primary to secondary to be able

> to upgrade. This needs to be fixed if we want to use this procedure. A

> defect was filed months ago.
>=20
> -----Original Message-----
> From: Rich LaReau
> Sent: Thursday, January 24, 2008 4:49 PM
> To: Larry Scheer; Andy Sharp; Sandrine Boulanger; Eric Barrett
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew=20
> LeFebvre; Caeli Collins; Paul Hammer; Brian Stark
> Subject: RE: "system upgrade" vs. script for Cougar
>=20
>=20
>=20
> Hi all,
>=20
> After further discussion with Larry, we think that we have a simple=20
> and robust solution with the current implementation of the "system=20
> upgrade" command.  In particular, we should consider making our=20
> primary supported upgrade process the following:
>=20
> < get logs, insure backups are good... etc.>
>=20
> Then run the two commands:
>=20
> > system copy init
> > system upgrade ...
>=20
> This essentially replicates what the script is doing.  The first=20
> command will make a clean CF with empty partitions.  The second will=20
> make a clean installation.  (I've tested this in 3.2, and there's=20
> still a wasted slow step of "checking to see what's there" before the=20
> install, but Larry says we can clean that up easily. I'll file the
> ECR.)
>=20
> We could consider standardizing on this method for 3.2 patch releases,

> and make any subsequent changes in time for Cougar, if necessary.
>=20
> Would this meet everyone's requirements?
>=20
> Rich
>=20
>=20
> -----Original Message-----
> From: Rich LaReau
> Sent: Thursday, January 24, 2008 9:04 AM
> To: Larry Scheer; Andy Sharp
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew=20
> LeFebvre; Caeli Collins; Paul Hammer; Brian Stark
> Subject: RE: "system upgrade" vs. script for Cougar
>=20
>=20
> Thanks Andy and Larry, for the comments.  I guess it's hard for me to=20
> say one way or the other whether the current "system upgrade" process=20
> is good enough for everyone's concerns.  I believe that customers have

> been using the scripts exclusively since they were available, so we=20
> have no data otherwise.  The admittedly old perceptions of the "system

> upgrade" commands have not been replaced by good experience in the=20
> field.
>=20
> We could try to use the non-script process for patch releases for 3.2=20
> and see how well they go.  Although any feedback we get might be too=20
> late to incorporate back into Cougar, if needed.
>=20
> Anybody else have other suggestions?
>=20
> Rich
>=20
>=20
> -----Original Message-----
> From: Larry Scheer
> Sent: Wednesday, January 23, 2008 5:42 PM
> To: Andy Sharp; Rich LaReau
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew=20
> LeFebvre; Caeli Collins; Paul Hammer; Brian Stark
> Subject: RE: "system upgrade" vs. script for Cougar
>=20
> Thanks Andy for forwarding this message to me; shame on you Rich, for=20
> not sending me a copy of your original message. ;-)
>=20
> I placed my comments in line below.
>=20
> Larry
>=20
> -----Original Message-----
> From: Andy Sharp
> Sent: Wednesday, January 23, 2008 3:32 PM
> To: Rich LaReau
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew=20
> LeFebvre; Larry Scheer; Caeli Collins; Paul Hammer; Brian Stark
> Subject: Re: "system upgrade" vs. script for Cougar
>=20
> On Wed, 23 Jan 2008 15:02:01 -0800 "Rich LaReau"
> <rich.lareau@onstor.com> wrote:
>=20
> >=20
> > Hi team,
> >=20
> > Yesterday after the core team meeting I reviewed the issues that=20
> > Support had with the "system upgrade" procedure.  The concerns and=20
> > requirements appear to come down to a few issues.  I wanted to get=20
> > them out for review, and then ask you what our next steps need to=20
> > be.
> >=20
> > The issues which are of CS concern are:
> >=20
> > 1)   Failed upgrades.  Before we used the scripts, it was common
> > that we would get upgrades which outright failed, or worse, appeared

> > to work but contained some corruption.  If we are to rely on "system

> > upgrade" then these issues need to be resolved.  By comparison, the=20
> > scripts completely blank the secondary flash and then copy=20
> > everything in fresh.  This avoids problems with copying cruft (old=20
> > core files in odd places, repair attempts, lost+found files, etc.)
>=20
> I think this is largely an old perception.  I personally know of all=20
> these problems you mentioned being fixed at least by the 3.0 release=20
> if not sooner. At this point in time, and consider that this is the=20
> person who originally wrote the install script saying this, if I it=20
> was my customer, and I was upgrading from a 3.x to a 3.x, I would use=20
> system upgrade.  Or flash_install with auto-upgrade.
>=20
> Note that there is a workflow that uses system upgrade that does what=20
> you mention in terms of wiping the CF card and installing fresh.
> Doing a _system copy init_ followed by a _system upgrade_.  Code was=20
> specifically added to make this possible.
>=20
> [LCS] I completely agree with Andy. Your perceptions are very old. The

> problems with system upgrade corrupting installations were fixed in=20
> Release 2.1. The ability to erase a flash and install (as Andy
> mentions) was introduced in release 3.0 when the layout of the flash=20
> changed.
>=20
> > 2)  Time of upgrade.  This problem was primarily the result of=20
> > needing
>=20
> > to download the entire tarball twice from the ftp server.  The=20
> > scripts
>=20
> > are able to work with only one download.
> >=20
>=20
> [LCS] The double download problem was fixed in 3.0. The cw_install=20
> script was created because the 2.X system upgrade could not reliably=20
> upgrade the system to 3.0 due to the SSC running out of memory. A side

> benefit of the cw_install script is it ran in less time. But it was=20
> designed to get customers from 2.X and earlier to 3.0 in a single=20
> unmonitored step. I am working on a faster upgrade but I can't promise

> anything for system W (BSD based releases.)
>=20
> > 3) Ease of use.  We do a lot of upgrades, so the process needs to be
> > simple-- both in the upgrade itself and the recovery process in case

> > something fails.
>=20
> As I understand it, many SEs make a flash with flash_install.sh before

> they head to the customer site, and set it to auto-upgrade when it is=20
> booted. That means that you can just insert the flash card in the=20
> secondary slot, and do a reboot.  When it boots, it copies the=20
> configuration info from the other flash card.  For down time, this is=20
> definitely the fastest way to go, but obviously there are a lot of=20
> caveats to that as well, not the least of which is logistical, which=20
> prevents us (I'm told) from making this our default upgrade procedure.
>=20
> [LCS] I'm with you on this one Rich!
>=20
> > Some people had suggestions about how we might implement a good=20
> > system
>=20
> > upgrade process:
>=20
> Excellent.  Ideas always most welcome.  Keep 'em coming.  Often.
>=20
> > A)  The system upgrade command should look in the downloaded tarball

> > for a script to execute.  If it exists, the script is run; if not,=20
> > it uses the old method.  That means if we do other flash card=20
> > layouts, or
>=20
> > even more drastic changes, the script can manage all of that, and it

> > will be tied to the release bundle rather than having to be inserted

> > earlier in the development cycle.
>=20
> This is the way it actually works now.  Believe me when I say that=20
> this has been long thought of and desired.  Because of the current=20
> design of executing a script from the upgrade tarball itself, many=20
> changes can be made in one release, but not all.  For instance, being=20
> able to use HTTP as well as FTP, well, that's going into the next=20
> release, but it won't help when upgrading to that release.  But=20
> eventually it will help.
>=20
> [LCS] This is the way system upgrade has been working since 2.1. A=20
> change was just checked into the dev branch to allow the System W (BSD
> based) release to use HTTP. But remember, hen you run system upgrade=20
> it is the older nfxsh without the new capability that is doing the=20
> download.
>=20
> > B)  Standardize to the WebUI and replace the need for an FTP server=20
> > all together.  We could specify that the tar ball and md5 file are=20
> > in the same directory pick them both up and check them before=20
> > install commences, therefore ruling out a corrupt tar ball as a=20
> > problem for the installation. If we also had a new upgrade script we

> > could have the process look for this before commencing the=20
> > installation as well, replace the current install script and then=20
> > start the process again. So the process would be, go to the WebUI,=20
> > open the system upgrade page, point it the directory containing the=20
> > scripts and click on the version to install, then sit back get a=20
> > coffee/tea and watch the status bar go straight up to 90% before=20
> > waiting another 30 minutes for
>=20
> > the process to complete (sorry this isn't MS is it!)
>=20
> Sounds good to me!  Let us know when you've got something to test!
> And yes, that is on my list of faves but earliest would be post Cougar

> release, and no, it's not on any schedule.  Keep in mind that=20
> implementing this is no small project, and I'm not even including the=20
> WebUI development part.  Ease of use is not always easy to do ~:^)
>=20
> Cheers,
>=20
> a
