X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C85EEC.A23CC248@onstor-exch02.onstor.net>; Thu, 24 Jan 2008 17:53:00 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-class: urn:content-classes:message
Subject: RE: "system upgrade" vs. script for Cougar
Date: Thu, 24 Jan 2008 17:52:59 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E07E5FA34@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E07E5FA2B@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: "system upgrade" vs. script for Cougar
Thread-Index: AcheGCModWzkBXy1QRSuLILN5GEsWgACT0zAACH5qCAADa2Y4AADEMSw
From: "Tim Gardner" <tim.gardner@onstor.com>
To: "Rich LaReau" <rich.lareau@onstor.com>,
	"Larry Scheer" <larry.scheer@onstor.com>,
	"Andy Sharp" <andy.sharp@onstor.com>,
	"Sandrine Boulanger" <sandrine.boulanger@onstor.com>,
	"Eric Barrett" <eric.barrett@onstor.com>
Cc: "Vikas Saini" <vikas.saini@onstor.com>,
	"Sudheesh Nair" <sudheesh.nair@onstor.com>,
	"Dennis Arellano" <dennis.arellano@onstor.com>,
	"Andrew LeFebvre" <andrew.lefebvre@onstor.com>,
	"Caeli Collins" <caeli.collins@onstor.com>,
	"Paul Hammer" <paul.hammer@onstor.com>,
	"Brian Stark" <brian.stark@onstor.com>


Other things to consider:
We tell everyone to not upgrade to the primary flash but yet we still
support it. Should we just remove this support? Or perhaps make -s the
default option and add a -p (for primary) option that maybe is only
available as diag?

Why not just incorporate the system copy init functionality into
system upgrade? Is there ever a case where you would not want to
first run system copy init when running the system upgrade to
the secondary flash?

> -----Original Message-----
> From: Rich LaReau
> Sent: Thursday, January 24, 2008 4:49 PM
> To: Larry Scheer; Andy Sharp; Sandrine Boulanger; Eric Barrett
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew
> LeFebvre; Caeli Collins; Paul Hammer; Brian Stark
> Subject: RE: "system upgrade" vs. script for Cougar
>=20
>=20
>=20
> Hi all,
>=20
> After further discussion with Larry, we think that we have a simple
and
> robust solution with the current implementation of the "system
upgrade"
> command.  In particular, we should consider making our primary
supported
> upgrade process the following:
>=20
> < get logs, insure backups are good... etc.>
>=20
> Then run the two commands:
>=20
> > system copy init
> > system upgrade ...
>=20
> This essentially replicates what the script is doing.  The first
command
> will make a clean CF with empty partitions.  The second will make a
clean
> installation.  (I've tested this in 3.2, and there's still a wasted
slow
> step of "checking to see what's there" before the install, but Larry
says
> we can clean that up easily. I'll file the ECR.)
>=20
> We could consider standardizing on this method for 3.2 patch releases,
and
> make any subsequent changes in time for Cougar, if necessary.
>=20
> Would this meet everyone's requirements?
>=20
> Rich
>=20
>=20
> -----Original Message-----
> From: Rich LaReau
> Sent: Thursday, January 24, 2008 9:04 AM
> To: Larry Scheer; Andy Sharp
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew
> LeFebvre; Caeli Collins; Paul Hammer; Brian Stark
> Subject: RE: "system upgrade" vs. script for Cougar
>=20
>=20
> Thanks Andy and Larry, for the comments.  I guess it's hard for me to
say
> one way or the other whether the current "system upgrade" process is
good
> enough for everyone's concerns.  I believe that customers have been
using
> the scripts exclusively since they were available, so we have no data
> otherwise.  The admittedly old perceptions of the "system upgrade"
> commands have not been replaced by good experience in the field.
>=20
> We could try to use the non-script process for patch releases for 3.2
and
> see how well they go.  Although any feedback we get might be too late
to
> incorporate back into Cougar, if needed.
>=20
> Anybody else have other suggestions?
>=20
> Rich
>=20
>=20
> -----Original Message-----
> From: Larry Scheer
> Sent: Wednesday, January 23, 2008 5:42 PM
> To: Andy Sharp; Rich LaReau
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew
> LeFebvre; Caeli Collins; Paul Hammer; Brian Stark
> Subject: RE: "system upgrade" vs. script for Cougar
>=20
> Thanks Andy for forwarding this message to me; shame on you Rich, for
not
> sending me a copy of your original message. ;-)
>=20
> I placed my comments in line below.
>=20
> Larry
>=20
> -----Original Message-----
> From: Andy Sharp
> Sent: Wednesday, January 23, 2008 3:32 PM
> To: Rich LaReau
> Cc: Vikas Saini; Sudheesh Nair; Tim Gardner; Dennis Arellano; Andrew
> LeFebvre; Larry Scheer; Caeli Collins; Paul Hammer; Brian Stark
> Subject: Re: "system upgrade" vs. script for Cougar
>=20
> On Wed, 23 Jan 2008 15:02:01 -0800 "Rich LaReau"
> <rich.lareau@onstor.com> wrote:
>=20
> >
> > Hi team,
> >
> > Yesterday after the core team meeting I reviewed the issues that
> > Support had with the "system upgrade" procedure.  The concerns and
> > requirements appear to come down to a few issues.  I wanted to get
> > them out for review, and then ask you what our next steps need to
be.
> >
> > The issues which are of CS concern are:
> >
> > 1)   Failed upgrades.  Before we used the scripts, it was common
that
> > we would get upgrades which outright failed, or worse, appeared to
> > work but contained some corruption.  If we are to rely on "system
> > upgrade" then these issues need to be resolved.  By comparison, the
> > scripts completely blank the secondary flash and then copy
everything
> > in fresh.  This avoids problems with copying cruft (old core files
in
> > odd places, repair attempts, lost+found files, etc.)
>=20
> I think this is largely an old perception.  I personally know of all
these
> problems you mentioned being fixed at least by the 3.0 release if not
> sooner. At this point in time, and consider that this is the person
who
> originally wrote the install script saying this, if I it was my
customer,
> and I was upgrading from a 3.x to a 3.x, I would use system upgrade.
Or
> flash_install with auto-upgrade.
>=20
> Note that there is a workflow that uses system upgrade that does what
you
> mention in terms of wiping the CF card and installing fresh. Doing a
> _system copy init_ followed by a _system upgrade_.  Code was
specifically
> added to make this possible.
>=20
> [LCS] I completely agree with Andy. Your perceptions are very old. The
> problems with system upgrade corrupting installations were fixed in
> Release 2.1. The ability to erase a flash and install (as Andy
mentions)
> was introduced in release 3.0 when the layout of the flash changed.
>=20
> > 2)  Time of upgrade.  This problem was primarily the result of
needing
> > to download the entire tarball twice from the ftp server.  The
scripts
> > are able to work with only one download.
> >
>=20
> [LCS] The double download problem was fixed in 3.0. The cw_install
script
> was created because the 2.X system upgrade could not reliably upgrade
the
> system to 3.0 due to the SSC running out of memory. A side benefit of
the
> cw_install script is it ran in less time. But it was designed to get
> customers from 2.X and earlier to 3.0 in a single unmonitored step. I
am
> working on a faster upgrade but I can't promise anything for system W
(BSD
> based releases.)
>=20
> > 3) Ease of use.  We do a lot of upgrades, so the process needs to be
> > simple-- both in the upgrade itself and the recovery process in case
> > something fails.
>=20
> As I understand it, many SEs make a flash with flash_install.sh before
> they head to the customer site, and set it to auto-upgrade when it is
> booted. That means that you can just insert the flash card in the
> secondary slot, and do a reboot.  When it boots, it copies the
> configuration info from the other flash card.  For down time, this is
> definitely the fastest way to go, but obviously there are a lot of
caveats
> to that as well, not the least of which is logistical, which prevents
us
> (I'm told) from making this our default upgrade procedure.
>=20
> [LCS] I'm with you on this one Rich!
>=20
> > Some people had suggestions about how we might implement a good
system
> > upgrade process:
>=20
> Excellent.  Ideas always most welcome.  Keep 'em coming.  Often.
>=20
> > A)  The system upgrade command should look in the downloaded tarball
> > for a script to execute.  If it exists, the script is run; if not,
it
> > uses the old method.  That means if we do other flash card layouts,
or
> > even more drastic changes, the script can manage all of that, and it
> > will be tied to the release bundle rather than having to be inserted
> > earlier in the development cycle.
>=20
> This is the way it actually works now.  Believe me when I say that
this
> has been long thought of and desired.  Because of the current design
of
> executing a script from the upgrade tarball itself, many changes can
be
> made in one release, but not all.  For instance, being able to use
HTTP as
> well as FTP, well, that's going into the next release, but it won't
help
> when upgrading to that release.  But eventually it will help.
>=20
> [LCS] This is the way system upgrade has been working since 2.1. A
change
> was just checked into the dev branch to allow the System W (BSD based)
> release to use HTTP. But remember, hen you run system upgrade it is
the
> older nfxsh without the new capability that is doing the download.
>=20
> > B)  Standardize to the WebUI and replace the need for an FTP server
> > all together.  We could specify that the tar ball and md5 file are
in
> > the same directory pick them both up and check them before install
> > commences, therefore ruling out a corrupt tar ball as a problem for
> > the installation. If we also had a new upgrade script we could have
> > the process look for this before commencing the installation as
well,
> > replace the current install script and then start the process again.
> > So the process would be, go to the WebUI, open the system upgrade
> > page, point it the directory containing the scripts and click on the
> > version to install, then sit back get a coffee/tea and watch the
> > status bar go straight up to 90% before waiting another 30 minutes
for
> > the process to complete (sorry this isn't MS is it!)
>=20
> Sounds good to me!  Let us know when you've got something to test!
And
> yes, that is on my list of faves but earliest would be post Cougar
> release, and no, it's not on any schedule.  Keep in mind that
implementing
> this is no small project, and I'm not even including the WebUI
development
> part.  Ease of use is not always easy to do ~:^)
>=20
> Cheers,
>=20
> a
