X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C7A92C.DB152373@onstor-exch02.onstor.net>; Thu, 7 Jun 2007 10:54:12 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-class: urn:content-classes:message
Subject: Notes from our discussion of What We Learned from Delorean
Date: Thu, 7 Jun 2007 10:54:11 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E040F0914@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: Notes from our discussion of What We Learned from Delorean
Thread-Index: AcepLNrdRLcGgv5VSMi+YT2GvqxpTg==
From: "Jay Michlin" <jay.michlin@onstor.com>
To: "dl-Software" <dl-software@onstor.com>

Hello all,

Here is a first draft of my notes from yesterday's discussion. I'd
appreciate any comments, correcions or additions.

Thanks.
jay

_._._._._._._._._._

Delorean Post-Mortem
What Have We Learned from the Delorean Experience?
Software Development
June 6, 2007


*	We should improve our internal software development quality.
-	Do more development testing.
-	Be tougher in code reviews
-	Don't allow schedule compression to compromise development
quality
*	Raise code quality before we provide it to QA.
*	Teamwork with QA on defects must improve
-	Joint planning of tests
-	Stop the game of "email tennis" on defects=20
-	Work to make sure we share a common goal
*	Ramping up testing near the end of the project was destructive
-	In general, testing on this project was unusually back-end
loaded
-	Especially ill conceived with so much new functionality
-	This alone assures project slips and loss of definitive project
control since serious defects surface when the schedule has no remaining
flexibility.
*	Some of the new testing regimes were "debug unfriendly"
-	That is, they found problems in such a way that addressing them
was difficult or impossible
-	Defects were often filed in the form, "The product failed, but
we don't know where, why or how"
-	The testing regime produced a large number of "can't reproduce"
defects
-	The testing regime had too many variables to handle productively
in a project context. Perhaps TTE testing ought to be organized as a
standalone project.
*	Delays in the lab delayed testing of complex new functionality.
*	Defect triage set too low a threshold for Must Fix
-	In some cases fixing defects labeled Must Fix caused regressions
worse than the defects themselves
-	SW Developers repeatedly raised this with QA and were repeatedly
rejected or subjected to long volleys of "email tennis"=20
-	This should be a factor in code reviews; we should ask, "Does
including this defect fix in the project generate more risk than it's
worth?"=20
*	We should not change "core things" at the end of a project
*	Supersoak's ongoing failure and lack of reliable operation for
much of the project deprived us of the opportunity to identify and fix
defects early
-	Then it generated a large cascade of defects near the end of the
project
-	And many of the defects were questionable because of Supersoak
lab problems
