X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C8840F.4324E774@onstor-exch02.onstor.net>; Wed, 12 Mar 2008 00:04:06 -0700
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C8840F.4324E774"
Content-class: urn:content-classes:message
Subject: RE: testing status of new branch
Date: Wed, 12 Mar 2008 00:04:06 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E08D2A2B5@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E08D2A2B3@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: testing status of new branch
Thread-Index: AciEDqz17SM225xWRL2fVoVc9Iq8HQAABKXg
From: "Tim Gardner" <tim.gardner@onstor.com>
To: "Jonathan Goldick" <jonathan.goldick@onstor.com>,
	"dl-Cougar" <dl-Cougar@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C8840F.4324E774
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Plan for tomorrow (today actually):

Bill and I will work on the first issue. This is a blocker. No
integration of fb-jong-perf2 into dev and thus no submittal until this
has been fixed.

Mike will continue to work the dump crash problem. May need some help
from Max on instrumenting edesc double free.
This issue is not a blocker for the submittal.

Would like Jobi/Amit to work issue 3. Since there is a workaround, this
is not a blocker for the submittal.

Hope to start the build for the submittal by mid afternoon.

Tim


> _____________________________________________=20
> From: 	Jonathan Goldick =20
> Sent:	Tuesday, March 11, 2008 11:00 PM
> To:	dl-Cougar
> Subject:	testing status of new branch
>=20
> Here are the problems we are hitting and the progress on resolution:
> 1.	TED00022751 SCSI Timers are not SMP safe - Bill has coded up a
> change to make the Cougar scsi timer code get locks.  Tim will review
> it and get it into the branch.  This has been causing dump/restore to
> crash about 25% of the time.
> 2.	We had a crash in dump which looks like a double free of an
> e-descriptor in the tape write path.  The root cause remains to be
> identified.  Mike Lee fixed a similar about a month ago.
> 3.	There is a significant slowdown that happens under a combined
> restore load, core dump copy to mgmt volume, and a dd write.  There is
> little dirty data but we are spending huge times waiting for file
> system locks.  Basically there is some throttle that I have not found
> in my testing.  In the interim I have checked in a change that
> defaults the I/O coalescing for log writes to the 'dev' branch
> behavior.  When Tim and I turned the knob to this on a machine that
> was crawling, it started running fast again.  I will continue to work
> this issue in my fb-jong-perf2 branch after the submittal is done, a
> QA resource will be needed to reproduce the problem and I could use
> Jobi and/or Amit's help to find the cause of the slowdown.
>=20
> I think that only number 1 above is blocking the submittal at this
> point.
>=20

------_=_NextPart_001_01C8840F.4324E774
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7653.38">
<TITLE>RE: testing status of new branch</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P><FONT COLOR=3D"#0000FF" SIZE=3D2 FACE=3D"Arial">Plan for tomorrow =
(today actually):</FONT>
</P>

<P><FONT COLOR=3D"#0000FF" SIZE=3D2 FACE=3D"Arial">Bill and I will work =
on the first issue. This is a blocker. No integration of fb-jong-perf2 =
into dev and thus no submittal until this has been fixed.</FONT></P>

<P><FONT COLOR=3D"#0000FF" SIZE=3D2 FACE=3D"Arial">Mike will continue to =
work the dump crash problem. May need some help from Max on =
instrumenting edesc double free.</FONT>

<BR><FONT COLOR=3D"#0000FF" SIZE=3D2 FACE=3D"Arial">This issue is not a =
blocker for the submittal.</FONT>
</P>

<P><FONT COLOR=3D"#0000FF" SIZE=3D2 FACE=3D"Arial">Would like Jobi/Amit =
to work issue 3. Since there is a workaround, this is not a blocker for =
the submittal.</FONT>
</P>

<P><FONT COLOR=3D"#0000FF" SIZE=3D2 FACE=3D"Arial">Hope to start the =
build for the submittal by mid afternoon.</FONT>
</P>

<P><FONT COLOR=3D"#0000FF" SIZE=3D2 FACE=3D"Arial">Tim</FONT>
</P>
<BR>
<UL>
<P><FONT SIZE=3D1 =
FACE=3D"Tahoma">_____________________________________________ </FONT>

<BR><B><FONT SIZE=3D1 FACE=3D"Tahoma">From: &nbsp;</FONT></B> <FONT =
SIZE=3D1 FACE=3D"Tahoma">Jonathan Goldick&nbsp; </FONT>

<BR><B><FONT SIZE=3D1 FACE=3D"Tahoma">Sent:&nbsp;&nbsp;</FONT></B> <FONT =
SIZE=3D1 FACE=3D"Tahoma">Tuesday, March 11, 2008 11:00 PM</FONT>

<BR><B><FONT SIZE=3D1 =
FACE=3D"Tahoma">To:&nbsp;&nbsp;&nbsp;&nbsp;</FONT></B> <FONT SIZE=3D1 =
FACE=3D"Tahoma">dl-Cougar</FONT>

<BR><B><FONT SIZE=3D1 =
FACE=3D"Tahoma">Subject:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</FONT>=
</B> <FONT SIZE=3D1 FACE=3D"Tahoma">testing status of new branch</FONT>
</P>

<P><FONT SIZE=3D2 FACE=3D"Arial">Here are the problems we are hitting =
and the progress on resolution:</FONT>
<UL>
<OL TYPE=3D1>
<LI><FONT SIZE=3D2 FACE=3D"Courier New">TED00022751 SCSI Timers are not =
SMP safe &#8211; Bill has coded up a change to make the Cougar scsi =
timer code get locks.&nbsp; Tim will review it and get it into the =
branch.&nbsp; This has been causing dump/restore to crash about 25% of =
the time.</FONT></LI>

<LI><FONT SIZE=3D2 FACE=3D"Arial">We had a crash in dump which looks =
like a double free of an e-descriptor in the tape write path.&nbsp; The =
root cause remains to be identified.&nbsp; Mike Lee fixed a similar =
about a month ago.</FONT></LI>

<LI><FONT SIZE=3D2 FACE=3D"Arial">There is a significant slowdown that =
happens under a combined restore load, core dump copy to mgmt volume, =
and a dd write.&nbsp; There is little dirty data but we are spending =
huge times waiting for file system locks.&nbsp; Basically there is some =
throttle that I have not found in my testing.&nbsp; In the interim I =
have checked in a change that defaults the I/O coalescing for log writes =
to the &#8216;dev&#8217; branch behavior.&nbsp; When Tim and I turned =
the knob to this on a machine that was crawling, it started running fast =
again.&nbsp; I will continue to work this issue in my fb-jong-perf2 =
branch after the submittal is done, a QA resource will be needed to =
reproduce the problem and I could use Jobi and/or Amit&#8217;s help to =
find the cause of the slowdown.</FONT></LI>
<BR>
</OL></UL>
<P><FONT SIZE=3D2 FACE=3D"Arial">I think that only number 1 above is =
blocking the submittal at this point.</FONT>
</P>
</UL>
</BODY>
</HTML>
------_=_NextPart_001_01C8840F.4324E774--
