X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C7AA63.964965D9@onstor-exch02.onstor.net>; Fri, 8 Jun 2007 23:58:30 -0700
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C7AA63.964965D9"
Content-class: urn:content-classes:message
Subject: Delorean status (06/07)
Date: Fri, 8 Jun 2007 23:58:30 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E02F14A58@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: Delorean status (06/07)
Thread-Index: AceqXHTje5M9QAYoTvGf6I+z6B7fCw==
From: "Raj Kumar" <raj.kumar@onstor.com>
To: "dl-Delorean" <dl-Delorean@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C7AA63.964965D9
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

1. Defects:
=3D=3D=3D=3D=3D=3D=3D
Closed: 10
Opened: 13
Backlog: 3

2. Issues / Concerns:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
1. S-Soak, Issue 1: Volumes in one of the node (SF2220) are down due to =
the LUNS missing at the SCSI layer. These luns are missing only from =
these node but are visible from other nodes (PR# 19485)
2. S-Soak, Issue 2: Later this evening the cluster went down and all 4 =
nodes were not operational, PR#19497 . There were several cores on the 3 =
nodes (cluster_contlr, spm, ncmd). Had to reboot all the nodes to get =
the cluster working.=20
3 . 40 TB: Looks like we have hit the practical limit of 33 TB with =
Delorean. After 33 TB we ran into several issues (Super low performance =
- 19465, Continuous FP crash - 19472). When we reached 32 TB, we added =
some additional tests (nested quotas, auditing) on this volume based on =
the request from marketing and at this point its not clear whether these =
failures are due to those tests. Plan is to disable those features, =
disable snapshots, delete snapshots, EEK the volume and start the data =
generation.

3. Tasks:
=3D=3D=3D=3D=3D=3D
(\\10.0.0.222\qe\Projects\Delorean\TestPlans\SubmittalPlans\Submittal_21_=
FCS_Tasks.htm =
<file://10.0.0.222/qe/Projects/Delorean/TestPlans/SubmittalPlans/Submitta=
l_21_FCS_Tasks.htm =
<file://10.0.0.222/qe/Projects/Delorean/TestPlans/SubmittalPlans/Submitta=
l_21_FCS_Tasks.htm> > )

1. Standalone upgrade - Ran tests on the 4 setups running 1.2.8.11, =
1.3.3.17, 2.1.0.4 and 2.2.2.7. Out of 7 defects critical defect is =
PR#19489 the most critical one where it rebooted by itself to the same =
flash before the install is complete, this may be a BSD crash but not =
sure at this point. We are trying to reproduce this problem again.
We also ran into a clusterDB upgrade problem for which we already have a =
defect (PR# 19239) which is already a P1 for Delorean.
=20
2. 1.5 TB mirror (create /promote) - Mirror for a promoted-mirror =
volume, took 12 hours to complete.

3  FS, Log replay - Done

4. FS Trashcan regression - Data set  is ready, need to complete the =
tests by monday

5. NDMP remote backups of 500K, 1 million directories - Backups are in =
progress, results will be available in the next status

4. S-Soak:
=3D=3D=3D=3D=3D=3D=3D

-------------------------------------------------------------------------=
-------------------------
| VS             | Speed (Ops/sec)                   | Throughput =
(Bytes/sec)                    |
-------------------------------------------------------------------------=
-------------------------
|                |    NFS |   CIFS |    NFS |   CIFS |      NFS |     =
CIFS |      NFS |     CIFS |
|                |   (IN) |   (IN) |  (OUT) |  (OUT) |     (IN) |     =
(IN) |    (OUT) |    (OUT) |
-------------------------------------------------------------------------=
-------------------------
| -N/A-          |   1138 |  13618 |   1146 |  13618 |   16.88M |  =
803.47K |  304.32K |  857.94K |
| -N/A-          |   2138 |  10422 |   2137 |  10422 |   35.26M |  =
614.95K |  470.20K |  656.64K |
| -N/A-          |   2110 |  13064 |   2111 |  13064 |   37.32M |  =
770.80K |  442.32K |  823.06K |
| -N/A-          |   1403 |  11221 |   1399 |  11221 |   27.03M |  =
662.04K |  356.76K |  706.93K |
| -N/A-          |   1927 |  11013 |   1930 |  11013 |   43.46M |  =
649.81K |  448.37K |  693.86K |
| -N/A-          |   4038 |  17285 |   4038 |  17285 |   70.80M |    =
1.02M |  689.50K |    1.09M |
| -N/A-          |   3299 |  23281 |   3298 |  23281 |   64.43M |    =
1.37M |  496.20K |    1.47M |
| -N/A-          |   3168 |  23276 |   3174 |  23276 |   49.85M |    =
1.37M |  466.24K |    1.47M |
| -N/A-          |   4117 |  23886 |   4116 |  23886 |   73.53M |    =
1.41M |  610.87K |    1.50M |


Thanks.

--kumar :)


------_=_NextPart_001_01C7AA63.964965D9
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<HTML dir=3Dltr><HEAD>=0A=
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Dunicode">=0A=
<META content=3D"MSHTML 6.00.2800.1555" name=3DGENERATOR></HEAD>=0A=
<BODY>=0A=
<DIV><FONT face=3DArial color=3D#000000 size=3D2><FONT face=3D"Times New =
Roman">1. Defects:<BR>=3D=3D=3D=3D=3D=3D=3D<BR>Closed: 10<BR>Opened: =
13<BR>Backlog: 3<BR><BR>2. Issues / =
Concerns:<BR>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<BR>1. S-Soak, =
Issue 1: Volumes in one of the node (SF2220) are down due to the LUNS =
missing at the SCSI layer. These luns are missing only from these node =
but are visible from other nodes (PR# 19485)</FONT></FONT></DIV>=0A=
<DIV><FONT color=3D#ff0000 size=3D2>2. S-Soak, Issue 2: Later this =
evening the cluster went down and all 4 nodes were not operational, =
PR#19497 . There were several cores on the 3 nodes (cluster_contlr, spm, =
ncmd). Had to reboot all the nodes to get the cluster working. =
</FONT></DIV>=0A=
<DIV><FONT face=3DArial color=3D#000000><FONT face=3D"Times New =
Roman"><FONT size=3D2>3 . 40 TB: Looks like we have hit the practical =
limit of 33 TB with Delorean. After 33 TB we ran into several issues =
(Super low performance - 19465, Continuous FP crash -&nbsp;19472). When =
we reached 32 TB, we added some additional tests (nested quotas, =
auditing) on this volume based on the request from marketing and at this =
point its not clear whether these failures are due to those tests. Plan =
is to disable those features, disable snapshots, delete snapshots, EEK =
the volume and start the data =
generation.</FONT></FONT></FONT></DIV><FONT face=3DArial =
color=3D#000000><FONT face=3D"Times New Roman"><FONT size=3D2>=0A=
<DIV><BR>3. =
Tasks:<BR>=3D=3D=3D=3D=3D=3D<BR>(\\10.0.0.222\qe\Projects\Delorean\TestPl=
ans\SubmittalPlans\Submittal_21_FCS_Tasks.htm &lt;</FONT></FONT><A =
href=3D"file://10.0.0.222/qe/Projects/Delorean/TestPlans/SubmittalPlans/S=
ubmittal_21_FCS_Tasks.htm" target=3D_blank><FONT face=3D"Times New =
Roman" =
size=3D2>file://10.0.0.222/qe/Projects/Delorean/TestPlans/SubmittalPlans/=
Submittal_21_FCS_Tasks.htm</FONT></A><FONT size=3D2><FONT face=3D"Times =
New Roman">&gt; )<BR><BR>1. Standalone upgrade - Ran tests on the 4 =
setups running 1.2.8.11, 1.3.3.17, 2.1.0.4 and 2.2.2.7. Out of 7 defects =
<FONT color=3D#ff0000>critical defect is PR#19489 the most critical one =
where it rebooted by itself to the same flash before the install is =
complete, this may be a BSD crash but not sure at this point. We are =
trying to reproduce this problem again.</FONT></FONT></FONT></DIV>=0A=
<DIV><FONT face=3D"Times New Roman" size=3D2>We also ran into a =
clusterDB upgrade problem for which we already have a defect (PR# 19239) =
which is already&nbsp;a P1 for Delorean.</FONT></DIV>=0A=
<DIV><FONT size=3D2><FONT face=3D"Times New =
Roman"></FONT></FONT>&nbsp;</DIV>=0A=
<DIV><FONT size=3D2><FONT face=3D"Times New Roman">2. 1.5 TB mirror =
(create /promote) - Mirror for a promoted-mirror volume, took 12 hours =
to complete.<BR><BR>3 &nbsp;FS, Log replay - Done<BR><BR>4. FS Trashcan =
regression - Data set&nbsp; is ready, need to complete the tests by =
monday<BR><BR>5. NDMP remote backups of 500K, 1 million directories - =
Backups are in progress, results will be available in the next =
status</FONT></FONT></DIV>=0A=
<P><FONT size=3D2><FONT face=3D"Times New Roman"></FONT></FONT><FONT =
size=3D2><FONT face=3D"Times New Roman">4. =
S-Soak:<BR>=3D=3D=3D=3D=3D=3D=3D</FONT></FONT></P>=0A=
<P><FONT size=3D2><FONT face=3D"Times New =
Roman">------------------------------------------------------------------=
--------------------------------<BR>| =
VS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
; | Speed =
(Ops/sec)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | Throughput =
(Bytes/sec)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|<BR>--------------------------------------------------------------------=
------------------------------<BR>|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; =
NFS |&nbsp;&nbsp; CIFS |&nbsp;&nbsp;&nbsp; NFS |&nbsp;&nbsp; CIFS =
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; NFS |&nbsp;&nbsp;&nbsp;&nbsp; CIFS =
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; NFS |&nbsp;&nbsp;&nbsp;&nbsp; CIFS =
|<BR>|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; (IN) |&nbsp;&nbsp; (IN) |&nbsp; =
(OUT) |&nbsp; (OUT) |&nbsp;&nbsp;&nbsp;&nbsp; (IN) =
|&nbsp;&nbsp;&nbsp;&nbsp; (IN) |&nbsp;&nbsp;&nbsp; (OUT) =
|&nbsp;&nbsp;&nbsp; (OUT) =
|<BR>--------------------------------------------------------------------=
------------------------------<BR>| =
-N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 1138 |&nbsp; 13618 |&nbsp;&nbsp; 1146 |&nbsp; 13618 =
|&nbsp;&nbsp; 16.88M |&nbsp; 803.47K |&nbsp; 304.32K |&nbsp; 857.94K =
|<BR>| -N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 2138 |&nbsp; 10422 |&nbsp;&nbsp; 2137 |&nbsp; 10422 =
|&nbsp;&nbsp; 35.26M |&nbsp; 614.95K |&nbsp; 470.20K |&nbsp; 656.64K =
|<BR>| -N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 2110 |&nbsp; 13064 |&nbsp;&nbsp; 2111 |&nbsp; 13064 =
|&nbsp;&nbsp; 37.32M |&nbsp; 770.80K |&nbsp; 442.32K |&nbsp; 823.06K =
|<BR>| -N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 1403 |&nbsp; 11221 |&nbsp;&nbsp; 1399 |&nbsp; 11221 =
|&nbsp;&nbsp; 27.03M |&nbsp; 662.04K |&nbsp; 356.76K |&nbsp; 706.93K =
|<BR>| -N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 1927 |&nbsp; 11013 |&nbsp;&nbsp; 1930 |&nbsp; 11013 =
|&nbsp;&nbsp; 43.46M |&nbsp; 649.81K |&nbsp; 448.37K |&nbsp; 693.86K =
|<BR>| -N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 4038 |&nbsp; 17285 |&nbsp;&nbsp; 4038 |&nbsp; 17285 =
|&nbsp;&nbsp; 70.80M |&nbsp;&nbsp;&nbsp; 1.02M |&nbsp; 689.50K =
|&nbsp;&nbsp;&nbsp; 1.09M |<BR>| =
-N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 3299 |&nbsp; 23281 |&nbsp;&nbsp; 3298 |&nbsp; 23281 =
|&nbsp;&nbsp; 64.43M |&nbsp;&nbsp;&nbsp; 1.37M |&nbsp; 496.20K =
|&nbsp;&nbsp;&nbsp; 1.47M |<BR>| =
-N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 3168 |&nbsp; 23276 |&nbsp;&nbsp; 3174 |&nbsp; 23276 =
|&nbsp;&nbsp; 49.85M |&nbsp;&nbsp;&nbsp; 1.37M |&nbsp; 466.24K =
|&nbsp;&nbsp;&nbsp; 1.47M |<BR>| =
-N/A-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
|&nbsp;&nbsp; 4117 |&nbsp; 23886 |&nbsp;&nbsp; 4116 |&nbsp; 23886 =
|&nbsp;&nbsp; 73.53M |&nbsp;&nbsp;&nbsp; 1.41M |&nbsp; 610.87K =
|&nbsp;&nbsp;&nbsp; 1.50M |</FONT></FONT></P>=0A=
<P><FONT size=3D2><FONT face=3D"Times New =
Roman"><BR>Thanks.</FONT><BR><BR>--kumar =
:)</P></FONT></FONT></BODY></HTML>
------_=_NextPart_001_01C7AA63.964965D9--
