X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C73064.0D314122@onstor-exch02.onstor.net>; Thu, 4 Jan 2007 16:54:27 -0800
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C73064.0D314122"
Content-class: urn:content-classes:message
Subject: RE: Interesting file corruption pattern
Date: Thu, 4 Jan 2007 16:54:27 -0800
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E0A9451@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E0A9450@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: Interesting file corruption pattern
thread-index: AccwYEeMc05KTNPxSJGElQw9iLsblgAAzmnw
From: "Larry Scheer" <larry.scheer@onstor.com>
To: "Larry Scheer" <larry.scheer@onstor.com>,
	"Andy Sharp" <andy.sharp@onstor.com>,
	"Tim Gardner" <tim.gardner@onstor.com>
Cc: "Jay Michlin" <jay.michlin@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C73064.0D314122
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I have just verified (somewhat) that the much of the text corruption is
coming from the environment variables set in the shell such TERMCAP and
others that I have exported in my test scripts.

So the rhetorical question is how is data from the environment variables
getting into the file when it is being installed?

Larry

_____________________________________________
From: Larry Scheer=20
Sent: Thursday, January 04, 2007 4:27 PM
To: Andy Sharp; Tim Gardner
Cc: Jay Michlin
Subject: Interesting file corruption pattern

I changed the upgrade program to report file differences after the
upgrade complete using upgrade's own built in verification code.

I ran one pass of my upgrade test that installs 2 different builds to
the secondary flash one after another.

When the second upgrade command was run 23 files were reported as being
different from the version on the source tar file. An examination of all
23 files shown they were corrupt. The files were a combination of files
from both BSD and our applications.

Here are some interesting patterns of  the corruption:

Good source (/usr/local/agile/bin/ tape-copy):
< 000029e0  60 80 82 8f 00 00 00 00  00 00 42 8c 00 00 00 00
|`.........B.....|
< 000029f0  0d 00 40 10 00 00 00 00  1c 80 84 8f 00 00 00 00
|..@.............|

Corrupt target (/usr/local/agile/bin/ tape-copy):
> 000029e0  61 6c 2f 61 67 69 6c 65  2f 62 69 6e 2f 74 61 70
|al/agile/bin/tap|
> 000029f0  65 2d 63 6f 70 79 20 2f  6d 6e 74 31 2f 75 73 72  |e-copy
/mnt1/usr|

Note the nature of the corruption is a partial path name of the source
file and its location on the mfs file system where the source was
stored.

Good source (/usr/local/agile/bin/ tape-copy):
< 00002b60  1c 16 84 24 00 00 00 00  20 00 c5 8f 00 00 00 00  |...$....
.......|
< 00002b70  58 80 99 8f 00 00 00 00  09 f8 20 03 00 00 00 00
|X......... .....|
< 00002b80  18 00 dc 8f 00 00 00 00  20 00 c4 8f 00 00 00 00  |........
.......|
< 00002b90  7c 80 99 8f 00 00 00 00  09 f8 20 03 00 00 00 00
||......... .....|
< 00002ba0  18 00 dc 8f 00 00 00 00  24 00 c2 af 24 00 c2 8f
|........$...$...|
< 00002bb0  00 00 00 00 ff ff 03 24  1a 00 43 14 00 00 00 00
|.......$..C.....|

Corrupt target (/usr/local/agile/bin/ tape-copy):
> 00002b60  4f 4b 5f 50 52 49 4e 43  44 4f 4d 41 49 4e 3d 00
|OK_PRINCDOMAIN=3D.|
> 00002b70  41 55 54 48 47 45 4e 54  4f 4b 5f 50 52 49 4e 43
|AUTHGENTOK_PRINC|
> 00002b80  54 59 50 45 3d 4c 4f 43  41 4c 46 49 4c 45 52 00
|TYPE=3DLOCALFILER.|
> 00002b90  54 45 52 4d 43 41 50 3d  78 74 65 72 6d 3a 61 6d
|TERMCAP=3Dxterm:am|
> 00002ba0  3a 62 73 3a 6b 6d 3a 6d  69 3a 6d 73 3a 78 6e 3a
|:bs:km:mi:ms:xn:|
> 00002bb0  63 6f 23 38 30 3a 69 74  23 38 3a 6c 69 23 32 34
|co#80:it#8:li#24|

Part of this corruption looks like it comes from the file:
/usr/share/misc/termcap.

I have collected the hex dumps and differences between the source and
corrupted files and I am see this pattern repeated in many of the files.
By pattern I mean the source file paths and sections from the termcap
file. The data is not identical in the files.

Most of the time the corruption is zeros where other data should be. For
example: (/usr/local/agile/bin/evm_cfgd)

< 000419e0  18 00 c2 8f 00 00 00 00  ff 03 03 3c ff ff 63 34
|...........<..c4|
< 000419f0  24 10 43 00 7f 00 43 30  0c 00 c3 af 0c 00 c2 8f
|$.C...C0........|
---
> 000419e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
|................|
> *

Larry



------_=_NextPart_001_01C73064.0D314122
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7650.28">
<TITLE>RE: Interesting file corruption pattern</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">I have just verified (somewhat) that the much of the text =
corruption is coming from the environment variables set in the shell =
such TERMCAP and others that I have exported in my test =
scripts.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">So the</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">rhetorical</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial"></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">question =
is how is data from the environment variables getting into the file when =
it is being installed?</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Larry</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Larry Scheer<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Thursday, January 04, =
2007 4:27 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Andy Sharp; Tim =
Gardner<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Cc:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jay Michlin<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> Interesting file corruption pattern</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">I =
changed the upgrade program to report file differences after the upgrade =
complete using upgrade&#8217;s own built in verification =
code.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">I ran =
one pass of my upgrade test that installs 2 different builds to the =
secondary flash one after another.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">When =
the second upgrade command was run 23 files were reported as being =
different from the version on the source tar file. An examination of all =
23 files shown they were corrupt. The files were a combination of files =
from both BSD and our applications.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Here =
are some interesting patterns of&nbsp; the corruption:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Good =
source (/usr/local/agile/bin/</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT SIZE=3D2 =
FACE=3D"Arial">tape-copy):</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
000029e0&nbsp; 60 80 82 8f 00 00 00 00&nbsp; 00 00 42 8c 00 00 00 =
00&nbsp; |`.........B.....|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
000029f0&nbsp; 0d 00 40 10 00 00 00 00&nbsp; 1c 80 84 8f 00 00 00 =
00&nbsp; |..@.............|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Corrupt target (/usr/local/agile/bin/</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT SIZE=3D2 =
FACE=3D"Arial">tape-copy):</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
000029e0&nbsp; 61 6c 2f 61 67 69 6c 65&nbsp; 2f 62 69 6e 2f 74 61 =
70&nbsp; |al/agile/bin/tap|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
000029f0&nbsp; 65 2d 63 6f 70 79 20 2f&nbsp; 6d 6e 74 31 2f 75 73 =
72&nbsp; |e-copy /mnt1/usr|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Note =
the nature of the corruption is a partial path name of the source file =
and its location on the mfs file system where the source was =
stored.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Good =
source (/usr/local/agile/bin/</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT SIZE=3D2 =
FACE=3D"Arial">tape-copy):</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
00002b60&nbsp; 1c 16 84 24 00 00 00 00&nbsp; 20 00 c5 8f 00 00 00 =
00&nbsp; |...$.... .......|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
00002b70&nbsp; 58 80 99 8f 00 00 00 00&nbsp; 09 f8 20 03 00 00 00 =
00&nbsp; |X......... .....|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
00002b80&nbsp; 18 00 dc 8f 00 00 00 00&nbsp; 20 00 c4 8f 00 00 00 =
00&nbsp; |........ .......|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
00002b90&nbsp; 7c 80 99 8f 00 00 00 00&nbsp; 09 f8 20 03 00 00 00 =
00&nbsp; ||......... .....|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
00002ba0&nbsp; 18 00 dc 8f 00 00 00 00&nbsp; 24 00 c2 af 24 00 c2 =
8f&nbsp; |........$...$...|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
00002bb0&nbsp; 00 00 00 00 ff ff 03 24&nbsp; 1a 00 43 14 00 00 00 =
00&nbsp; |.......$..C.....|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Corrupt target (/usr/local/agile/bin/</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT SIZE=3D2 =
FACE=3D"Arial">tape-copy):</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
00002b60&nbsp; 4f 4b 5f 50 52 49 4e 43&nbsp; 44 4f 4d 41 49 4e 3d =
00&nbsp; |OK_PRINCDOMAIN=3D.|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
00002b70&nbsp; 41 55 54 48 47 45 4e 54&nbsp; 4f 4b 5f 50 52 49 4e =
43&nbsp; |AUTHGENTOK_PRINC|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
00002b80&nbsp; 54 59 50 45 3d 4c 4f 43&nbsp; 41 4c 46 49 4c 45 52 =
00&nbsp; |TYPE=3DLOCALFILER.|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
00002b90&nbsp; 54 45 52 4d 43 41 50 3d&nbsp; 78 74 65 72 6d 3a 61 =
6d&nbsp; |TERMCAP=3Dxterm:am|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
00002ba0&nbsp; 3a 62 73 3a 6b 6d 3a 6d&nbsp; 69 3a 6d 73 3a 78 6e =
3a&nbsp; |:bs:km:mi:ms:xn:|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
00002bb0&nbsp; 63 6f 23 38 30 3a 69 74&nbsp; 23 38 3a 6c 69 23 32 =
34&nbsp; |co#80:it#8:li#24|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Part =
of this corruption looks like it comes from the file: =
/usr/share/misc/termcap.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">I =
have collected the hex dumps and differences between the source and =
corrupted files and I am see this pattern repeated in many of the files. =
By pattern I mean the source file paths and sections from the termcap =
file. The data is not identical in the files.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Most =
of the time the corruption is zeros where other data should be. For =
example: (/usr/local/agile/bin/evm_cfgd)</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
000419e0&nbsp; 18 00 c2 8f 00 00 00 00&nbsp; ff 03 03 3c ff ff 63 =
34&nbsp; |...........&lt;..c4|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&lt; =
000419f0&nbsp; 24 10 43 00 7f 00 43 30&nbsp; 0c 00 c3 af 0c 00 c2 =
8f&nbsp; |$.C...C0........|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">---</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
000419e0&nbsp; 00 00 00 00 00 00 00 00&nbsp; 00 00 00 00 00 00 00 =
00&nbsp; |................|</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&gt; =
*</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Larry</FONT></SPAN></P>
<BR>

</BODY>
</HTML>
------_=_NextPart_001_01C73064.0D314122--
