X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C7E5C9.63A32F4B@onstor-exch02.onstor.net>; Thu, 23 Aug 2007 13:05:52 -0800
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C7E5C9.63A32F4B"
Content-class: urn:content-classes:message
Subject: Nightly Build Info and Failure Notifications
Date: Thu, 23 Aug 2007 13:05:52 -0800
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E02F3D6B5@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: Nightly Build Info and Failure Notifications
Thread-Index: AcflxtCeZdv7jA2rTu2EB65w8kWHeQ==
From: "Ken Renshaw" <ken.renshaw@onstor.com>
To: "dl-Cougar" <dl-Cougar@onstor.com>,
	"Brian DeForest" <brian.deforest@onstor.com>,
	"Paul Hammer" <paul.hammer@onstor.com>,
	"Sandrine Boulanger" <sandrine.boulanger@onstor.com>
Cc: "Ken Renshaw" <ken.renshaw@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C7E5C9.63A32F4B
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi all. I'm working on revamping all automated build services, and while =
I'm not quite finished with the whole enchilda I'd at least like to =
serve some nachos to keep your stomachs from grumbling too much while =
you're waiting.

The nightly build runs at 3 AM every night, and compiles all targets and =
variants for bobcat, cheetah, cougar, and bobcat-linux. All runs are =
logged and all logs available here ( sorry for the dumb front-end for =
browsing, I'm working on something else ):

http://k2 or http://10.0.0.123

From here you click on the branch, e.g. dev, then on the day of the week =
bearing in mind the runs are done at 3 AM. You'll get a list of logs for =
every run like this, e.g. from last night's run:

[build@k2 build]$ ls -lrt /perforce/buildlogs/dev/Thursday
total 26700
-rw-rw-r--    1 build    build          31 Aug 23 03:00 =
opt_image_filename
-rw-rw-r--    1 build    build          49 Aug 23 03:00 location
-rw-rw-r--    1 build    build           0 Aug 23 03:00 FULL_BUILD
-rw-rw-r--    1 build    build        1158 Aug 23 03:00 env
-rw-rw-r--    1 build    build          34 Aug 23 03:00 =
dbg_image_filename
-rw-rw-r--    1 build    build           6 Aug 23 03:00 =
current_changelist
-rw-rw-r--    1 build    build     3587112 Aug 23 03:27 =
ch-opt-compile.log
-rw-rw-r--    1 build    build      388861 Aug 23 03:30 =
ch-opt-release.log
-rw-rw-r--    1 build    build     4209093 Aug 23 03:55 =
ch-dbg-compile.log
-rw-rw-r--    1 build    build      385438 Aug 23 03:57 =
ch-dbg-release.log
-rw-rw-r--    1 build    build     3406540 Aug 23 04:25 =
bc-opt-compile.log
-rw-rw-r--    1 build    build      386408 Aug 23 04:28 =
bc-opt-release.log
-rw-rw-r--    1 build    build     4028938 Aug 23 04:53 =
bc-dbg-compile.log
-rw-rw-r--    1 build    build      384063 Aug 23 04:56 =
bc-dbg-release.log
-rw-rw-r--    1 build    build     2731453 Aug 23 05:05 =
cg-dbg-compile.log
-rw-rw-r--    1 build    build     2113954 Aug 23 05:16 =
cg-opt-compile.log
-rw-rw-r--    1 build    build     3110215 Aug 23 05:25 =
bl-dbg-compile.log
-rw-rw-r--    1 build    build     2493578 Aug 23 05:36 =
bl-opt-compile.log
-rw-rw-r--    1 build    build           0 Aug 23 05:36 =
FATAL_COMPILATION_BUILD_ABORTING
-rw-rw-r--    1 build    build        1708 Aug 23 05:36 err.out
-rw-rw-r--    1 build    build         234 Aug 23 05:36 =
ch-warnings-opt.txt
-rw-rw-r--    1 build    build         234 Aug 23 05:36 =
ch-warnings-dbg.txt
-rw-rw-r--    1 build    build         234 Aug 23 05:36 =
bc-warnings-opt.txt
-rw-rw-r--    1 build    build         234 Aug 23 05:36 =
bc-warnings-dbg.txt
-rw-rw-r--    1 build    build        2917 Aug 23 05:36 auditlog

There's lots of stuff there, but the auditlog file contains the =
top-level build info. Had the build passed it would have went on to =
upgrade a cluster and start build verification regressions. the =
*-warnings-*.txt files contain reports of all non-fatal compilation =
warnings as we've decided we can't tolerate them and they should be =
fixed.

The *-compile.log files are the result of running the 'make all' target =
for a particular variant and prod combination, and the *-release.log =
files are the result of running 'make release' for the same set of =
targets. This is currently unimplemented on the linux-based builds.

The FATAL_COMPILATION_BUILD_ABORTING is just a semaphore that things =
exploded, and the err.out file contains short context on what the matter =
might be, e.g. in last night's case:

/perforce/buildlogs/dev/Thursday/bl-dbg-compile.log:make[3]: *** =
[../../Build/bl/dbg/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 1
/perforce/buildlogs/dev/Thursday/bl-dbg-compile.log:make[2]: *** =
[../../Build/bl/dbg/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 2
/perforce/buildlogs/dev/Thursday/bl-dbg-compile.log:make[1]: *** =
[default] Error 1
/perforce/buildlogs/dev/Thursday/bl-dbg-compile.log:make: *** [ssc] =
Error 2
/perforce/buildlogs/dev/Thursday/bl-opt-compile.log:make[3]: *** =
[../../Build/bl/opt/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 1
/perforce/buildlogs/dev/Thursday/bl-opt-compile.log:make[2]: *** =
[../../Build/bl/opt/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 2
/perforce/buildlogs/dev/Thursday/bl-opt-compile.log:make[1]: *** =
[default] Error 1
/perforce/buildlogs/dev/Thursday/bl-opt-compile.log:make: *** [ssc] =
Error 2
/perforce/buildlogs/dev/Thursday/cg-dbg-compile.log:make[3]: *** =
[../../Build/cg/dbg/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 1
/perforce/buildlogs/dev/Thursday/cg-dbg-compile.log:make[2]: *** =
[../../Build/cg/dbg/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 2
/perforce/buildlogs/dev/Thursday/cg-dbg-compile.log:make[1]: *** =
[default] Error 1
/perforce/buildlogs/dev/Thursday/cg-dbg-compile.log:make: *** [ssc] =
Error 2
/perforce/buildlogs/dev/Thursday/cg-opt-compile.log:make[3]: *** =
[../../Build/cg/opt/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 1
/perforce/buildlogs/dev/Thursday/cg-opt-compile.log:make[2]: *** =
[../../Build/cg/opt/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 2
/perforce/buildlogs/dev/Thursday/cg-opt-compile.log:make[1]: *** =
[default] Error 1
/perforce/buildlogs/dev/Thursday/cg-opt-compile.log:make: *** [ssc] =
Error 2

This breakage has been fixed today incidentally.

There are other things captured at the time of build like env settings.

Starting tonight any compilation errors will be sent to the group of =
people on this email. Please let me know if the list should change.

Things will be changing and growing much from here, OS build automation =
is going in this afternoon for instance, but at least we have something =
going since things are moving way too fast all of a sudden to continue =
without it. My apologies for not disentangling sooner this work from the =
other things I'm integrating into the system ( code coverage, coverity, =
test harness expansion, etc. ).

Thanks,

-Ken






------_=_NextPart_001_01C7E5C9.63A32F4B
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7652.24">
<TITLE>Nightly Build Info and Failure Notifications</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->

<P><FONT SIZE=3D2>Hi all. I'm working on revamping all automated build =
services, and while I'm not quite finished with the whole enchilda I'd =
at least like to serve some nachos to keep your stomachs from grumbling =
too much while you're waiting.<BR>
<BR>
The nightly build runs at 3 AM every night, and compiles all targets and =
variants for bobcat, cheetah, cougar, and bobcat-linux. All runs are =
logged and all logs available here ( sorry for the dumb front-end for =
browsing, I'm working on something else ):<BR>
<BR>
<A HREF=3D"http://k2">http://k2</A> or <A =
HREF=3D"http://10.0.0.123">http://10.0.0.123</A><BR>
<BR>
From here you click on the branch, e.g. dev, then on the day of the week =
bearing in mind the runs are done at 3 AM. You'll get a list of logs for =
every run like this, e.g. from last night's run:<BR>
<BR>
[build@k2 build]$ ls -lrt /perforce/buildlogs/dev/Thursday<BR>
total 26700<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 31 Aug 23 =
03:00 opt_image_filename<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 49 Aug 23 =
03:00 location<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0 Aug =
23 03:00 FULL_BUILD<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1158 Aug 23 03:00 =
env<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 34 Aug 23 =
03:00 dbg_image_filename<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6 Aug =
23 03:00 current_changelist<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp; 3587112 Aug 23 03:27 =
ch-opt-compile.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 388861 Aug 23 03:30 =
ch-opt-release.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp; 4209093 Aug 23 03:55 =
ch-dbg-compile.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 385438 Aug 23 03:57 =
ch-dbg-release.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp; 3406540 Aug 23 04:25 =
bc-opt-compile.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 386408 Aug 23 04:28 =
bc-opt-release.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp; 4028938 Aug 23 04:53 =
bc-dbg-compile.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 384063 Aug 23 04:56 =
bc-dbg-release.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp; 2731453 Aug 23 05:05 =
cg-dbg-compile.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp; 2113954 Aug 23 05:16 =
cg-opt-compile.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp; 3110215 Aug 23 05:25 =
bl-dbg-compile.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp; 2493578 Aug 23 05:36 =
bl-opt-compile.log<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0 Aug =
23 05:36 FATAL_COMPILATION_BUILD_ABORTING<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1708 Aug 23 05:36 =
err.out<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 234 Aug 23 05:36 =
ch-warnings-opt.txt<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 234 Aug 23 05:36 =
ch-warnings-dbg.txt<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 234 Aug 23 05:36 =
bc-warnings-opt.txt<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 234 Aug 23 05:36 =
bc-warnings-dbg.txt<BR>
-rw-rw-r--&nbsp;&nbsp;&nbsp; 1 build&nbsp;&nbsp;&nbsp; =
build&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2917 Aug 23 05:36 =
auditlog<BR>
<BR>
There's lots of stuff there, but the auditlog file contains the =
top-level build info. Had the build passed it would have went on to =
upgrade a cluster and start build verification regressions. the =
*-warnings-*.txt files contain reports of all non-fatal compilation =
warnings as we've decided we can't tolerate them and they should be =
fixed.<BR>
<BR>
The *-compile.log files are the result of running the 'make all' target =
for a particular variant and prod combination, and the *-release.log =
files are the result of running 'make release' for the same set of =
targets. This is currently unimplemented on the linux-based builds.<BR>
<BR>
The FATAL_COMPILATION_BUILD_ABORTING is just a semaphore that things =
exploded, and the err.out file contains short context on what the matter =
might be, e.g. in last night's case:<BR>
<BR>
/perforce/buildlogs/dev/Thursday/bl-dbg-compile.log:make[3]: *** =
[../../Build/bl/dbg/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 1<BR>
/perforce/buildlogs/dev/Thursday/bl-dbg-compile.log:make[2]: *** =
[../../Build/bl/dbg/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 2<BR>
/perforce/buildlogs/dev/Thursday/bl-dbg-compile.log:make[1]: *** =
[default] Error 1<BR>
/perforce/buildlogs/dev/Thursday/bl-dbg-compile.log:make: *** [ssc] =
Error 2<BR>
/perforce/buildlogs/dev/Thursday/bl-opt-compile.log:make[3]: *** =
[../../Build/bl/opt/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 1<BR>
/perforce/buildlogs/dev/Thursday/bl-opt-compile.log:make[2]: *** =
[../../Build/bl/opt/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 2<BR>
/perforce/buildlogs/dev/Thursday/bl-opt-compile.log:make[1]: *** =
[default] Error 1<BR>
/perforce/buildlogs/dev/Thursday/bl-opt-compile.log:make: *** [ssc] =
Error 2<BR>
/perforce/buildlogs/dev/Thursday/cg-dbg-compile.log:make[3]: *** =
[../../Build/cg/dbg/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 1<BR>
/perforce/buildlogs/dev/Thursday/cg-dbg-compile.log:make[2]: *** =
[../../Build/cg/dbg/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 2<BR>
/perforce/buildlogs/dev/Thursday/cg-dbg-compile.log:make[1]: *** =
[default] Error 1<BR>
/perforce/buildlogs/dev/Thursday/cg-dbg-compile.log:make: *** [ssc] =
Error 2<BR>
/perforce/buildlogs/dev/Thursday/cg-opt-compile.log:make[3]: *** =
[../../Build/cg/opt/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 1<BR>
/perforce/buildlogs/dev/Thursday/cg-opt-compile.log:make[2]: *** =
[../../Build/cg/opt/Objects/SSC/ssc-authen-lib/authen-lib.o] Error 2<BR>
/perforce/buildlogs/dev/Thursday/cg-opt-compile.log:make[1]: *** =
[default] Error 1<BR>
/perforce/buildlogs/dev/Thursday/cg-opt-compile.log:make: *** [ssc] =
Error 2<BR>
<BR>
This breakage has been fixed today incidentally.<BR>
<BR>
There are other things captured at the time of build like env =
settings.<BR>
<BR>
Starting tonight any compilation errors will be sent to the group of =
people on this email. Please let me know if the list should change.<BR>
<BR>
Things will be changing and growing much from here, OS build automation =
is going in this afternoon for instance, but at least we have something =
going since things are moving way too fast all of a sudden to continue =
without it. My apologies for not disentangling sooner this work from the =
other things I'm integrating into the system ( code coverage, coverity, =
test harness expansion, etc. ).<BR>
<BR>
Thanks,<BR>
<BR>
-Ken<BR>
<BR>
<BR>
<BR>
<BR>
<BR>
</FONT>
</P>

</BODY>
</HTML>
------_=_NextPart_001_01C7E5C9.63A32F4B--
