X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C794D1.B8031B8C@onstor-exch02.onstor.net>; Sat, 12 May 2007 13:11:25 -0700
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C794D1.B8031B8C"
Content-class: urn:content-classes:message
Subject: RE: Test upgrade fom 222 to 30 prior to MD upgrade, still seeing cluster errors
Date: Sat, 12 May 2007 13:11:25 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E03A8E85D@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E03A8E854@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: Test upgrade fom 222 to 30 prior to MD upgrade, still seeing cluster errors
Thread-Index: AceUzHDmrKQBNO3gTRuD2SAA03/ulgABSM9A
From: "Raj Kumar" <raj.kumar@onstor.com>
To: "Sandrine Boulanger" <sandrine.boulanger@onstor.com>,
	"Chris Vandever" <chris.vandever@onstor.com>,
	"Brian DeForest" <brian.deforest@onstor.com>
Cc: "Andy Sharp" <andy.sharp@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C794D1.B8031B8C
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I think we can live with GNS failures for MD.

The cluster errors are concerning. Can we try some failovers/vsvr moves
and make sure HA is functional?

Thanks Sandrine.

_____________________________________________
From: Sandrine Boulanger=20
Sent: Saturday, May 12, 2007 12:34 PM
To: Chris Vandever; Brian DeForest; Raj Kumar
Cc: Andy Sharp
Subject: Test upgrade fom 222 to 30 prior to MD upgrade, still seeing
cluster errors

I tested Andy's fix for the copy of the config files. I used the files
as standalone since we don't have a new build. It worked fine. Prior to
the DBs being upgraded from 220 to 30, all commands were running OK. I
tried to add a gns root and it refused because the DBs were not upgraded
yet, which is good.
Then the DBs were upgraded to 30, and at that time I saw the same type
of errors we saw last Thursday: cluster show summary on g10r9, or gns
show cifs was failing with cluster2 errors, see elog below. The commands
are OK on g11r9.
G10r9 was the first to be rebooted on 3.0 flash, g11r9 was second.

G10r9 ssc ip is 10.2.9.10
I'll leave it in this state, let me know if I can help debugging.


# dbtools show /onstor/conf/cluster.db.DB0 -c=20
Cluster DB Header
------------------------------------
dbHdr.version =3D 0x30000
dbHdr.signature =3D 0x434c5553
dbHdr.blockSize =3D 256
dbHdr.freeOffset =3D 0
dbHdr.endOffset =3D 2336000
dbHdr.recCount =3D 0
dbHdr.hashTblSize =3D 4096
dbHdr.hashTblOffset =3D 256
dbHdr.DebugMode =3D 0

# nfxsh

Welcome to the ONStor NAS Gateway.

g10r9 diag> gns show
% Command incomplete.
g10r9 diag> gns show=20
  cifs  Show GNS CIFS
g10r9 diag> gns show cifs=20
  ROOTNAME[\Path]  Root and optional path
  all
g10r9 diag> gns show cifs all
This command is unavailable until after the cluster database has been
upgraded.
% Command failure.
g10r9 diag> gns show cifs all
This command is unavailable until after the cluster database has been
upgraded.
% Command failure.
g10r9 diag> cluster show summary=20
Cluster Name: g10r9       Cluster State:   On
------------------------------------------------------
NAS Gateways        IP              State   PCC
------------------------------------------------------
g10r9               10.2.9.10       UP      YES
g11r9               10.2.9.11       UP      NO
Getting cluster information failed
% Command failure.
g10r9 diag>


# cat /var/agile/messages=20
May 12 12:00:02 g10r9 newsyslog[23005]: logfile turned over
May 12 12:00:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:00:29 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList:
cannot get cluster rec, rcode 30=20
May 12 12:00:38 g10r9 : 0:0:spm:ERROR: failed to get filer list from
cluster daemon=20
May 12 12:00:39 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:01:09 g10r9 last message repeated 3 times
May 12 12:01:13 g10r9 : 0:0:spm:ERROR: failed to get filer list from
cluster daemon=20
May 12 12:01:19 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR: cluster_request_upgrade_db:
no reply bck -1=20
May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR:
ClusterCtrl_ProcessTimerFunc: Request to upgrade clusDb failed, rc 30=20
May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR:
ClusterCtrl_ProcessTimerFunc: Running in degraded mode on version
0x20200, will retry upgrade to version 0x30000 later...=20
May 12 12:01:39 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:01:40 g10r9 : 0:0:cluster2:ERROR: cluster_set_lock_on_rpc: no
reply bck -1=20
May 12 12:01:49 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:02:00 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:03:24 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:03:24 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList:
cannot get cluster rec, rcode 30=20
May 12 12:03:35 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:03:57 g10r9 last message repeated 3 times
May 12 12:03:57 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get
vsId list for node g10r9
May 12 12:04:06 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:04:07 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:04:07 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get
vsId list for node g10r9
May 12 12:04:16 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:04:26 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:04:27 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.2.9.11=20
May 12 12:04:36 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:04:42 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.12.9.11=20
May 12 12:04:46 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:04:56 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:05:28 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.2.9.11=20
May 12 12:05:33 g10r9 : 0:0:cluster2:NOTICE: node 10.2.9.11 is down=20
May 12 12:05:34 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.2.9.11=20
May 12 12:05:34 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name
'g11r9', State Up, Msg ''
May 12 12:05:41 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.12.9.11=20
May 12 12:05:50 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.2.9.11=20
May 12 12:05:54 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.12.9.11=20
May 12 12:05:59 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.2.9.11=20
May 12 12:06:11 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.12.9.11=20
May 12 12:06:15 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.2.9.11=20
May 12 12:06:20 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.12.9.11=20
May 12 12:06:24 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.2.9.11=20
May 12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList:
cannot get cluster rec, rcode 30=20
May 12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no
reply bck -1=20
May 12 12:06:29 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get
vsId list for node g10r9
May 12 12:06:33 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.12.9.11=20
May 12 12:06:40 g10r9 : 0:0:cluster2:ERROR: cluster_upgrade_db_runtime:
ClusDb update complete; successfully updated version from 0x20200 to
0x30000=20
May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send
new file in progress, remote ver 1178995860(554), sending to 10.2.9.11=20
May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: cluster
db version matches=20
May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send
new file end, code 0=20
May 12 12:07:45 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: gns show : status[4]
May 12 12:08:05 g10r9 : 0:0:cluster2:ERROR: cluster_get_db_ver: no reply
bck -1=20
May 12 12:08:05 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: gns show cifs all :
status[11]
May 12 12:10:12 g10r9 : 0:0:cluster2:ERROR: cluster_get_db_ver: no reply
bck -1=20
May 12 12:10:12 g10r9 : 0:0:nfxsh:NOTICE: cmd[2]: gns show cifs all :
status[11]



# zcat /var/agile/messages.0.gz

May 12 11:38:00 g10r9 : 0:0:cm:NOTICE: CHASSISD: Chassis Manager:
Started
May 12 11:38:00 g10r9 : 0:0:cm:NOTICE: CM: opened kseg1
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1,
CPU 0, State Up
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1,
CPU 1, State Up
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1,
CPU 2, State Up
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1,
CPU 3, State Up
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 2,
CPU 0, State Up
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.0
State Up, Msg ''
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.1
State Down, Msg ''
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.2
State Down, Msg ''
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.3
State Down, Msg ''
May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open
failed
May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open
failed
May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_closeSess : NCM session
closed
May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_closeSess : SDM session
closed
May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID
5=20
May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID
6=20
May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID
8=20
May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID
9=20
May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID
7=20
May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name
'g10r9', State Up, Msg ''
May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open
success
May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open
success
May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_closeSess : NCM session
closed
May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_closeSess : SDM session
closed
May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open
success
May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open
success
May 12 11:38:17 g10r9 : 0:0:cluster2:NOTICE: Using 10.2.9.10 as my
primary address=20
May 12 11:38:17 g10r9 : 0:0:cluster2:NOTICE: ubik init with buff size 64

May 12 11:38:17 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'nis-vol', Id 0x000002000000006a, Event 'Create', State 'Down'
May 12 11:38:18 g10r9 : 1:2:sanm_ag:WARNING: 2: sanm_doRMCLocalClose:
reopening session to sanmd=20
May 12 11:38:18 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'jumbo-vol', Id 0x0000020000000073, Event 'Create', State 'Down'
May 12 11:38:19 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'win-vol', Id 0x000002000000006d, Event 'Create', State 'Down'
May 12 11:38:19 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: -> rc.agile: booting
version 3.0.0.0b : status[2]
May 12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'openldap-vol', Id 0x0000020000000071, Event 'Create', State 'Down'
May 12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'local-nis-vol', Id 0x000002000000006e, Event 'Create', State 'Down'
May 12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'authdom-vol', Id 0x0000020000000072, Event 'Create', State 'Down'
May 12 11:38:21 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'local', Id 0x0000020000000075, Event 'Create', State 'Down'
May 12 11:38:22 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: -> rc.agile: booting
from /dev/wd0a : status[2]
May 12 11:38:22 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'vol_mgmt_512', Id 0x0000020000000066, Event 'Create', State 'Down'
May 12 11:38:23 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'LSI_E4600A_R46_g10r9_core', Id 0x0000020000000074, Event 'Create',
State 'Down'
May 12 11:38:32 g10r9 : 0:0:eventd:WARNING: Process-EVENT 0.0.0.0: Mgmt
Port 0.0.0.0 PCC, State Up
May 12 11:38:39 g10r9 : 0:0:ea:NOTICE: ea_rcvRmcMsg : NCM session open
failed
May 12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 00000001026a8a93, WWN 100000068d002a40, LUN 1, vendor 'IBM', model
'ULTRIUM-TD3'=20
May 12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000000299256570, WWN 100000068d002a40, LUN 2, vendor 'IBM', model
'ULTRIUM-TD3'=20
May 12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000000474cbbcf7, WWN 100000068d002a40, LUN 4, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 00000005020ee656, WWN 100000068d002a40, LUN 5, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 00000007527722f4, WWN 100000068d002a40, LUN 7, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 00000008c30a5552, WWN 100000068d002a40, LUN 8, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000000ad8b479e3, WWN 100000068d002a40, LUN 10, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000000b326471cc, WWN 100000068d002a40, LUN 11, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000000d7663d86b, WWN 100000068d002a40, LUN 13, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000000e50bb2aad, WWN 100000068d002a40, LUN 14, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 00000010cecbc163, WWN 100000068d002a40, LUN 16, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000001101709690, WWN 100000068d002a40, LUN 17, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:ea:NOTICE: ea_closeSess : NCM session closed
May 12 11:38:44 g10r9 : 0:0:ea:NOTICE: ea_rcvRmcMsg : NCM session open
success
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 000000120c18b3b8, WWN 100000068d002a40, LUN 18, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000001483e7a0c2, WWN 100000068d002a40, LUN 20, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 000000154c5cf731, WWN 100000068d002a40, LUN 21, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 000000168d36b6dd, WWN 100000068d002a40, LUN 22, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 00000017428de12e, WWN 100000068d002a40, LUN 23, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 00000019533a4544, WWN 100000068d002a40, LUN 25, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device:
id 0000001ad886bb10, WWN 100000068d002a40, LUN 26, vendor 'IBM', model
'ULTRIUM-TD2'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new
media-changer: id 00000000b26acfa2, WWN 100000068d002a40, LUN 0, vendor
'ADIC', model 'Scalar-24'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new
media-changer: id 00000003632ebb68, WWN 100000068d002a40, LUN 3, vendor
'STK', model 'L180'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new
media-changer: id 000000061ed3ca31, WWN 100000068d002a40, LUN 6, vendor
'STK', model 'L180'=20
May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new
media-changer: id 000000093a855f24, WWN 100000068d002a40, LUN 9, vendor
'STK', model 'L180'=20
May 12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new
media-changer: id 0000000ca1e082bf, WWN 100000068d002a40, LUN 12, vendor
'STK', model 'L180'=20
May 12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new
media-changer: id 0000000ff746fa9b, WWN 100000068d002a40, LUN 15, vendor
'STK', model 'L180'=20
May 12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new
media-changer: id 00000013719221e7, WWN 100000068d002a40, LUN 19, vendor
'STK', model 'L180'=20
May 12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new
media-changer: id 000000182eb0d71b, WWN 100000068d002a40, LUN 24, vendor
'STK', model 'L180'=20
May 12 11:38:46 g10r9 : 0:0:auth_agent:NOTICE: main: auth-agent started
May 12 11:38:49 g10r9 : 1:2:sanm_ag:WARNING: 3: sanm_doRMCLocalClose:
reopening session to sanmd=20
May 12 11:38:50 g10r9 : 1:0:ipm:NOTICE: 10: vstack_vlink_create:294 : vl
FO active 1 prefered 1=20
May 12 11:38:50 g10r9 : 1:0:ipm:NOTICE: 11: vstack_vlink_create:329:
vlink lp link op-state DOWN=20
May 12 11:38:52 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.1.1, Port bp0, State Up
May 12 11:38:55 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'jumbo-vol', Id 0x0000020000000073, Event 'Online', was offline for
roughly 255 sec.
May 12 11:38:55 g10r9 : 0:0:sanm:NOTICE: SANM: ONStor Data Mirror
(c)2006: Started
May 12 11:38:57 g10r9 : 1:0:bsdrl:WARNING: 12: route dst 0 mask 0
gateway 100030a fail to be added/deleted. error =3D 51=20
May 12 11:38:57 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.8.1, Port bp0, State Up
May 12 11:38:59 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'VS_MGMT_512', Id 1, State 'Up'
May 12 11:38:59 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for
vsId[1]
May 12 11:39:00 g10r9 : 0:0:ea:ERROR: ea_procMovInReq[1586] : Volume
(vol_mgmt_512) is locked for Online operation
May 12 11:39:01 g10r9 : 0:0:vsd:ERROR: vsd_mountVolProc[4941] : Failed
to mount volume ID=3D0x20000000066. Operation not allowed.
May 12 11:39:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'vol_mgmt_512', Id 0x0000020000000066, Event 'Online', was offline for
roughly 266 sec.
May 12 11:39:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'JUMBO_G10R9', Id 8, State 'Up'
May 12 11:39:02 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for
vsId[8]
May 12 11:44:43 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: system version  :
status[0]
May 12 11:44:48 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: cl sh cl : status[0]
May 12 11:45:29 g10r9 : 0:0:nfxsh:NOTICE: cmd[2]: interface show ip -n
g11r9 : status[0]
May 12 11:45:37 g10r9 : 0:0:nfxsh:NOTICE: cmd[3]: cluster sho gr :
status[0]
May 12 11:45:41 g10r9 : 0:0:nfxsh:NOTICE: cmd[4]: cluster show summary
: status[0]
May 12 11:45:52 g10r9 : 0:0:nfxsh:NOTICE: cmd[5]: vsvr show all :
status[0]
May 12 11:49:23 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: elog show config  :
status[0]
May 12 11:50:35 g10r9 : 0:0:cluster2:NOTICE: node 10.2.9.11 is down=20
May 12 11:50:35 g10r9 : 0:0:eventd:WARNING: Process-EVENT 0.0.0.0: Mgmt
Port 0.0.0.0 PCC, State Up
May 12 11:50:51 g10r9 : 0:0:cluster2:WARNING:
ClusterCtrl_ProcessTimerFunc: post node down hostname g11r9=20
May 12 11:50:51 g10r9 : 0:0:eventd:CRITICAL: Process-EVENT Node: Name
'g11r9', State Down, Msg ''
May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1
g10r9 has matching (vlink#=3D1, vol#=3D2), num_vsvr 2=20
May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 3=20
May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 4=20
May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 5=20
May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 6=20
May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'NIS_G10R9', Id 3, State 'Enabled'
May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'WINDOWS_G10R9', Id 4, State 'Enabled'
May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'LOCAL_NIS_G10R9', Id 5, State 'Enabled'
May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'OPENLDAP_G10R9', Id 6, State 'Enabled'
May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'AUTHDOM_G10R9', Id 7, State 'Enabled'
May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'NONETBIOS_G10R9', Id 10, State 'Enabled'
May 12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'nis-vol', Id 0x000002000000006a, Event 'Takeover', State 'Up'
May 12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'local', Id 0x0000020000000075, Event 'Takeover', State 'Up'
May 12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'NONETBIOS_G10R9', Id 10, State 'Up'
May 12 11:51:02 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for
vsId[10]
May 12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'nis-vol', Id 0x000002000000006a, Event 'Online', was offline for
roughly 69 sec.
May 12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'win-vol', Id 0x000002000000006d, Event 'Takeover', State 'Up'
May 12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'local-nis-vol', Id 0x000002000000006e, Event 'Takeover', State 'Up'
May 12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'openldap-vol', Id 0x0000020000000071, Event 'Takeover', State 'Up'
May 12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'authdom-vol', Id 0x0000020000000072, Event 'Takeover', State 'Up'
May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'local', Id 0x0000020000000075, Event 'Online', was offline for roughly
52 sec.
May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'openldap-vol', Id 0x0000020000000071, Event 'Online', was offline for
roughly 62 sec.
May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'win-vol', Id 0x000002000000006d, Event 'Online', was offline for
roughly 66 sec.
May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'local-nis-vol', Id 0x000002000000006e, Event 'Online', was offline for
roughly 59 sec.
May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'authdom-vol', Id 0x0000020000000072, Event 'Online', was offline for
roughly 55 sec.
May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.3.1, Port bp0, State Up
May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.6.1, Port bp0, State Up
May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.5.1, Port bp0, State Up
May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.4.1, Port bp0, State Up
May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.4.2, Port bp0, State Up
May 12 11:51:07 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer:
ncm_open_filer(g11r9) succeeded=20
May 12 11:51:07 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'NIS_G10R9', Id 3, State 'Up'
May 12 11:51:07 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for
vsId[3]
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'OPENLDAP_G10R9', Id 6, State 'Up'
May 12 11:51:08 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for
vsId[6]
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.1, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.2, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.3, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.4, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.5, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.6, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.7, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.8, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.9, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.10, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.11, Port bp0, State Up
May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.12, Port bp0, State Up
May 12 11:51:09 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.13, Port bp0, State Up
May 12 11:51:09 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.14, Port bp0, State Up
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.15, Port bp0, State Up
May 12 11:51:10 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer:
ncm_open_filer(g11r9) succeeded=20
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.16, Port bp0, State Up
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.17, Port bp0, State Up
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.18, Port bp0, State Up
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.19, Port bp0, State Up
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -14=20
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.20, Port bp0, State Up
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -14=20
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.21, Port bp0, State Up
May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:11 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.22, Port bp0, State Up
May 12 11:51:11 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.23, Port bp0, State Up
May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.24, Port bp0, State Up
May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.25, Port bp0, State Up
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.26, Port bp0, State Up
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.27, Port bp0, State Up
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.28, Port bp0, State Up
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.29, Port bp0, State Up
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.30, Port bp0, State Up
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.31, Port bp0, State Up
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.7.32, Port bp0, State Up
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: IP i/f RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'LOCAL_NIS_G10R9', Id 5, State 'Up'
May 12 11:51:13 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for
vsId[5]
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP:
asd_rmc EVENT: Vsvr RMC Error -3=20
May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'AUTHDOM_G10R9', Id 7, State 'Up'
May 12 11:51:13 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for
vsId[7]
May 12 11:51:14 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer:
retries to filer(g11r9) failed - aborting attemtpts
May 12 11:51:14 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_handle_net_msg_done:
ncm_repoen filer failed - -1
May 12 11:51:18 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'WINDOWS_G10R9', Id 4, State 'Up'
May 12 11:51:18 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for
vsId[4]
May 12 11:51:43 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer:
ncm_open_filer(g11r9) succeeded=20
May 12 11:51:46 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer:
ncm_open_filer(g11r9) succeeded=20
May 12 11:51:49 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer:
retries to filer(g11r9) failed - aborting attemtpts
May 12 11:51:49 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_handle_net_msg_done:
ncm_repoen filer failed - -1
May 12 11:53:48 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name
'g11r9', State Up, Msg ''
May 12 11:53:53 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.2.9.11=20
May 12 11:53:53 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send
new file in progress, remote ver 1178991276(64), sending to 10.2.9.11=20
May 12 11:54:01 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send
new file end, code 0=20
May 12 11:54:29 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: cl sh cl : status[0]
May 12 11:58:08 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: elog find cluster2 :
status[0]
May 12 11:59:10 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is
back up: will be contacted through 10.12.9.11=20
May 12 11:59:12 g10r9 : 0:0:cluster2:NOTICE: ubik init with buff size 64

May 12 11:59:32 g10r9 : 0:0:cluster2:ERROR: cluster_get_db_ver: no reply
bck -1=20
May 12 11:59:32 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: gns add root cifs
gnsroot  : status[11] =3D=3D> tried this before the DB was upgraded
May 12 12:00:03 g10r9 : 0:0:spm:ERROR: failed to get filer list from
cluster daemon=20
May 12 12:00:02 g10r9 newsyslog[23005]: logfile turned over
#




------_=_NextPart_001_01C794D1.B8031B8C
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7652.24">
<TITLE>RE: Test upgrade fom 222 to 30 prior to MD upgrade, still seeing =
cluster errors</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">I think we can live with GNS failures for =
MD.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">The</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">cluster</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial"> errors =
are concerning. Can we try some failovers/vsvr moves and make sure HA is =
functional?</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Thanks Sandrine.</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Sandrine Boulanger<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Saturday, May 12, 2007 =
12:34 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Chris Vandever; Brian =
DeForest; Raj Kumar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Cc:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Andy Sharp<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> Test upgrade fom 222 to 30 prior to MD upgrade, still =
seeing cluster errors</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">I =
tested Andy&#8217;s fix for the copy of the config files. I used the =
files as standalone since we don&#8217;t have a new build. It worked =
fine. Prior to the DBs being upgraded from 220 to 30, all commands were =
running OK. I tried to add a gns root and it refused because the DBs =
were not upgraded yet, which is good.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Then =
the DBs were upgraded to 30, and at that time I saw the same type of =
errors we saw last Thursday: cluster show summary on g10r9, or gns show =
cifs was failing with cluster2 errors, see elog below. The commands are =
OK on g11r9.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">G10r9 =
was the first to be rebooted on 3.0 flash, g11r9 was =
second.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">G10r9 =
ssc ip is 10.2.9.10</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">I&#8217;ll leave it in this state, let me know if I can =
help debugging.</FONT></SPAN></P>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial"># =
dbtools show /onstor/conf/cluster.db.DB0 -c </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Cluster DB Header</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">------------------------------------</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.version =3D 0x30000</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.signature =3D 0x434c5553</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.blockSize =3D 256</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.freeOffset =3D 0</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.endOffset =3D 2336000</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.recCount =3D 0</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.hashTblSize =3D 4096</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.hashTblOffset =3D 256</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">dbHdr.DebugMode =3D 0</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial"># =
nfxsh</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Welcome to the ONStor NAS Gateway.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">g10r9 =
diag&gt; gns show</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">% =
Command incomplete.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">g10r9 =
diag&gt; gns show </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">&nbsp; cifs&nbsp; Show GNS CIFS</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">g10r9 =
diag&gt; gns show cifs </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">&nbsp; ROOTNAME[\Path]&nbsp; Root and optional =
path</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">&nbsp; all</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">g10r9 =
diag&gt; gns show cifs all</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">This =
command is unavailable until after the cluster database has been =
upgraded.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">% =
Command failure.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">g10r9 =
diag&gt; gns show cifs all</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">This =
command is unavailable until after the cluster database has been =
upgraded.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">% =
Command failure.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">g10r9 =
diag&gt; cluster show summary </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Cluster Name: g10r9&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
Cluster State:&nbsp;&nbsp; On</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">------------------------------------------------------</FO=
NT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">NAS =
Gateways&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
IP&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp; State&nbsp;&nbsp; PCC</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">------------------------------------------------------</FO=
NT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">g10r9&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
10.2.9.10&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
UP&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; YES</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">g11r9&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
10.2.9.11&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
UP&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; NO</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Getting cluster information failed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">% =
Command failure.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">g10r9 =
diag&gt;</FONT></SPAN></P>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial"># cat =
/var/agile/messages </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:00:02 g10r9 newsyslog[23005]: logfile turned =
over</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#FF0000" SIZE=3D2 =
FACE=3D"Arial">May 12 12:00:29 g10r9 : 0:0:cluster2:ERROR: =
cluster_getRecordIdByKey: no reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#FF0000" SIZE=3D2 =
FACE=3D"Arial">May 12 12:00:29 g10r9 : 0:0:cluster2:ERROR: =
cluster_getFilerNameList: cannot get cluster rec, rcode 30 =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:00:38 g10r9 : 0:0:spm:ERROR: failed to get filer list from cluster =
daemon </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:00:39 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:01:09 g10r9 last message repeated 3 times</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:01:13 g10r9 : 0:0:spm:ERROR: failed to get filer list from cluster =
daemon </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:01:19 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:01:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:01:29 g10r9 : 0:0:cluster2:ERROR: cluster_request_upgrade_db: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR: =
ClusterCtrl_ProcessTimerFunc: Request to upgrade clusDb failed, rc 30 =
</FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR: =
ClusterCtrl_ProcessTimerFunc: Running in degraded mode on version =
0x20200, will retry upgrade to version 0x30000 later... =
</FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:01:39 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:01:40 g10r9 : 0:0:cluster2:ERROR: cluster_set_lock_on_rpc: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:01:49 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:02:00 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:03:24 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:03:24 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList: cannot =
get cluster rec, rcode 30 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:03:35 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:03:57 g10r9 last message repeated 3 times</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:03:57 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get =
vsId list for node g10r9</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:06 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:07 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:07 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get =
vsId list for node g10r9</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:16 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:26 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:27 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.2.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:36 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:42 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.12.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:46 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:04:56 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:05:28 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.2.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:05:33 g10r9 : 0:0:cluster2:NOTICE: node 10.2.9.11 is down =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:05:34 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.2.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:05:34 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name 'g11r9', =
State Up, Msg ''</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:05:41 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.12.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:05:50 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.2.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:05:54 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.12.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:05:59 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.2.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:06:11 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.12.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:06:15 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.2.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:06:20 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.12.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:06:24 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.2.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList: cannot =
get cluster rec, rcode 30 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:06:29 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get =
vsId list for node g10r9</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 12:06:33 g10r9 : 0:0:cluster2:NOTICE: ubik:server =
10.2.9.11 is back up: will be contacted through 10.12.9.11 =
</FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 12:06:40 g10r9 : 0:0:cluster2:ERROR: =
cluster_upgrade_db_runtime: ClusDb update complete; successfully updated =
version from 0x20200 to 0x30000 </FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: =
urecovery_Interact: send new file in progress, remote ver =
1178995860(554), sending to 10.2.9.11 </FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: =
urecovery_Interact: cluster db version matches </FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: =
urecovery_Interact: send new file end, code 0 </FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:07:45 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: gns show : =
status[4]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#FF0000" SIZE=3D2 =
FACE=3D"Arial">May 12 12:08:05 g10r9 : 0:0:cluster2:ERROR: =
cluster_get_db_ver: no reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:08:05 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: gns show cifs all : =
status[11]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#FF0000" SIZE=3D2 =
FACE=3D"Arial">May 12 12:10:12 g10r9 : 0:0:cluster2:ERROR: =
cluster_get_db_ver: no reply bck -1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:10:12 g10r9 : 0:0:nfxsh:NOTICE: cmd[2]: gns show cifs all : =
status[11]</FONT></SPAN></P>
<BR>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial"># =
zcat =
/var/agile/messages.0.gz&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:00 g10r9 : 0:0:cm:NOTICE: CHASSISD: Chassis Manager: =
Started</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:00 g10r9 : 0:0:cm:NOTICE: CM: opened kseg1</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, CPU 0, =
State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, CPU 1, =
State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, CPU 2, =
State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, CPU 3, =
State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 2, CPU 0, =
State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.0 State =
Up, Msg ''</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.1 State =
Down, Msg ''</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.2 State =
Down, Msg ''</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.3 State =
Down, Msg ''</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
failed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
failed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_closeSess : NCM session =
closed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_closeSess : SDM session =
closed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 5 =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 6 =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 8 =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 9 =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 7 =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name 'g10r9', =
State Up, Msg ''</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
success</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
success</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_closeSess : NCM session =
closed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_closeSess : SDM session =
closed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
success</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
success</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:17 g10r9 : 0:0:cluster2:NOTICE: Using 10.2.9.10 as my primary =
address </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:17 g10r9 : 0:0:cluster2:NOTICE: ubik init with buff size 64 =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:17 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Create', State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:18 g10r9 : 1:2:sanm_ag:WARNING: 2: sanm_doRMCLocalClose: =
reopening session to sanmd </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:18 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'jumbo-vol', Id 0x0000020000000073, Event 'Create', State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:19 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Create', State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:19 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: -&gt; rc.agile: booting =
version 3.0.0.0b : status[2]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Create', State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Create', State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Create', State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:21 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local', Id 0x0000020000000075, Event 'Create', State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:22 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: -&gt; rc.agile: booting =
from /dev/wd0a : status[2]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:22 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'vol_mgmt_512', Id 0x0000020000000066, Event 'Create', State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:23 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'LSI_E4600A_R46_g10r9_core', Id 0x0000020000000074, Event 'Create', =
State 'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:32 g10r9 : 0:0:eventd:WARNING: Process-EVENT 0.0.0.0: Mgmt Port =
0.0.0.0 PCC, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:39 g10r9 : 0:0:ea:NOTICE: ea_rcvRmcMsg : NCM session open =
failed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000001026a8a93, WWN 100000068d002a40, LUN 1, vendor 'IBM', model =
'ULTRIUM-TD3' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000299256570, WWN 100000068d002a40, LUN 2, vendor 'IBM', model =
'ULTRIUM-TD3' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000474cbbcf7, WWN 100000068d002a40, LUN 4, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000005020ee656, WWN 100000068d002a40, LUN 5, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000007527722f4, WWN 100000068d002a40, LUN 7, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000008c30a5552, WWN 100000068d002a40, LUN 8, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000ad8b479e3, WWN 100000068d002a40, LUN 10, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000b326471cc, WWN 100000068d002a40, LUN 11, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000d7663d86b, WWN 100000068d002a40, LUN 13, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000e50bb2aad, WWN 100000068d002a40, LUN 14, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000010cecbc163, WWN 100000068d002a40, LUN 16, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000001101709690, WWN 100000068d002a40, LUN 17, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:ea:NOTICE: ea_closeSess : NCM session =
closed</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:ea:NOTICE: ea_rcvRmcMsg : NCM session open =
success</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
000000120c18b3b8, WWN 100000068d002a40, LUN 18, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000001483e7a0c2, WWN 100000068d002a40, LUN 20, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
000000154c5cf731, WWN 100000068d002a40, LUN 21, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
000000168d36b6dd, WWN 100000068d002a40, LUN 22, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000017428de12e, WWN 100000068d002a40, LUN 23, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000019533a4544, WWN 100000068d002a40, LUN 25, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000001ad886bb10, WWN 100000068d002a40, LUN 26, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
00000000b26acfa2, WWN 100000068d002a40, LUN 0, vendor 'ADIC', model =
'Scalar-24' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
00000003632ebb68, WWN 100000068d002a40, LUN 3, vendor 'STK', model =
'L180' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
000000061ed3ca31, WWN 100000068d002a40, LUN 6, vendor 'STK', model =
'L180' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
000000093a855f24, WWN 100000068d002a40, LUN 9, vendor 'STK', model =
'L180' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
0000000ca1e082bf, WWN 100000068d002a40, LUN 12, vendor 'STK', model =
'L180' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
0000000ff746fa9b, WWN 100000068d002a40, LUN 15, vendor 'STK', model =
'L180' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
00000013719221e7, WWN 100000068d002a40, LUN 19, vendor 'STK', model =
'L180' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
000000182eb0d71b, WWN 100000068d002a40, LUN 24, vendor 'STK', model =
'L180' </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:46 g10r9 : 0:0:auth_agent:NOTICE: main: auth-agent =
started</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:49 g10r9 : 1:2:sanm_ag:WARNING: 3: sanm_doRMCLocalClose: =
reopening session to sanmd </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:50 g10r9 : 1:0:ipm:NOTICE: 10: vstack_vlink_create:294 : vl FO =
active 1 prefered 1 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:50 g10r9 : 1:0:ipm:NOTICE: 11: vstack_vlink_create:329: vlink =
lp link op-state DOWN </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:52 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.1.1, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:55 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'jumbo-vol', Id 0x0000020000000073, Event 'Online', was offline for =
roughly 255 sec.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:55 g10r9 : 0:0:sanm:NOTICE: SANM: ONStor Data Mirror (c)2006: =
Started</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:57 g10r9 : 1:0:bsdrl:WARNING: 12: route dst 0 mask 0 gateway =
100030a fail to be added/deleted. error =3D 51 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:57 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.8.1, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:59 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'VS_MGMT_512', Id 1, State 'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:38:59 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[1]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:39:00 g10r9 : 0:0:ea:ERROR: ea_procMovInReq[1586] : Volume =
(vol_mgmt_512) is locked for Online operation</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:39:01 g10r9 : 0:0:vsd:ERROR: vsd_mountVolProc[4941] : Failed to =
mount volume ID=3D0x20000000066. Operation not =
allowed.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:39:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'vol_mgmt_512', Id 0x0000020000000066, Event 'Online', was offline for =
roughly 266 sec.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:39:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'JUMBO_G10R9', Id 8, State 'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:39:02 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[8]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:44:43 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: system version&nbsp; : =
status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:44:48 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: cl sh cl : =
status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:45:29 g10r9 : 0:0:nfxsh:NOTICE: cmd[2]: interface show ip -n g11r9 =
: status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:45:37 g10r9 : 0:0:nfxsh:NOTICE: cmd[3]: cluster sho gr : =
status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:45:41 g10r9 : 0:0:nfxsh:NOTICE: cmd[4]: cluster show summary&nbsp; =
: status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:45:52 g10r9 : 0:0:nfxsh:NOTICE: cmd[5]: vsvr show all : =
status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:49:23 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: elog show config&nbsp; : =
status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:35 g10r9 : 0:0:cluster2:NOTICE: node 10.2.9.11 is down =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:35 g10r9 : 0:0:eventd:WARNING: Process-EVENT 0.0.0.0: Mgmt Port =
0.0.0.0 PCC, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:51 g10r9 : 0:0:cluster2:WARNING: ClusterCtrl_ProcessTimerFunc: =
post node down hostname g11r9 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:51 g10r9 : 0:0:eventd:CRITICAL: Process-EVENT Node: Name =
'g11r9', State Down, Msg ''</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D2), num_vsvr 2 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 4 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 5 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 6 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'NIS_G10R9', Id 3, State 'Enabled'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'WINDOWS_G10R9', Id 4, State 'Enabled'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'LOCAL_NIS_G10R9', Id 5, State 'Enabled'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'OPENLDAP_G10R9', Id 6, State 'Enabled'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'AUTHDOM_G10R9', Id 7, State 'Enabled'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'NONETBIOS_G10R9', Id 10, State 'Enabled'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Takeover', State =
'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local', Id 0x0000020000000075, Event 'Takeover', State =
'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'NONETBIOS_G10R9', Id 10, State 'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:02 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[10]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Online', was offline for =
roughly 69 sec.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Takeover', State =
'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Takeover', State =
'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Takeover', State =
'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Takeover', State =
'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local', Id 0x0000020000000075, Event 'Online', was offline for roughly =
52 sec.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Online', was offline for =
roughly 62 sec.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Online', was offline for =
roughly 66 sec.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Online', was offline for =
roughly 59 sec.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Online', was offline for =
roughly 55 sec.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.3.1, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.6.1, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.5.1, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.4.1, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.4.2, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:07 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:07 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'NIS_G10R9', Id 3, State 'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:07 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[3]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'OPENLDAP_G10R9', Id 6, State 'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[6]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.1, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.2, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.3, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.4, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.5, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.6, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.7, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.8, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.9, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.10, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.11, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.12, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:09 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.13, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:09 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.14, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.15, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.16, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.17, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.18, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.19, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -14 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.20, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -14 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.21, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:11 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.22, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:11 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.23, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:12 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.24, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:12 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.25, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.26, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.27, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.28, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.29, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.30, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.31, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.32, Port bp0, State Up</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'LOCAL_NIS_G10R9', Id 5, State 'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[5]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: Vsvr RMC Error -3 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'AUTHDOM_G10R9', Id 7, State 'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:13 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[7]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:14 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: retries to =
filer(g11r9) failed - aborting attemtpts</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:14 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_handle_net_msg_done: =
ncm_repoen filer failed - -1</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:18 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'WINDOWS_G10R9', Id 4, State 'Up'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:18 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[4]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:43 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:46 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:49 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: retries to =
filer(g11r9) failed - aborting attemtpts</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:51:49 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_handle_net_msg_done: =
ncm_repoen filer failed - -1</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:53:48 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name 'g11r9', =
State Up, Msg ''</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 11:53:53 g10r9 : 0:0:cluster2:NOTICE: ubik:server =
10.2.9.11 is back up: will be contacted through 10.2.9.11 =
</FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 11:53:53 g10r9 : 0:0:cluster2:NOTICE: =
urecovery_Interact: send new file in progress, remote ver =
1178991276(64), sending to 10.2.9.11 </FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 11:54:01 g10r9 : 0:0:cluster2:NOTICE: =
urecovery_Interact: send new file end, code 0 </FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:54:29 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: cl sh cl : =
status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:58:08 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: elog find cluster2 : =
status[0]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:59:10 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back =
up: will be contacted through 10.12.9.11 </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 11:59:12 g10r9 : 0:0:cluster2:NOTICE: ubik init with buff size 64 =
</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 11:59:32 g10r9 : 0:0:cluster2:ERROR: =
cluster_get_db_ver: no reply bck -1 </FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 11:59:32 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: gns add =
root cifs gnsroot&nbsp; : status[11]</FONT></B></SPAN><SPAN =
LANG=3D"en-us"><B></B></SPAN><SPAN LANG=3D"en-us"><B> <FONT =
FACE=3D"Wingdings" SIZE=3D2>&#232;</FONT></B></SPAN><SPAN =
LANG=3D"en-us"><B></B></SPAN><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial"> tried this before the DB was =
upgraded</FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">May 12 12:00:03 g10r9 : 0:0:spm:ERROR: failed to get =
filer list from cluster daemon </FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">May =
12 12:00:02 g10r9 newsyslog[23005]: logfile turned =
over</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">#</FONT></SPAN></P>
<BR>
<BR>

</BODY>
</HTML>
------_=_NextPart_001_01C794D1.B8031B8C--
