X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C794F7.2FE44638@onstor-exch02.onstor.net>; Sat, 12 May 2007 17:39:38 -0700
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C794F7.2FE44638"
References: <BB375AF679D4A34E9CA8DFA650E2B04E03A8E854@onstor-exch02.onstor.net>
Content-class: urn:content-classes:message
Subject: RE: Test upgrade fom 222 to 30 prior to MD upgrade, still seeing cluster errors
Date: Sat, 12 May 2007 17:36:26 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E0353B352@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: Test upgrade fom 222 to 30 prior to MD upgrade, still seeing cluster errors
Thread-Index: AceUzHDmrKQBNO3gTRuD2SAA03/ulgAKkyw3
From: "Chris Vandever" <chris.vandever@onstor.com>
To: "Sandrine Boulanger" <sandrine.boulanger@onstor.com>,
	"Brian DeForest" <brian.deforest@onstor.com>,
	"Raj Kumar" <raj.kumar@onstor.com>
Cc: "Andy Sharp" <andy.sharp@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C794F7.2FE44638
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

I'll need the exact procedure that's being used for the upgrade, and =
access to any scripts, etc.  Clearly, the clustering code is confused =
since the clusDb shows it was successfully upgraded, yet the clustering =
code is still trying (unsuccessfully) to upgrade it.
=20
I won't be able to look at it this evening, but I will try to do so =
tomorrow.
=20
ChrisV

________________________________

From: Sandrine Boulanger
Sent: Sat 5/12/2007 12:33 PM
To: Chris Vandever; Brian DeForest; Raj Kumar
Cc: Andy Sharp
Subject: Test upgrade fom 222 to 30 prior to MD upgrade, still seeing =
cluster errors



I tested Andy's fix for the copy of the config files. I used the files =
as standalone since we don't have a new build. It worked fine. Prior to =
the DBs being upgraded from 220 to 30, all commands were running OK. I =
tried to add a gns root and it refused because the DBs were not upgraded =
yet, which is good.

Then the DBs were upgraded to 30, and at that time I saw the same type =
of errors we saw last Thursday: cluster show summary on g10r9, or gns =
show cifs was failing with cluster2 errors, see elog below. The commands =
are OK on g11r9.

G10r9 was the first to be rebooted on 3.0 flash, g11r9 was second.

G10r9 ssc ip is 10.2.9.10

I'll leave it in this state, let me know if I can help debugging.

# dbtools show /onstor/conf/cluster.db.DB0 -c=20

Cluster DB Header

------------------------------------

dbHdr.version =3D 0x30000

dbHdr.signature =3D 0x434c5553

dbHdr.blockSize =3D 256

dbHdr.freeOffset =3D 0

dbHdr.endOffset =3D 2336000

dbHdr.recCount =3D 0

dbHdr.hashTblSize =3D 4096

dbHdr.hashTblOffset =3D 256

dbHdr.DebugMode =3D 0

# nfxsh

Welcome to the ONStor NAS Gateway.

g10r9 diag> gns show

% Command incomplete.

g10r9 diag> gns show=20

  cifs  Show GNS CIFS

g10r9 diag> gns show cifs=20

  ROOTNAME[\Path]  Root and optional path

  all

g10r9 diag> gns show cifs all

This command is unavailable until after the cluster database has been =
upgraded.

% Command failure.

g10r9 diag> gns show cifs all

This command is unavailable until after the cluster database has been =
upgraded.

% Command failure.

g10r9 diag> cluster show summary=20

Cluster Name: g10r9       Cluster State:   On

------------------------------------------------------

NAS Gateways        IP              State   PCC

------------------------------------------------------

g10r9               10.2.9.10       UP      YES

g11r9               10.2.9.11       UP      NO

Getting cluster information failed

% Command failure.

g10r9 diag>

# cat /var/agile/messages=20

May 12 12:00:02 g10r9 newsyslog[23005]: logfile turned over

May 12 12:00:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:00:29 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList: =
cannot get cluster rec, rcode 30=20

May 12 12:00:38 g10r9 : 0:0:spm:ERROR: failed to get filer list from =
cluster daemon=20

May 12 12:00:39 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:01:09 g10r9 last message repeated 3 times

May 12 12:01:13 g10r9 : 0:0:spm:ERROR: failed to get filer list from =
cluster daemon=20

May 12 12:01:19 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR: cluster_request_upgrade_db: =
no reply bck -1=20

May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR: =
ClusterCtrl_ProcessTimerFunc: Request to upgrade clusDb failed, rc 30=20

May 12 12:01:29 g10r9 : 0:0:cluster2:ERROR: =
ClusterCtrl_ProcessTimerFunc: Running in degraded mode on version =
0x20200, will retry upgrade to version 0x30000 later...=20

May 12 12:01:39 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:01:40 g10r9 : 0:0:cluster2:ERROR: cluster_set_lock_on_rpc: no =
reply bck -1=20

May 12 12:01:49 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:02:00 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:03:24 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:03:24 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList: =
cannot get cluster rec, rcode 30=20

May 12 12:03:35 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:03:57 g10r9 last message repeated 3 times

May 12 12:03:57 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get =
vsId list for node g10r9

May 12 12:04:06 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:04:07 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:04:07 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get =
vsId list for node g10r9

May 12 12:04:16 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:04:26 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:04:27 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.2.9.11=20

May 12 12:04:36 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:04:42 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.12.9.11=20

May 12 12:04:46 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:04:56 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:05:28 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.2.9.11=20

May 12 12:05:33 g10r9 : 0:0:cluster2:NOTICE: node 10.2.9.11 is down=20

May 12 12:05:34 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.2.9.11=20

May 12 12:05:34 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name =
'g11r9', State Up, Msg ''

May 12 12:05:41 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.12.9.11=20

May 12 12:05:50 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.2.9.11=20

May 12 12:05:54 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.12.9.11=20

May 12 12:05:59 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.2.9.11=20

May 12 12:06:11 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.12.9.11=20

May 12 12:06:15 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.2.9.11=20

May 12 12:06:20 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.12.9.11=20

May 12 12:06:24 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.2.9.11=20

May 12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList: =
cannot get cluster rec, rcode 30=20

May 12 12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no =
reply bck -1=20

May 12 12:06:29 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get =
vsId list for node g10r9

May 12 12:06:33 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.12.9.11=20

May 12 12:06:40 g10r9 : 0:0:cluster2:ERROR: cluster_upgrade_db_runtime: =
ClusDb update complete; successfully updated version from 0x20200 to =
0x30000=20

May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send =
new file in progress, remote ver 1178995860(554), sending to 10.2.9.11=20

May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: cluster =
db version matches=20

May 12 12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send =
new file end, code 0=20

May 12 12:07:45 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: gns show : status[4]

May 12 12:08:05 g10r9 : 0:0:cluster2:ERROR: cluster_get_db_ver: no reply =
bck -1=20

May 12 12:08:05 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: gns show cifs all : =
status[11]

May 12 12:10:12 g10r9 : 0:0:cluster2:ERROR: cluster_get_db_ver: no reply =
bck -1=20

May 12 12:10:12 g10r9 : 0:0:nfxsh:NOTICE: cmd[2]: gns show cifs all : =
status[11]

# zcat /var/agile/messages.0.gz                                          =
                                                             =20

May 12 11:38:00 g10r9 : 0:0:cm:NOTICE: CHASSISD: Chassis Manager: =
Started

May 12 11:38:00 g10r9 : 0:0:cm:NOTICE: CM: opened kseg1

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, =
CPU 0, State Up

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, =
CPU 1, State Up

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, =
CPU 2, State Up

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, =
CPU 3, State Up

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 2, =
CPU 0, State Up

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.0 =
State Up, Msg ''

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.1 =
State Down, Msg ''

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.2 =
State Down, Msg ''

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.3 =
State Down, Msg ''

May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
failed

May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
failed

May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_closeSess : NCM session =
closed

May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_closeSess : SDM session =
closed

May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID =
5=20

May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID =
6=20

May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID =
8=20

May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID =
9=20

May 12 11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID =
7=20

May 12 11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name =
'g10r9', State Up, Msg ''

May 12 11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
success

May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
success

May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_closeSess : NCM session =
closed

May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_closeSess : SDM session =
closed

May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
success

May 12 11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
success

May 12 11:38:17 g10r9 : 0:0:cluster2:NOTICE: Using 10.2.9.10 as my =
primary address=20

May 12 11:38:17 g10r9 : 0:0:cluster2:NOTICE: ubik init with buff size 64 =


May 12 11:38:17 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Create', State 'Down'

May 12 11:38:18 g10r9 : 1:2:sanm_ag:WARNING: 2: sanm_doRMCLocalClose: =
reopening session to sanmd=20

May 12 11:38:18 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'jumbo-vol', Id 0x0000020000000073, Event 'Create', State 'Down'

May 12 11:38:19 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Create', State 'Down'

May 12 11:38:19 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: -> rc.agile: booting =
version 3.0.0.0b : status[2]

May 12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Create', State 'Down'

May 12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Create', State 'Down'

May 12 11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Create', State 'Down'

May 12 11:38:21 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local', Id 0x0000020000000075, Event 'Create', State 'Down'

May 12 11:38:22 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: -> rc.agile: booting =
from /dev/wd0a : status[2]

May 12 11:38:22 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'vol_mgmt_512', Id 0x0000020000000066, Event 'Create', State 'Down'

May 12 11:38:23 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'LSI_E4600A_R46_g10r9_core', Id 0x0000020000000074, Event 'Create', =
State 'Down'

May 12 11:38:32 g10r9 : 0:0:eventd:WARNING: Process-EVENT 0.0.0.0: Mgmt =
Port 0.0.0.0 PCC, State Up

May 12 11:38:39 g10r9 : 0:0:ea:NOTICE: ea_rcvRmcMsg : NCM session open =
failed

May 12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 00000001026a8a93, WWN 100000068d002a40, LUN 1, vendor 'IBM', model =
'ULTRIUM-TD3'=20

May 12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000000299256570, WWN 100000068d002a40, LUN 2, vendor 'IBM', model =
'ULTRIUM-TD3'=20

May 12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000000474cbbcf7, WWN 100000068d002a40, LUN 4, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 00000005020ee656, WWN 100000068d002a40, LUN 5, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 00000007527722f4, WWN 100000068d002a40, LUN 7, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 00000008c30a5552, WWN 100000068d002a40, LUN 8, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000000ad8b479e3, WWN 100000068d002a40, LUN 10, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000000b326471cc, WWN 100000068d002a40, LUN 11, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000000d7663d86b, WWN 100000068d002a40, LUN 13, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000000e50bb2aad, WWN 100000068d002a40, LUN 14, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 00000010cecbc163, WWN 100000068d002a40, LUN 16, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000001101709690, WWN 100000068d002a40, LUN 17, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:ea:NOTICE: ea_closeSess : NCM session closed

May 12 11:38:44 g10r9 : 0:0:ea:NOTICE: ea_rcvRmcMsg : NCM session open =
success

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 000000120c18b3b8, WWN 100000068d002a40, LUN 18, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000001483e7a0c2, WWN 100000068d002a40, LUN 20, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 000000154c5cf731, WWN 100000068d002a40, LUN 21, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 000000168d36b6dd, WWN 100000068d002a40, LUN 22, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 00000017428de12e, WWN 100000068d002a40, LUN 23, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 00000019533a4544, WWN 100000068d002a40, LUN 25, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: =
id 0000001ad886bb10, WWN 100000068d002a40, LUN 26, vendor 'IBM', model =
'ULTRIUM-TD2'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new =
media-changer: id 00000000b26acfa2, WWN 100000068d002a40, LUN 0, vendor =
'ADIC', model 'Scalar-24'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new =
media-changer: id 00000003632ebb68, WWN 100000068d002a40, LUN 3, vendor =
'STK', model 'L180'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new =
media-changer: id 000000061ed3ca31, WWN 100000068d002a40, LUN 6, vendor =
'STK', model 'L180'=20

May 12 11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new =
media-changer: id 000000093a855f24, WWN 100000068d002a40, LUN 9, vendor =
'STK', model 'L180'=20

May 12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new =
media-changer: id 0000000ca1e082bf, WWN 100000068d002a40, LUN 12, vendor =
'STK', model 'L180'=20

May 12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new =
media-changer: id 0000000ff746fa9b, WWN 100000068d002a40, LUN 15, vendor =
'STK', model 'L180'=20

May 12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new =
media-changer: id 00000013719221e7, WWN 100000068d002a40, LUN 19, vendor =
'STK', model 'L180'=20

May 12 11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new =
media-changer: id 000000182eb0d71b, WWN 100000068d002a40, LUN 24, vendor =
'STK', model 'L180'=20

May 12 11:38:46 g10r9 : 0:0:auth_agent:NOTICE: main: auth-agent started

May 12 11:38:49 g10r9 : 1:2:sanm_ag:WARNING: 3: sanm_doRMCLocalClose: =
reopening session to sanmd=20

May 12 11:38:50 g10r9 : 1:0:ipm:NOTICE: 10: vstack_vlink_create:294 : vl =
FO active 1 prefered 1=20

May 12 11:38:50 g10r9 : 1:0:ipm:NOTICE: 11: vstack_vlink_create:329: =
vlink lp link op-state DOWN=20

May 12 11:38:52 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.1.1, Port bp0, State Up

May 12 11:38:55 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'jumbo-vol', Id 0x0000020000000073, Event 'Online', was offline for =
roughly 255 sec.

May 12 11:38:55 g10r9 : 0:0:sanm:NOTICE: SANM: ONStor Data Mirror =
(c)2006: Started

May 12 11:38:57 g10r9 : 1:0:bsdrl:WARNING: 12: route dst 0 mask 0 =
gateway 100030a fail to be added/deleted. error =3D 51=20

May 12 11:38:57 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.8.1, Port bp0, State Up

May 12 11:38:59 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'VS_MGMT_512', Id 1, State 'Up'

May 12 11:38:59 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[1]

May 12 11:39:00 g10r9 : 0:0:ea:ERROR: ea_procMovInReq[1586] : Volume =
(vol_mgmt_512) is locked for Online operation

May 12 11:39:01 g10r9 : 0:0:vsd:ERROR: vsd_mountVolProc[4941] : Failed =
to mount volume ID=3D0x20000000066. Operation not allowed.

May 12 11:39:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'vol_mgmt_512', Id 0x0000020000000066, Event 'Online', was offline for =
roughly 266 sec.

May 12 11:39:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'JUMBO_G10R9', Id 8, State 'Up'

May 12 11:39:02 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[8]

May 12 11:44:43 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: system version  : =
status[0]

May 12 11:44:48 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: cl sh cl : status[0]

May 12 11:45:29 g10r9 : 0:0:nfxsh:NOTICE: cmd[2]: interface show ip -n =
g11r9 : status[0]

May 12 11:45:37 g10r9 : 0:0:nfxsh:NOTICE: cmd[3]: cluster sho gr : =
status[0]

May 12 11:45:41 g10r9 : 0:0:nfxsh:NOTICE: cmd[4]: cluster show summary  =
: status[0]

May 12 11:45:52 g10r9 : 0:0:nfxsh:NOTICE: cmd[5]: vsvr show all : =
status[0]

May 12 11:49:23 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: elog show config  : =
status[0]

May 12 11:50:35 g10r9 : 0:0:cluster2:NOTICE: node 10.2.9.11 is down=20

May 12 11:50:35 g10r9 : 0:0:eventd:WARNING: Process-EVENT 0.0.0.0: Mgmt =
Port 0.0.0.0 PCC, State Up

May 12 11:50:51 g10r9 : 0:0:cluster2:WARNING: =
ClusterCtrl_ProcessTimerFunc: post node down hostname g11r9=20

May 12 11:50:51 g10r9 : 0:0:eventd:CRITICAL: Process-EVENT Node: Name =
'g11r9', State Down, Msg ''

May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D2), num_vsvr 2=20

May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 3=20

May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 4=20

May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 5=20

May 12 11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 =
g10r9 has matching (vlink#=3D1, vol#=3D1), num_vsvr 6=20

May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'NIS_G10R9', Id 3, State 'Enabled'

May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'WINDOWS_G10R9', Id 4, State 'Enabled'

May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'LOCAL_NIS_G10R9', Id 5, State 'Enabled'

May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'OPENLDAP_G10R9', Id 6, State 'Enabled'

May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'AUTHDOM_G10R9', Id 7, State 'Enabled'

May 12 11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'NONETBIOS_G10R9', Id 10, State 'Enabled'

May 12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Takeover', State 'Up'

May 12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local', Id 0x0000020000000075, Event 'Takeover', State 'Up'

May 12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'NONETBIOS_G10R9', Id 10, State 'Up'

May 12 11:51:02 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[10]

May 12 11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Online', was offline for =
roughly 69 sec.

May 12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Takeover', State 'Up'

May 12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Takeover', State 'Up'

May 12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Takeover', State 'Up'

May 12 11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Takeover', State 'Up'

May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local', Id 0x0000020000000075, Event 'Online', was offline for roughly =
52 sec.

May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Online', was offline for =
roughly 62 sec.

May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Online', was offline for =
roughly 66 sec.

May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Online', was offline for =
roughly 59 sec.

May 12 11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Online', was offline for =
roughly 55 sec.

May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.3.1, Port bp0, State Up

May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.6.1, Port bp0, State Up

May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.5.1, Port bp0, State Up

May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.4.1, Port bp0, State Up

May 12 11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.4.2, Port bp0, State Up

May 12 11:51:07 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded=20

May 12 11:51:07 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'NIS_G10R9', Id 3, State 'Up'

May 12 11:51:07 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[3]

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'OPENLDAP_G10R9', Id 6, State 'Up'

May 12 11:51:08 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[6]

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.1, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.2, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.3, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.4, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.5, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.6, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.7, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.8, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.9, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.10, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.11, Port bp0, State Up

May 12 11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.12, Port bp0, State Up

May 12 11:51:09 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.13, Port bp0, State Up

May 12 11:51:09 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.14, Port bp0, State Up

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.15, Port bp0, State Up

May 12 11:51:10 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded=20

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.16, Port bp0, State Up

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.17, Port bp0, State Up

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.18, Port bp0, State Up

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.19, Port bp0, State Up

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -14=20

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.20, Port bp0, State Up

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -14=20

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.21, Port bp0, State Up

May 12 11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:11 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.22, Port bp0, State Up

May 12 11:51:11 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.23, Port bp0, State Up

May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.24, Port bp0, State Up

May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.25, Port bp0, State Up

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.26, Port bp0, State Up

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.27, Port bp0, State Up

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.28, Port bp0, State Up

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.29, Port bp0, State Up

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.30, Port bp0, State Up

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.31, Port bp0, State Up

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.32, Port bp0, State Up

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: IP i/f RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'LOCAL_NIS_G10R9', Id 5, State 'Up'

May 12 11:51:13 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[5]

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: =
asd_rmc EVENT: Vsvr RMC Error -3=20

May 12 11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'AUTHDOM_G10R9', Id 7, State 'Up'

May 12 11:51:13 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[7]

May 12 11:51:14 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
retries to filer(g11r9) failed - aborting attemtpts

May 12 11:51:14 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_handle_net_msg_done: =
ncm_repoen filer failed - -1

May 12 11:51:18 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual =
server 'WINDOWS_G10R9', Id 4, State 'Up'

May 12 11:51:18 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[4]

May 12 11:51:43 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded=20

May 12 11:51:46 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded=20

May 12 11:51:49 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
retries to filer(g11r9) failed - aborting attemtpts

May 12 11:51:49 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_handle_net_msg_done: =
ncm_repoen filer failed - -1

May 12 11:53:48 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name =
'g11r9', State Up, Msg ''

May 12 11:53:53 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.2.9.11=20

May 12 11:53:53 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send =
new file in progress, remote ver 1178991276(64), sending to 10.2.9.11=20

May 12 11:54:01 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send =
new file end, code 0=20

May 12 11:54:29 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: cl sh cl : status[0]

May 12 11:58:08 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: elog find cluster2 : =
status[0]

May 12 11:59:10 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is =
back up: will be contacted through 10.12.9.11=20

May 12 11:59:12 g10r9 : 0:0:cluster2:NOTICE: ubik init with buff size 64 =


May 12 11:59:32 g10r9 : 0:0:cluster2:ERROR: cluster_get_db_ver: no reply =
bck -1=20

May 12 11:59:32 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: gns add root cifs =
gnsroot  : status[11] =3D=3D> tried this before the DB was upgraded

May 12 12:00:03 g10r9 : 0:0:spm:ERROR: failed to get filer list from =
cluster daemon=20

May 12 12:00:02 g10r9 newsyslog[23005]: logfile turned over

#


------_=_NextPart_001_01C794F7.2FE44638
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<HTML dir=3Dltr><HEAD><TITLE>Test upgrade fom 222 to 30 prior to MD =
upgrade, still seeing cluster errors</TITLE>=0A=
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Dunicode">=0A=
<META content=3D"MSHTML 6.00.5730.11" name=3DGENERATOR></HEAD>=0A=
<BODY>=0A=
<DIV id=3DidOWAReplyText10557 dir=3Dltr>=0A=
<DIV dir=3Dltr><FONT face=3DArial color=3D#000000 size=3D2>I'll need the =
exact procedure that's being used for the upgrade, and access to any =
scripts, etc.&nbsp; Clearly, the clustering code is confused since the =
clusDb shows it was successfully upgraded, yet the clustering code is =
still trying (unsuccessfully) to upgrade it.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2>I won't be able to look at it =
this evening, but I will try to do so tomorrow.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial size=3D2>ChrisV</FONT></DIV></DIV>=0A=
<DIV dir=3Dltr><BR>=0A=
<HR tabIndex=3D-1>=0A=
<FONT face=3DTahoma size=3D2><B>From:</B> Sandrine =
Boulanger<BR><B>Sent:</B> Sat 5/12/2007 12:33 PM<BR><B>To:</B> Chris =
Vandever; Brian DeForest; Raj Kumar<BR><B>Cc:</B> Andy =
Sharp<BR><B>Subject:</B> Test upgrade fom 222 to 30 prior to MD upgrade, =
still seeing cluster errors<BR></FONT><BR></DIV>=0A=
<DIV>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>I tested Andy</FONT></SPAN><SPAN =
lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>&#8217;</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us><FONT face=3DArial size=3D2>s fix for the copy of the =
config files.</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us> =
<FONT face=3DArial size=3D2>I used the files as standalone since we =
don</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>&#8217;</FONT></SPAN><SPAN =
lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT face=3DArial size=3D2>t =
have a new build.</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us> <FONT face=3DArial size=3D2>It worked fine. Prior to the =
DBs being upgraded from 220 to 30, all commands were running OK. I tried =
to add a gns root and it refused because the DBs were not upgraded yet, =
which is good.</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>Then the DBs were upgraded to =
30</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>, and at that time I saw the same type of errors =
we saw last Thursday: cluster show summary on g10r9, or gns show =
cifs</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us> <FONT =
face=3DArial size=3D2>was failing with cluster2 errors, see elog below. =
The commands are OK on g11r9.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>G10r9 was =
the fi</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>r</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us><FONT face=3DArial size=3D2>st to be rebooted on 3.0 =
flash,</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us> <FONT =
face=3DArial size=3D2>g</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us><FONT face=3DArial size=3D2>11r9 was =
second.</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>G10r9 ssc ip =
is 10.2.9.10</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>I</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us><FONT face=3DArial size=3D2>&#8217;</FONT></SPAN><SPAN =
lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT face=3DArial size=3D2>ll =
leave it in this state, let me know if</FONT></SPAN><SPAN =
lang=3Den-us></SPAN><SPAN lang=3Den-us> <FONT face=3DArial =
size=3D2>I</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us><FONT face=3DArial size=3D2> can help =
debugging.</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2># dbtools show /onstor/conf/cluster.db.DB0 -c =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Cluster DB =
Header</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>------------------------------------</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.version =3D 0x30000</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.signature =3D 0x434c5553</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.blockSize =3D 256</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.freeOffset =3D 0</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.endOffset =3D 2336000</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.recCount =3D 0</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.hashTblSize =3D 4096</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.hashTblOffset =3D 256</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>dbHdr.DebugMode =3D 0</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2># =
nfxsh</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Welcome to =
the ONStor NAS Gateway.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>g10r9 =
diag&gt; gns show</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>% Command =
incomplete.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>g10r9 =
diag&gt; gns show </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>&nbsp; =
cifs&nbsp; Show GNS CIFS</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>g10r9 =
diag&gt; gns show cifs </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>&nbsp; =
ROOTNAME[\Path]&nbsp; Root and optional path</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>&nbsp; =
all</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>g10r9 =
diag&gt; gns show cifs all</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>This command =
is unavailable until after the cluster database has been =
upgraded.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>% Command =
failure.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>g10r9 =
diag&gt; gns show cifs all</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>This command =
is unavailable until after the cluster database has been =
upgraded.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>% Command =
failure.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>g10r9 =
diag&gt; cluster show summary </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Cluster =
Name: g10r9&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Cluster =
State:&nbsp;&nbsp; On</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>------------------------------------------------------</FONT></S=
PAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>NAS =
Gateways&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
IP&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp; State&nbsp;&nbsp; PCC</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>------------------------------------------------------</FONT></S=
PAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>g10r9&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp; 10.2.9.10&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
UP&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; YES</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>g11r9&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp; 10.2.9.11&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
UP&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; NO</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Getting =
cluster information failed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>% Command =
failure.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>g10r9 =
diag&gt;</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2># cat /var/agile/messages </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:00:02 g10r9 newsyslog[23005]: logfile turned over</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial color=3D#ff0000 size=3D2>May 12 12:00:29 g10r9 : =
0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply bck -1 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial color=3D#ff0000 =
size=3D2>May 12 12:00:29 g10r9 : 0:0:cluster2:ERROR: =
cluster_getFilerNameList: cannot get cluster rec, rcode 30 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>May 12 12:00:38 g10r9 : 0:0:spm:ERROR: failed to =
get filer list from cluster daemon </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:00:39 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:01:09 g10r9 last message repeated 3 times</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:01:13 g10r9 : 0:0:spm:ERROR: failed to get filer list from cluster =
daemon </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:01:19 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:01:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:01:29 g10r9 : 0:0:cluster2:ERROR: cluster_request_upgrade_db: no =
reply bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B></B></SPAN><SPAN =
lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 12:01:29 g10r9 : =
0:0:cluster2:ERROR: ClusterCtrl_ProcessTimerFunc: Request to upgrade =
clusDb failed, rc 30 </FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
12:01:29 g10r9 : 0:0:cluster2:ERROR: ClusterCtrl_ProcessTimerFunc: =
Running in degraded mode on version 0x20200, will retry upgrade to =
version 0x30000 later... </FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>May 12 12:01:39 g10r9 : 0:0:cluster2:ERROR: =
cluster_getRecordIdByKey: no reply bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:01:40 g10r9 : 0:0:cluster2:ERROR: cluster_set_lock_on_rpc: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:01:49 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:02:00 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:03:24 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:03:24 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList: cannot =
get cluster rec, rcode 30 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:03:35 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:03:57 g10r9 last message repeated 3 times</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:03:57 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get vsId =
list for node g10r9</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:06 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:07 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:07 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get vsId =
list for node g10r9</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:16 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:26 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:27 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.2.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:36 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:42 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.12.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:46 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:04:56 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:05:28 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.2.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:05:33 g10r9 : 0:0:cluster2:NOTICE: node 10.2.9.11 is down =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:05:34 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.2.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:05:34 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name 'g11r9', =
State Up, Msg ''</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:05:41 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.12.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:05:50 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.2.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:05:54 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.12.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:05:59 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.2.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:06:11 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.12.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:06:15 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.2.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:06:20 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.12.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:06:24 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.2.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getFilerNameList: cannot =
get cluster rec, rcode 30 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:06:29 g10r9 : 0:0:cluster2:ERROR: cluster_getRecordIdByKey: no reply =
bck -1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
12:06:29 g10r9 : 0:0:vsd:ERROR: vsd_procGetVsListReq : Cannot get vsId =
list for node g10r9</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B></B></SPAN><SPAN =
lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 12:06:33 g10r9 : =
0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: will be contacted =
through 10.12.9.11 </FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
12:06:40 g10r9 : 0:0:cluster2:ERROR: cluster_upgrade_db_runtime: ClusDb =
update complete; successfully updated version from 0x20200 to 0x30000 =
</FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send new file =
in progress, remote ver 1178995860(554), sending to 10.2.9.11 =
</FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: cluster db =
version matches </FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
12:06:40 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send new file =
end, code 0 </FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>May 12 12:07:45 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: =
gns show : status[4]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial color=3D#ff0000 size=3D2>May 12 12:08:05 g10r9 : =
0:0:cluster2:ERROR: cluster_get_db_ver: no reply bck -1 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>May 12 12:08:05 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: =
gns show cifs all : status[11]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial color=3D#ff0000 size=3D2>May 12 12:10:12 g10r9 : =
0:0:cluster2:ERROR: cluster_get_db_ver: no reply bck -1 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>May 12 12:10:12 g10r9 : 0:0:nfxsh:NOTICE: cmd[2]: =
gns show cifs all : status[11]</FONT></SPAN><SPAN =
lang=3Den-us></SPAN><SPAN lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2># zcat =
/var/agile/messages.0.gz&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:00 g10r9 : 0:0:cm:NOTICE: CHASSISD: Chassis Manager: =
Started</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:00 g10r9 : 0:0:cm:NOTICE: CM: opened kseg1</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, CPU 0, =
State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, CPU 1, =
State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, CPU 2, =
State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 1, CPU 3, =
State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT CPU: Slot 2, CPU 0, =
State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.0 State Up, =
Msg ''</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.1 State =
Down, Msg ''</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.2 State =
Down, Msg ''</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Port: fp1.3 State =
Down, Msg ''</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
failed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
failed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:evm:NOTICE: evm_closeSess : NCM session =
closed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:evm:NOTICE: evm_closeSess : SDM session =
closed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 5 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 6 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 8 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 9 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:sdm:ERROR: sdm_procScsiRelayRsp: Invalid TID 7 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name 'g10r9', =
State Up, Msg ''</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:16 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
success</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
success</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:17 g10r9 : 0:0:evm:NOTICE: evm_closeSess : NCM session =
closed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:17 g10r9 : 0:0:evm:NOTICE: evm_closeSess : SDM session =
closed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : NCM session open =
success</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:17 g10r9 : 0:0:evm:NOTICE: evm_rcvRmcMsg : SDM session open =
success</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:17 g10r9 : 0:0:cluster2:NOTICE: Using 10.2.9.10 as my primary =
address </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:17 g10r9 : 0:0:cluster2:NOTICE: ubik init with buff size 64 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:17 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Create', State =
'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:18 g10r9 : 1:2:sanm_ag:WARNING: 2: sanm_doRMCLocalClose: reopening =
session to sanmd </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:18 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'jumbo-vol', Id 0x0000020000000073, Event 'Create', State =
'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:19 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Create', State =
'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:19 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: -&gt; rc.agile: booting =
version 3.0.0.0b : status[2]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Create', State =
'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Create', State =
'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:20 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Create', State =
'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:21 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name 'local', =
Id 0x0000020000000075, Event 'Create', State 'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:22 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: -&gt; rc.agile: booting from =
/dev/wd0a : status[2]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:22 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'vol_mgmt_512', Id 0x0000020000000066, Event 'Create', State =
'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:23 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'LSI_E4600A_R46_g10r9_core', Id 0x0000020000000074, Event 'Create', =
State 'Down'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:32 g10r9 : 0:0:eventd:WARNING: Process-EVENT 0.0.0.0: Mgmt Port =
0.0.0.0 PCC, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:39 g10r9 : 0:0:ea:NOTICE: ea_rcvRmcMsg : NCM session open =
failed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000001026a8a93, WWN 100000068d002a40, LUN 1, vendor 'IBM', model =
'ULTRIUM-TD3' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000299256570, WWN 100000068d002a40, LUN 2, vendor 'IBM', model =
'ULTRIUM-TD3' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000474cbbcf7, WWN 100000068d002a40, LUN 4, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:42 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000005020ee656, WWN 100000068d002a40, LUN 5, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000007527722f4, WWN 100000068d002a40, LUN 7, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000008c30a5552, WWN 100000068d002a40, LUN 8, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000ad8b479e3, WWN 100000068d002a40, LUN 10, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000b326471cc, WWN 100000068d002a40, LUN 11, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000d7663d86b, WWN 100000068d002a40, LUN 13, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000000e50bb2aad, WWN 100000068d002a40, LUN 14, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:43 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000010cecbc163, WWN 100000068d002a40, LUN 16, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000001101709690, WWN 100000068d002a40, LUN 17, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:ea:NOTICE: ea_closeSess : NCM session =
closed</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:ea:NOTICE: ea_rcvRmcMsg : NCM session open =
success</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
000000120c18b3b8, WWN 100000068d002a40, LUN 18, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000001483e7a0c2, WWN 100000068d002a40, LUN 20, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
000000154c5cf731, WWN 100000068d002a40, LUN 21, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
000000168d36b6dd, WWN 100000068d002a40, LUN 22, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000017428de12e, WWN 100000068d002a40, LUN 23, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
00000019533a4544, WWN 100000068d002a40, LUN 25, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new tape device: id =
0000001ad886bb10, WWN 100000068d002a40, LUN 26, vendor 'IBM', model =
'ULTRIUM-TD2' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
00000000b26acfa2, WWN 100000068d002a40, LUN 0, vendor 'ADIC', model =
'Scalar-24' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
00000003632ebb68, WWN 100000068d002a40, LUN 3, vendor 'STK', model =
'L180' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
000000061ed3ca31, WWN 100000068d002a40, LUN 6, vendor 'STK', model =
'L180' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:44 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
000000093a855f24, WWN 100000068d002a40, LUN 9, vendor 'STK', model =
'L180' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
0000000ca1e082bf, WWN 100000068d002a40, LUN 12, vendor 'STK', model =
'L180' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
0000000ff746fa9b, WWN 100000068d002a40, LUN 15, vendor 'STK', model =
'L180' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
00000013719221e7, WWN 100000068d002a40, LUN 19, vendor 'STK', model =
'L180' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:45 g10r9 : 0:0:tape-driver:NOTICE: Adding new media-changer: id =
000000182eb0d71b, WWN 100000068d002a40, LUN 24, vendor 'STK', model =
'L180' </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:46 g10r9 : 0:0:auth_agent:NOTICE: main: auth-agent =
started</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:49 g10r9 : 1:2:sanm_ag:WARNING: 3: sanm_doRMCLocalClose: reopening =
session to sanmd </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:50 g10r9 : 1:0:ipm:NOTICE: 10: vstack_vlink_create:294 : vl FO =
active 1 prefered 1 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:50 g10r9 : 1:0:ipm:NOTICE: 11: vstack_vlink_create:329: vlink lp =
link op-state DOWN </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:52 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.1.1, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:55 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'jumbo-vol', Id 0x0000020000000073, Event 'Online', was offline for =
roughly 255 sec.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:55 g10r9 : 0:0:sanm:NOTICE: SANM: ONStor Data Mirror (c)2006: =
Started</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:57 g10r9 : 1:0:bsdrl:WARNING: 12: route dst 0 mask 0 gateway =
100030a fail to be added/deleted. error =3D 51 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:57 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.8.1, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:59 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'VS_MGMT_512', Id 1, State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:38:59 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[1]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:39:00 g10r9 : 0:0:ea:ERROR: ea_procMovInReq[1586] : Volume =
(vol_mgmt_512) is locked for Online operation</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:39:01 g10r9 : 0:0:vsd:ERROR: vsd_mountVolProc[4941] : Failed to mount =
volume ID=3D0x20000000066. Operation not allowed.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:39:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'vol_mgmt_512', Id 0x0000020000000066, Event 'Online', was offline for =
roughly 266 sec.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:39:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'JUMBO_G10R9', Id 8, State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:39:02 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[8]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:44:43 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: system version&nbsp; : =
status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:44:48 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: cl sh cl : =
status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:45:29 g10r9 : 0:0:nfxsh:NOTICE: cmd[2]: interface show ip -n g11r9 : =
status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:45:37 g10r9 : 0:0:nfxsh:NOTICE: cmd[3]: cluster sho gr : =
status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:45:41 g10r9 : 0:0:nfxsh:NOTICE: cmd[4]: cluster show summary&nbsp; : =
status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:45:52 g10r9 : 0:0:nfxsh:NOTICE: cmd[5]: vsvr show all : =
status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:49:23 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: elog show config&nbsp; : =
status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:35 g10r9 : 0:0:cluster2:NOTICE: node 10.2.9.11 is down =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:35 g10r9 : 0:0:eventd:WARNING: Process-EVENT 0.0.0.0: Mgmt Port =
0.0.0.0 PCC, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:51 g10r9 : 0:0:cluster2:WARNING: ClusterCtrl_ProcessTimerFunc: =
post node down hostname g11r9 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:51 g10r9 : 0:0:eventd:CRITICAL: Process-EVENT Node: Name 'g11r9', =
State Down, Msg ''</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 g10r9 =
has matching (vlink#=3D1, vol#=3D2), num_vsvr 2 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 g10r9 =
has matching (vlink#=3D1, vol#=3D1), num_vsvr 3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 g10r9 =
has matching (vlink#=3D1, vol#=3D1), num_vsvr 4 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 g10r9 =
has matching (vlink#=3D1, vol#=3D1), num_vsvr 5 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:50:52 g10r9 : 0:0:vtm:NOTICE: vtm_get_better_node[729]: node1 g10r9 =
has matching (vlink#=3D1, vol#=3D1), num_vsvr 6 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'NIS_G10R9', Id 3, State 'Enabled'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'WINDOWS_G10R9', Id 4, State 'Enabled'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'LOCAL_NIS_G10R9', Id 5, State 'Enabled'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'OPENLDAP_G10R9', Id 6, State 'Enabled'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'AUTHDOM_G10R9', Id 7, State 'Enabled'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:01 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'NONETBIOS_G10R9', Id 10, State 'Enabled'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Takeover', State =
'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name 'local', =
Id 0x0000020000000075, Event 'Takeover', State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'NONETBIOS_G10R9', Id 10, State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:02 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[10]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:02 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'nis-vol', Id 0x000002000000006a, Event 'Online', was offline for =
roughly 69 sec.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Takeover', State =
'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Takeover', State =
'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Takeover', State =
'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:03 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Takeover', State =
'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name 'local', =
Id 0x0000020000000075, Event 'Online', was offline for roughly 52 =
sec.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'openldap-vol', Id 0x0000020000000071, Event 'Online', was offline for =
roughly 62 sec.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'win-vol', Id 0x000002000000006d, Event 'Online', was offline for =
roughly 66 sec.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'local-nis-vol', Id 0x000002000000006e, Event 'Online', was offline for =
roughly 59 sec.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:04 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Volume: name =
'authdom-vol', Id 0x0000020000000072, Event 'Online', was offline for =
roughly 55 sec.</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.3.1, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.6.1, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.5.1, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.4.1, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:06 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.4.2, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:07 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:07 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'NIS_G10R9', Id 3, State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:07 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[3]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'OPENLDAP_G10R9', Id 6, State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[6]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.1, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.2, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.3, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.4, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.5, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.6, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.7, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.8, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.9, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.10, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.11, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:08 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.12, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:09 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.13, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:09 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.14, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.15, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.16, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.17, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.18, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.19, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -14 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.20, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -14 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.21, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:10 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:11 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.22, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:11 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.23, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:12 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.24, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:12 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:12 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.25, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.26, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.27, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.28, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.29, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.30, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.31, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP =
192.167.7.32, Port bp0, State Up</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: IP i/f RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'LOCAL_NIS_G10R9', Id 5, State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[5]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: forwardToLocalApps dstAPP: asd_rmc =
EVENT: Vsvr RMC Error -3 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'AUTHDOM_G10R9', Id 7, State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:13 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[7]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:14 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: retries to =
filer(g11r9) failed - aborting attemtpts</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:14 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_handle_net_msg_done: =
ncm_repoen filer failed - -1</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:18 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual server =
'WINDOWS_G10R9', Id 4, State 'Up'</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:18 g10r9 : 0:0:sanm:NOTICE: Processing fail-over event for =
vsId[4]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:43 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:46 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: =
ncm_open_filer(g11r9) succeeded </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:49 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer: retries to =
filer(g11r9) failed - aborting attemtpts</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:51:49 g10r9 : 0:0:ncm:WARNING: ncmd : ncm_handle_net_msg_done: =
ncm_repoen filer failed - -1</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:53:48 g10r9 : 0:0:eventd:NOTICE: Process-EVENT Node: Name 'g11r9', =
State Up, Msg ''</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B></B></SPAN><SPAN =
lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 11:53:53 g10r9 : =
0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: will be contacted =
through 10.2.9.11 </FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
11:53:53 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send new file =
in progress, remote ver 1178991276(64), sending to 10.2.9.11 =
</FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
11:54:01 g10r9 : 0:0:cluster2:NOTICE: urecovery_Interact: send new file =
end, code 0 </FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>May 12 11:54:29 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: =
cl sh cl : status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:58:08 g10r9 : 0:0:nfxsh:NOTICE: cmd[0]: elog find cluster2 : =
status[0]</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:59:10 g10r9 : 0:0:cluster2:NOTICE: ubik:server 10.2.9.11 is back up: =
will be contacted through 10.12.9.11 </FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial size=3D2>May 12 =
11:59:12 g10r9 : 0:0:cluster2:NOTICE: ubik init with buff size 64 =
</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B></B></SPAN><SPAN =
lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 11:59:32 g10r9 : =
0:0:cluster2:ERROR: cluster_get_db_ver: no reply bck -1 =
</FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
11:59:32 g10r9 : 0:0:nfxsh:NOTICE: cmd[1]: gns add root cifs =
gnsroot&nbsp; : status[11]</FONT></B></SPAN><SPAN =
lang=3Den-us><B></B></SPAN><SPAN lang=3Den-us><B><FONT face=3DArial =
size=3D2></FONT></B></SPAN><SPAN lang=3Den-us><B></B></SPAN><SPAN =
lang=3Den-us><B> <FONT face=3DWingdings =
size=3D2>=E8</FONT></B></SPAN><SPAN lang=3Den-us><B></B></SPAN><SPAN =
lang=3Den-us><B><FONT face=3DArial size=3D2> tried this before the DB =
was upgraded</FONT></B></SPAN><SPAN lang=3Den-us><B></B></SPAN><SPAN =
lang=3Den-us><B></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><B><FONT face=3DArial size=3D2>May 12 =
12:00:03 g10r9 : 0:0:spm:ERROR: failed to get filer list from cluster =
daemon </FONT></B></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us><FONT =
face=3DArial size=3D2>May 12 12:00:02 g10r9 newsyslog[23005]: logfile =
turned over</FONT></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us><FONT face=3DArial =
size=3D2>#</FONT></SPAN><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN lang=3Den-us></SPAN></P>=0A=
<P dir=3Dltr><SPAN lang=3Den-us></SPAN><SPAN =
lang=3Den-us></SPAN></P></DIV></BODY></HTML>
------_=_NextPart_001_01C794F7.2FE44638--
