X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C898D8.ECF79848@onstor-exch02.onstor.net>; Mon, 7 Apr 2008 10:58:03 -0700
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C898D8.ECF79848"
Content-class: urn:content-classes:message
Subject: RE: why eventd sends 'Node going down for reboot' 3 times in a row?
Date: Mon, 7 Apr 2008 10:58:03 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E03B1C175@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E05C74662@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: why eventd sends 'Node going down for reboot' 3 times in a row?
Thread-Index: AciY1+MCc6Bgjl4eSP6AHMY/SKBDSQAANqgw
From: "Rendell Fong" <rendell.fong@onstor.com>
To: "Sandrine Boulanger" <sandrine.boulanger@onstor.com>,
	"Andy Sharp" <andy.sharp@onstor.com>,
	"Larry Scheer" <larry.scheer@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C898D8.ECF79848
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

The chassis daemon is checking every TXRX/FP cpu and sending the node
event multiple times since multiple cpus are down.  I suppose once is
enough.=20

_____________________________________________
From: Sandrine Boulanger=20
Sent: Monday, April 07, 2008 10:51 AM
To: Andy Sharp; Larry Scheer; Rendell Fong
Subject: why eventd sends 'Node going down for reboot' 3 times in a row?

I don't know yet if vol create triggered the crash, but I don't
understand why eventd sends 3 times that the node is going down for
reboot (I got 3 autosupport emails too)

Apr  7 09:45:26 g14r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'vol_5', Id 0x000107540000012f, Event 'Create', State 'Down'
Apr  7 09:45:30 g14r10 : 0:0:eventd:WARNING: Process-EVENT CPU: Slot 1,
CPU 0, State Down
Apr  7 09:45:30 g14r10 : 0:0:eventd:WARNING: Process-EVENT CPU: Slot 1,
CPU 1, State Down
Apr  7 09:45:30 g14r10 : 0:0:vtm:NOTICE: Vtm_ProcEvntMsg: NFX_EVENT_CPU,
state 2, slot 1, cpu 0
Apr  7 09:45:30 g14r10 : 0:0:vtm:INFO: vtm_sendCardStateMsg: Sending
card DOWN to g14r10
Apr  7 09:45:31 g14r10 : 0:0:vtm:NOTICE: Vtm_ProcEvntMsg: NFX_EVENT_CPU,
state 2, slot 1, cpu 1
Apr  7 09:45:31 g14r10 : 0:0:vtm:INFO: Vtm_ProcCardStateMsg: card state
DOWN message from g14r10
Apr  7 09:45:31 g14r10 : 0:0:auth_agent:INFO: uninitCifsd:
Uninitializing CIFS daemon
Apr  7 09:45:31 g14r10 : 0:0:auth_agent:INFO: killCifsd[213]: Killing
CIFS daemon (pid=3D5204) for VS=3D2[G14R10-1] intentionally.
alreadyRebooting=3D0, force=3D0
Apr  7 09:45:31 g14r10 : 0:0:eventd:WARNING: Process-EVENT CPU: Slot 1,
CPU 2, State Down
Apr  7 09:45:32 g14r10 : 0:0:evm:ERROR: evm_sendCreateLvReq : Timeout
creating run time instance of volume vol_5
Apr  7 09:45:32 g14r10 : 0:0:evm:ERROR: evm_onlineEvmSrvr : Cannot
create LV instance on FP for volume[vol_5].
Apr  7 09:45:32 g14r10 : 0:0:evm:ERROR: evm_putVolOnline: Cannot create
run time instance of volume[vol_5]
Apr  7 09:45:32 g14r10 : 0:0:ea:ERROR: ea_evmCreateVol[145]: Internal
EVM error. Operation failed on volume[vol_5].
Apr  7 09:45:32 g14r10 : 0:0:sanm:ERROR: SANM: FP NIM down. Aborting all
mirror sessions.
Apr  7 09:45:32 g14r10 : 0:0:vtm:NOTICE: Vtm_ProcEvntMsg: NFX_EVENT_CPU,
state 2, slot 1, cpu 2
Apr  7 09:45:32 g14r10 : 0:0:eventd:WARNING: Process-EVENT CPU: Slot 1,
CPU 3, State Down
Apr  7 09:45:32 g14r10 : 0:0:sanm:ERROR: SANM: FP NIM down. Aborting all
mirror sessions.
Apr  7 09:45:32 g14r10 : 0:0:vtm:NOTICE: Vtm_ProcEvntMsg: NFX_EVENT_CPU,
state 2, slot 1, cpu 3
Apr  7 09:45:32 g14r10 : 0:0:eventd:WARNING: Process-EVENT CPU: Slot 1,
CPU 4, State Down
Apr  7 09:45:32 g14r10 : 0:0:vtm:NOTICE: Vtm_ProcEvntMsg: NFX_EVENT_CPU,
state 2, slot 1, cpu 4
Apr  7 09:45:32 g14r10 : 0:0:eventd:WARNING: Process-EVENT CPU: Slot 1,
CPU 5, State Down
Apr  7 09:45:32 g14r10 : 0:0:vtm:NOTICE: Vtm_ProcEvntMsg: NFX_EVENT_CPU,
state 2, slot 1, cpu 5
Apr  7 09:45:32 g14r10 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.1.1, Port bp0, State Remove
Apr  7 09:45:32 g14r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'vol_mgmt_67412', Id 0x000107540000006d, Event 'Offline', State 'Down'
Apr  7 09:45:32 g14r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'sand', Id 0x0000075400000068, Event 'Offline', State 'Down'
Apr  7 09:45:32 g14r10 : 0:0:eventd:WARNING: Process-EVENT Vsvr: Virtual
server 'VS_MGMT_67412', Id 1, State 'Down'
Apr  7 09:45:33 g14r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'sand2-m', Id 0x0001075400000099, Event 'Offline', State 'Down'
Apr  7 09:45:33 g14r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'sand1', Id 0x000107540000006e, Event 'Offline', State 'Down'
Apr  7 09:45:33 g14r10 : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.2.1, Port bp0, State Remove
Apr  7 09:45:33 g14r10 : 0:0:ea:INFO: Error string (Internal EVM error.
Operation failed on volume[vol_5].) len (54)
Apr  7 09:45:33 g14r10 : 0:0:eventd:WARNING: Process-EVENT Vsvr: Virtual
server 'G14R10-1', Id 2, State 'Down'
Apr  7 09:45:33 g14r10 : 0:0:nfxsh:NOTICE: cmd[16]: vol create vol_5
3PARdata_0565_000900565 : status[11]
Apr  7 09:45:34 g14r10 : 0:0:vtm:ERROR: vtm_gatherInfo_proc: gather info
failed for request from g14r10
Apr  7 09:45:35 g14r10 : 0:0:vtm:ERROR: vtm_failover_vsvr_proc: no
alternative filer found for vs 2 failover
Apr  7 09:45:35 g14r10 : 0:0:vtm:ERROR: vtm_failover_vsvr_proc: no
alternative filer found for vs 3 failover
Apr  7 09:46:02 g14r10 : 0:0:eventd:NOTICE: handleMessageFailure: Msg
failure (-19) for session 0x4add20 app  unused 0. Closing connection.
Apr  7 09:46:02 g14r10 : 0:0:eventd:INFO: close_session: Lost
connection. Close rmc session 0x4add20 for event session (nil) app
unused 0.
Apr  7 09:46:02 g14r10 : 0:0:eventd:NOTICE: handleMessageFailure: Msg
failure (-19) for session 0x4ae278 app  unused 0. Closing connection.
Apr  7 09:46:02 g14r10 : 0:0:eventd:INFO: close_session: Lost
connection. Close rmc session 0x4ae278 for event session (nil) app
unused 0.
Apr  7 09:46:03 g14r10 : 0:0:sdm:INFO: sdm_rcvRmcMsg:RMC Msg type [4]
{sdm:Agile1301}->{:scsi2.2.0} msg flags[1004] msg_id[221] msg_seq[11028]
status[-19] rc[-20]
Apr  7 09:48:36 g14r10 : 0:0:evm:ERROR: evm_readVolumeLabelReq: Unable
to read primary label for LUN[3PARdata_0565_000900565]
Apr  7 09:48:36 g14r10 : 0:0:evm:ERROR: evm_deleteVol: Cannot read label
for LUN[3PARdata_0565_000900565]
Apr  7 09:48:36 g14r10 : 0:0:evm:ERROR: evm_volCreateReqProc: Delete
failed[4294961669] for volume[vol_5]
Apr  7 10:09:24 g14r10 : 0:0:cm:WARNING: TXRX/FP module is down,
rebooting system.
Apr  7 10:09:24 g14r10 : 0:0:eventd:CRITICAL: Process-EVENT Node: Name
'local', State Down, Msg 'Node going down for reboot! (chassisd: because
FP/TXRX module is down).'
Apr  7 10:09:25 g14r10 : 0:0:spm:NOTICE: spm_ncmNodeEvent: Lost connect
for
Apr  7 10:09:25 g14r10 : 0:0:spm:NOTICE: spm_ncmNodeEvent:  disconnected

Broadcast message from root@g14r10 (Mon Apr  7 10:09:26 2008):

The system is going down for reboot NOW!
Apr  7 10:09:27 g14r10 : 0:0:eventd:CRITICAL: Process-EVENT Node: Name
'local', State Down, Msg 'Node going down for reboot! (chassisd: because
FP/TXRX module is down).'
Apr  7 10:09:28 g14r10 : 0:0:spm:NOTICE: spm_ncmNodeEvent: Lost connect
for
Apr  7 10:09:28 g14r10 : 0:0:spm:NOTICE: spm_ncmNodeEvent:  disconnected
Apr  7 10:09:28 g14r10 : 0:0:cm:WARNING: TXRX/FP module is down,
rebooting system.
Apr  7 10:09:28 g14r10 : 0:0:eventd:INFO: close_session: Lost
connection. Close rmc session 0x4aa4d0 for event session (nil) app
unused 0.

Broadcast message from root@g14r10 (Mon Apr  7 10:09:29 2008):

The system is going down for reboot NOW!
Apr  7 10:09:32 g14r10 : 0:0:eventd:CRITICAL: Process-EVENT Node: Name
'local', State Down, Msg 'Node going down for reboot! (chassisd: because
FP/TXRX module is down).'
Apr  7 10:09:33 g14r10 : 0:0:spm:NOTICE: spm_ncmNodeEvent: Lost connect
for
Connection to 10.2.10.14 closed by remote host.
Connection to 10.2.10.14 closed.
[sandrineb@sandrineb tst-evm]$

------_=_NextPart_001_01C898D8.ECF79848
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7653.38">
<TITLE>RE: why eventd sends 'Node going down for reboot' 3 times in a =
row?</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">The chassis daemon is checking every TXRX/FP =
cpu</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial"> and sending the node event =
multiple times since multiple cpus are down</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">.</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">&nbsp; I =
suppose once is enough.</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial"></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> </SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Sandrine Boulanger<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Monday, April 07, 2008 =
10:51 AM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Andy Sharp; Larry Scheer; =
Rendell Fong<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> why eventd sends 'Node going down for reboot' 3 times =
in a row?</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">I =
don&#8217;t know yet if vol create triggered the crash, but I =
don&#8217;t understand why eventd sends 3 times that the node is going =
down for reboot (I got 3 autosupport emails too)</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:26 g14r10 : 0:0:eventd:NOTICE: =
Process-EVENT Volume: name 'vol_5', Id 0x000107540000012f, Event =
'Create', State 'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:30 g14r10 : 0:0:eventd:WARNING: =
Process-EVENT CPU: Slot 1, CPU 0, State Down</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:30 g14r10 : 0:0:eventd:WARNING: =
Process-EVENT CPU: Slot 1, CPU 1, State Down</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:30 g14r10 : 0:0:vtm:NOTICE: =
Vtm_ProcEvntMsg: NFX_EVENT_CPU, state 2, slot 1, cpu 0</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:30 g14r10 : 0:0:vtm:INFO: =
vtm_sendCardStateMsg: Sending card DOWN to g14r10</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:31 g14r10 : 0:0:vtm:NOTICE: =
Vtm_ProcEvntMsg: NFX_EVENT_CPU, state 2, slot 1, cpu 1</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:31 g14r10 : 0:0:vtm:INFO: =
Vtm_ProcCardStateMsg: card state DOWN message from =
g14r10</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:31 g14r10 : 0:0:auth_agent:INFO: =
uninitCifsd: Uninitializing CIFS daemon</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:31 g14r10 : 0:0:auth_agent:INFO: =
killCifsd[213]: Killing CIFS daemon (pid=3D5204) for VS=3D2[G14R10-1] =
intentionally. alreadyRebooting=3D0, force=3D0</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:31 g14r10 : 0:0:eventd:WARNING: =
Process-EVENT CPU: Slot 1, CPU 2, State Down</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:evm:ERROR: =
evm_sendCreateLvReq : Timeout creating run time instance of volume =
vol_5</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:evm:ERROR: =
evm_onlineEvmSrvr : Cannot create LV instance on FP for =
volume[vol_5].</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:evm:ERROR: =
evm_putVolOnline: Cannot create run time instance of =
volume[vol_5]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:ea:ERROR: =
ea_evmCreateVol[145]: Internal EVM error. Operation failed on =
volume[vol_5].</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:sanm:ERROR: SANM: FP =
NIM down. Aborting all mirror sessions.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:vtm:NOTICE: =
Vtm_ProcEvntMsg: NFX_EVENT_CPU, state 2, slot 1, cpu 2</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:eventd:WARNING: =
Process-EVENT CPU: Slot 1, CPU 3, State Down</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:sanm:ERROR: SANM: FP =
NIM down. Aborting all mirror sessions.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:vtm:NOTICE: =
Vtm_ProcEvntMsg: NFX_EVENT_CPU, state 2, slot 1, cpu 3</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:eventd:WARNING: =
Process-EVENT CPU: Slot 1, CPU 4, State Down</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:vtm:NOTICE: =
Vtm_ProcEvntMsg: NFX_EVENT_CPU, state 2, slot 1, cpu 4</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:eventd:WARNING: =
Process-EVENT CPU: Slot 1, CPU 5, State Down</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:vtm:NOTICE: =
Vtm_ProcEvntMsg: NFX_EVENT_CPU, state 2, slot 1, cpu 5</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:eventd:NOTICE: =
Process-EVENT IP i/f: IP 192.167.1.1, Port bp0, State =
Remove</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:eventd:NOTICE: =
Process-EVENT Volume: name 'vol_mgmt_67412', Id 0x000107540000006d, =
Event 'Offline', State 'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:eventd:NOTICE: =
Process-EVENT Volume: name 'sand', Id 0x0000075400000068, Event =
'Offline', State 'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:32 g14r10 : 0:0:eventd:WARNING: =
Process-EVENT Vsvr: Virtual server 'VS_MGMT_67412', Id 1, State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:33 g14r10 : 0:0:eventd:NOTICE: =
Process-EVENT Volume: name 'sand2-m', Id 0x0001075400000099, Event =
'Offline', State 'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:33 g14r10 : 0:0:eventd:NOTICE: =
Process-EVENT Volume: name 'sand1', Id 0x000107540000006e, Event =
'Offline', State 'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:33 g14r10 : 0:0:eventd:NOTICE: =
Process-EVENT IP i/f: IP 192.167.2.1, Port bp0, State =
Remove</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:33 g14r10 : 0:0:ea:INFO: Error string =
(Internal EVM error. Operation failed on volume[vol_5].) len =
(54)</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:33 g14r10 : 0:0:eventd:WARNING: =
Process-EVENT Vsvr: Virtual server 'G14R10-1', Id 2, State =
'Down'</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:33 g14r10 : 0:0:nfxsh:NOTICE: cmd[16]: =
vol create vol_5 3PARdata_0565_000900565 : status[11]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:34 g14r10 : 0:0:vtm:ERROR: =
vtm_gatherInfo_proc: gather info failed for request from =
g14r10</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:35 g14r10 : 0:0:vtm:ERROR: =
vtm_failover_vsvr_proc: no alternative filer found for vs 2 =
failover</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:45:35 g14r10 : 0:0:vtm:ERROR: =
vtm_failover_vsvr_proc: no alternative filer found for vs 3 =
failover</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:46:02 g14r10 : 0:0:eventd:NOTICE: =
handleMessageFailure: Msg failure (-19) for session 0x4add20 app&nbsp; =
unused 0. Closing connection.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:46:02 g14r10 : 0:0:eventd:INFO: =
close_session: Lost connection. Close rmc session 0x4add20 for event =
session (nil) app unused 0.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:46:02 g14r10 : 0:0:eventd:NOTICE: =
handleMessageFailure: Msg failure (-19) for session 0x4ae278 app&nbsp; =
unused 0. Closing connection.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:46:02 g14r10 : 0:0:eventd:INFO: =
close_session: Lost connection. Close rmc session 0x4ae278 for event =
session (nil) app unused 0.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:46:03 g14r10 : 0:0:sdm:INFO: =
sdm_rcvRmcMsg:RMC Msg type [4] {sdm:Agile1301}-&gt;{:scsi2.2.0} msg =
flags[1004] msg_id[221] msg_seq[11028] status[-19] =
rc[-20]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:48:36 g14r10 : 0:0:evm:ERROR: =
evm_readVolumeLabelReq: Unable to read primary label for =
LUN[3PARdata_0565_000900565]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:48:36 g14r10 : 0:0:evm:ERROR: =
evm_deleteVol: Cannot read label for =
LUN[3PARdata_0565_000900565]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 09:48:36 g14r10 : 0:0:evm:ERROR: =
evm_volCreateReqProc: Delete failed[4294961669] for =
volume[vol_5]</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:24 g14r10 : 0:0:cm:WARNING: TXRX/FP =
module is down, rebooting system.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:24 g14r10 : 0:0:eventd:CRITICAL: =
Process-EVENT Node: Name 'local', State Down, Msg 'Node going down for =
reboot! (chassisd: because FP/TXRX module is =
down).'</FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:25 g14r10 : 0:0:spm:NOTICE: =
spm_ncmNodeEvent: Lost connect for</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:25 g14r10 : 0:0:spm:NOTICE: =
spm_ncmNodeEvent:&nbsp; disconnected</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Broadcast message from root@g14r10 (Mon Apr&nbsp; 7 =
10:09:26 2008):</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">The =
system is going down for reboot NOW!</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:27 g14r10 : 0:0:eventd:CRITICAL: =
Process-EVENT Node: Name 'local', State Down, Msg 'Node going down for =
reboot! (chassisd: because FP/TXRX module is =
down).'</FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:28 g14r10 : 0:0:spm:NOTICE: =
spm_ncmNodeEvent: Lost connect for</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:28 g14r10 : 0:0:spm:NOTICE: =
spm_ncmNodeEvent:&nbsp; disconnected</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:28 g14r10 : 0:0:cm:WARNING: TXRX/FP =
module is down, rebooting system.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:28 g14r10 : 0:0:eventd:INFO: =
close_session: Lost connection. Close rmc session 0x4aa4d0 for event =
session (nil) app unused 0.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Broadcast message from root@g14r10 (Mon Apr&nbsp; 7 =
10:09:29 2008):</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">The =
system is going down for reboot NOW!</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:32 g14r10 : 0:0:eventd:CRITICAL: =
Process-EVENT Node: Name 'local', State Down, Msg 'Node going down for =
reboot! (chassisd: because FP/TXRX module is =
down).'</FONT></B></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Apr&nbsp; 7 10:09:33 g14r10 : 0:0:spm:NOTICE: =
spm_ncmNodeEvent: Lost connect for</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Connection to 10.2.10.14 closed by remote =
host.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Connection to 10.2.10.14 closed.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">[sandrineb@sandrineb tst-evm]$</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

</BODY>
</HTML>
------_=_NextPart_001_01C898D8.ECF79848--
