X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C76827.927E0562@onstor-exch02.onstor.net>; Fri, 16 Mar 2007 17:02:37 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-class: urn:content-classes:message
Subject: FW: NFX CPU event - Dogfood
Date: Fri, 16 Mar 2007 17:02:37 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E02E3FC7F@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: NFX CPU event - Dogfood
Thread-Index: AcdoJtoTeMudeKHVQne3/kq8fCYUvQAAHJlQ
X-Priority: 1
Priority: Urgent
Importance: high
From: "Kevin Matthews" <kevin.matthews@onstor.com>
To: "dl-Customer Service Group" <csgroup@onstor.com>,
	"dl-cstech" <dl-cstech@onstor.com>

Partial elog showing the core generated.

This is on the Dogfood node - 10.0.2.2

There are no virtual servers running on it now, but I'm planning on
moving them back over.

Mar 16 16:55:54 Dogfood : 0:0:ncm:WARNING: ncmd :
ncm_handle_net_msg_done: fwd_msg[0x10093f00] flags[1004] sz[2576]
len[2576] agile[0] status[-19] failed
Mar 16 16:55:54 Dogfood : 0:0:ncm:WARNING: ncmd :
ncm_handle_net_msg_done: fwd_msg[0x100a3000] flags[1004] sz[528]
len[528] agile[0] status[-19] failed
Mar 16 16:55:54 Dogfood : 0:0:ncm:WARNING: ncmd :
ncm_handle_net_msg_done: fwd_msg[0x100a3100] flags[1004] sz[528]
len[528] agile[0] status[-19] failed
Mar 16 16:55:54 Dogfood : 0:0:ncm:WARNING: ncmd :
ncm_handle_net_msg_done: fwd_msg[0x100a3200] flags[1004] sz[528]
len[528] agile[0] status[-19] failed
Mar 16 16:55:54 Dogfood : 0:0:ncm:WARNING: ncmd :
ncm_handle_net_msg_done: fwd_msg[0x100a3300] flags[4] sz[528] len[528]
agile[0] status[-19] failed
Mar 16 16:55:54 Dogfood : 1:2:sanm_ag:WARNING: 4: sanm_doRMCLocalClose:
reopening session to sanmd
Mar 16 16:55:54 Dogfood : 0:0:cluster2:INFO: cluster_clientSendRmcRpc:
Error sending rpc to clusterrpc, flags 820a, name elog, rc -20,
retrying...
Mar 16 16:55:56 Dogfood : 0:0:pm:INFO: /usr/local/agile/bin/snmpd:
finished initialization.
Mar 16 16:56:01 Dogfood : 0:0:pm:INFO: /usr/local/agile/bin/sscccc:
finished initialization.
Mar 16 16:56:06 Dogfood : 0:0:spm:WARNING: spm_sendNcmMsg: attempt send
before open completed
Mar 16 16:56:06 Dogfood : 0:0:spm:WARNING: spm_sendNcmMsg: attempt send
before open completed
Mar 16 16:56:10 Dogfood : 0:0:auth_agent:NOTICE: main: auth-agent
started
Mar 16 16:56:10 Dogfood : 0:0:pm:INFO: /usr/local/agile/bin/auth-agent:
finished initialization.
Mar 16 16:56:20 Dogfood : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer:
ncm_open_filer(mktg3) succeeded
Mar 16 16:56:22 Dogfood : 1:2:sanm_ag:WARNING: 5: sanm_doRMCLocalClose:
reopening session to sanmd
Mar 16 16:56:22 Dogfood : 0:0:vsd:INFO: vsd_ensureNisFileCoherence[1220]
: vs=3D1, file=3D71, #masterRec=3D0
Mar 16 16:56:22 Dogfood : 0:0:vsd:INFO: vsd_ensureNisFileCoherence[1220]
: vs=3D1, file=3D72, #masterRec=3D0
Mar 16 16:56:22 Dogfood : 0:0:vsd:INFO: vsd_ensureNisFileCoherence[1220]
: vs=3D1, file=3D74, #masterRec=3D0
Mar 16 16:56:23 Dogfood : 0:0:pm:INFO: /usr/local/agile/bin/vsd:
finished initialization.
Mar 16 16:56:26 Dogfood : 1:0:ipm:NOTICE: 10: vstack_vlink_create:325:
vlink trunk1 link op-state UP
Mar 16 16:56:27 Dogfood : 0:0:cluster2:INFO: ClusterCtrl_releaseFiler:
called by vtm
Mar 16 16:56:28 Dogfood : 0:0:vtm:INFO: vtm_sendCardStateMsg: Sending
card UP to Dogfood
Mar 16 16:56:28 Dogfood : 0:0:pm:INFO: /usr/local/agile/bin/vtmd:
finished initialization.
Mar 16 16:56:29 Dogfood : 0:0:vtm:INFO: Vtm_ProcCardStateMsg: card state
UP message from Dogfood
Mar 16 16:56:30 Dogfood : 0:0:eventd:NOTICE: Process-EVENT IP i/f: IP
192.167.1.1, Port bp0, State Up
Mar 16 16:56:31 Dogfood : 0:0:spm:WARNING: spm_sendNcmMsg: attempt send
before open completed
Mar 16 16:56:31 Dogfood : 0:0:spm:WARNING: spm_sendNcmMsg: attempt send
before open completed
Mar 16 16:56:35 Dogfood : 0:0:ea:INFO: ea_newAcptSess: creating new
session
Mar 16 16:56:35 Dogfood : 0:0:ea:INFO: ea_openSess: open for
{vsd:Agile1843.0.0}
Mar 16 16:56:35 Dogfood : 0:0:auth_agent:INFO:
authAgent_checkForReqTimeouts: 5825: Timedifference is computed to be
greater than 1 sec: 4
Mar 16 16:56:36 Dogfood : 0:0:pm:INFO: /usr/local/agile/bin/sanmd:
finished initialization.
Mar 16 16:56:36 Dogfood : 0:0:sanm:NOTICE: SANM: ONStor Data Mirror
(c)2006: Started
Mar 16 16:56:39 Dogfood : 0:0:pm:INFO:
/usr/local/agile/bin/cluster_relay: finished initialization.
Mar 16 16:56:46 Dogfood : 0:0:eventd:NOTICE: Process-EVENT Vsvr: Virtual
server 'VS_MGMT_258', Id 1, State 'Up'
Mar 16 16:56:46 Dogfood : 0:0:sanm:NOTICE: Processing fail-over event
for vsId[1]
Mar 16 16:56:54 Dogfood : 1:2:efs:INFO: 6: FS: vol_mgmt_258
0x10200000155 - mount - log - replay successfull [Empty Log]
Mar 16 16:56:55 Dogfood : 1:3:efs:INFO: 7: FS: vol_mgmt_258
0x10200000155 - mount - req - Volume online
Mar 16 16:56:55 Dogfood : 0:0:eventd:NOTICE: Process-EVENT Volume: name
'vol_mgmt_258', Id 0x0000010200000155, Event 'Online', was offline for
roughly 1459 s
ec.
Mar 16 16:57:01 Dogfood : 1:3:coredump:INFO: 8: core dump state: on -
volume: core_258
Mar 16 16:57:03 Dogfood : 1:1:nfs:INFO: 11: Finished NLM grace period on
virtual server 1
Mar 16 16:57:10 Dogfood : 0:0:pm:INFO: /usr/local/agile/bin/sendmail:
finished initialization.
Mar 16 16:57:12 Dogfood : 0:0:spm:WARNING: spm_sendNcmMsg: attempt send
before open completed
Mar 16 16:57:12 Dogfood : 0:0:spm:WARNING: spm_sendNcmMsg: attempt send
before open completed
Mar 16 16:57:13 Dogfood : 0:0:ncm:WARNING: ncmd : ncm_reopen_filer:
ncm_open_filer(mktg3) succeeded
Mar 16 16:58:33 Dogfood : 0:0:ssh:INFO: 'root' logged in through remote
host(192.168.212.93)
Mar 16 16:58:50 Dogfood : 0:0:nfxsh:NOTICE: cmd[0]: system show chassis
: status[0]

-----Original Message-----
From: Dogfood@onstor.com [mailto:Dogfood@onstor.com]=20
Sent: Friday, March 16, 2007 4:57 PM
To: Kevin Matthews
Cc: sysadm
Subject: NFX CPU event - Dogfood

Auto support mail:
	NFX Node   : Dogfood
	Event      : CPU
	Application: crashsave
	Message    : /var/crash/1.0
	Time       : Mar 16 16:55:28 2007
-----
-----
THIS MAIL HAS BEEN AUTOMATICALLY GENERATED.
PLEASE DO NOT REPLY.
-----
