AF:
NF:0
PS:10
SRH:1
SFN:
DSR:
MID:<20080505100626.315eadc7@ripper.onstor.net>
CFG:
PT:0
S:andy.sharp@onstor.com
RQ:
SSV:onstor-exch02.onstor.net
NSV:
SSH:
R:<john.keiffer@onstor.com>
MAID:1
X-Sylpheed-Privacy-System:
X-Sylpheed-Sign:0
SCF:#mh/Mailbox/sent
RMID:#imap/andys@onstor.net@onstor-exch02.onstor.net/INBOX	0	BB375AF679D4A34E9CA8DFA650E2B04E09CA5912@onstor-exch02.onstor.net
X-Sylpheed-End-Special-Headers: 1
Date: Mon, 5 May 2008 10:06:35 -0700
From: Andrew Sharp <andy.sharp@onstor.com>
To: "John Keiffer" <john.keiffer@onstor.com>
Subject: Re: oops, it did it again... (go Britney)
Message-ID: <20080505100635.6763e4db@ripper.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E09CA5912@onstor-exch02.onstor.net>
References: <BB375AF679D4A34E9CA8DFA650E2B04E09CA5912@onstor-exch02.onstor.net>
Organization: Onstor
X-Mailer: Sylpheed-Claws 2.6.0 (GTK+ 2.8.20; x86_64-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Grr, OK, I have another fix to try.  I'll spin that one and put it on
there in about 45 minutes.

We'll get some panties on our filer yet.

Cheers,

a

On Mon, 5 May 2008 09:12:47 -0700 "John Keiffer"
<john.keiffer@onstor.com> wrote:

> Hey Andy,
> 
>  
> 
> This morning I removed g5r10 from the g6r10 cluster. Then (a little
> later) I did a 'system reboot -y' on g6r10, and I hit an oops...
> 
>  
> 
> May  5 09:09:06 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:06 g6r10 : 0:0:eventd:CRITICAL: Process-EVENT Node: Name
> 'local', State Down, Msg 'Node going down for reboot! ('system reboot'
> issued from nfxsh).'
> 
> May  5 09:09:06 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:06 g6r10 : 0:0:vtm:DEBUG:
> vtm_get_filer_config_and_start_vsvr_trans: start collecting failover
> vsvr
> 
> May  5 09:09:06 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: nfxsh
> EVENT: Node
> 
> May  5 09:09:06 g6r10 : 0:0:vtm:DEBUG:
> vtm_get_filer_config_and_start_vsvr_trans: end collecting failover
> vsvr
> 
> May  5 09:09:06 g6r10 : 0:0:spm:NOTICE: spm_ncmNodeEvent: Lost connect
> for
> 
> May  5 09:09:07 g6r10 : 0:0:spm:NOTICE: spm_ncmNodeEvent:
> disconnected
> 
> May  5 09:09:07 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: vsvr show all :
> status[0]
> 
> May  5 09:09:07 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 1 : status[0]
> 
> May  5 09:09:07 g6r10 : 1:1:nfs:INFO: 611: Finished failover of NSM
> file after volume event on virtual server 3
> 
> May  5 09:09:08 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 2 : status[0]
> 
> May  5 09:09:08 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 3 : status[0]
> 
> May  5 09:09:08 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 4 : status[0]
> 
> May  5 09:09:08 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 6 : status[0]
> 
> May  5 09:09:09 g6r10 : 1:2:efs:INFO: 15004: FS: win-vol-g6r10
> 0x74000000069 - unmount - req - Unmounting volume
> 
> May  5 09:09:09 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 7 : status[0]
> 
> May  5 09:09:09 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:09 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'win-vol-g6r10', Id 0x0000074000000069, Event 'Offline', State 'Down'
> 
> May  5 09:09:09 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:09 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:09 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 8 : status[0]
> 
> May  5 09:09:09 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 9 : status[0]
> 
> May  5 09:09:09 g6r10 : 1:1:nfs:INFO: 612: Finished failover of NSM
> file after volume event on virtual server 7
> 
> May  5 09:09:10 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 10 : status[0]
> 
> May  5 09:09:10 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 11 : status[0]
> 
> May  5 09:09:10 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx kpi
> get_summary 16 : status[0]
> 
> May  5 09:09:11 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: stats txrx io sizes :
> status[0]
> 
> May  5 09:09:11 g6r10 : 1:2:efs:INFO: 15005: FS: openldap-vol-g6r10
> 0x7400000006e - unmount - req - Unmounting volume
> 
> May  5 09:09:11 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:11 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'openldap-vol-g6r10', Id 0x000007400000006e, Event 'Offline', State
> 'Down'
> 
> May  5 09:09:11 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:11 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:12 g6r10 : 1:1:nfs:INFO: 613: Finished failover of NSM
> file after volume event on virtual server 2
> 
> May  5 09:09:12 g6r10 : 0:0:nfxsh:DEBUG: cmd[0]: volume show :
> status[0]
> 
> May  5 09:09:13 g6r10 : 1:2:efs:INFO: 15006: FS: nis-vol-g6r10
> 0x7400000012b - unmount - req - Unmounting volume
> 
> May  5 09:09:13 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:13 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'nis-vol-g6r10', Id 0x000007400000012b, Event 'Offline', State 'Down'
> 
> May  5 09:09:13 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:13 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:14 g6r10 : 1:1:nfs:INFO: 614: Finished failover of NSM
> file after volume event on virtual server 10
> 
> May  5 09:09:14 g6r10 : 1:2:efs:INFO: 15007: FS: vol1-g5r10
> 0x10757000001c6 - unmount - req - Unmounting volume
> 
> May  5 09:09:15 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:15 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'vol1-g5r10', Id 0x00010757000001c6, Event 'Offline', State 'Down'
> 
> May  5 09:09:15 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:15 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:15 g6r10 : 1:1:nfs:INFO: 615: Finished failover of NSM
> file after volume event on virtual server 10
> 
> May  5 09:09:16 g6r10 : 1:2:efs:INFO: 15008: FS: vol2-g5r10
> 0x10757000001c7 - unmount - req - Unmounting volume
> 
> May  5 09:09:16 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:16 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'vol2-g5r10', Id 0x00010757000001c7, Event 'Offline', State 'Down'
> 
> May  5 09:09:17 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:17 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:17 g6r10 : 1:1:nfs:INFO: 616: Finished failover of NSM
> file after volume event on virtual server 10
> 
> May  5 09:09:18 g6r10 : 1:2:efs:INFO: 15009: FS: vol3-g5r10
> 0x10757000001c8 - unmount - req - Unmounting volume
> 
> May  5 09:09:18 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:18 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'vol3-g5r10', Id 0x00010757000001c8, Event 'Offline', State 'Down'
> 
> May  5 09:09:19 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:19 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:20 g6r10 : 1:2:efs:INFO: 15010: FS: local-win-target
> 0x74000000108 - unmount - req - Unmounting volume
> 
> May  5 09:09:20 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:20 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'local-win-target', Id 0x0000074000000108, Event 'Offline', State
> 'Down'
> 
> May  5 09:09:20 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:20 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:20 g6r10 : 1:1:nfs:INFO: 617: Finished failover of NSM
> file after volume event on virtual server 10
> 
> May  5 09:09:21 g6r10 : 1:2:efs:INFO: 15011: FS: vol4-g5r10
> 0x10757000001c9 - unmount - req - Unmounting volume
> 
> May  5 09:09:22 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:22 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'vol4-g5r10', Id 0x00010757000001c9, Event 'Offline', State 'Down'
> 
> May  5 09:09:22 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:22 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:23 g6r10 : 1:3:efs:INFO: 15012: FS: jk-source2
> 0x740000000bb - unmount - req - Unmounting volume
> 
> May  5 09:09:24 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:24 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'jk-source2', Id 0x00000740000000bb, Event 'Offline', State 'Down'
> 
> May  5 09:09:24 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:24 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:25 g6r10 : 1:1:nfs:INFO: 618: Finished failover of NSM
> file after volume event on virtual server 16
> 
> May  5 09:09:31 g6r10 : 1:2:efs:INFO: 15013: FS: jk-source1
> 0x740000000ba - unmount - req - Unmounting volume
> 
> May  5 09:09:31 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:31 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'jk-source1', Id 0x00000740000000ba, Event 'Offline', State 'Down'
> 
> May  5 09:09:31 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:31 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:31 g6r10 : 1:1:nfs:INFO: 619: Finished failover of NSM
> file after volume event on virtual server 6
> 
> May  5 09:09:33 g6r10 : 1:2:efs:INFO: 15014: FS: local-nis-vol-g6r10
> 0x7400000006d - unmount - req - Unmounting volume
> 
> May  5 09:09:33 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:33 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'local-nis-vol-g6r10', Id 0x000007400000006d, Event 'Offline', State
> 'Down'
> 
> May  5 09:09:33 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:33 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:33 g6r10 : 1:1:nfs:INFO: 620: Finished failover of NSM
> file after volume event on virtual server 8
> 
> May  5 09:09:34 g6r10 : 1:2:efs:INFO: 15015: FS: authdom-vol-g6r10
> 0x7400000006f - unmount - req - Unmounting volume
> 
> May  5 09:09:34 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:34 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'authdom-vol-g6r10', Id 0x000007400000006f, Event 'Offline', State
> 'Down'
> 
> May  5 09:09:35 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:35 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:35 g6r10 : 1:1:nfs:INFO: 621: Finished failover of NSM
> file after volume event on virtual server 4
> 
> May  5 09:09:36 g6r10 : 1:2:efs:INFO: 15016: FS:
> 255_Characters_Super_long_name_length_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> 22222222222221111111111111111111111111111111111111111111111xxxxxxxxxxxxx
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx55555555555555555555_123456789
> _987654321_123456789_987654321_12345678 0x7400000006c - unmount - req
> - Unmounting volume
> 
> May  5 09:09:36 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:36 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> '255_Characters_Super_long_name_length_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> x22222222222221111111111111111111111111111111111111111111111xxxxxxxxxxxx
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx55555555555555555555_12345678
> 9_987654321_123456789_987654321_12345678', Id 0x000007400000006c,
> Event 'Offline', State 'Down'
> 
> May  5 09:09:36 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:36 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:37 g6r10 : 1:1:nfs:INFO: 622: Finished failover of NSM
> file after volume event on virtual server 9
> 
> May  5 09:09:38 g6r10 : 1:2:efs:INFO: 15017: FS: nonetbios-vol-g6r10
> 0x74000000070 - unmount - req - Unmounting volume
> 
> May  5 09:09:38 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:38 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'nonetbios-vol-g6r10', Id 0x0000074000000070, Event 'Offline', State
> 'Down'
> 
> May  5 09:09:38 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:38 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:39 g6r10 : 1:2:efs:INFO: 15018: FS: 225_Char_mirror
> 0x740000001c5 - unmount - req - Unmounting volume
> 
> May  5 09:09:40 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:40 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> '225_Char_mirror', Id 0x00000740000001c5, Event 'Offline', State
> 'Down'
> 
> May  5 09:09:40 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:40 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:40 g6r10 : 1:1:nfs:INFO: 623: Finished failover of NSM
> file after volume event on virtual server 1
> 
> May  5 09:09:41 g6r10 : 1:2:efs:INFO: 15019: FS: vol_mgmt_1856
> 0x7400000010b - unmount - req - Unmounting volume
> 
> May  5 09:09:41 g6r10 : 0:0:eventd:DEBUG: > ems_logEvent()
> 
> May  5 09:09:41 g6r10 : 0:0:eventd:NOTICE: Process-EVENT Volume: name
> 'vol_mgmt_1856', Id 0x000007400000010b, Event 'Offline', State 'Down'
> 
> May  5 09:09:42 g6r10 : 0:0:eventd:DEBUG: < ems_logEvent()
> 
> May  5 09:09:42 g6r10 : 0:0:asd:DEBUG: Rcvd post request APP: evm
> EVENT: Volume
> 
> May  5 09:09:42 g6r10 : 0:0:pm:ERROR: pm_sig_handler: got SIGTERM
> 
> Data bus error, epc == ffffffff8218afe4, ra == ffffffff8204f614
> 
> Oops[#1]:
> 
> Cpu 0
> 
> $ 0   : 0000000000000000 0000000010001fe1 ffffffffffffffff
> 9000000041001000
> 
> $ 4   : 0000000000000039 a8000000049bc800 0200000000000000
> 0000000000000000
> 
> $ 8   : a8000000049bc800 9000000000000000 ffffffff822f019c
> d04f32062ba8360f
> 
> $12   : 0000000010001fe0 000000001000001f 0000000074656e6f
> d005080000000004
> 
> $16   : a80000000490e520 0000000000000000 0000000000000039
> 0000000000000000
> 
> $20   : 0000000000000001 900000f81a7084a0 0000000000000000
> 000000002ae8f440
> 
> $24   : 0000000000000005 ffffffff82195ec0
> 
> $28   : a80000008d7b4000 a80000008d7b7a30 0000000000472be8
> ffffffff8204f614
> 
> Hi    : 0000000000000000
> 
> Lo    : 0000000000001398
> 
> epc   : ffffffff8218afe4 yenta_interrupt+0x14/0x118     Not tainted
> 
> ra    : ffffffff8204f614 handle_IRQ_event+0x6c/0xe8
> 
> Status: 10001fe3    KX SX UX KERNEL EXL IE
> 
> Cause : 0080841c
> 
> PrId  : 00040103
> 
> Modules linked in: autofs4
> 
> Process pm (pid: 1023, threadinfo=a80000008d7b4000,
> task=a80000008e7a6d18)
> 
> Stack : ffffffff8204f614 0000000000000800 ffffffff822b6878
> 0000000000000039
> 
>         a80000000490e520 fffffffffffffbff ffffffff82310000
> ffffffff8204f764
> 
>         0200000000000000 a80000008e0df83e a8000000891a8e28
> 900000008f0012e0
> 
>         900000008f004b40 ffffffff820011a4 a80000008d7b7eb0
> ffffffff82001840
> 
>         0000000000000000 0000000000000008 900000f81a5a2700
> 00000000005a2700
> 
>         900000f81a5a2780 a80000008e0df8c8 0000000000000005
> 0000000000000000
> 
>         ff800803000a4002 62df80030000a249 0a08010100002f5c
> d04f32062ba8360f
> 
>         424d53ff29000000 ffffffff8212e6b4 0000000074656e6f
> d005080000000004
> 
>         000000000000008d a80000008e0df83e a8000000891a8e28
> 900000008f0012e0
> 
>         900000008f004b40 900000f81a7084a0 0000000000000000
> 000000002ae8f440
> 
>         ...
> 
> Call Trace:
> 
> [<ffffffff8218afe4>] yenta_interrupt+0x14/0x118
> 
> [<ffffffff8204f614>] handle_IRQ_event+0x6c/0xe8
> 
> [<ffffffff8204f764>] __do_IRQ+0xd4/0x160
> 
> [<ffffffff820011a4>] plat_irq_dispatch+0x1e4/0x1f0
> 
> [<ffffffff82001840>] ret_from_irq+0x0/0x4
> 
> [<ffffffff8212e378>] less_than_4units+0x14/0x48
> 
> [<ffffffff82195fdc>] mgmtbus_hard_start_xmit+0x11c/0x188
> 
> [<ffffffff821a9eb4>] dev_queue_xmit+0x30c/0x458
> 
> [<ffffffff82220164>] relay_hard_start_xmit+0x164/0x1e0
> 
> [<ffffffff821a9eb4>] dev_queue_xmit+0x30c/0x458
> 
> [<ffffffff821c88ec>] ip_queue_xmit+0x21c/0x438
> 
> [<ffffffff821dc0e0>] tcp_transmit_skb+0x500/0xa10
> 
> [<ffffffff821dd4c0>] tcp_retransmit_skb+0x100/0x778
> 
> [<ffffffff821e09a4>] tcp_write_timer+0x3cc/0x7b8
> 
> [<ffffffff8202efe4>] run_timer_softirq+0x164/0x258
> 
> [<ffffffff8202a4d4>] __do_softirq+0x94/0x140
> 
> [<ffffffff8202a610>] do_softirq+0x90/0x98
> 
> [<ffffffff82001840>] ret_from_irq+0x0/0x4
> 
>  
> 
>  
> 
> Code: ffbf0000  dca30010  8c620000 <0040202d> ac620000  dca60010
> 8cc30000  90c20804  1480001f
> 
> Kernel panic - not syncing: Fatal exception in interrupt
> 
> Rebooting in 5 seconds..<2>SiByte Watchdog in danger of initiating
> system reset in 7.8 seconds
> 
>  
> 
>  
> 
>  
> 
> PowerOn Self Test........OK
> 
>  
> 
> Initializing System......please wait
> 
>  
> 
> Thank you,
> 
> John Keiffer
> 
>  
> 
>  
> 
