X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C83C3F.F937333F@onstor-exch02.onstor.net>; Tue, 11 Dec 2007 13:51:24 -0800
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C83C3F.F937333F"
Content-class: urn:content-classes:message
Subject: cougar todo
Date: Tue, 11 Dec 2007 13:51:23 -0800
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E07099682@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: cougar todo
Thread-Index: Acg8P/kMvGHpexMIT+qq/u5avqK1Mg==
From: "Maxim Kozlovsky" <maxim.kozlovsky@onstor.com>
To: "dl-Cougar" <dl-Cougar@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C83C3F.F937333F
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

If anybody looking for things to do, here is something:

We get the following crash because of spinlock recursion when the EVM
has requests waiting in the pending queue and tries to send one of those
requests when a response from scsi is received. This causes spinlock
recursion because the scsi code is called again. To fix this the
recursion must be broken somewhere, for example by using of an
intermediate queue of responses to send which is processed without
holding ifpsc spinlock.
=20
#0  spinlock_error (lockp=3D0x1002a80710, format=3D0xffffffff8349dc98
"deadlock in spinlock_getlock: lk=3D%p owner=3D%p ra=3D%p cpu=3D%d\n") =
at
spinlock-api.c:78
#1  0x83050238 in spinlock_getlock (lockp=3D0x1002a80710) at
spinlock-api.c:223
#2  0x8344b598 in ispfc_ioctl (code=3D4114, cb=3D0x1002a60000) at
ispfc_ioctl.c:666
#3  0x83468aa0 in scsi_receiveMsg (edesc=3D0x4004125900, rmcMsg=3D0x0,
sess=3D0x0) at scsi-msg.c:301
#4  0x8347bb80 in scsi_eeeReceiveMsg (edesc=3D0x4004125900) at
scsi-msg.c:7274
#5  0x83011eec in eee_deliverPacketToApp (edesc=3D0x4004125900, =
app_id=3D7)
at eee-fwd.c:2005
#6  0x830123f8 in eee_forwardPacket (edesc=3D0x4004125900) at
eee-fwd.c:2280
#7  0x832df93c in evm_sendScsiMsg (qEle=3D0x1002ec8240, =
lcb=3D0x1002ec8240,
pcb=3D0x1002ec83b0, retry=3Dfalse) at evm-msg.c:293
#8  0x832d0d08 in evm_io_sendPioReq (qEle=3D0x1002ec8240,
lcb=3D0x1002ec8240, pcb=3D0x1002ec83b0) at evm-io.c:784
#9  0x832d5c94 in evm_io_scsiRsp (msgp=3D0x401a84fa48 "\001\004",
msgLen=3D8248, srcPort=3D135175) at evm-io.c:2703
#10 0x832c9094 in evm_rcvMsg (edesc=3D0x400411c380) at evm-api.c:321
#11 0x83011eec in eee_deliverPacketToApp (edesc=3D0x400411c380, =
app_id=3D6)
at eee-fwd.c:2005
#12 0x830123f8 in eee_forwardPacket (edesc=3D0x400411c380) at
eee-fwd.c:2280
#13 0x83473768 in scsi_sendReadRspWithData (sd=3D0x10029aec60, =
eCode=3D0,
csum=3D0, transSeqId=3D0, dataOffset=3D0x0) at scsi-msg.c:4419
#14 0x8347aeb4 in scsi_sendDeviceRspData (pdev=3D0x101f1ba000,
sd=3D0x10029aec60, smpd=3D0x0, ecode=3D0, total_csum=3D0) at =
scsi-msg.c:6819
#15 0x8347a648 in scsi_processDeviceRsp (pdev=3D0x101f1ba000,
sd=3D0x10029aec60) at scsi-msg.c:6574
#16 0x834675fc in scsi_llReceive (sd=3D0x10029aec60) at scsi-ll.c:279
#17 0x8345ac04 in ispfc_dispose_of_scsi_desc (cb=3D0x1002a60000,
dev=3D0x1000a04000, sd=3D0x10029aec60, retry=3D0) at ispfc_scsi.c:308
#18 0x8344860c in ispfc_process_iocb_completions (cb=3D0x1002a60000,
response_throttle=3D16) at ispfc_iocb.c:1430
#19 0x83432f2c in ispfc_completion_handler (cb=3D0x1002a60000,
timer_ref=3D2) at ispfc.c:823
#20 0x83016588 in eee_poll (num_loops=3D60) at eee-poll.c:551
#21 0x8304d8a4 in getchar () at serio-api.c:333
#22 0x830420e4 in get_line (p=3D0xffffffff864f2f08 "", usehist=3D1) at
hist.c:145
#23 0x83042830 in get_input (p=3D0xffffffff864f2f08 "") at hist.c:259
#24 0x83042868 in get_cmd (p=3D0xffffffff864f2f08 "") at hist.c:284
#25 0x8304c29c in runtime_prompt () at test.c:554
#26 0x8304c1c0 in _main () at test.c:537

------_=_NextPart_001_01C83C3F.F937333F
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7652.24">
<TITLE>cougar todo</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 FACE=3D"Arial">If a</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">nybody looking for things to</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial"></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT SIZE=3D2 FACE=3D"Arial">do</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">, here is something:</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">We get =
the following crash because of spinlock recursion =
when</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> =
<FONT SIZE=3D2 FACE=3D"Arial">the EVM has requests waiting in the =
pending queue and tries to send one of those requests when a response =
from scsi is received. This causes spinlock recursion because the scsi =
code is called again.</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT SIZE=3D2 FACE=3D"Arial">To fix this the recursion =
must be broken somewhere, for example by using of an intermediate queue =
of response</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">s</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial"> to send which is processed without</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT SIZE=3D2 =
FACE=3D"Arial">holding</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT SIZE=3D2 FACE=3D"Arial">ifpsc</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial"> spinlock.</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 FACE=3D"Arial"></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us">&nbsp;</SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 FACE=3D"Arial">#0&nbsp; spinlock_error (lockp=3D0x1002a80710, =
format=3D0xffffffff8349dc98 &quot;deadlock in spinlock_getlock: lk=3D%p =
owner=3D%p ra=3D%p cpu=3D%d\n&quot;) at =
spinlock-api.c:78</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#1&nbsp; =
0x83050238 in spinlock_getlock (lockp=3D0x1002a80710) at =
spinlock-api.c:223</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#2&nbsp; =
0x8344b598 in ispfc_ioctl (code=3D4114, cb=3D0x1002a60000) at =
ispfc_ioctl.c:666</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#3&nbsp; =
0x83468aa0 in scsi_receiveMsg (edesc=3D0x4004125900, rmcMsg=3D0x0, =
sess=3D0x0) at scsi-msg.c:301</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#4&nbsp; =
0x8347bb80 in scsi_eeeReceiveMsg (edesc=3D0x4004125900) at =
scsi-msg.c:7274</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#5&nbsp; =
0x83011eec in eee_deliverPacketToApp (edesc=3D0x4004125900, app_id=3D7) =
at eee-fwd.c:2005</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#6&nbsp; =
0x830123f8 in eee_forwardPacket (edesc=3D0x4004125900) at =
eee-fwd.c:2280</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#7&nbsp; =
0x832df93c in evm_sendScsiMsg (qEle=3D0x1002ec8240, lcb=3D0x1002ec8240, =
pcb=3D0x1002ec83b0, retry=3Dfalse) at evm-msg.c:293</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#8&nbsp; =
0x832d0d08 in evm_io_sendPioReq (qEle=3D0x1002ec8240, =
lcb=3D0x1002ec8240, pcb=3D0x1002ec83b0) at =
evm-io.c:784</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#9&nbsp; =
0x832d5c94 in evm_io_scsiRsp (msgp=3D0x401a84fa48 &quot;\001\004&quot;, =
msgLen=3D8248, srcPort=3D135175) at evm-io.c:2703</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#10 =
0x832c9094 in evm_rcvMsg (edesc=3D0x400411c380) at =
evm-api.c:321</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#11 =
0x83011eec in eee_deliverPacketToApp (edesc=3D0x400411c380, app_id=3D6) =
at eee-fwd.c:2005</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#12 =
0x830123f8 in eee_forwardPacket (edesc=3D0x400411c380) at =
eee-fwd.c:2280</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#13 =
0x83473768 in scsi_sendReadRspWithData (sd=3D0x10029aec60, eCode=3D0, =
csum=3D0, transSeqId=3D0, dataOffset=3D0x0) at =
scsi-msg.c:4419</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#14 =
0x8347aeb4 in scsi_sendDeviceRspData (pdev=3D0x101f1ba000, =
sd=3D0x10029aec60, smpd=3D0x0, ecode=3D0, total_csum=3D0) at =
scsi-msg.c:6819</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#15 =
0x8347a648 in scsi_processDeviceRsp (pdev=3D0x101f1ba000, =
sd=3D0x10029aec60) at scsi-msg.c:6574</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#16 =
0x834675fc in scsi_llReceive (sd=3D0x10029aec60) at =
scsi-ll.c:279</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#17 =
0x8345ac04 in ispfc_dispose_of_scsi_desc (cb=3D0x1002a60000, =
dev=3D0x1000a04000, sd=3D0x10029aec60, retry=3D0) at =
ispfc_scsi.c:308</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#18 =
0x8344860c in ispfc_process_iocb_completions (cb=3D0x1002a60000, =
response_throttle=3D16) at ispfc_iocb.c:1430</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#19 =
0x83432f2c in ispfc_completion_handler (cb=3D0x1002a60000, =
timer_ref=3D2) at ispfc.c:823</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#20 =
0x83016588 in eee_poll (num_loops=3D60) at =
eee-poll.c:551</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#21 =
0x8304d8a4 in getchar () at serio-api.c:333</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#22 =
0x830420e4 in get_line (p=3D0xffffffff864f2f08 &quot;&quot;, =
usehist=3D1) at hist.c:145</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#23 =
0x83042830 in get_input (p=3D0xffffffff864f2f08 &quot;&quot;) at =
hist.c:259</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#24 =
0x83042868 in get_cmd (p=3D0xffffffff864f2f08 &quot;&quot;) at =
hist.c:284</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#25 =
0x8304c29c in runtime_prompt () at test.c:554</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">#26 =
0x8304c1c0 in _main () at test.c:537</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

</BODY>
</HTML>
------_=_NextPart_001_01C83C3F.F937333F--
