X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C7D09C.92348521@onstor-exch02.onstor.net>; Fri, 27 Jul 2007 14:22:08 -0800
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C7D09C.92348521"
Content-class: urn:content-classes:message
Subject: RE: SMP FC locking - version 3
Date: Fri, 27 Jul 2007 14:22:08 -0800
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E018A69D3@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E04AFA695@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: SMP FC locking - version 3
Thread-Index: AcfLI2WbSODYorRwRPa2C6RVMJ0hMQAA2evwAAAZjjAAACfg0AD5asJAAAGbSSAAJVV+kAA8eXmQ
From: "Jeff Miller" <jeff.miller@onstor.com>
To: "Maxim Kozlovsky" <maxim.kozlovsky@onstor.com>,
	"dl-Cougar" <dl-Cougar@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C7D09C.92348521
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I think this will work - basically option two of the previous version
which was preferred by Bill and Max.  My only problem with it was how to
implement the new message queue - existing link fields can be used and
the RMC session can be fetched from the scsiCtrl structure, so it seems
ok.  I added notes on the timers.


The general idea with moving the FC code to the FP is just remove the
network hardware and drivers. Using existing features of the EEE
forwarding code, a send from the FP ends up just calling the recieve
function that had been on the FC. The EEE code is modified to consider
the FC slot the same as the FP slot.  The FC polling functions are just
moved to the FP.

This is all fine except the FC was a single processor but the FP is SMP,
two CPUs on Cheetah/Bobcat and soon four on Cougar.  The general idea
for SMP-izing the FC code is that FC ports are not tied to CPUs but that
they owned by any available CPU only for the period time that the port
needs attention from a CPU.  So there has to be some sort of locking on
the hardware and related data.

Three types of locks are needed:  port message receive queue lock, FC
port lock, and scsiCtrl lock.

Messages are queued to a per port FIFO message queue by the message
receive functions, both scsi_eeeReceiveMsg() and scsi_rmcReceiveMsg().
A per port spin lock protects the message queue while messages are added
or removed.  The receive functions only queue the messages - they do not
process them.  All messages are removed and processed by a polling
function.  The messages are a mixture of RMC messages and edescriptors,
so the queuing code needs to be able to handle both types.  Existing
link fields in RMC messages and edescriptors are used for queueing.

A second set of per port spin locks protects port control structure and
hardware access.  The polling functions use trylock - if the lock is
acquired the port is checked, otherwise return without doing anything.
The timer functions use spinlock and will wait to gain access to the
port.  If this turns out to have significant performance impact then an
alternative is to have the timer functions do nothing except note that
some operation should be done by a polling function.

The third lock is a single lock on the global scsiCtrl structure.


Change details:
Add port spin lock to ispfc_cb_t structure.
Add port queue spin lock to ispfc_cb_t structure.
Add global spinlock to scsiCtrl structure.

Change message receive to only queue messages (using port queue lock).

Add polling function to remove message from receive queue (acquiring
first port lock and then queue lock).  Call scsi_receiveMsg() using
edesc or rmc message depending on message type.  RMC session is taken
from scsiCtrl.  Queue lock is released after removing message, port lock
is kept until return.

do NFX_SPIN_TRYLOCK() at beginning of polling functions, do nothing
(don't wait) if lock not acquired.
Release lock at function returns.

Lock port with trylock in these polling routines:
ispfc_ct_desc_start_handler()
ispfc_completion_handler()
ispfc_link_management_handler()
ispfc_device_management_handler()
scsi_test_daemon()

Lock port with spinlock in these timer routines:
scsi_devO_procGenIOTimer in devO_sm_funcs.c
scsi_devO_procWriteGateTimer in devO_sm_funcs.c
scsi_genIOQueueTimerHandler in scsi-ll.c
scsi_test_timer in scsi-test.c
scsi_gatherStats in scsi.c  Uses each dev on each port.  Lock ports
while accessing their devices.
scsi_ack_timeout_handler in scsi.c
scsi_exch_timeout_handler in scsi.c
scsi_req_timeout_handler in scsi-msg.c

Lock scsiCtrl in these timer routines:
scsi_sdm_poll_handler in scsi.c
scsi_msg_q_timer_handler in scsi.c


_____________________________________________
From: Maxim Kozlovsky=20
Sent: Thursday, July 26, 2007 10:29 AM
To: Jeff Miller; dl-Cougar
Subject: RE: SMP FC locking - version 2

Some additional work that needs to be done:

The SCSI code creates some timers that receive pointers to internal
structures as arguments. These structures are addressed directly from
the timer handler. With SMP this will not work for obvious reasons.=20

_____________________________________________
From: Maxim Kozlovsky=20
Sent: Wednesday, July 25, 2007 4:40 PM
To: Jeff Miller; dl-Cougar
Subject: RE: SMP FC locking - version 2

Observe that you still have to solve the problem of putting an unlimited
number of messages which can be both RMC and EEE on the device queue in
the first case, so we can strike out the Disadvantages clause in the
second case.

I like the second choice because it so much simpler, there are just 8
locks and 4 queues and most of the code can remain unchanged. There is
no point in trying to achieve greater parallelism if it does not provide
you greater performance.

_____________________________________________
From: Jeff Miller=20
Sent: Wednesday, July 25, 2007 4:21 PM
To: Maxim Kozlovsky; dl-Cougar
Subject: RE: SMP FC locking - version 2

Here is a variation on the first idea that solves the delete device
problem.

Add a lock per handle per port.  This is a lock for each possible
device.  The lock must be acquired for any device access including
deleting the device.=20
scsi_getDevice() locks the port-handle, returns device if it exists
(keeping lock), otherwise unlocks port-handle.  Lock must be released
after device processing.
Message is queued to device queue.
Try to acquire port lock, if available then dispatch to QLogic.  If port
not available and no previous I/O queued, then put device on 'needs
attention' queue.

Add polling of 'needs attention' queue.

Only need locks for handles up to SCSI_MAX_DEV_PER_PORT (2000), not
maximum for the 12 bit field (4096).

Advantages:
Messages are queued using a per device lock rather than per port.
Somewhat less latency starting I/O in certain cases since sender can
initiate I/O.

Disadvantages:
Device lock per port per handle uses about 500k memory.
Need to add polling for inactive devices with new I/Os.


A second possibility:
Add a per port message queue that replaces the current network queue.
Spinlock per port protects queue.  Messages are taken off this queue by
polling function after obtaining a per port lock.=20

Advantages:
Simple code change except maybe for new message queue implementation.

Disadvantages:
The new message queue implementation is not so straight forward.  Needs
to handle any number of messages.  Messages are edesc or RMC. Could add
fields to each, or maybe there are existing fields that could be used.

I don't like this as much.

_____________________________________________
From: Maxim Kozlovsky=20
Sent: Friday, July 20, 2007 4:48 PM
To: Jeff Miller; dl-Cougar
Subject: RE: SMP FC locking

Not exactly, because to be able to use the pointer to the device
structure you have to have a lock which keeps the device structure from
being deleted, or otherwise made unable to accept the scsi descriptors.
This lock can be only a port lock. You can use a reference count to
prevent the device from being deleted, but then again to find the device
and increment the reference count you need to use the port lock.

_____________________________________________
From: Jeff Miller=20
Sent: Friday, July 20, 2007 4:43 PM
To: Maxim Kozlovsky; dl-Cougar
Subject: RE: SMP FC locking

Getting the device lock lets you queue an I/O even while another CPU
owns the port.  You don't have to wait for the port lock, which may be
unavailable for a while.  The device lock is only ever kept for a very
short time.

_____________________________________________
From: Maxim Kozlovsky=20
Sent: Friday, July 20, 2007 4:39 PM
To: Jeff Miller; dl-Cougar
Subject: RE: SMP FC locking

What exactly does the device lock buy you? Wouldn't you already have the
port lock anyway by the time you have to get the device lock in most of
the cases?=20

_____________________________________________
From: Jeff Miller=20
Sent: Friday, July 20, 2007 4:12 PM
To: dl-Cougar
Subject: SMP FC locking

This is what I am thinking for a SMP FC.  Any comments?
Thanks,
Jeff

MP locking strategy
Lock port during port control structure or hardware access.  A FC port
is not permanantly tied to a particular CPU as in Cheetah, but is owned
by a CPU only for the period time that it needs attention from a CPU. =20

Lock device only to protect device queues while adding or removing scsi
descriptors.  All other device fields are protected by the port lock.

A CPU may have to wait (in spin lock) for another CPU, but only for a
very short time while a scsi descriptor is being linked or unlinked.
The lock for the port is held for a much longer time, but CPUs do not
wait at all for the port lock. =20


Some assumptions:
A device is associated with only one port.


Changes
Add port spin lock to ispfc_cb_t structure.
do NFX_SPIN_TRYLOCK() at beginning of polling functions and before
issuing command to hardware in receive function, do nothing (don't wait)
if lock not acquired.
Release lock at function returns.

Add device spin lock to scsi_dev_t structure.  Add lock/unlock only
around code that manipulates the device queues.  Do lock and unlock
within loops that handle multiple scsi descriptors.

Polling functions that need to lock port:
ispfc_ct_desc_start_handler()
ispfc_completion_handler()
ispfc_link_management_handler()
ispfc_device_management_handler()
scsi_test_daemon() ?

Port also needs to be locked for calls to scsi_sendNextCmd().

Add device lock only to protect device queues. All other device fields
are protected by the port locks.

These functions manipulate the device queues and need to use the device
lock:
scsi_devO_procWriteGateTimer()
scsi_sendNextCmd()
scsi_flushReqQ()
scsi_addToReqQ()

------_=_NextPart_001_01C7D09C.92348521
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7652.24">
<TITLE>RE: SMP FC locking - version 3</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">I think this will work</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">&#8211;</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial"></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">basically</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial"> option =
two of the previous version which was</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">preferred</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial"></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">by Bill =
and Max.&nbsp; My only problem with it was how to implement the new =
message queue</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">&#8211;</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial"> =
existing</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> =
<FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">link =
fields</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> =
<FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">can be used and the RMC =
session can be fetched from the</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">scsiCtrl structure</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">, so it seems ok.</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">&nbsp; I added notes on the =
timers.</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">The general idea with moving the FC code to the FP is =
just remove the network hardware and drivers. Using existing features of =
the EEE forwarding code, a send from the FP ends up just calling the =
recieve function that had been on the FC. The EEE code is modified to =
consider the FC slot the same as the FP slot.&nbsp; The FC polling =
functions are just moved to the FP.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">This is all fine except the FC was a single processor but =
the FP is SMP, two CPUs on Cheetah/Bobcat and soon four on Cougar.&nbsp; =
The general idea for SMP-izing the FC code is that FC ports are not tied =
to CPUs but that they owned by any available CPU only for the period =
time that the port needs attention from a CPU.&nbsp; So there has to be =
some sort of locking on the hardware and related data.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Three types of locks are needed:&nbsp; port message =
receive queue lock, FC port lock, and scsiCtrl lock.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Messages are queued to a per port FIFO message queue by =
the message receive functions, both scsi_eeeReceiveMsg() and =
scsi_rmcReceiveMsg().&nbsp; A per port spin lock protects the message =
queue while messages are added or removed.&nbsp; The receive functions =
only queue the messages - they do not process them.&nbsp; All messages =
are removed and processed by a polling function.&nbsp; The messages are =
a mixture of RMC messages and edescriptors, so the queuing code needs to =
be able to handle both types.&nbsp; Existing link fields in RMC messages =
and edescriptors are used for queueing.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">A second set of per port spin locks protects port control =
structure and hardware access.&nbsp; The polling functions use trylock - =
if the lock is acquired the port is checked, otherwise return without =
doing anything.&nbsp; The timer functions use spinlock and will wait to =
gain access to the port.&nbsp; If this turns out to have significant =
performance impact then an alternative is to have the timer functions do =
nothing except note that some operation should be done by a polling =
function.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">The third lock is a single lock on the global scsiCtrl =
structure.</FONT></SPAN></P>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Change details:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add port spin lock to ispfc_cb_t =
structure.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add port queue spin lock to ispfc_cb_t =
structure.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add global spinlock to scsiCtrl =
structure.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Change message receive to only queue messages (using port =
queue lock).</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add polling function to remove message from receive queue =
(acquiring first port lock and then queue lock).&nbsp; Call =
scsi_receiveMsg() using ede</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">s</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">c or rmc =
message depending on message type.&nbsp; RMC session is taken from =
scsiCtrl.&nbsp; Queue lock is released after removing message, port lock =
is kept until return.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">do NFX_SPIN_TRYLOCK() at beginning of polling functions, =
do nothing (don't wait) if lock not acquired.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Release lock at function returns.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Lock port with trylock in these polling =
routines:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">ispfc_ct_desc_start_handler()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">ispfc_completion_handler()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">ispfc_link_management_handler()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">ispfc_device_management_handler()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_test_daemon()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Lock port with spinlock in these timer =
routines:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_devO_procGenIOTimer in =
devO_sm_funcs.c</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_devO_procWriteGateTimer in =
devO_sm_funcs.c</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_genIOQueueTimerHandler in =
scsi-ll.c</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_test_timer in scsi-test.c</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_gatherStats in scsi.c&nbsp; Uses each dev on each =
port.&nbsp; Lock ports while accessing their devices.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_ack_timeout_handler in scsi.c</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_exch_timeout_handler in scsi.c</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_req_timeout_handler in scsi-msg.c</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Lock scsiCtrl in these timer routines:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_sdm_poll_handler in scsi.c</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_msg_q_timer_handler in scsi.c</FONT></SPAN></P>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Thursday, July 26, 2007 =
10:29 AM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking - version 2</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Some additional work that needs to be =
done:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">The SCSI code creates some timers that receive pointers =
to internal structures as arguments. These structures are addressed =
directly from the timer handler. With SMP this will not work for obvious =
reasons. </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Wednesday, July 25, 2007 =
4:40 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking - version 2</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Observe that you still have to solve the problem of =
putting an unlimited number of messages which can be both RMC and EEE on =
the device queue in the first case, so we can strike out the =
Disadvantages clause in the second case.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">I like the second choice because it so much simpler, =
there are just 8 locks and 4 queues and most of the code can remain =
unchanged. There is no point in trying to achieve greater parallelism if =
it does not provide you greater performance.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Wednesday, July 25, 2007 =
4:21 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking - version 2</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Here is a variation on the first idea that solves the =
delete device problem.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add a lock per handle per port.&nbsp; This is a lock for =
each possible device.&nbsp; The lock must be acquired for any device =
access including deleting the device. </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_getDevice() locks the port-handle, returns device if =
it exists (keeping lock), otherwise unlocks port-handle.&nbsp; Lock must =
be released after device processing.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Message is queued to device queue.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Try to acquire port lock, if available then dispatch to =
QLogic.&nbsp; If port not available and no previous I/O queued, then put =
device on 'needs attention' queue.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add polling of 'needs attention' queue.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Only need locks for handles up to SCSI_MAX_DEV_PER_PORT =
(2000), not maximum for the 12 bit field (4096).</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Advantages:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Messages are queued using a per device lock rather than =
per port.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Somewhat less latency starting I/O in certain cases since =
sender can initiate I/O.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Disadvantages:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Device lock per port per handle uses about 500k =
memory.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Need to add polling for inactive devices with new =
I/Os.</FONT></SPAN></P>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">A second possibility:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add a per port message queue that replaces the current =
network queue. Spinlock per port protects queue.&nbsp; Messages are =
taken off this queue by polling function after obtaining a per port =
lock. </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Advantages:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Simple code change except maybe for new message queue =
implementation.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Disadvantages:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">The new message queue implementation is not so straight =
forward.&nbsp; Needs to handle any number of messages.&nbsp; Messages =
are edesc or RMC. Could add fields to each, or maybe there are existing =
fields that could be used.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">I don&#8217;t like this as much.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:48 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Not exactly, because to be able to use the pointer to the =
device structure you have to have a lock which keeps the device =
structure from being deleted, or otherwise made unable to accept the =
scsi descriptors. This lock can be only a port lock. You can use a =
reference count to prevent the device from being deleted, but then again =
to find the device and increment the reference count you need to use the =
port lock.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:43 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Getting the device lock lets you queue an I/O even while =
another CPU owns the port.&nbsp; You don&#8217;t have to wait for the =
port lock, which may be unavailable for a while.&nbsp; The device lock =
is only ever kept for a very short time.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:39 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">What exactly does the device lock buy you? Wouldn&#8217;t =
you already have the port lock anyway by the time you have to get the =
device lock in most of the cases? </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:12 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">This =
is what I am thinking for a SMP FC.&nbsp; Any =
comments?</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Thanks,</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Jeff</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">MP =
locking strategy</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Lock =
port during port control structure or hardware access.&nbsp; A FC port =
is not permanantly tied to a particular CPU as in Cheetah, but is owned =
by a CPU only for the period time that it needs attention from a =
CPU.&nbsp; </FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Lock =
device only to protect device queues while adding or removing scsi =
descriptors.&nbsp; All other device fields are protected by the port =
lock.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">A CPU =
may have to wait (in spin lock) for another CPU, but only for a very =
short time while a scsi descriptor is being linked or unlinked.&nbsp; =
The lock for the port is held for a much longer time, but CPUs do not =
wait at all for the port lock.&nbsp; </FONT></SPAN></P>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Some =
assumptions:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">A =
device is associated with only one port.</FONT></SPAN></P>
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Changes</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add =
port spin lock to ispfc_cb_t structure.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">do =
NFX_SPIN_TRYLOCK() at beginning of polling functions and before issuing =
command to hardware in receive function, do nothing (don't wait) if lock =
not acquired.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Release lock at function returns.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add =
device spin lock to scsi_dev_t structure.&nbsp; Add lock/unlock only =
around code that manipulates the device queues.&nbsp; Do lock and unlock =
within loops that handle multiple scsi descriptors.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Polling functions that need to lock =
port:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_ct_desc_start_handler()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_completion_handler()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_link_management_handler()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_device_management_handler()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_test_daemon() ?</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Port =
also needs to be locked for calls to =
scsi_sendNextCmd().</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add =
device lock only to protect device queues. All other device fields are =
protected by the port locks.</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">These =
functions manipulate the device queues and need to use the device =
lock:</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_devO_procWriteGateTimer()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_sendNextCmd()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_flushReqQ()</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_addToReqQ()</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

</BODY>
</HTML>
------_=_NextPart_001_01C7D09C.92348521--
