X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C7CF15.256E7223@onstor-exch02.onstor.net>; Wed, 25 Jul 2007 15:40:13 -0800
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C7CF15.256E7223"
Content-class: urn:content-classes:message
Subject: RE: SMP FC locking - version 2
Date: Wed, 25 Jul 2007 15:40:13 -0800
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E04AFA3BA@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E018A69D2@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: SMP FC locking - version 2
Thread-Index: AcfLI2WbSODYorRwRPa2C6RVMJ0hMQAA2evwAAAZjjAAACfg0AD5asJAAAGbSSA=
References: <BB375AF679D4A34E9CA8DFA650E2B04E049DE811@onstor-exch02.onstor.net> <BB375AF679D4A34E9CA8DFA650E2B04E018A69D2@onstor-exch02.onstor.net>
From: "Maxim Kozlovsky" <maxim.kozlovsky@onstor.com>
To: "Jeff Miller" <jeff.miller@onstor.com>,
	"dl-Cougar" <dl-Cougar@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C7CF15.256E7223
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Observe that you still have to solve the problem of putting an unlimited
number of messages which can be both RMC and EEE on the device queue in
the first case, so we can strike out the Disadvantages clause in the
second case.

I like the second choice because it so much simpler, there are just 8
locks and 4 queues and most of the code can remain unchanged. There is
no point in trying to achieve greater parallelism if it does not provide
you greater performance.

_____________________________________________
From: Jeff Miller=20
Sent: Wednesday, July 25, 2007 4:21 PM
To: Maxim Kozlovsky; dl-Cougar
Subject: RE: SMP FC locking - version 2

Here is a variation on the first idea that solves the delete device
problem.

Add a lock per handle per port.  This is a lock for each possible
device.  The lock must be acquired for any device access including
deleting the device.=20
scsi_getDevice() locks the port-handle, returns device if it exists
(keeping lock), otherwise unlocks port-handle.  Lock must be released
after device processing.
Message is queued to device queue.
Try to acquire port lock, if available then dispatch to QLogic.  If port
not available and no previous I/O queued, then put device on 'needs
attention' queue.

Add polling of 'needs attention' queue.

Only need locks for handles up to SCSI_MAX_DEV_PER_PORT (2000), not
maximum for the 12 bit field (4096).

Advantages:
Messages are queued using a per device lock rather than per port.
Somewhat less latency starting I/O in certain cases since sender can
initiate I/O.

Disadvantages:
Device lock per port per handle uses about 500k memory.
Need to add polling for inactive devices with new I/Os.


A second possibility:
Add a per port message queue that replaces the current network queue.
Spinlock per port protects queue.  Messages are taken off this queue by
polling function after obtaining a per port lock.=20

Advantages:
Simple code change except maybe for new message queue implementation.

Disadvantages:
The new message queue implementation is not so straight forward.  Needs
to handle any number of messages.  Messages are edesc or RMC. Could add
fields to each, or maybe there are existing fields that could be used.

I don't like this as much.

_____________________________________________
From: Maxim Kozlovsky=20
Sent: Friday, July 20, 2007 4:48 PM
To: Jeff Miller; dl-Cougar
Subject: RE: SMP FC locking

Not exactly, because to be able to use the pointer to the device
structure you have to have a lock which keeps the device structure from
being deleted, or otherwise made unable to accept the scsi descriptors.
This lock can be only a port lock. You can use a reference count to
prevent the device from being deleted, but then again to find the device
and increment the reference count you need to use the port lock.

_____________________________________________
From: Jeff Miller=20
Sent: Friday, July 20, 2007 4:43 PM
To: Maxim Kozlovsky; dl-Cougar
Subject: RE: SMP FC locking

Getting the device lock lets you queue an I/O even while another CPU
owns the port.  You don't have to wait for the port lock, which may be
unavailable for a while.  The device lock is only ever kept for a very
short time.

_____________________________________________
From: Maxim Kozlovsky=20
Sent: Friday, July 20, 2007 4:39 PM
To: Jeff Miller; dl-Cougar
Subject: RE: SMP FC locking

What exactly does the device lock buy you? Wouldn't you already have the
port lock anyway by the time you have to get the device lock in most of
the cases?=20

_____________________________________________
From: Jeff Miller=20
Sent: Friday, July 20, 2007 4:12 PM
To: dl-Cougar
Subject: SMP FC locking

This is what I am thinking for a SMP FC.  Any comments?
Thanks,
Jeff

MP locking strategy
Lock port during port control structure or hardware access.  A FC port
is not permanantly tied to a particular CPU as in Cheetah, but is owned
by a CPU only for the period time that it needs attention from a CPU. =20

Lock device only to protect device queues while adding or removing scsi
descriptors.  All other device fields are protected by the port lock.

A CPU may have to wait (in spin lock) for another CPU, but only for a
very short time while a scsi descriptor is being linked or unlinked.
The lock for the port is held for a much longer time, but CPUs do not
wait at all for the port lock. =20


Some assumptions:
A device is associated with only one port.


Changes
Add port spin lock to ispfc_cb_t structure.
do NFX_SPIN_TRYLOCK() at beginning of polling functions and before
issuing command to hardware in receive function, do nothing (don't wait)
if lock not acquired.
Release lock at function returns.

Add device spin lock to scsi_dev_t structure.  Add lock/unlock only
around code that manipulates the device queues.  Do lock and unlock
within loops that handle multiple scsi descriptors.

Polling functions that need to lock port:
ispfc_ct_desc_start_handler()
ispfc_completion_handler()
ispfc_link_management_handler()
ispfc_device_management_handler()
scsi_test_daemon() ?

Port also needs to be locked for calls to scsi_sendNextCmd().

Add device lock only to protect device queues. All other device fields
are protected by the port locks.

These functions manipulate the device queues and need to use the device
lock:
scsi_devO_procWriteGateTimer()
scsi_sendNextCmd()
scsi_flushReqQ()
scsi_addToReqQ()

------_=_NextPart_001_01C7CF15.256E7223
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7652.24">
<TITLE>RE: SMP FC locking - version 2</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">Observe that you still have to =
solve the problem of putting an unlimited number of messages which can =
be both RMC and EEE on the device queue in</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">the</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial"></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">first =
case, so we can strike out the Disadvantages clause in the second =
case.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">I like the second choice =
because it so much simpler, there are just 8 locks and 4 =
queues</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial"> and most =
of the code can remain unchanged</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">. There is no point</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial"> in</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">trying to achieve greater parallelism if it does =
not provide you greater performance</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">.</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Wednesday, July 25, 2007 =
4:21 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking - version 2</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">Here is a variation on the =
first idea that solves the delete device problem.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add a lock per handle per port.&nbsp; This is a lock for =
each possible device.&nbsp; The lock must be acquired for any device =
access including deleting the device. </FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">scsi_getDevice() locks the port-handle, returns device if =
it exists (keeping lock), otherwise unlocks port-handle.&nbsp; Lock must =
be released after device processing.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Message is queued to device queue.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Try to acquire port lock, if available then dispatch to =
QLogic.&nbsp; If port not available and no previous I/O queued, then put =
device on 'needs attention' queue.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add polling of 'needs attention' queue.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Only need locks for handles up to SCSI_MAX_DEV_PER_PORT =
(2000), not maximum for the 12 bit field (4096).</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Advantages:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Messages are queued using a per device lock rather than =
per port.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Somewhat less latency starting I/O in certain cases since =
sender can initiate I/O.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Disadvantages:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Device lock per port per handle uses about 500k =
memory.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Need to add polling for inactive devices with new =
I/Os.</FONT></SPAN></P>
<BR>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">A second possibility:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Add a per port message queue that replaces the current =
network queue. Spinlock per port protects queue.&nbsp; Messages are =
taken off this queue by polling function after obtaining a per port =
lock. </FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Advantages:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Simple code change except maybe for new message queue =
implementation.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">Disadvantages:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">The new message queue implementation is not so straight =
forward.&nbsp; Needs to handle any number of messages.&nbsp; Messages =
are edesc or RMC. Could add fields to each, or maybe there are existing =
fields that could be used.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">I don&#8217;t like this as much.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:48 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">Not exactly, because to be =
able to use the pointer to the device structure you have to have a lock =
which keeps the device structure from being deleted, or otherwise made =
unable to accept the scsi descriptors. This lock can be only a port =
lock. You can use a reference count to prevent the device from being =
deleted, but then again to find the device and increment the reference =
count you need to use the port lock.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:43 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">Getting the device lock lets =
you queue an I/O even while another CPU owns the port.&nbsp; You =
don&#8217;t have to wait for the port lock, which may be unavailable for =
a while.&nbsp; The device lock is only ever kept for a very short =
time.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:39 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">What exactly does the device =
lock buy you? Wouldn&#8217;t you already have the port lock anyway by =
the time you have to get the device lock in most of the cases? =
</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:12 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 FACE=3D"Arial">This is what I am thinking for a SMP FC.&nbsp; =
Any comments?</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Thanks,</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Jeff</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">MP =
locking strategy</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Lock =
port during port control structure or hardware access.&nbsp; A FC port =
is not permanantly tied to a particular CPU as in Cheetah, but is owned =
by a CPU only for the period time that it needs attention from a =
CPU.&nbsp; </FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Lock =
device only to protect device queues while adding or removing scsi =
descriptors.&nbsp; All other device fields are protected by the port =
lock.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">A CPU =
may have to wait (in spin lock) for another CPU, but only for a very =
short time while a scsi descriptor is being linked or unlinked.&nbsp; =
The lock for the port is held for a much longer time, but CPUs do not =
wait at all for the port lock.&nbsp; </FONT></SPAN></P>
<BR>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Some =
assumptions:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">A device =
is associated with only one port.</FONT></SPAN></P>
<BR>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Changes</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add port =
spin lock to ispfc_cb_t structure.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">do =
NFX_SPIN_TRYLOCK() at beginning of polling functions and before issuing =
command to hardware in receive function, do nothing (don't wait) if lock =
not acquired.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Release =
lock at function returns.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add =
device spin lock to scsi_dev_t structure.&nbsp; Add lock/unlock only =
around code that manipulates the device queues.&nbsp; Do lock and unlock =
within loops that handle multiple scsi descriptors.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Polling =
functions that need to lock port:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_ct_desc_start_handler()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_completion_handler()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_link_management_handler()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_device_management_handler()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_test_daemon() ?</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Port =
also needs to be locked for calls to =
scsi_sendNextCmd().</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add =
device lock only to protect device queues. All other device fields are =
protected by the port locks.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">These =
functions manipulate the device queues and need to use the device =
lock:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_devO_procWriteGateTimer()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_sendNextCmd()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_flushReqQ()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_addToReqQ()</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

</BODY>
</HTML>
------_=_NextPart_001_01C7CF15.256E7223--
