X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C7CB27.215709DB@onstor-exch02.onstor.net>; Fri, 20 Jul 2007 15:38:52 -0800
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C7CB27.215709DB"
Content-class: urn:content-classes:message
Subject: RE: SMP FC locking
Date: Fri, 20 Jul 2007 15:38:50 -0800
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E049DE7FA@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E018A69C8@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: SMP FC locking
Thread-Index: AcfLI2WbSODYorRwRPa2C6RVMJ0hMQAA2evw
References: <BB375AF679D4A34E9CA8DFA650E2B04E018A69C8@onstor-exch02.onstor.net>
From: "Maxim Kozlovsky" <maxim.kozlovsky@onstor.com>
To: "Jeff Miller" <jeff.miller@onstor.com>,
	"dl-Cougar" <dl-Cougar@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C7CB27.215709DB
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

What exactly does the device lock buy you? Wouldn't you already have the
port lock anyway by the time you have to get the device lock in most of
the cases?=20

_____________________________________________
From: Jeff Miller=20
Sent: Friday, July 20, 2007 4:12 PM
To: dl-Cougar
Subject: SMP FC locking

This is what I am thinking for a SMP FC.  Any comments?
Thanks,
Jeff

MP locking strategy
Lock port during port control structure or hardware access.  A FC port
is not permanantly tied to a particular CPU as in Cheetah, but is owned
by a CPU only for the period time that it needs attention from a CPU. =20

Lock device only to protect device queues while adding or removing scsi
descriptors.  All other device fields are protected by the port lock.

A CPU may have to wait (in spin lock) for another CPU, but only for a
very short time while a scsi descriptor is being linked or unlinked.
The lock for the port is held for a much longer time, but CPUs do not
wait at all for the port lock. =20


Some assumptions:
A device is associated with only one port.


Changes
Add port spin lock to ispfc_cb_t structure.
do NFX_SPIN_TRYLOCK() at beginning of polling functions and before
issuing command to hardware in receive function, do nothing (don't wait)
if lock not acquired.
Release lock at function returns.

Add device spin lock to scsi_dev_t structure.  Add lock/unlock only
around code that manipulates the device queues.  Do lock and unlock
within loops that handle multiple scsi descriptors.

Polling functions that need to lock port:
ispfc_ct_desc_start_handler()
ispfc_completion_handler()
ispfc_link_management_handler()
ispfc_device_management_handler()
scsi_test_daemon() ?

Port also needs to be locked for calls to scsi_sendNextCmd().

Add device lock only to protect device queues. All other device fields
are protected by the port locks.

These functions manipulate the device queues and need to use the device
lock:
scsi_devO_procWriteGateTimer()
scsi_sendNextCmd()
scsi_flushReqQ()
scsi_addToReqQ()

------_=_NextPart_001_01C7CB27.215709DB
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7652.24">
<TITLE>RE: SMP FC locking</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">What exactly does the device =
lock buy you? Wouldn</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">&#8217;</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">t you =
already have the port lock anyway by the time you have to =
get</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">the</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial"></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">device =
lock in most of the cases?</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> </SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jeff Miller<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Friday, July 20, 2007 =
4:12 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> SMP FC locking</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 FACE=3D"Arial">This is what I am thinking for a SMP FC.&nbsp; =
Any comments?</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Thanks,</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Jeff</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">MP =
locking strategy</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Lock =
port during port control structure or hardware access.&nbsp; A FC port =
is not permanantly tied to a particular CPU as in Cheetah, but is owned =
by a CPU only for the period time that it needs attention from a =
CPU.&nbsp; </FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Lock =
device only to protect device queues while adding or removing scsi =
descriptors.&nbsp; All other device fields are protected by the port =
lock.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">A CPU =
may have to wait (in spin lock) for another CPU, but only for a very =
short time while a scsi descriptor is being linked or unlinked.&nbsp; =
The lock for the port is held for a much longer time, but CPUs do not =
wait at all for the port lock.&nbsp; </FONT></SPAN></P>
<BR>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Some =
assumptions:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">A device =
is associated with only one port.</FONT></SPAN></P>
<BR>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Changes</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add port =
spin lock to ispfc_cb_t structure.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">do =
NFX_SPIN_TRYLOCK() at beginning of polling functions and before issuing =
command to hardware in receive function, do nothing (don't wait) if lock =
not acquired.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Release =
lock at function returns.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add =
device spin lock to scsi_dev_t structure.&nbsp; Add lock/unlock only =
around code that manipulates the device queues.&nbsp; Do lock and unlock =
within loops that handle multiple scsi descriptors.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Polling =
functions that need to lock port:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_ct_desc_start_handler()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_completion_handler()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_link_management_handler()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">ispfc_device_management_handler()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_test_daemon() ?</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Port =
also needs to be locked for calls to =
scsi_sendNextCmd().</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Add =
device lock only to protect device queues. All other device fields are =
protected by the port locks.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">These =
functions manipulate the device queues and need to use the device =
lock:</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_devO_procWriteGateTimer()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_sendNextCmd()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_flushReqQ()</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">scsi_addToReqQ()</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

</BODY>
</HTML>
------_=_NextPart_001_01C7CB27.215709DB--
