AF:
NF:0
PS:10
SRH:1
SFN:
DSR:
MID:
CFG:
PT:0
S:andy.sharp@lsi.com
RQ:
SSV:mhbs.lsil.com
NSV:
SSH:
R:<Rendell.Fong@lsi.com>
MAID:2
X-Sylpheed-Privacy-System:
X-Sylpheed-Sign:0
SCF:#mh/Mailbox/sent
RMID:#imap/LSI/INBOX	0	1254862002.14287.235.camel@rendellf
X-Sylpheed-End-Special-Headers: 1
Date: Tue, 6 Oct 2009 14:03:23 -0700
From: Andrew Sharp <andy.sharp@lsi.com>
To: Rendell Fong <Rendell.Fong@lsi.com>
Subject: Re: vsd change description
Message-ID: <20091006140323.092a9b49@ripper.onstor.net>
In-Reply-To: <1254862002.14287.235.camel@rendellf>
References: <1254759281.14287.169.camel@rendellf>
	<20091005093247.7d7bade0@ripper.onstor.net>
	<1254766688.14287.180.camel@rendellf>
	<20091005115557.7b22a164@ripper.onstor.net>
	<1254852979.14287.230.camel@rendellf>
	<20091006130843.030931d6@ripper.onstor.net>
	<1254862002.14287.235.camel@rendellf>
Organization: LSI
X-Mailer: Sylpheed-Claws 2.6.0 (GTK+ 2.8.20; x86_64-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

It's gotta be possible.  If not, we can stipulate that no IP addresses
overlap between vsvrs.  I hope.  I think Max once told me that would be
OK.

On Tue, 6 Oct 2009 14:46:42 -0600 Rendell Fong <Rendell.Fong@lsi.com>
wrote:

> On Tue, 2009-10-06 at 14:08 -0600, Andrew Sharp wrote:
> > The answer is simple: it has to specify the interface.  's got to
> > be a way.
> > 
> 
> Meaning you don't know if it's possible?
> I guess we're stalled until this is resolved.
> 
> 
> > On Tue, 6 Oct 2009 12:16:18 -0600 Rendell Fong
> > <Rendell.Fong@lsi.com> wrote:
> > 
> > > I've got some questions:
> > > 
> > > If we don't use private IPs, then how can user apps determine the
> > > vsvr context when the vsvr IPs are not unique?  Socket info
> > > doesn't include any info regarding the interface from which
> > > incoming packets were received.
> > > 
> > > When user apps are sending packets relative to a vsvr context,
> > > how do they specify the routing table wrt the socket call to
> > > ensure that proper routing occurs?  Is there some socket option
> > > or bind option that must be set?
> > > 
> > > 
> > > On Mon, 2009-10-05 at 12:55 -0600, Andrew Sharp wrote:
> > > > Some useless comments inline.  See what you can dig up
> > > > regarding how DNS will be handled, then give me a "final"
> > > > version and I will throw it on the wiki.  Or you can do it
> > > > yourself, your choice.
> > > > 
> > > > On Mon, 5 Oct 2009 12:18:08 -0600 Rendell Fong
> > > > <Rendell.Fong@lsi.com> wrote:
> > > > 
> > > > > On Mon, 2009-10-05 at 10:32 -0600, Andrew Sharp wrote:
> > > > > > On Mon, 5 Oct 2009 10:14:41 -0600 Rendell Fong
> > > > > > <Rendell.Fong@lsi.com> wrote:
> > > > > > 
> > > > > > > This is a first cut at describing the tentative VSD
> > > > > > > changes.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > Description:
> > > > > > > 
> > > > > > > In Tuxstor, the user space daemons and TXRX/FP kernel code
> > > > > > > will be co-resident in a unified SMP target platform.
> > > > > > > Virtual server interfaces, IP addresses, and routes can be
> > > > > > > configured from user space. So rather than issuing request
> > > > > > > messages to IPM like in the existing implementation, VSD
> > > > > > > can perform this function itself to reduce complexity.
> > > > > > > IPM will be eliminated but IPMD shall continue to be
> > > > > > > utilized for processing stats requests originating from
> > > > > > > local daemons (and remote ones when in cluster mode).
> > > > > > > 
> > > > > > > Virtual server interfaces shall be created by setting up
> > > > > > > virtual interfaces in Linux with the ifconfig utility.
> > > > > > > The virtual interface
> > > > > > 
> > > > > > The ip utility, actually.
> > > > > 
> > > > > You won't be able to get stats per virtual server if you do it
> > > > > strictly with the ip utility.  Using virtual interfaces will
> > > > > allow IP packet/byte stats to be collected as expected.
> > > > 
> > > > Hm, why would the utility make a difference?  But I ask
> > > > questions I don't care about.  Whatever is the right choice.
> > > > 
> > > > > > > name shall be generated based on the base network device
> > > > > > > name, virtual server Id, and virtual server interface
> > > > > > > index (e.g. eth0:8-1 where eth0=NetDevice, 8=VsId, and
> > > > > > > 1=VsIfIndex).  By utilizing a virtual interface, stats
> > > > > > > for incoming/outgoing IP packet and data bytes will be
> > > > > > > collected on an individual virtual interface basis.  The
> > > > > > > stats for a virtual server will be derived from a
> > > > > > > summation of its virtual interface stats (or realm
> > > > > > > stats).  We still need to determine how to get ICMP, UDP,
> > > > > > > and TCP stats on a virtual server basis.
> > > > > > > 
> > > > > > > When a virtual server is configured to use vlan tagging,
> > > > > > > an additional virtual network interface device shall be
> > > > > > > created in Linux with the vconfig utility.  The name of
> > > > > > > the network interface device is automatically created by
> > > > > > > the utility based on the base network device name and
> > > > > > > vlan Id (e.g. eth0:8-1.10 where 10=vlanId). 
> > > > > > > 
> > > > > > > When a virtual server is configured to use an aggregated
> > > > > > > L-Port, a network interface device shall be created in
> > > > > > > Linux with the ifenslave utility to attach it to the
> > > > > > > bonding device.  The bonding device will
> > > > > > 
> > > > > > can't the ip utility do all this?  not that I care.
> > > > > 
> > > > > I don't know.  These are things to figure out once its
> > > > > implemented. 
> > > > 
> > > > Agreed.
> > > > 
> > > > > > 
> > > > > > > act like a normal Ethernet network device to the kernel,
> > > > > > > but will send out the packets via the slave devices using
> > > > > > > a simple round-robin scheduler. This supports simple
> > > > > > > load-balancing. Setup for the failover L-Port is to be
> > > > > > > determined.  
> > > > > > > 
> > > > > > > IP routing for each virtual server shall be administered
> > > > > > > by RPDB in Linux.  A separate routing table will be setup
> > > > > > > for each virtual server using the ip and iptables
> > > > > > > utilities. Once the routes are added, the ip route cache
> > > > > > > shall be flushed to commit the changes.  The existing
> > > > > > > private virtual server IP address for each of its
> > > > > > > interfaces shall be
> > > > > > 
> > > > > > The wha?  Private virtual server IP address?  Wha be dis?
> > > > > > Why be dis? Das history.
> > > > > > 
> > > > > 
> > > > > It might be easier to get things working this way initially.
> > > > > Forcing use the the virtual server external IP addresses
> > > > > directly may make it more difficult to correlate to the
> > > > > virtual server since the IP addresses may not be unique
> > > > > except per network interface.  As a result, extra
> > > > > parameter(s) may need to be passed around.
> > > > 
> > > > Nah.  We're gonna give the private IP think das boot.
> > > > 
> > > > > > > supported using the iptable nat function to translate
> > > > > > > this IP address to/from its external IP address for
> > > > > > > outgoing/incoming packets.
> > > > > > > 
> > > > > > > 
> > > > > > > Issues:
> > > > > > > 
> > > > > > > 1. Need TCP/UDP/ICMP stats per vsvr.
> > > > > > 
> > > > > > low priority details ~:^)
> > > > > > 
> > > > > > > 2. Virtual network device names will used.  Existing code
> > > > > > > that references network interface info will probably be
> > > > > > > affected. 3. Is the length of the network device name a
> > > > > > > problem?
> > > > > > 
> > > > > > There's got to be a better naming scheme that isn't
> > > > > > so...cyborgish.
> > > > > > 
> > > > > 
> > > > > I'll leave it to you since it sounds like you have a better
> > > > > idea.
> > > > 
> > > > No, I'm leaving it to you to wrack your brain to improve upon
> > > > your original idea ~:^)
> > > > 
> > > > > > > 4. Certain stat counters shall probably change based on
> > > > > > > whatever exists.
> > > > > > 
> > > > > > But of course.
> > > > > > 
> > > > > > 5.  Library code that does the 192.167.major.bites thing
> > > > > > will have to be converted to the new world order.  Same
> > > > > > goes for any daemon code that messes under the covers
> > > > > > directly with those addresses. Minor, but should be
> > > > > > mentioned.
> > > > > > 
> > > > > > 
> > > > > > > Example:
> > > > > > > 
> > > > > > > IP=11.1.1.1  NETMSK=255.0.0.0  GW=11.0.0.1
> > > > > > > NETIP=11.0.0.0  NETMB=8
> > > > > > > VSID=8  IDX=1  [VLAN=10]
> > > > > > > DEV=eth0
> > > > > > > EDEV=DEV:VSID-IDX
> > > > > > > VDEV=EDEV.VLAN
> > > > > > > EVDEV=EDEV [| VDEV]
> > > > > > > 
> > > > > > > Virtual Server Enable Operation:
> > > > > > > 
> > > > > > > ifconfig EDEV IP netmask NETMSK up
> > > > > > > [vconfig add EDEV 10]
> > > > > > > ip route add to NETIP/NETMB src IP dev EVDEV table VSID
> > > > > > > realm VSID [ip route add default via GW dev EVDEV table
> > > > > > > VSID realm VSID] ip rule add from NETIP/NETMB to IP iif
> > > > > > > EVDEV table VSID realms VSID/VSID ip addr add
> > > > > > > 192.167.VSID.IDX/32 scope host dev EVDEV ip rule add from
> > > > > > > 192.167.VSDID.IDX/32 table VSID realms VSID/VSID iptables
> > > > > > > -t nat -A POSTROUTING -s 192.167.VSID.IDX/32 -o EVDEV -j
> > > > > > > SNAT --to-source IP ip route flush cache
> > > > > > > 
> > > > > > > 
> > > > > > > Virtual Server Disable Operation:
> > > > > > > 
> > > > > > > iptables -t nat -D POSTROUTING -s 192.167.VSID.IDX/32 -o
> > > > > > > EVDEV -j SNAT --to-source IP
> > > > > > 
> > > > > > nack.  192.167.bite.me is history.
> > > > > > 
> > > > > > > ip rule delete from 192.167.VSID.IDX/32 table VSID realms
> > > > > > > VSID/VSID ip addr delete 192.167.VSID.IDX/32 scope host
> > > > > > > dev EVDEV ip rule delete from NETIP/NETMB to IP iif EVDEV
> > > > > > > table VSID realms VSID/VSID
> > > > > > > [ip route delete default via NETIP dev EVDEV table VSID
> > > > > > > realm VSID] ip route delete to NETIP/NETMB src IP dev
> > > > > > > EVDEV table VSID realm VSID [vconfig rem VDEV]
> > > > > > > ifconfig EDEV down
> > > > > > > ip route flush cache
> > > > > > > 
> > > > > > > 
> > > > > 
> > > 
> 
