AF:
NF:0
PS:10
SRH:1
SFN:
DSR:
MID:
CFG:
PT:0
S:andy.sharp@lsi.com
RQ:
SSV:mhbs.lsil.com
NSV:
SSH:
R:<Rendell.Fong@lsi.com>
MAID:2
X-Sylpheed-Privacy-System:
X-Sylpheed-Sign:0
SCF:#mh/Mailbox/sent
RMID:#imap/LSI/INBOX	0	1254759281.14287.169.camel@rendellf
X-Sylpheed-End-Special-Headers: 1
Date: Mon, 5 Oct 2009 09:32:47 -0700
From: Andrew Sharp <andy.sharp@lsi.com>
To: Rendell Fong <Rendell.Fong@lsi.com>
Subject: Re: vsd change description
Message-ID: <20091005093247.7d7bade0@ripper.onstor.net>
In-Reply-To: <1254759281.14287.169.camel@rendellf>
References: <1254759281.14287.169.camel@rendellf>
Organization: LSI
X-Mailer: Sylpheed-Claws 2.6.0 (GTK+ 2.8.20; x86_64-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Mon, 5 Oct 2009 10:14:41 -0600 Rendell Fong <Rendell.Fong@lsi.com>
wrote:

> This is a first cut at describing the tentative VSD changes.
> 
> 
> 
> Description:
> 
> In Tuxstor, the user space daemons and TXRX/FP kernel code will be 
> co-resident in a unified SMP target platform.  Virtual server
> interfaces, IP addresses, and routes can be configured from user
> space. So rather than issuing request messages to IPM like in the
> existing implementation, VSD can perform this function itself to
> reduce complexity.  IPM will be eliminated but IPMD shall continue to
> be utilized for processing stats requests originating from local
> daemons (and remote ones when in cluster mode).
> 
> Virtual server interfaces shall be created by setting up virtual
> interfaces in Linux with the ifconfig utility.  The virtual interface

The ip utility, actually.

> name shall be generated based on the base network device name, virtual
> server Id, and virtual server interface index (e.g. eth0:8-1 where
> eth0=NetDevice, 8=VsId, and 1=VsIfIndex).  By utilizing a virtual
> interface, stats for incoming/outgoing IP packet and data bytes will
> be collected on an individual virtual interface basis.  The stats for
> a virtual server will be derived from a summation of its virtual
> interface stats (or realm stats).  We still need to determine how to
> get ICMP, UDP, and TCP stats on a virtual server basis.
> 
> When a virtual server is configured to use vlan tagging, an additional
> virtual network interface device shall be created in Linux with the
> vconfig utility.  The name of the network interface device is
> automatically created by the utility based on the base network device
> name and vlan Id (e.g. eth0:8-1.10 where 10=vlanId). 
> 
> When a virtual server is configured to use an aggregated L-Port, a
> network interface device shall be created in Linux with the ifenslave
> utility to attach it to the bonding device.  The bonding device will

can't the ip utility do all this?  not that I care.

> act like a normal Ethernet network device to the kernel, but will
> send out the packets via the slave devices using a simple round-robin
> scheduler. This supports simple load-balancing.  Setup for the
> failover L-Port is to be determined.  
> 
> IP routing for each virtual server shall be administered by RPDB in
> Linux.  A separate routing table will be setup for each virtual server
> using the ip and iptables utilities.  Once the routes are added, the
> ip route cache shall be flushed to commit the changes.  The existing
> private virtual server IP address for each of its interfaces shall be

The wha?  Private virtual server IP address?  Wha be dis?  Why be dis?
Das history.

> supported using the iptable nat function to translate this IP address
> to/from its external IP address for outgoing/incoming packets.
> 
> 
> Issues:
> 
> 1. Need TCP/UDP/ICMP stats per vsvr.

low priority details ~:^)

> 2. Virtual network device names will used.  Existing code that
> references network interface info will probably be affected.
> 3. Is the length of the network device name a problem?

There's got to be a better naming scheme that isn't so...cyborgish.

> 4. Certain stat counters shall probably change based on whatever
> exists.

But of course.

5.  Library code that does the 192.167.major.bites thing will have to be
converted to the new world order.  Same goes for any daemon code that
messes under the covers directly with those addresses.  Minor, but
should be mentioned.


> Example:
> 
> IP=11.1.1.1  NETMSK=255.0.0.0  GW=11.0.0.1
> NETIP=11.0.0.0  NETMB=8
> VSID=8  IDX=1  [VLAN=10]
> DEV=eth0
> EDEV=DEV:VSID-IDX
> VDEV=EDEV.VLAN
> EVDEV=EDEV [| VDEV]
> 
> Virtual Server Enable Operation:
> 
> ifconfig EDEV IP netmask NETMSK up
> [vconfig add EDEV 10]
> ip route add to NETIP/NETMB src IP dev EVDEV table VSID realm VSID
> [ip route add default via GW dev EVDEV table VSID realm VSID]
> ip rule add from NETIP/NETMB to IP iif EVDEV table VSID realms
> VSID/VSID ip addr add 192.167.VSID.IDX/32 scope host dev EVDEV
> ip rule add from 192.167.VSDID.IDX/32 table VSID realms VSID/VSID
> iptables -t nat -A POSTROUTING -s 192.167.VSID.IDX/32 -o EVDEV -j SNAT
> --to-source IP
> ip route flush cache
> 
> 
> Virtual Server Disable Operation:
> 
> iptables -t nat -D POSTROUTING -s 192.167.VSID.IDX/32 -o EVDEV -j SNAT
> --to-source IP

nack.  192.167.bite.me is history.

> ip rule delete from 192.167.VSID.IDX/32 table VSID realms VSID/VSID
> ip addr delete 192.167.VSID.IDX/32 scope host dev EVDEV
> ip rule delete from NETIP/NETMB to IP iif EVDEV table VSID realms
> VSID/VSID
> [ip route delete default via NETIP dev EVDEV table VSID realm VSID]
> ip route delete to NETIP/NETMB src IP dev EVDEV table VSID realm VSID
> [vconfig rem VDEV]
> ifconfig EDEV down
> ip route flush cache
> 
> 
