Description: ------------ For Tuxstor, configuration of the virtual server interfaces, IP addresses, and routes can be performed from user space. This facet of the virtual server code changes will vary depending on whether the non-merged or merged approach is chosen for running User Apps and TXRX/FP code. The user space apps (daemons) and TXRX/FP kernel code will be co-resident in a unified target platform. VSD no longer needs to issue messages to the TXRX code (now in the kernel) and have it perform configuration operations. Now it will perform the operations directly by by system calls or other means. The code may be implemented within the IPMD library since the API already exists. By putting the code in this library, it can be shared by VSD, IPMD, and perhaps test utilities. No current functionality that is using messages between ssc daemons will be optimized to perform the function directly or via a linkable library so as to not make extra work for ourselves if that actually does become part of the software model for Orion. IPMD Library APIs to be used: ----------------------------- ipmd_CreateVifCluster ipmd_DeleteVifCluster ipmd_PrefVifCluster ipmd_AddLinkVifCluster ipmd_RmLinkVifCluster ipmd_SetIntfMtuCluster ipmd_AddIpCluster ipmd_DelIpCluster ipmd_AddRtCluster ipmd_DelRtCluster ipmd_GetIfStatsCluster ipmd_GetIpStatsCluster ipmd_GetIcmpStatsCluster ipmd_GetUdpStatsCluster ipmd_GetTcpStatsCluster ipmd_ClearTxrxIntfStat ipmd_ClearTxrxICMPStat ipmd_ClearTxrxIpStat ipmd_ClearTxrxUdpStat ipmd_ClearTxrxTcpStat APIs to be removed: ----------------------------- ipmd_EnableIntfCluster ipmd_DisableIntfCluster ipmd_EnableIntfAutoCluster ipmd_DisableIntfAutoCluster ipmd_RestartCluster ipmd_ShutDownCluster ipmd_GetTcpConnCluster ipmd_GetUdpConnCluster ipmd_AddArpCluster ipmd_DelArpCluster (and any others that aren't in use) IP routing for each virtual server shall be administered by RPDB in Linux. A separate routing table will be setup for each virtual server using the ip command utility. The ip route cache shall be flushed to commit any changes once made. Rules shall be configured with respect to each virtual server context for processing network traffic. No changes are expected for dealing with network traffic via mgmt ports routed via the main routing table. Since the mgmt bus interface is going away, private virtual server IPs will no longer be needed. Instead, apps will set/get the virtual server context using setsockopt/getsockopt to set/get the parameters necessary for the kernel to know which routing table to invoke. These parameters are the interface and source/destination IP address, prior to sending or after receiving a packet, or just the interface name in the case of broadcasts or generic listeners. This functionality will be implemented in a new api. IPMD: IPMD shall continue to be utilized for processing IPM message requests originating from remote daemon when in cluster mode. But rather than relaying message requests to the TXRX code, it will handle them itself. VSD: The majority of changes are concentrated in IPM interface routines (residing in vs-ipm.c). The routines will be changed to reference IPMD library API instead of sending message requests to the TXRX code. Virtual Server API / Data Structure: For the most part, changes to the existing virtual server api and data structure (vs_t) will be minimal if any. A few routines may be changed to pass virtual server Id instead of a vs_t address pointer to reduce reference count dependencies (as pointed out by Bill). Other changes may be related to where some tpl connection info will now be stored. Virtual Server Stats: The incoming/outgoing IP packet and data byte stats will be collected for each virtual server based on packet classification by realm. Each of the routes associated with a virtual server shall specify a realm for accounting purposes. These stats shall be accessible via the rtacct utility. ICMP, UDP, TCP, and other stats by protocol shall be collected by configuring a set of iptables rules for each virtual server interface. The protocol specific stats shall be accessible via the proc filesystem (/proc/net/{tcp,udp,snmp,...}). The summation of these protocol specific stats may be used to derive IP stats rather than collection of IP stats by realm as previously described. These stats shall be retrieved upon request by ipmd using the existing api. Network Device Stats: IP stats for each physical network device (ethN) shall be collected by parsing /proc/net content or via network device ioctl. Stats for vlan enabled and bonded network interface devices may be included as well. These stats shall be retrieved upon request by ipmd using the existing api. --------------------------------------------------------------------------- Example: IP=11.1.1.1 NETMSK=255.0.0.0 GW=11.0.0.1 NETIP=11.0.0.0 NETMB=8 BDCS=11.255.255.255 VSID=8 IDX=1 [VLAN=10] DEV=interface BDEV=bonded device EDEV=DEV [| BDEV] VDEV=EDEV.VLAN EVDEV=EDEV [| VDEV] Virtual Server Enable Operation: [vconfig add EDEV 10] ip addr add IP/NETMB broadcast BDCS dev EVDEV ip route del to NETIP/NETMB src IP dev EVDEV (remove auto generated route?) ip route add to NETIP/NETMB src IP dev EVDEV table VSID realm VSID ip rule add from NETIP/NETMB to IP iif EVDEV table VSID realms VSID/VSID iptables -A INPUT -m account -aname vs.VSID -aaddr IP iptables -A OUTPUT -m account -aname vs.VSID -aaddr IP ip route flush cache Virtual Server Disable Operation: iptables -D INPUT -m account -aname vs.VSID -aaddr IP iptables -D OUTPUT -m account -aname vs.VSID -aaddr IP ip rule del from NETIP/NETMB to IP iif EVDEV table VSID realms VSID/VSID ip route del to NETIP/NETMB src IP dev EVDEV table VSID realm VSID ip addr del IP/NETMB dev EVDEV [vconfig rem VDEV] ip route flush cache Interface Enable/Disable Operations: ip link set DEV up|down Route Setup/Shutdown Operation: ip route add|del default|IP/NETMB via GW table VSID realm VSID ip route flush cache