AF:
NF:0
PS:10
SRH:1
SFN:
DSR:
MID:<20080619182640.7c48dc56@ripper.onstor.net>
CFG:
PT:0
S:andy.sharp@onstor.com
RQ:
SSV:onstor-exch02.onstor.net
NSV:
SSH:
R:<john.rogers@onstor.com>,<sandrine.boulanger@onstor.com>,<vikas.saini@onstor.com>,<larry.scheer@onstor.com>
MAID:1
X-Sylpheed-Privacy-System:
X-Sylpheed-Sign:0
SCF:#mh/Mailbox/sent
RMID:#imap/andys@onstor.net@onstor-exch02.onstor.net/INBOX	0	BB375AF679D4A34E9CA8DFA650E2B04E09C42599@onstor-exch02.onstor.net
X-Sylpheed-End-Special-Headers: 1
Date: Thu, 19 Jun 2008 18:27:23 -0700
From: Andrew Sharp <andy.sharp@onstor.com>
To: "John Rogers" <john.rogers@onstor.com>
Cc: "Sandrine Boulanger" <sandrine.boulanger@onstor.com>, "Vikas Saini"
 <vikas.saini@onstor.com>, "Larry Scheer" <larry.scheer@onstor.com>
Subject: Re: How come we did not upgrade MD today to sub#26 beta?
Message-ID: <20080619182723.3d490991@ripper.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E09C42599@onstor-exch02.onstor.net>
References: <BB375AF679D4A34E9CA8DFA650E2B04E09C42599@onstor-exch02.onstor.net>
Organization: Onstor
X-Mailer: Sylpheed-Claws 2.6.0 (GTK+ 2.8.20; x86_64-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Excuse me?  My yammering rules.  But OK, here you go:


Project Cougar Submittal Notes

 

 

 

 

 

Project

Cougar 4.0

Submittal#

26

Date

June 17 2008

 

 

 




 

TABLE OF CONTENTS

 

1.       Features & Functions included in this submittal 4

2.       Specific Installation/Configuration Information. 4

2.1.      Configuration files that are the same for both Beta and
Development Build. 5

2.2.      Configuration file changes specific to cougar Beta Build. 6

2.3.      Configuration file changes specific to cougar Development
Build. 6

2.4.      HTTP and FTP use with the upgrade server 8

2.5.      Enabling DMALLOC in the runtime environment 8

2.6.      Configuring the automounter and ypbind. 9

3.       Source tree information. 9

3.1.      Perforce label 9

3.1.1.    Submittal Beta Candidate. 9

3.1.2.    Submittal Development Build. 9

3.2.      Last EverON Perforce change list 9

3.2.1.    Submittal Beta Candidate. 9

3.2.2.    Submittal Development Build. 9

3.3.      Source tree location. 9

3.3.1.    Submittal Beta Candidate. 9

3.3.2.    Submittal Development Build. 9

3.4.      BSD distribution images for System W upgrade. 9

3.4.1.    Submittal Beta Candidate. 9

3.4.2.    Submittal Development Build. 10

4.       System upgrade information. 10

4.1.      System upgrade information for EverON.. 10

4.2.      Location of Cougar distribution images. 10

4.2.1.    Submittal Beta Candidate. 10

4.2.2.    Submittal Development Build. 10

4.3.      Prom settings for compact flash boot 11

5.       SSC PROM Upgrade instructions. 11

6.       Runtime PROM Upgrade instructions. 12

7.       Restrictions/limitations in this submittal 13

7.1.      Restrictions in this submittal 13

7.2.      Restrictions from previous submittal that still apply to this
submittal 13

7.3.      Problems encountered during Acceptance Testing. 13

8.       Details of New Features introduced in submittal 12. 13

8.1.      I/O Coalescing. 13

8.2.      Root level storage command. 13

8.3.      TXRX command: nfsperftest 13

8.4.      FC diag command: scsiadmin trace. 13

8.5.      New Analytics. 13

9.       Requirements to boot from Compact Flash. 14

10.     New/Changed/Obsolete modules/binaries. 14

10.1.       Changed in submittal 18. 14

10.2.       Introduced in submittal 2: 14

10.3.       Introduced in submittal 14: 14

11.     Troubleshooting tips. 14

12.     Special focus/concerns/additional testing. 14

13.     Pre-submittal tests results. 15

13.1.       Build verification test results. 15

13.2.       Acceptance test results. 15

 

 




 


1.       Features & Functions included in this submittal


New features

 

None

 

Configuration files have changed; see Section 2 for important
configuration changes and information regarding the changes.

 

If you haven't upgraded to prom version: Cougar-prom-1.0.6: Mon May 12
14:38:16 2008 Follow the instructions in section 5 to upgrade your SSC
PROM; follow the instructions in section 6 to upgrade the PROMS of the
embedded processors.

 

The cougar builds made in the beta branch are built with DMALLLOC
capability. The DMALLOC libraries are installed in the Linux root file
system. However, the default runtime state has DMALLOC is turned off.
See section 2.1 for instructions on how to boot your cougar system with
DMALLOC enabled in the runtime.

 

 

Particular fixes Beta

 

TED 23108 - NFS performance occassionally drops to zero for a second or
two

TED 23748 - Cougar- /etc/hosts file gets corrupted ( doesnt look similar
in cluster)

TED 23894 - CSoak (G12R10) - Volume exception - modify lock not being
released by some thread

TED 23984 - cougar streaming write performance is low

TED 24014 - dirty user data can stay in cache for long time

TED 24174 - (8394 - Onstor) Several volume exceptions due to array issue

TED 24203 - fp hits assert in fs_rdwrUpdateResourceCountsPreChecks()
while creating a volume

TED 24238 - dbg FP crash due to missing lock acquisition

 

Perforce change lists integrated into beta branch

 

Change 29693 on 2008/06/14 by larrys@larrys 'Integrate from dev branch
chang'

Change 29692 on 2008/06/14 by maximk@maximk-13 'integrate 29691.   Fix
compilat'

Change 29688 on 2008/06/13 by jobia@jobi:jobi '   Integrate the
following chan'

Change 29685 on 2008/06/13 by jong@jong-jong-cifs 'Integration from dev
branch    '

Change 29683 on 2008/06/13 by jong@jong-jong-cifs 'Integration from dev
branch    '

Change 29682 on 2008/06/13 by jobia@jobi:jobi '    Integrate following
changes'

Change 29647 on 2008/06/12 by jong@jong-jong-cifs 'Integration from dev
branch.   '

Change 29623 on 2008/06/10 by jobia@jobi:jobi '   Integrate the
following chan'

 


2.       Specific Installation/Configuration Information


Verify your SSC, TXRX, and FP PROMs are version: Cougar-prom-1.0.6: Mon
May 12 14:38:16 2008.

If not follow the instructions in section 5 to upgrade your SSC PROM and
the instructions in section 6 to upgrade your FP and TXRX PROMs.

 

You must have you SSC, TXRX, and FP PROM autoload and autoreboot
settings configured identically.

 

Recommended settings:

env set autoload on

env set autoreboot off

Configure your SSC prom in the SSC console as explained in section 4.3

 

Configure your TXRX PROM with rcon; example:

rcon  1 0

TXRX0:1 > env set autoload on

TXRX0:2 > env set autoreboot off

<ctrl> c

Configure your FP PROM with rcon; example:

rcon  1 2

FP0:1 > env set autoload on

FP0:2 > env set autoreboot off

<ctrl> c

 

Before doing an upgrade or a flash install check your configuration
files for up-to-date format changes and proper configuration as outlined
in sections 2.1, 2.2, and 2.3. 

 


2.1.      Configuration files that are the same for both Beta and
Development Build 


The format of configuration files preserved between upgrades and
flash_install.sh have changed over time. Check the following files for
correct configuration as indicated.

Hosts:

/etc/hosts (default example with no SC interfaces configured shown)

            127.0.0.1               localhost

 

Hosts.user:

Remove /etc/hosts.user if its contents look like this:

127.0.0.1               localhost

 

NTP Conf file (shown configured with two ntp servers)

/etc/ntp.conf

# /etc/ntp.conf, configuration for ntpd

 

driftfile /var/lib/ntp/ntp.drift

statsdir /var/log/ntpstats/

 

statistics loopstats peerstats clockstats

filegen loopstats file loopstats type day enable

filegen peerstats file peerstats type day enable

filegen clockstats file clockstats type day enable

 

 

# By default, exchange time with everybody, but don't allow
configuration.

# See /usr/share/doc/ntp-doc/html/accopt.html for details.

restrict -4 default kod notrap nomodify nopeer noquery

restrict -6 default kod notrap nomodify nopeer noquery

 

# Local users may interrogate the ntp server more closely.

restrict 127.0.0.1

restrict ::1

 

# You do need to talk to an NTP server or two (or three).

#server ntp.your-provider.example

server 10.0.0.18 iburst

server 10.2.0.2 iburst

 

Time Zone file (should be an ascii file; example shows the time zone for
Campbell, CA)

/etc/timezone 

US/Pacific

 

Mail Name file (an unconfigured file will contain the name localhost)

/etc/mailname

   <your filer's hostname>

 


2.2.      Configuration file changes specific to cougar Beta Build 


Default Route using addroutes:

/etc/network/if-up.d/addroutes 

#!/bin/sh

route add -net 0.0.0.0 netmask 0.0.0.0 gw 10.2.0.1

 

SSC Network Interfaces (single port example shown)

/etc/network/interfaces 

# Used by ifup(8) and ifdown(8). See the interfaces(5) manpage or

# /usr/share/doc/ifupdown/examples for more information.

 

# The loopback network interface

auto lo

iface lo inet loopback

 

# The primary network interface

auto eth0

iface eth0 inet static

  address 10.2.203.12

  netmask 255.255.0.0

 

# The secondary network interface

auto eth1

iface eth1 inet static

# address not_configured

# netmask not_configured

 

 


2.3.      Configuration file changes specific to cougar Development
Build


Addroutes:

 

Before flash_install or system upgrade remove
/etc/network/if-up.d/addroutes on the primary flash.

 

After flash_install or system upgrade and a reboot -s, verify
/etc/network/if-up.d/addroutes is as follows:

 

#!/bin/sh

#

#  File:  /etc/network/if-up.d/addroutes 

# 

# This file is executed by if-up (via run-parts) for every interface
configured

# as auto in /etc/network/interfaces.  A success status must be returned
to

# run-parts upon completion to prevent from reporting false negatives.
Route

# add completion status is captured in $rtlog.

# 

# The route-up files created in /tmp indicate the interface that a route
was 

# successfully added.

#

 

rtlog=/tmp/route-add.log

 

# Only add routes to the eth0 and eth1 interfaces 

if [[ $IFACE == eth? ]] ; then

    touch $rtlog

    

    #if defaultroute exists and route is not up read its contents to
setup

    #the default route

    if [ -f /etc/defaultroute -a ! -f /tmp/default-route-up ] ; then

        while read MYGATE ; do

 

            # only create the tmp file if the route succeeds for this
interface

            echo -e "Adding default route on ${IFACE}... \c" >>$rtlog

            route add default gw $MYGATE 2>>$rtlog && {

                echo "$IFACE" >/tmp/default-route-up 

                echo "done." >>$rtlog

            } || true

        done < /etc/defaultroute

    fi

 

    #if static routes were set by nfxsh read the route data

    if [ -f /etc/routes.ssc ] ; then

        while read NET MASK GW ; do 

 

            # if route is already up skip it

            if [ -f /tmp/${NET}-route-up ] ; then

                continue

            fi

 

            # only create the tmp file if the route succeeds for this
interface

            echo -e "Adding $NET route on ${IFACE}... \c" >>$rtlog

            route add -net $NET netmask $MASK gw $GW 2>>$rtlog && {

                echo "$IFACE" >/tmp/${NET}-route-up

                echo "done." >>$rtlog

            } || true

        done < /etc/routes.ssc

    fi

fi

 

Default Route:

 

Upgrade or flash_install.sh will add /etc/defaultroute with the
following gateway configured:

 

/etc/defaultroute

            10.2.0.1

 

If your filer uses some other host as its default gateway change
/etc/defaultroute to match your configuration

 

SSC Network Interfaces (single port example shown) Note that auto eth1
is commented out.

/etc/network/interfaces 

# Used by ifup(8) and ifdown(8). See the interfaces(5) manpage or

# /usr/share/doc/ifupdown/examples for more information.

 

# The loopback network interface

auto lo

iface lo inet loopback

 

# The primary network interface

auto eth0

iface eth0 inet static

  address 10.2.203.12

  netmask 255.255.0.0

 

# The secondary network interface

#auto eth1

iface eth1 inet static

# address not_configured

# netmask not_configured

 

If you are running a release older than submittal 19 it is recommended
you save your cluster files and use flash_install to update your NAS
Gateway without copying the configuration files. Restore your cluster
files after the flash install and running the FTI.

 

A system copy init followed by system upgrade -s will efficiently
upgrade an initialized flash card. It will copy the configuration data
and files from the primary flash to the standby flash.

 


2.4.      HTTP and FTP use with the upgrade server


A new upgrade server is available that supports both FTP and HTTP
download. 

 

If your system has DNS services configured you can use its hostname
jump.onstor.lab. Because not all systems have DNS configured all URLs
published in this document will use the IP address for jump.onstor.lab.

Upgrade example using HTTP URL:

g4r5 diag> system upgrade -f -s
http://10.2.0.21/upgrade/R4.0.0.0CG-0601408.tar.gz
<http://10.2.0.21/upgrade/R4.0.0.0CG-050608.tar.gz> 

 

If you need to use the FTP syntax with this server use the same paths as
the previous server. Only the IP address has changed. Example of new
server FTP URL:

Bobcat Optimized

ftp://upgrade@10.2.0.21/home/upgrade/R3.3.0.0BC-0601408.tar.gz
<ftp://upgrade@10.2.0.21/home/upgrade/R3.3.0.0BC-050608.tar.gz> 

 

Cougar Debug:

ftp://upgrade@10.2.0.21/R4.0.0.0CGDBG-0601408.tar.gz
<ftp://upgrade@10.2.0.21/R4.0.0.0CGDBG-050608.tar.gz> 

 


2.5.      Enabling DMALLOC in the runtime environment


This submittal is built with DMALLOC support. To turn on DMALLOC's
built-in memory monitoring and analysis perform the following steps.

 Edit /etc/default/onstor

Change line 4 from:

#export DMALLOC_OPTIONS=debug=0x4f4ed03,inter=100

Change line 4 to:

export DMALLOC_OPTIONS=debug=0x4f4ed03,inter=100

Reboot the system

 


2.6.      Configuring the automounter and ypbind


Remove /etc/defaultdomain if you have no desire to run the automounter
and you are having ypbind problems. Production root file systems will
not be configured with /etc/defaultdomain.

 

If you want to run the automounter and ypbind make sure these two
configuration files agree with each other.

The correct settings for /etc/defualtdomain and /etc/yp.conf are as
follows:

 

/etc/defaultdomain:

  agilestorage

/etc/yp.conf:

  domain agilestorage server 10.0.0.21

 

/etc/defaultdomain:

  onstorlab

/etc/yp.conf:

  domain agilestorage server 10.2.0.4

 

 


3.       Source tree information


 


3.1.      Perforce label


3.1.1.        Submittal Beta Candidate 


cougar-sub26


3.1.2.        Submittal Development Build 


None, use Change list # 29694

 


3.2.      Last EverON Perforce change list


3.2.1.        Submittal Beta Candidate


               //depot/cg_beta/...@29693


3.2.2.        Submittal Development Build


  //depot/dev/...@29694

 


3.3.      Source tree location


3.3.1.        Submittal Beta Candidate


/n/Build-Trees/R4.0.0.0/R4.0.0.0-0601408-sub26/nfx-tree

/n/Build-Trees/R4.0.0.0/R4.0.0.0-0601408-sub26/linux


 


3.3.2.        Submittal Development Build


/n/Build-Trees/R4.0.0.0/R4.0.0.0DEV-060708-sub26/nfx-tree

/n/Build-Trees/R4.0.0.0/R4.0.0.0DEV-060708-sub26/linux

 


3.4.      BSD distribution images for System W upgrade


3.4.1.        Submittal Beta Candidate


 

R3.3.0.0 images are here:

Cheetah optimized:

http://10.2.0.21/upgrade/R3.3.0.0-0601408.tar.gz
<http://10.2.0.21/upgrade/R3.3.0.0-052008.tar.gz> 

Cheetah debug:

http://10.2.0.21/upgrade/R3.3.0.0DBG-0601408.tar.gz
<http://10.2.0.21/upgrade/R3.3.0.0DBG-052008.tar.gz> 

 

Bobcat optimized:

http://10.2.0.21/upgrade/R3.3.0.0BC-0601408.tar.gz
<http://10.2.0.21/upgrade/R3.3.0.0BC-051308.tar.gz> 

Bobcat debug:

http://10.2.0.21/upgrade/R3.3.0.0BCDBG-0601408.tar.gz
<http://10.2.0.21/upgrade/R3.3.0.0BCDBG-042908.tar.gz> 

 

FTP example for 3.2.0.4 and older nfxsh programs that don't support
http:

ftp://upgrade@10.2.0.21/home/upgrade/R3.3.0.0BC-0601408.tar.gz
<ftp://upgrade@10.2.0.21/home/upgrade/R3.3.0.0BC-050608.tar.gz> 

 


3.4.2.        Submittal Development Build


 

R3.3.0.0 images are here:

Cheetah optimized:

http://10.2.0.21/upgrade/R3.3.0.DEV-060708.tar.gz

Cheetah debug:

http://10.2.0.21/upgrade/R3.3.0.DEV-DBG-060708.tar.gz

 

Bobcat optimized:

http://10.2.0.21/upgrade/R3.3.0.DEV-BC-060708.tar.gz 

Bobcat debug:

http://10.2.0.21/upgrade/R3.3.0.DEV-BCDBG-060708.tar.gz 

 


4.       System upgrade information


 


4.1.      System upgrade information for EverON


 

System upgrade on Linux systems uses wget to pull files from ftp or http
servers. The syntax of wget is slightly different from ftp. You do not
add the ftp user directory to the path specified on the command line. 

 

Use system upgrade as you would normally for subsequent updates.

 


4.2.      Location of Cougar distribution images


4.2.1.        Submittal Beta Candidate


Cougar debug:

http://10.2.0.21/upgrade/R4.0.0.0CGDBG-0601408.tar.gz
<http://10.2.0.21/upgrade/R4.0.0.0CGDBG-042908.tar.gz> 

Cougar optimized:

http://10.2.0.21/upgrade/R4.0.0.0CG-0601408.tar.gz
<http://10.2.0.21/upgrade/R4.0.0.0CG-051308.tar.gz> 

 

FTP example for those who fear change:

ftp://upgrade@10.2.0.21/R4.0.0.0CG-0601408.tar.gz
<ftp://upgrade@10.2.0.21/R4.0.0.0CG-050608.tar.gz> 

 


4.2.2.        Submittal Development Build


Cougar debug:

http://10.2.0.21/upgrade/R4.0.0.DEV-CGDBG-060708.tar.gz

Cougar optimized:

http://10.2.0.21/upgrade/R4.0.0.DEV-CG-060708.tar.gz

 


4.3.      Prom settings for compact flash boot


Set your prom environment as follows: 

SSC-PROM> env view

      Current Environment Settings:

 

       autoload       = on         ( off on )

       autoreboot     = off        ( off on )

       load_src       = disk    ( disk network )

       load_opts      = mAt        ( m/s|A/a|d|t )

       boot_dev       = sda1        ( sda1 sdb1 sc1 )

       upgrade_prom   = yes        ( no yes )

       env_reset      = no         ( no yes )

       autolinkenable = f          (SP.0 enabled, SP.1 enabled, SP.2
enabled, SP.3 enabled)

       hw_qt          = off        ( off on )

       hw_ft          = off        ( off on )

       hw_diag        = off        ( off on )

 

SSC PROM> env set autoload on

SSC PROM> env set autoreboot off

SSC PROM> env set load_src disk

SC PROM> env set boot_dev sda1

   Note: The upper compact flash slot (compact flash 0) is boot_dev =
sda1. The lower compact flash slot (compact flash 1) is boot_dev = sdb1

 


5.       SSC PROM Upgrade instructions


This procedure updates the SSC PROM. It is done while the system is down
and from the SSC console.

Connect to your SSC console via a telnet session. (Find something else
to do  if you don't know how to do this.)

Shut the system down from the NFX Shell:

-          nfxsh

-          system reboot -y

When the system starts rebooting type:

-          <ctrl> e

Upgrade the standard and recovery SSC PROMs

-          bsd rt add 10.0.0.0 255.255.0.0 10.2.0.1

-          bsd ip add fp0 <ip-address> 255.255.0.0 -  Use the ip-address
of your cougar - 

-          prom upgrade 10.0.0.143 rel-proms/sibfimg_cg.bin

-          prom upgrade 10.0.0.143 rel-proms/sibyte_cg.bin

 


6.       Runtime PROM Upgrade instructions


Close all rcon sessions you have open before you run this procedure.

This procedure upgrades the FP and TXRX PROM images from runtime.

-          Login as root

-          nfxsh

-          port show all - Note what links are up, you will need to
re-enable the sp2.n interfaces at the end of this procedure.

-          prom upgrade files
http://10.2.0.21/upgrade/PROM-1.0.6-CG.tar.gz

This will get the PROM files and place them in the /boot dir on CF 

-          prom upgrade standard

Takes about 5 minutes, system will reboot upon successful PROM upgrade 

 

After the system reboots upgrade the Recovery PROMs. The PROM
distribution file is on the compact flash.

-          Login as root

-          nfxsh

-          if necessary: port enable sp.2.n - where n equals the last
digit - for all interfaces noted at the beginning of this procedure

-          prom upgrade recovery

Takes about 1.5 minutes, system will reboot upon successful PROM upgrade

 

Due to a prom upgrade bug the ports are disabled and not re-enabled
automatically. Your volumes are not visible to the virtual servers after
the reboots. You need to manually enable the links by performing the
following steps.

After the system reboots

-          Login is root

-          nfxsh

-          port enable sp.2.n - where n equals the last digit - for all
interfaces noted at the beginning of this procedure.

 


7.       Restrictions/limitations in this submittal


7.1.      Restrictions in this submittal


*         You must upgrade your systems with the latest firmware; follow
the instructions in sections 5 and 6.

*         It is recommended you run eek on your volumes to test for
corruption

 


7.2.      Restrictions from previous submittal that still apply to this
submittal


DMIP mirror sessions between cougar and cheetah will fail with: SANM_AG:
no listening session.

 

You need to have your autoload and autoreboot settings on your TXRX and
FP cores match your SSC settings.

*         For the SSC you need to be in the SSC console to set the prom
enviroment

*         For the TXRX and FP you can use rcon  to set the prom
environment

 


7.3.      Problems encountered during Acceptance Testing


*         The mirror tests in the test harness reported 14 out of 15
tests passing. Checking the logs the failures appear to be bugs in the
test harness mirror tests and not problems with the filer.

 

 


8.       Details of New Features introduced in submittal 12


8.1.      I/O Coalescing


Coalescing is only enabled for user read and write.

Do not test diag commands

Do not use nfxsh to modify the file system defaults

 


8.2.      Root level storage command


Usage: storage view dev stats -d

Output changed substantially; it only lists LUNS that are non-zero

 


8.3.      TXRX command: nfsperftest


Run this using rcon to the ACPU. The options have changed, run without
arguments to get online help.

You can specify volume names.

This exercises the whole system including the NCPU. 

 


8.4.      FC diag command: scsiadmin trace


New option: sds sds 

This gives copious amount of output. It shows every SCSI descriptor.

 


8.5.      New Analytics


A great deal of analytics have been added the SCSI code. Expect to see
more asserts in SCSI code when the SCSI debug analytics are tripped.

 


9.       Requirements to boot from Compact Flash


Booting from compact flash requires the latest prom release see sections
5 and 6 for instructions on how to upgrade your PROMs.

QA cougar systems should already be running the latest PROM revision as
it was a requirement for previous submittals.

 

Booting from compact flash requires you to install 1 Gbyte Compact Flash
cards into the upper and lower slots of your cougar. Run the
flash_install.sh script from your NFS root per the instructions found
here: http://wiki.onstor.net/wiki/Flash_initialization

 

Use the feature of flash_install.sh that copies the configuration files
from the running OS to the compact flash.

 When you see:

     Is this an upgrade? If you answer yes the current configuration
will be copied to the new flash [N] 

Answer 'Y' to copy your configuration information from the NFS root to
your compact flash.

 

A system copy init followed by system upgrade -s will efficiently
upgrade an initialized flash card. It will copy the configuration data
and files from the primary flash to the standby flash.

 


10. New/Changed/Obsolete modules/binaries


10.1.   Changed in submittal 18


Change 28776 reordered the startup of the SSC daemons. Clustering starts
before sdm and evm, since both use clustering. Sdm now queries
clustering at initialization rather than waiting 30 sec. Snmpd and asd
are delayed until just before the start of sscccc.

 


10.2.  Introduced in submittal 2:


Changes 27104 and 27105 remove the reference to the name "agile" from
nearly everywhere.

It also changes the name and location of trace files and logs.

 

The elog is now in /var/log/onstor/messages

The trace files are stored in /var/onstor

Samba logs are in /var/log/onstor/samba

 


10.3.   Introduced in submittal 14:


The cluster_contrl process was removed from /onstor/etc/pmtab and is now
started by the cluster_server process. You should be able to kill any
cluster_server or cluster_contrl process and all 3 will be restarted in
the correct sequence.

 


11. Troubleshooting tips


Turn elog level to info.

Watch elogs: Have elog display enable and watch elogs on ssc console, or
run tail -f /var/log/onstor/messages

 


12. Special focus/concerns/additional testing


Development is interested in looking at SSC crashes with DMALLOC
enabled. If you encounter SSC daemon problems contact development.

 


13. Pre-submittal tests results


13.1.  Build verification test results


 

Test run

Pass/Fail/Not run

Upgrade 

Pass

NFS traffic (no jumbo frames)

Pass

CIFS traffic (no jumbo frames)

Pass

Failover

Not run

 

 


13.2.  Acceptance test results

Test Run

Pass/Fail/Not run

Snapshot 

Pass

Mirror 

Pass

Use of NCM for Configuration and Setup

N/A

NFS and CIFS Traffic

Pass 

Upgrade/Flash_install  on Cheetah and Bobcat

Pass

Smbtorture

Pass

Online eek

N/A

Quota reporting

N/A

FTI

Not tested

NDMP Backup/Restore

N/A

 





On Thu, 19 Jun 2008 18:19:33 -0700 "John Rogers"
<john.rogers@onstor.com> wrote:

> I much prefer submitall notes to andy's yammering. Build strings and
> tar file exact locations, a list of gotchas along with any particular
> procedure eng would like me to follow. I'll have it done today if you
> guys can pull that together for me. 
> 
> -----Original Message-----
> From: Andy Sharp
> To: John Rogers
> CC: Sandrine Boulanger; Vikas Saini; Larry Scheer
> Sent: Thu Jun 19 18:08:21 2008
> Subject: Re: How come we did not upgrade MD today to sub#26 beta?
> 
> On Thu, 19 Jun 2008 17:59:12 -0700 "John Rogers"
> <john.rogers@onstor.com> wrote:
> 
> > Not sure. Did I miss the announcement that it was ready?
> 
> I thought Vikas volunteered to do the upgrade.  Or may just if John
> didn't make it back today.
> 
> John: please take care of upgrading MD to sub#26beta today.
> Thankyouvermuch.
> 
> Problem solved!
> 
> But seriously, our finely tuned schedule calculations sorta rely on MD
> being upgraded to sub#26 today.  Is that possible?  Would you need any
> help?  No system copy all -i needed ~:^)  No eek needed either, nes
> paux?  Ho yeah, I can speak french all right.  But you probably wanted
> to run eek anyway, just to show off.
> 
> Cheers,
> 
> a
> 
> > -----Original Message-----
> > From: Sandrine Boulanger
> > To: Vikas Saini; John Rogers
> > CC: Andy Sharp
> > Sent: Thu Jun 19 17:49:41 2008
> > Subject: How come we did not upgrade MD today to sub#26 beta?
> > 
> > 
