AF:
NF:0
PS:10
SRH:1
SFN:
DSR:
MID:<20070917103323.16e3cd2a@ripper.onstor.net>
CFG:
PT:0
S:andy.sharp@onstor.com
RQ:
SSV:onstor-exch02.onstor.net
NSV:
SSH:
R:<deepak.veliath@onstor.com>,<tim.gardner@onstor.com>,<jeyaramg@onstor.com>,<dl-cougar>
MAID:1
X-Sylpheed-Privacy-System:
X-Sylpheed-Sign:0
SCF:#mh/Mailbox/sent
RMID:#imap/andys@onstor.net@onstor-exch02.onstor.net/INBOX	0	BB375AF679D4A34E9CA8DFA650E2B04E0587B974@onstor-exch02.onstor.net
X-Sylpheed-End-Special-Headers: 1
Date: Mon, 17 Sep 2007 10:35:11 -0700
From: Andrew Sharp <andy.sharp@onstor.com>
To: "Deepak Veliath" <deepak.veliath@onstor.com>
Cc: "Tim Gardner" <tim.gardner@onstor.com>, "Jeyaram Sankar Gurusamy (HCL)"
 <jeyaramg@onstor.com>, dl-cougar
Subject: Re: migration actions items
Message-ID: <20070917103511.02ee3fc3@ripper.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E0587B974@onstor-exch02.onstor.net>
References: <BB375AF679D4A34E9CA8DFA650E2B04E0579D760@onstor-exch02.onstor.net>
	<BB375AF679D4A34E9CA8DFA650E2B04E0587B7A8@onstor-exch02.onstor.net>
	<BB375AF679D4A34E9CA8DFA650E2B04E0587B974@onstor-exch02.onstor.net>
Organization: Onstor
X-Mailer: Sylpheed-Claws 2.6.0 (GTK+ 2.8.20; x86_64-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Sounds like we can use this to implement the "take the bobcat flash and
stick it in the cougar" method of migrating config data.  We can
automate everything in here in a script to import bobcat data.

I still think we should migrate/import data via the storage or network,
but in cases where that isn't possible, this sounds like it will work.

Cheers,

a


On Mon, 17 Sep 2007 10:21:26 -0700 "Deepak Veliath"
<deepak.veliath@onstor.com> wrote:

> Hello Tim and Andy,
> I'm including a write-up I'd sent Jeyaram on the problem you'll have
> of migrating data from the OpenBSD based Bobcat to Linux based Cougar.
> 
> 
> 
> 	_____________________________________________ 
> 	From: 	Deepak Veliath  
> 	Sent:	Friday, September 14, 2007 1:40 AM
> 	To:	Jeyaram Sankar Gurusamy (HCL); Narain Ramadass
> 	Subject:	RE: migration actions items
> 
> 	Hello Jeyaram and Narain,
> 	 
> 	The problem really is in two parts as Andy said.
> 	The first is being able to read OpenBSD or ONStor's OpenBSD's
> disklabels in linux.
> 	 
> 	I poked around the disk and found that ONStor's disklabels do
> not reside in the boot sector as the default BSD disklabels do.
> 
> 		[BTW a disklabel is an extended partitioning scheme
> used by the BSDs.  You give BSD a primary partition and it
> sub-divides this into its own logical partitions.  All Unices except
> Linux have their own extended partitioning scheme including MP-RAS,
> Solaris, MS-DOS]
> 
> 	So because of the non-std disklabel location the Linux kernel
> -- which can detect BSD disklabels -- does not see these and the /dev/
> entries for these partitions are not created.
> 	 
> 	The disklabels in ONStor's CFs (OCF) reside in the sector
> after the boot-sector.  Its offset within this sector has also been
> shifted to 0.
> 	 
> 	 <<minlabel-ONStor.tbz2>> 
> 	I looked around for a tiny disklabel tool for Linux and found
> one called minlabel.  I modified it with the info above.  Running it
> on the OCF shows me the cylinder boundaries of all the ONStor
> partitions (All screen logs begin with my "hanging" shell prompt.
> The "@" means I'm running whatever follows in quotes as root):
> 
> 		<veliath@mleccha:/home/veliath/Downloads/minlabel-1.2>
> 		@ 'minlabel -r /dev/sda'
> 		# /dev/sda:
> 		type: ESDI
> 		disk: ESDI/IDE disk, label: cf488.label
> 		flags:
> 		bytes/sector: 512    sectors/track: 63
> 		tracks/cylinder: 16    sectors/cylinder: 1008
> 		cylinders: 993    sectors/unit: 1000944
> 		rpm: 3600    interleave: 1
> 		trackskew: 0    cylinderskew: 0
> 		headswitch: 0           # milliseconds
> 		track-to-track seek: 0  # milliseconds
> 		drivedata: 0
> 		 
> 		16 partitions:
> 		#        size   offset    fstype   Size(Kb)   Size(Mb)
> 		  1:   643104        0    4.2BSD     321552     314.0
> # (Cyl.    0 - 637)
> 		  2:    59472   643104      swap      29736      29.0
> # (Cyl.  638 - 696)
> 		  3:  1000944        0    unused     500472     488.7
> # (Cyl.    0 - 992)
> 		  4:   159264   702576    4.2BSD      79632      77.7
> # (Cyl.  697 - 854)
> 		  5:    39312   861840    4.2BSD      19656      19.1
> # (Cyl.  855 - 893)
> 		  6:    99792   901152    4.2BSD      49896      48.7
> # (Cyl.  894 - 992)
> 		  7:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  8:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  9:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  ::        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  ;:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  <:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  =:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  >:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  ?:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		  @:        0        0    unused          0       0.0
> # (Cyl.    0 - -1)
> 		
> 	A key feature of the BSD disklabeling scheme is that the 3rd
> partition represents the whole disk or primary BSD partition
> (depending on which level the BSD disklabel has been used).
> 	 
> 	Now the boot sector's partition table (MBR) that Linux sees
> does not seem to be used in the OCF.  So I created the last three
> partitions' information into the MBR using fdisk.  The MBR looks as
> below:
> 
> 		<veliath@mleccha:/mnt>
> 		@ 'fdisk -l /dev/sda'
> 		This disk has both DOS and BSD magic.
> 		Give the 'b' command to go to BSD mode.
> 
> 		Disk /dev/sda: 512 MB, 512483328 bytes
> 		16 heads, 63 sectors/track, 993 cylinders Units =
> cylinders of 1008 * 512 = 516096 bytes
> 
> 		   Device Boot      Start         End      Blocks   Id
> System
> 		/dev/sda1             698         855       79632   83
> Linux
> 		/dev/sda2             856         894       19656   83
> Linux
> 		/dev/sda3             895         993       49896   83
> Linux
> 	 
> 	Note that the BSD disklabeling numbers cylinders from 0, while
> Linux numbers from 1.  The reason I did not duplicate the 1st BSD
> partition is that when the Linux fdisk is asked to use cylinder 1, it
> always leaves the first 63 sectors out (it can be changed by entering
> expert mode, but who cares).  So we lose these sectors and cannot
> access the FS in BSD partition a.  However, we can always see BSD
> partition a's contents using the device node for the whole disk
> -- /dev/sda in this case.
> 	 
> 	So after creating these partitions all the four partitions and
> their FS contents are visible:
> 
> 		<veliath@mleccha:/mnt>
> 		@ 'file - < /dev/sda1'
> 		/dev/stdin: Unix Fast File system [v1]
> (little-endian), last mounted on /var, last written at Sat Sep  8
> 01:28:42 2007, clean flag 1, number of blocks  79632, number of data
> blocks 77055, number of cylinder groups 10, block size 8192, fragment
> size 1024, minimum percentage of free blocks 0, rotational  delay
> 0ms, disk rotational speed 60rps, SPACE optimization
> 
> 		<veliath@mleccha:/mnt>
> 		@ 'file - < /dev/sda2'
> 		/dev/stdin: Unix Fast File system [v1]
> (little-endian), last mounted on /tmp, last written at Sat Sep  8
> 01:28:41 2007, clean flag 1, number of blocks  19656, number of data
> blocks 18871, number of cylinder groups 3, block size 8192, fragment
> size 1024, minimum percentage of free blocks 0, rotational  delay
> 0ms, disk rotational speed 60rps, SPACE optimization
> 
> 		<veliath@mleccha:/mnt>
> 		@ 'file - < /dev/sda3'
> 		/dev/stdin: Unix Fast File system [v1]
> (little-endian), last mounted on /onstor/conf, last written at Sat
> Sep  8 01:28:41 2007, clean flag 1, number  of blocks 49896, number
> of data blocks 48087, number of cylinder groups 7, block size 8192,
> fragment size 1024, minimum percentage of free blocks 0,  rotational
> delay 0ms, disk rotational speed 60rps, SPACE optimization
> 
> 		<veliath@mleccha:/mnt>
> 		@ 'file - < /dev/sda'
> 		/dev/stdin: Unix Fast File system [v1]
> (little-endian), last mounted on /, last written at Sat Sep  8
> 01:28:42 2007, clean flag 1, number of blocks  321552, number of data
> blocks 311295, number of cylinder groups 40, block size 8192,
> fragment size 1024, minimum percentage of free blocks 0,  rotational
> delay 0ms, disk rotational speed 60rps, SPACE optimization 
> 	 
> 	The second problem is mounting these file-systems.  Linux 2.6
> has read-only UFS support in the kernel.  So I compiled it in in to
> mine and am able to mount the filesystems.  I then configured the
> kernel for MIPS and ensured that the UFS file-system support exists
> in the MIPS little-endian version as well.  BTW UFS is Sun's name for
> BSD's FFS. The FFS type has to be passed in as a mount option:
> 
> 		<veliath@mleccha:/home/veliath/Downloads/minlabel-1.2>
> 		@ 'mount -o ro,ufstype=44bsd /dev/sda3 /mnt'
> 	 
> 	So I hope the MO I'm suggesting is clear:
> 	1) We essentially mount the 1st BSD partition.  Its usually
> the root FS.  From it we find the associations of the partitons with
> their mount points.  We copy out what ever we need from this
> partition. 2) We dump the disklabels using minlabel and recreate them
> as MBR partitions visible to Linux using fdisk or sfdisk (we delete
> the existing FAT16 partition -- it's a place holder).  Note that we
> can only create 4 partitions in the MBR at a time.  The 1st disklabel
> partition is visible via the "whole disk" device node and so does not
> need a partition.  Most of the recent filers use only a total of 4
> partitions (a, d, e and f) -- so we have an MBR slot extra!  If we
> are to support more than 5 partitions we could consider using the std
> BSD labels or even extended-logical partitions.
> 	3) Force a re-read of the MBR partition table using the ioctl
> or worst-case Eject/Re-insert the OCF or reset the disk drive slot to
> force the updated MBR partitions to become visible.  We can optionally
> validate the partitions on Linux by running "file" on their
> corresponding device nodes to make sure FFS file-systems are visible
> there.
> 	4) Mount the partitions and copy off whatever is needed.
> 
> 	This can be scripted (hopefully including the eject &
> re-insert if it comes to it)
> 
> 	You'll could try duplicating it there at the GRO.  You'll will
> need an OCF from a bobcat, a USB CF reader and a Linux machine whose
> kernel you'll can rebuild.  I'm attaching the minlabel source tree and
> binary (built for my gentoo based Linux system).
> 
> 	I put the OCF I removed from my filer back into it and made
> sure the filer does not behave any differently now that its MBR has
> been changed.  I booted off it and everythings hunky-dory.
> 
> 	Let me know what you'll think.
> 
> 	veliath
> 
> 
> u> -----Original Message-----
> u> From: Jeyaram Sankar Gurusamy (HCL) 
> u> Sent: Monday, September 17, 2007 3:38 AM
> u> To: Tim Gardner
> u> Cc: Deepak Veliath
> u> Subject: RE: migration actions items
> u> 
> u> Tim,
> u> 
> u> Veliath has extensive experience in the similar areas.  I 
> u> have requested him to see if he can suggest a possible 
> u> solution for this problem and he has come up with a possible 
> u> approach.  Please feel free to talk him about this if this 
> u> problem hasn't been addressed.
> u> 
> u> Thanks and regards,
> u> G. Jeyaram Sankar
> u> HCL Technologies Ltd, 50-53 Greams Road, Chennai - 600006
> u> Phone: +91-44-28293298 x 2413
> u> Fax: +91-44-28294969
> u> Cell: +91 99401 30044
