X-Sylpheed-Account-Id:1
S:andy.sharp@onstor.com
SCF:#mh/Mailbox/sent
X-Sylpheed-Sign:0
X-Sylpheed-Encrypt:0
X-Sylpheed-Privacy-System:
RMID:#imap/andys@onstor.net@exch1.onstor.net/INBOX	0	2779531E7C760D4491C96305019FEEB52AC8BFF1B1@exch1.onstor.net
X-Sylpheed-End-Special-Headers: 1
Date: Thu, 7 May 2009 10:09:57 -0700
From: Andrew Sharp <andy.sharp@onstor.com>
To: Henry Lau <henry.lau@onstor.com>
Cc: Rich LaReau <richard.lareau@onstor.com>, "Alan Cooke (Glasshouse)"
 <acooke@css.glasshouse.com>, Larry Scheer <larry.scheer@onstor.com>, Bill
 Duffy <bill.duffy@onstor.com>, Doug Cook <doug.cook@onstor.com>, dl-cstech
 <dl-cstech@onstor.com>
Subject: Re: Truncated crontabs at OPM
Message-ID: <20090507100957.3dec45ea@ripper.onstor.net>
References: <20090507095907.5ec66171@ripper.onstor.net>
	<2779531E7C760D4491C96305019FEEB52AC8BFF1B1@exch1.onstor.net>
Organization: Onstor
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

I was told by Jobi some time ago that it is redundant now and doesn't
need it, especially since snapshots can be done at times not on the
hour.

On Thu, 7 May 2009 10:04:56 -0700 Henry Lau <henry.lau@onstor.com>
wrote:

> Snapadmin is needed to send the request to all file systems to check
> if it needs to perform schedule snapshot on the hour.
> 
> Regards,
> Henry
> 
> -----Original Message-----
> From: Andy Sharp 
> Sent: Thursday, May 07, 2009 9:59 AM
> To: Rich LaReau
> Cc: Alan Cooke (Glasshouse); Larry Scheer; Bill Duffy; Doug Cook;
> dl-cstech Subject: Re: Truncated crontabs at OPM
> 
> On Thu, 7 May 2009 08:45:36 -0700 Rich LaReau
> <richard.lareau@onstor.com> wrote:
> 
> > 
> > Yes, my guess is that any system which shipped to the field at
> > 3.3.0.0 will have this problem, as the upgrades to not seem to
> > resolve the problem.
> > 
> > Andy, isn't snapadmin also required?  In our hands, that entry is
> > not recreated when you configure a snapshot.
> 
> No, snapadmin is not required, the FS will do snapshots when they are
> scheduled, not when snapadmin happens to run ~:^)  At least that's
> what Jobi told me.  But don't mistake me, the newsyslog entry is very
> required and a bogus bug that it gets trashed.  I just hope it wasn't
> my bug ~:^)
> 
> > Rich
> > 
> > 
> > -----Original Message-----
> > From: Alan Cooke (Glasshouse) 
> > Sent: Thursday, May 07, 2009 6:22 AM
> > To: Andy Sharp; Larry Scheer
> > Cc: Rich LaReau; Bill Duffy; Doug Cook; dl-cstech
> > Subject: RE: Truncated crontabs at OPM
> > 
> > I have had a very similar problem both on the GH lab system bobcat1
> > and a customer system at Getronics. The result was that the log
> > files in /var never turned over and just kept growing. Neil Cook
> > identified the problem on the GH box and it was exactly the same at
> > Getronics. This is too much of a coincidence to not be a bug
> > somewhere in an upgrade. The GH lab box was running 3.3.2.7 at the
> > time and Getronics 3.3.2.5. Don't know what release they had come
> > from though.
> > 
> > Regards ... Alan.
> > 
> > -----Original Message-----
> > From: Andrew Sharp [mailto:andy.sharp@onstor.com]
> > Sent: 07 May 2009 00:37
> > To: Larry Scheer
> > Cc: Rich LaReau; Bill Duffy; Doug Cook; dl-cstech
> > Subject: Re: Truncated crontabs at OPM
> > 
> > On Wed, 6 May 2009 16:31:18 -0700 Larry Scheer
> > <larry.scheer@onstor.com> wrote:
> > 
> > > Yes, upgrade does not replace the crontab. It treats the crontab
> > > as part of the system configuration and copies the old file to
> > > the new installation.
> > 
> > Well, upgrade doesn't replace it exactly, but it does attempt to
> > "fix" it in a couple of minor ways during upgrade, but nothing in
> > this area. If memory servers, emrscron does some stuff with
> > crontab.  Like it wipes it and re-writes it whenever [feels like]
> > the system boots and at other times.  So possibly there is a bug in
> > emrscron where it's not recreating the file correctly.
> > 
> > BTW, the only entry that we should be keeping out of the ones you
> > highlighted is the one that runs newsyslog.  The rest of them are
> > nops.
> > 
> > 
> > > _____________________________________________
> > > From: Rich LaReau
> > > Sent: Wednesday, May 06, 2009 4:20 PM
> > > To: Bill Duffy
> > > Cc: Doug Cook; dl-cstech
> > > Subject: Truncated crontabs at OPM
> > > 
> > > 
> > > Hey Bill,
> > > 
> > > I had a look at all four nodes, they are all exactly the same.
> > > According to the logs I could find these systems were installed at
> > > 3.3.2.6 and then upgraded to 3.3.2.8.  They all are missing the
> > > parts highlighted in red below.  (This is a normal crontab that I
> > > got off our cslab 3.3.2.7 system.)
> > > 
> > > This couldn't possibly be a "corruption" on four different
> > > systems. If they were all bought and installed together then they
> > > might have all be initially upgraded from 3.3.0.0 where I believe
> > > there was crontab trouble.  Do you happen to know if this was the
> > > case?  I'll bet that the "system upgrade" keeps the old crontab,
> > > so if it's bad it doesn't get fixed with the upgrade.
> > > 
> > > This probably results in both their snapshot schedule problem and
> > > the lack of nightly elogs.  I wonder if it's possible that one of
> > > these maintenance scripts keeps the GUI from hanging as well?
> > > 
> > > Rich
> > > 
> > > 
> > > # crontab -l
> > > # DO NOT EDIT THIS FILE - edit the master and reinstall.
> > > # (/tmp/crontab.z15812 installed on Wed May  6 10:35:27 2009) #
> > > (Cron version -- $Id: crontab.c,v 1.1.1.1 2001/08/23 20:53:40
> > > maximk Exp $) # MAILTO=""
> > > SHELL=/bin/sh
> > > PATH=/bin:/sbin:/usr/bin:/usr/sbin:/onstor/bin
> > > HOME=/var/log
> > > #
> > > #minute hour    mday    month   wday    command
> > > #
> > > */10    *       *       *       *       /usr/libexec/atrun
> > > #
> > > # rotate log files every hour, if necessary
> > > 0       *       *       *       *       /usr/bin/newsyslog
> > > # send log file notifications, if necessary
> > > #*/1    *       *       *       *       /usr/bin/newsyslog -m
> > > #
> > > # do daily/weekly/monthly maintenance
> > > 30      1       *       *       *       /bin/sh /etc/daily 2>&1 |
> > > tee /var/log/daily.out # | mail -s "`/bin/hostname` daily output"
> > > root 30      3       *       *       6       /bin/sh /etc/weekly
> > > 2>&1 | tee /var/log/weekly.out # | mail -s "`/bin/hostname` weekly
> > > 2>output"
> > > root 30      5       1       *       *       /bin/sh /etc/monthly
> > > 2>&1 | tee /var/log/monthly.out # | mail -s "`/bin/hostname`
> > > 2>monthly output" root # do hourly snapshot maintenance 0   0-23
> > > 2>*  *
> > > 2>*   /onstor/etc/snapadmin
> > > 
> > > # gather config every day (at 23:05)
> > > 5       23      *       *       *       /onstor/bin/emrscron -g
> > > config
> > > 
> > > # gather stats every hour (on the half hour)
> > > 30      *       *       *       *       /onstor/bin/emrscron -g
> > > stats
> > > 
> > > # gather h_res_stats data every 3 min
> > > */3     *       *       *       *       /onstor/bin/emrscron -g
> > > h_res_stats
> > > 
> > > # keep the last 7 days of config and stats files (removing excess
> > > daily at 00:53) 53      0       *       *
> > > *       /onstor/bin/emrscron -t config; /onstor/bin/emrscron -t
> > > stats
> > > 
> > > # roll the log files for h_res_stats and keep only 7 log files
> > > around; send files to onstor 59      23      *       *
> > > *       /onstor/bin/emrscron -t h_res_stats; /onstor/bin/emrscron
> > > -s all
> > > 
> > > # send the kpi file every hour (the actual minute of this cron job
> > > is random; seeded by the mac address, so not all field machines
> > > send at the same time) 42      *       *       *
> > > *       /onstor/bin/emrscron -s kpi_stats #
> > > 
> > 
> > 
