AF:
NF:0
PS:10
SRH:1
SFN:
DSR:
MID:<20090507095830.12476443@ripper.onstor.net>
CFG:
PT:0
S:andy.sharp@onstor.com
RQ:
SSV:mail.onstor.net
NSV:
SSH:
R:<richard.lareau@onstor.com>,<acooke@css.glasshouse.com>,<larry.scheer@onstor.com>,<bill.duffy@onstor.com>,<doug.cook@onstor.com>,<dl-cstech@onstor.com>
MAID:1
X-Sylpheed-Privacy-System:
X-Sylpheed-Sign:0
SCF:#mh/Mailbox/sent
RMID:#imap/andys@onstor.net@exch1.onstor.net/INBOX	70441	102AB4F33EBBDB4C91915B145C8E9FB312972FB26D@exch1.onstor.net
X-Sylpheed-End-Special-Headers: 1
Date: Thu, 7 May 2009 09:59:07 -0700
From: Andrew Sharp <andy.sharp@onstor.com>
To: Rich LaReau <richard.lareau@onstor.com>
Cc: "Alan Cooke (Glasshouse)" <acooke@css.glasshouse.com>, Larry Scheer
 <larry.scheer@onstor.com>, Bill Duffy <bill.duffy@onstor.com>, Doug Cook
 <doug.cook@onstor.com>, dl-cstech <dl-cstech@onstor.com>
Subject: Re: Truncated crontabs at OPM
Message-ID: <20090507095907.5ec66171@ripper.onstor.net>
In-Reply-To: <102AB4F33EBBDB4C91915B145C8E9FB312972FB26D@exch1.onstor.net>
References: <20090506163704.7e006fb8@ripper.onstor.net>
 <200905071321.n47DLRL27397@mailhost-rtp.css.glasshouse.com>
 <102AB4F33EBBDB4C91915B145C8E9FB312972FB26D@exch1.onstor.net>
Organization: Onstor
X-Mailer: Sylpheed-Claws 2.6.0 (GTK+ 2.8.20; x86_64-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Thu, 7 May 2009 08:45:36 -0700 Rich LaReau
<richard.lareau@onstor.com> wrote:

> 
> Yes, my guess is that any system which shipped to the field at
> 3.3.0.0 will have this problem, as the upgrades to not seem to
> resolve the problem.
> 
> Andy, isn't snapadmin also required?  In our hands, that entry is not
> recreated when you configure a snapshot.

No, snapadmin is not required, the FS will do snapshots when they are
scheduled, not when snapadmin happens to run ~:^)  At least that's
what Jobi told me.  But don't mistake me, the newsyslog entry is very
required and a bogus bug that it gets trashed.  I just hope it wasn't
my bug ~:^)

> Rich
> 
> 
> -----Original Message-----
> From: Alan Cooke (Glasshouse) 
> Sent: Thursday, May 07, 2009 6:22 AM
> To: Andy Sharp; Larry Scheer
> Cc: Rich LaReau; Bill Duffy; Doug Cook; dl-cstech
> Subject: RE: Truncated crontabs at OPM
> 
> I have had a very similar problem both on the GH lab system bobcat1
> and a customer system at Getronics. The result was that the log files
> in /var never turned over and just kept growing. Neil Cook identified
> the problem on the GH box and it was exactly the same at Getronics.
> This is too much of a coincidence to not be a bug somewhere in an
> upgrade. The GH lab box was running 3.3.2.7 at the time and Getronics
> 3.3.2.5. Don't know what release they had come from though.
> 
> Regards ... Alan.
> 
> -----Original Message-----
> From: Andrew Sharp [mailto:andy.sharp@onstor.com]
> Sent: 07 May 2009 00:37
> To: Larry Scheer
> Cc: Rich LaReau; Bill Duffy; Doug Cook; dl-cstech
> Subject: Re: Truncated crontabs at OPM
> 
> On Wed, 6 May 2009 16:31:18 -0700 Larry Scheer
> <larry.scheer@onstor.com> wrote:
> 
> > Yes, upgrade does not replace the crontab. It treats the crontab as 
> > part of the system configuration and copies the old file to the new 
> > installation.
> 
> Well, upgrade doesn't replace it exactly, but it does attempt to "fix"
> it in a couple of minor ways during upgrade, but nothing in this area.
> If memory servers, emrscron does some stuff with crontab.  Like it
> wipes it and re-writes it whenever [feels like] the system boots and
> at other times.  So possibly there is a bug in emrscron where it's
> not recreating the file correctly.
> 
> BTW, the only entry that we should be keeping out of the ones you
> highlighted is the one that runs newsyslog.  The rest of them are
> nops.
> 
> 
> > _____________________________________________
> > From: Rich LaReau
> > Sent: Wednesday, May 06, 2009 4:20 PM
> > To: Bill Duffy
> > Cc: Doug Cook; dl-cstech
> > Subject: Truncated crontabs at OPM
> > 
> > 
> > Hey Bill,
> > 
> > I had a look at all four nodes, they are all exactly the same.
> > According to the logs I could find these systems were installed at
> > 3.3.2.6 and then upgraded to 3.3.2.8.  They all are missing the
> > parts highlighted in red below.  (This is a normal crontab that I
> > got off our cslab 3.3.2.7 system.)
> > 
> > This couldn't possibly be a "corruption" on four different systems.
> > If they were all bought and installed together then they might have 
> > all be initially upgraded from 3.3.0.0 where I believe there was 
> > crontab trouble.  Do you happen to know if this was the case?  I'll 
> > bet that the "system upgrade" keeps the old crontab, so if it's bad
> > it doesn't get fixed with the upgrade.
> > 
> > This probably results in both their snapshot schedule problem and
> > the lack of nightly elogs.  I wonder if it's possible that one of
> > these maintenance scripts keeps the GUI from hanging as well?
> > 
> > Rich
> > 
> > 
> > # crontab -l
> > # DO NOT EDIT THIS FILE - edit the master and reinstall.
> > # (/tmp/crontab.z15812 installed on Wed May  6 10:35:27 2009) #
> > (Cron version -- $Id: crontab.c,v 1.1.1.1 2001/08/23 20:53:40
> > maximk Exp $) # MAILTO=""
> > SHELL=/bin/sh
> > PATH=/bin:/sbin:/usr/bin:/usr/sbin:/onstor/bin
> > HOME=/var/log
> > #
> > #minute hour    mday    month   wday    command
> > #
> > */10    *       *       *       *       /usr/libexec/atrun
> > #
> > # rotate log files every hour, if necessary
> > 0       *       *       *       *       /usr/bin/newsyslog
> > # send log file notifications, if necessary
> > #*/1    *       *       *       *       /usr/bin/newsyslog -m
> > #
> > # do daily/weekly/monthly maintenance
> > 30      1       *       *       *       /bin/sh /etc/daily 2>&1 |
> > tee /var/log/daily.out # | mail -s "`/bin/hostname` daily output"
> > root 30      3       *       *       6       /bin/sh /etc/weekly
> > 2>&1 | tee /var/log/weekly.out # | mail -s "`/bin/hostname` weekly
> > 2>output"
> > root 30      5       1       *       *       /bin/sh /etc/monthly
> > 2>&1 | tee /var/log/monthly.out # | mail -s "`/bin/hostname` monthly
> > 2>output" root # do hourly snapshot maintenance 0   0-23 *  *
> > 2>*   /onstor/etc/snapadmin
> > 
> > # gather config every day (at 23:05)
> > 5       23      *       *       *       /onstor/bin/emrscron -g
> > config
> > 
> > # gather stats every hour (on the half hour)
> > 30      *       *       *       *       /onstor/bin/emrscron -g
> > stats
> > 
> > # gather h_res_stats data every 3 min
> > */3     *       *       *       *       /onstor/bin/emrscron -g
> > h_res_stats
> > 
> > # keep the last 7 days of config and stats files (removing excess
> > daily at 00:53) 53      0       *       *
> > *       /onstor/bin/emrscron -t config; /onstor/bin/emrscron -t
> > stats
> > 
> > # roll the log files for h_res_stats and keep only 7 log files
> > around; send files to onstor 59      23      *       *
> > *       /onstor/bin/emrscron -t h_res_stats; /onstor/bin/emrscron -s
> > all
> > 
> > # send the kpi file every hour (the actual minute of this cron job
> > is random; seeded by the mac address, so not all field machines
> > send at the same time) 42      *       *       *
> > *       /onstor/bin/emrscron -s kpi_stats #
> > 
> 
> 
