Apticron is very easy to set up, it’s in the repositories and requires basically no configuration.  It will drop a script in /etc/cron.daily and that is about it, emailing any reports to root.  Of course this can be modified through a .forward or an entry in /etc/aliases.

Logcheck is fairly simple to set up as well – it is also in the repositories.  Once installed, edit the /etc/logcheck/logcheck.conf file to configure.  The first thing you will want to set is the REPORTLEVEL setting, options are “workstation”, “server” (default value), or “paranoid”.  I use server on mine, which gives a good amount of detail. I would advise against using paranoid unless the server is extremely locked down and users do not typically login.  Workstation is good for a desktop environment.  The only other variable I edited was SENDMAILTO.  Logcheck works by basically comparing each  logentry against a set of regular expressions and generate a report if it does not match.  I had to modify one or two regex’s slightly to fix false positives, if you want my changes just ask and I’ll send them over.

One other small gem I want to mention : gkrellm.  I use this on both my desktop and server, it is invaluable for providing real-time system performance metrics.  Sure, it does not have any logging capabilities and thus unsuitable in a large-scale environment but for keeping an eye on one or two boxes it fits the bill quite nicely.

• Firstly, now you can you use the same script across multiple boxes – the UUIDs are configurable per hostname
• A bug was fixed where the script would fail if the destination disks were not already mounted.
• You can also customize the rsync invocation on a host-basis as well.  This was needed on my desktop machine where a /home account was mounted via NFS on a different file system causing IO errors and subsequently skipping the file deletion.

One final enhancement I want to add is the ability to spin the backup disk down after rsync is complete – this will not only help to increase drive life but also help reduce power use (however small it may be).  For some odd reason on my machines whenever I stop (umount, sync, then spin-down) an internal disk it works for a few seconds, then the drive spins back up again and I’m seeing ATA link reset messages as if it was just being plugged in.  External drives connected with eSATA seem to work just fine however.  Need to look into that more.

Also shortly available:

• A similar but different script I use for syncing my RAID storage array with an backup external drive connected via eSATA.  It’s a bit of a hack in some spots, but the nice thing is that it is almost fully automated.  Use this on a machine with one of those eSATA docks and you have a good way of making a quick backup of an array or disk.
• Sample service account script implementation with the ‘chattr’ command.

Download or wiki.

1. Run ssh-keygen -t rsa.  I specified a simple passphrase for general-purpose logins.  We will be adding the second phrase-less key later.

   (adechiaro@desktop:pts/6)-(4/0 @ 17k)-(09:22 PM)
   ~$ ssh-keygen -t rsa
   Generating public/private rsa key pair.
   Enter file in which to save the key (/home/adechiaro/.ssh/id_rsa):
   Enter passphrase (empty for no passphrase):
   Enter same passphrase again:
   Your identification has been saved in id_rsa.
   Your public key has been saved in id_rsa.pub.
   The key fingerprint is:
   5d:34:f1:de:ad:be:ef:65:34:36:a4:0d:75:d6:3c:47 adechiaro@desktop
   The key's randomart image is:
   +--[ RSA 2048]----+
   ...
   +-----------------+
   (adechiaro@desktop:pts/6)-(4/0 @ 17k)-(9:22 PM)
   ~$ cat .ssh/id_rsa.pub
   ssh-rsa <key> adechiaro@desktop

2. Any machine you want to be able to connect to with this key, login and copy the contents of your public key (id_rsa.pub) to the authorized_keys file.  These are all in your $HOME/.ssh/ directory.  There are various ways to do this: you could copy the file over with scp and cat/append it, you could remote in to the host and cut & paste the data, if you had a large infrastructure you could use ssh-copy-id or similar custom script.  It’s up to you, something like what is below should work in the general case.  Also the <key> is your public key in base64 encoded format.

   desktop:~$ scp .ssh/id_rsa.pub adechiaro@server:~
   id_service.pub                           100%  399     0.4KB/s   00:00
   desktop:~$ ssh adechiaro@server
   server:~$ cat id_rsa.pub >> ~/.ssh/authorized_keys

3. Now for an example of making the key more secure, you can add additional options to the authorized_keys file.  These come before the “ssh-rsa <key>” part of the entry (prefix the line with them):
   1. from=”host1,host2,10.0.0.1″ – This will prevent the key from authenticating except the hosts listed here, canonical name or IP.
   2. command=”/usr/local/bin/myscript.sh”, no-pty – Executes command upon login.  You might want to combine this with no-pty which prevents tty allocation (shell login) so you can securely execute a remote command and without granting direct login access (we will be doing this for our service account)
   3. There are a lot of more options – read the sshd manpage under the authorized_keys section for more.
4. Create the second public key saved with a different name.  Leave this with an empty passphrase as will be used for automation.

   ssh-keygen -f id_service

5. You will need to either write a simple script or use an existing one for linking with this key.  This will be run every time the account is logged into, regardless of what command may be passed on the SSH command line (the command line arguments will be read by the script but not necessarily executed).  Here is a simple generic one I threw together for general purpose use (download or view in wiki).   I do not make any promises this is completely bugfree/secure nor am I liable for any consequences!  If you use mine, you will want to configure the COMMANDS variable within the script.  These are the specific commands which the service account will be permitted to run.  A command specified on the SSH command line which does not match any entry in COMMANDS will not be run.  Also, make sure the file is chmod 500 after you are done configuring it, it’s basically a homebrew sudoers file and vital no other users can read the contents.
6. Copy your second key to the other machines as in step 2.  You will want to to prefix this new entry with ‘command=”/usr/local/bin/service.sh”, no-pty’, adjusting the path and script name as needed.  You may also want to specify ‘no-port-forwarding,no-X11-forwarding’ as additional security measures.
7. Done!

Now to run, simply connect via ssh.  It will default to your id_rsa key:

desktop:~$ ssh adechiaro@server
Enter passphrase for key '/home/adechiaro/.ssh/id_rsa':
server:~$

If you want to use your service account in a script, call it in the following manner:

desktop:~$ ssh adechiaro@server -i ~/.ssh/id_service /usr/bin/whoami
Executing command: "/usr/bin/whoami"
adechiaro
desktop:~$ ssh adechiaro@server -i ~/.ssh/id_service /bin/hostname
Executing command: "/bin/hostname"
server
desktop:~$

We need to specify the full path to the key in this case.  Replace whoami or hostname with whatever script/app you want to run and assuming you pre-configured it the service.sh script correctly, it should run just fine without prompting you for the passphrase.  Please feel free to leave any question and/or comments.

References:

http://www.securityfocus.com/infocus/1810
http://standalone-sysadmin.blogspot.com/2008/11/host-to-host-security-with-ssh-keys.html
http://standalone-sysadmin.blogspot.com/2008/11/wacky-ssh-authorized-keys-tricks.html
http://it.toolbox.com/blogs/unix-sysadmin/playing-with-openssh-public-keys-28377
http://blog.funnelfiasco.com/?p=44

escape ^||
msgwait 2

altscreen on
autodetach on
defscrollback 4000
startup_message off

# Message/bell/activity info
vbell off
vbell_msg " *beep* "
sorendition "+b kG"
bell "%c:bell -> %n%f %t^G"
activity "%c activity -> %n%f %t"

# Don't block screen session waiting for unresponsive window
nonblock on

hardstatus alwayslastline '%{= gk}%-Lw%{= rW}%50> %n%f* %t %{-}%+Lw%< %= %{= kG} %H %{= Bw} %l %{= kG} %Y-%m-%d %c %{g}' 

# Window number starts at 1, not 0
bind 'q' quit
bind c screen 1
bind 0 select 10

# Tell screen that you term can scroll and bind Shift+PgUp/PgDn
termcapinfo xterm ti@:te@
bindkey -m "^[[5;2~" stuff ^b
bindkey -m "^[[6;2~" stuff ^f

# Setup our default apps
screen -t htop 1 htop
screen -t iptraf 2 nice sudo iptraf
screen -t messages -M 3 tail -n 2000 -F /var/log/messages
screen 4 bash
select 1

OK, so firstly I like using Ctrl-Pipe for some reason – ‘^|’  I set some initial startup parameters, autodetach useful in case you need to kill X for some reason.  Then is the bell & activity customizations, nothing special to look at here.  My hardstatus is something I threw together based upon several I’ve seen online and the custom one I wrote for work.  I configured it to start numbering with ‘1′ instead of ‘0′ for window numbers and for xterm to recognize scrolling with my mousewheel.  Final section is just setting up some default windows.  I found the ‘-F’ flag to be vital versus the typical lowercase ‘-f’, as it implies ‘–retry’.  This is needed due to log rotation, it will force tail to retry opening file upon failure.  Otherwise once the file gets rotated, output from tail will just stop.

Screenshots:

Here are some basic configuration steps.  We will be using mail.myrelayhost.com and testing delivery to bob@aol.com as an example.  I have not tested this with DynDNS relaying in some time now but I see no reason that would prevent it from working if you use their service.  Note: These commands must be run with root permissions via sudo.

Configuration

1. Setup a password maps file /etc/postfix/sasl_passwd with the following:

   mail.myrelayhost.com    username:password

2. Secure it with the following:

   chown root:root /etc/postfix/sasl_passwd
   chmod 600 /etc/postfix/sasl_passwd
   postmap /etc/postfix/sasl_passwd

3. Edit /etc/postfix/main.cf, adding/or editing the following:

   relayhost = mail.myrelayhost.com:port
   smtp_sasl_auth_enable = yes
   smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd

   You can also edit /etc/mailname to change the domain/hostname outbound emails appeared to be delivered from (assuming main.cf has an entry “myorigin = /etc/mailname” which should be there by default)

4. Reload postfix: [ postfix reload ]
5. SASL libraries may need to be installed. On my Hardy machines, I have the following packages installed:

   root@eternal:~# dpkg -l *sasl* | grep -G '^ii'
   ii  libgsasl7                                 0.2.21-1                            GNU SASL library
   ii  libsasl2-2                                2.1.22.dfsg1-18ubuntu2              Cyrus SASL - authentication abstraction libr
   ii  libsasl2-modules                          2.1.22.dfsg1-18ubuntu2              Cyrus SASL - pluggable authentication module

6. Optional: Since we are going this far it makes sense to configure /etc/aliases as well.  This will allow us to forward the mail delivered to a local user to an external address instead:

   root@eternal:~# cat /etc/aliases
   # Added by installer for initial user
   root:   rigel@mydomain.net
   adechiaro:  adechiaro@mydomain.net

Testing/Debugging Steps

1. Check to see if Postfix recognizes your password maps (should return your username/password):

   postmap -q mail.myrelayhost.com /etc/postfix/sasl_passwd

2. Do a basic outbound email delivery test to a working email account:

   echo "relaying works!" | mailx bob@aol.com

   Check /var/log/mail.log (or similar) to see if the message was delivered correctly:

   Oct 18 13:01:04 eternal postfix/pickup[6410]: A88AF25E79: uid=1000 from=<anthony>
   Oct 18 13:01:04 eternal postfix/cleanup[6507]: A88AF25E79: message-id=<20081025170104.A88AF25E79@eternal>
   Oct 18 13:01:04 eternal postfix/qmgr[6411]: A88AF25E79: from=<anthony@eternal>, size=397, nrcpt=1 (queue active)
   Oct 18 13:01:05 eternal postfix/smtp[6521]: A88AF25E79: to=<bob@aol.com>, relay=myrelayhost.com[208.67.217.132]:25, delay=0.55, delays=0.03/0.01/0.22/0.29, dsn=2.0.0, status=sent (250 OK id=1KtmVL-0000ol-RU)
   Oct 18 13:01:05 eternal postfix/qmgr[6411]: A88AF25E79: removed

3. If getting SASL authentication errors, check your postfix options (either postconf command or main.cf file).  By default Postfix will not send cleartext passwords.  If your host only allows PLAIN or LOGIN methods, you will need to remove noplaintext from the SASL security options:

   postconf -e smtp_sasl_security_options=noanonymous
   postfix reload

4. If you are using GMail SMTP (or other large-scale host which uses load balancing) I’ve read the following might be necessary:

   postconf -e smtp_cname_overrides_servername=no

   If this resolves the problem, it’s due to the fact the server you specify in sasl_passwd might actually get delivered to a server with a different hostname.  Port 587 may be necessary for GMail as well.

*No affiliation with DynDNS, except for being a satified customer

Update (Nov 02): I also found an application called ssmtp which does a similar thing, it may be easier to configure and use although I have no experience with it.  You can see how to configure it here.

Update 2 (Nov 12): Just recently saw an article from my ubuntu-tutorials.com RSS feed on SMTP relaying with Postfix through GMail.

I did base my design upon some very good backup reference information.  Plus with this way it would be easy to boot from if ever needed and take tar archives of the data without dealing with file consistency or modification issues.

Here’s how it works:

Duplicate the partition table for your source/OS drive (or whichever drive you want to use) .  I did this manually with fdisk, but you could easily do it with:

dd if=/dev/<src> of=/dev/<dest> bs=512 count=1

Obviously replacing <src> and <dest> with the source and destination disks – it is critical you have them in the right order.  However this won’t work with extended partitions as they have additional tables.  You can read more here if you like.

Configure fstab to use UUID’s if not already done.  Have the partitions on the mirror/destination disk mounted under a common mount point, eg: /backup.

# /dev/sda2
UUID=50128bb8-7aa4-4b45-b2c6-5e406c641004 /               ext3
# /dev/sda1
UUID=f0cd2333-868c-4a31-8386-554070cdfc5e /boot           ext3
# /dev/sda3
UUID=62f0da84-24af-485b-8cab-81c90e740156 /home           ext3
UUID=adbd21df-2d93-4868-8272-74c4fd98ca71 /backup         ext3
UUID=32be4980-0bac-463c-bf54-2d2eb59e3942 /backup/boot    ext3
UUID=936c69b7-f076-4140-9e80-142858907862 /backup/home    ext3

I also have the /backup partitions mounted read-only and only change when actually doing the backup.

Configure the correct settings in the script.  We want to specify the source partitions to backup.

UUIDS=( 50128bb8-7aa4-4b45-b2c6-5e406c641004 f0cd2333-868c-4a31-8386-554070cdfc5e 62f0da84-24af-485b-8cab-81c90e740156 )

BKUP_MNT=/backup

You can download the script or view it in my wiki.  Feel free to leave comments and suggestions.  I did some basic debugging and seems to work just fine for me, but I am not held responsible for any consequences.

Update: See latest script version here