HSS UNIX Public Cookbook (Jan 09, 2006)

If the machine comes with its own diagnostic tools, then run a complete set of those on the machine.

If the machine doesn't come with its own diagnostic tools, use our copy of The Ultimate Boot CD v3.2 (a CD packed full of freely downloadable system diagnostic tools) to test the system.

After installing Linux on the system, run the following two benchmarks. These programs are installed by default as RPMs on all our systems.

First run /usr/bin/bonnie++.

touch /cf_hold
cd /var/tmp
mkdir bonnie++
cd bonnie++
bonnie++ -n 16
rm /cf_hold

Update /home/sysadmin/public_html/io-benchmarks.html with the results from the bonnie++ run.

Then run lmbench.

touch /cf_hold
cd /home/sysadmin/data/lmbench
/usr/bin/lmbench/config-run
/usr/bin/lmbench/results
cd results
make > /home/sysadmin/public_html/lmbench.txt
rm /cf_hold

If you have a Win98 or Win2000 (?) computer available, make a bootable DOS floppy, copy fdisk.exe to it (use Start -> Find to locate fdisk.exe on the Windows computer's hard drive). Then boot the GRUB/LILO afflicted box with that floppy, and do:

fdisk /mbr

You can also go out onto the 'Net and look at http://www.bootdisk.com: last time I checked, they had Win98 boot disk images which contained fdisk.exe.

If you have a Win XP install CD, you can boot from the Windows XP CD and press the "R" key during the setup in order to start the Recovery Console. Select your Windows XP installation from the list and enter the administrator password. At the input prompt, enter the command "FIXMBR" and confirm the query with "y". The MBR will be rewritten and GRUB/LILO will be erased. Press "exit" to reboot the computer. Note that for this, you will have to know the Administrator password for the afflicted computer. If you don't know it, this ain't a gonna help you.

If you use LILO as your boot loader, just before you reboot, do:

(as root)
lilo -R <desired once only image>

Example:

(as root)
lilo -R windows

If you use grub as your boot loader, just before you reboot, do:

(as root)
grub --batch <<EOT
1>/dev/null 2>/dev/null
savedefault --default=# --once
quit
EOT

Where "#" is the number of the boot image in /etc/grub.conf. The first boot image is 0, the second is 1, etc.

Example:

(as root)
#!/bin/sh
grub --batch <<EOT
1>/dev/null 2>/dev/null
savedefault --default=4 --once
quit
EOT

Connect the drive to the host, wait a bit and run dmesg to figure out what device in /dev you'll be accessing the drive via.

> dmesg
usb-storage: device found at 2
usb-storage: waiting for device to settle before scanning
  Vendor: LaCie     Model: Big Disk G379     Rev:
  Type:   Direct-Access                      ANSI SCSI revision: 04
usb-storage: device scan complete
SCSI device sda: 796594175 512-byte hdwr sectors (407856 MB)
sda: assuming drive cache: write through
SCSI device sda: 796594175 512-byte hdwr sectors (407856 MB)
sda: assuming drive cache: write through
 sda: sda1

You may have to do modprobe usb-storage before plugging in the drive.

Once Linux detects the drive, run fdisk on the device, delete the vfat partition, add a Linux partition (type 82), and write your changes.

Then format the new partition:

(as root)
mkfs.ext3 /dev/${device}

Get the RPM from here:

http://www.hss.caltech.edu/yum/fedora-core-3/en/os/i386/RPMS.extras/rsnapshot-1.1.6-1.noarch.rpm

Here's a sample /etc/rsnapshot.conf that keeps 30 days of backups, backs up /local to /misc/usbdrive/snapshots and logs what it's doing to /var/log/rsnapshot. If you want to do something different than this, you'll need to make a special copy of rsnapshot.conf for the host.

# rsnapshot.conf

snapshot_root   /misc/usbdrive/snapshots
no_create_root  0

cmd_cp          /bin/cp
cmd_rm          /bin/rm
cmd_rsync       /usr/bin/rsync
cmd_logger      /usr/bin/logger

interval        daily   30

link_dest       1
verbose         2
loglevel        3


logfile /var/log/rsnapshot

lockfile        /var/lock/subsys/rsnapshot

# Here's the important bit -- this is the directory we're backing up
# TAB SEPARATED!!!
backup  /local/         local/

Add this line to /etc/auto.master, if it doesn't already exist:

/misc           /etc/auto.misc

Then create /etc/auto.misc, if necessary, and add this line:

usbdrive -fstype=ext3  :/dev/$device"

where $device is the "sd[a-z][0-9]" device you discovered, above.

Restart the automounter:

(as root)
/sbin/service autofs restart

Create /etc/cron.daily/rsnapshot, and give it the following contents:

#!/bin/bash
/usr/bin/rsnapshot daily

rsnapshot is an implementation of Mike Rubel's rsync snapshot idea (http://www.mikerubel.org/computers/rsync_snapshots/). It uses rsync and hard links to keep a number of daily snapshots of the directory being backed up: go to the snapshot from 3 days ago, and you should see what the directory looked like 3 days ago.

The default configuration for rsnapshot is:

I make an automounter entry instead of relying on udevd to mount the drive under /media because I want a consistent name across all my systems which do this. udevd will mount the drive under /media/<token> where token is the disk label (maybe? not sure about where the name comes from) on the drive, which varies with manufacturer, model, etc.

The drive will come from the manufacturer formatted with a Windows (probably VFAT) filesystem. This doesn't do us any good, as if we copy our data to that filesystem, we lose our permissions and ownership information. So we need to repartition and reformat.

First, wait until the RAID rebuild has completed. If you look at the LCD panel on the front of the array, you'll see that it displays a percentage — this is how much of the rebuild it has completed.

If you touch any of the buttons on the front of the array, you'll lose the rebuild status display. So don't touch anything.

The RAID rebuild takes around 4.5 hours.

The drive caddies are locked into the array. Keys that will unlock them are attached to the drive case.

After the rebuild has completed, unlock the drive caddy with the red LED and remove it from the array. Remove the drive and toss it — I've tested drives the array rejects, and they sure are dead.

I store the cold spares on top of the array, so everyone will be able to find them easily.

Take one of the cold spares, make sure that it is an IDE master (by default, the drives are set to "Cable select", but I've made the appropriate modifications to the cold spares), and insert it into the caddy.

Then re-insert the caddy into the array, and lock it.

If you look at the LCD display on array after you do this, you should see the indicator for that slot change from "X" to "I" and at last to "S" ("No drive" -> "Identifying drive" -> "Hot spare").

The new drive is now the new hot spare.

I like to have two cold spares. Now that we've used one of the cold spares, we only have one left, and need to buy a new one.

The array was originally stocked with Seagate ST3120023A 120GB drives; I've replaced them with ST3120026A 120GB drives without problems.

This is the easier way if the drive is the primary drive in a computer on which you want to install Linux.

Boot the computer with a copy of the Ultimate Boot CD, go to the "Filesystem Utilities" section and choose FDISK.

Then just make your partitions, and presto, you have a new partition table.

Attach the drive to your Linux box, or boot the box it's in with a Linux Live CD (Knoppix, for example).

Zero out the partition table area of the disk with dd (using /dev/sdc for this example):

dd if=/dev/zero of=/dev/sdc bs=1024 count=10

Then just run fdisk on that disk and use "w" to write a new map — fdisk will write a fresh DOS partition map, like so:

> sudo fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 17274.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by
w(rite)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

cdrecord-ProDVD knows how to write to DVD+R and DVD+RW only drives.

This seems to be the distribution point for it:

http://gd.tuwien.ac.at/utils/schilling/cdrecord/ProDVD/

Go there and download the binary version with the latest version that is appropriate to your architecture.

cdrecord-ProDVD will look in ~/.prodvd.conf to figure out the device to write to, and what speed to use for that device.

You can reference files in /dev for the device. Here's an example ~/.prodvd.conf:

DVD RW DEVICE = /dev/hdd
DVD RW SPEED  = 4

Make your ISO image as usual with mksiofs. Then load a blank DVD+R or DVD+RW into your drive, and burn it like so:

(as root)
prodvd <iso image>

cdrecord-ProDVD is free only for private, non-commercial use and requires a security key to unlock write speeds greater than 1. It comes as a binary package only, and ships with a time limited key. When the key expires, you have to go back to the the distribution website and download a new copy of cdrecord-wrapper.sh: the key is in that file.

At least set DEFAULT_URL_HOST in mm_cfg.py.

Tar up the /data, /lists and /archives directories on the old server, and transfer the tar files to the new server.

(as root, on old server)
cd <mailman var directory>
# Probably /var/lib/mailman
tar zcf mmdata.tar.gz data
tar zcf mmlists.tar.gz lists
tar zcf mmarchives.tar.gz archives

Now copy mmdata.tar.gz, mmlists.tar.gz and mmarchives.tar.gz to the new server.

Now install the list data on the new server

(as root, on new server)
cd <mailman var directory>
# Probably /var/lib/mailman
tar zxf mmdata.tar.gz
tar zxf mmlists.tar.gz
tar zxf mmarchives.tar.gz

The mailman lists you copied over to the new server store the absolute path to their archive directories within the config files themselves. If the path to the mailman archive directories on the new server is different than the directory on the old server, you need to fix those paths. If you don't, you won't be able to find the list archives.

This bash script fixes those paths.

#!/bin/bash

newroot="<new mailman var directory>"
mailmanbin="<new mailman bin directory>"

for list in `ls $newroot/lists`; do
    python -i $mailmanbin/bin/withlist -l $list <<EOF
    m.private_archive_file_dir
    m.private_archive_file_dir='$newroot/archives/private/$list.mbox'
    m.public_archive_file_dir
    m.public_archive_file_dir='$newroot/archives/public'
    m.archive_directory
    m.archive_directory='$newroot/archives/private/$list'
    m.Save()
    EOF
done

The mailman lists you copied over to the new server store the URL to their admin pages, etc. within the config files. If the URL to the mailman CGIs are different on the new server than on the old server, you need to fix this. If you don't, none of your lists will show up on the "listinfo" page, you won't be able to configure any of the lists via the "admin" pages, or moderate messages on the "admindb" pages.

This bash script change the URLs in the config files to "mm_cfg.DEFAULT_URL_PATTERN % mm_cfg.DEFAULT_URL_HOST" (that's python code, FYI), where DEFAULT_URL_PATTERN and DEFAULT_URL_HOST are defined in mm_cfg.py.

#!/bin/bash

newroot="<new mailman var directory>"
mailmanbin="<new mailman bin directory>"

for list in `ls $newroot/lists`; do
    $mailmanbin/bin/withlist -l -r fix_url $list
done

The mailman binary may have a different path on the new server, so regenerate the aliases.

<new mailman bin directory>/genaliases

Be aware that the aliases file may show up in a different place on the new server than on the old.

If you want to allow people to get to the new list management pages via the old URLs, setup the following redirects on your old server for each list:

RedirectMatch /mailman/(.*)/mylist(.*) http://newlists.domain/mailman/$1/mylist$2
RedirectMatch /pipermail/mylist(.*) http://newlists.domain/pipermail/mylist$1

From the Mailman FAQ:

One caveat that might occur is you'll start getting notices about a missing stringIO modules. If this occurs to you then delete the following directories from the mailman directory and redo the above steps:

archives data lists locks logs qfiles spam

Under RH9, the sitewide admin password was stored in /var/mailman/data/adm.pw, but in FC3, it's stored in /etc/mailman/adm.pw.

Restore your admin password by running /usr/lib/mailman/bin/mmsitepass.

Under RH9, the Mailman aliases file was in /var/mailman/data/aliases, but in FC3, it's stored in /etc/mailman/aliases

Adjust your MTA's alias file settings appropriately.

Your milage may vary in regards to these instructions if your setup is different from this.

I'm going to store list data (membership, options, and archives) in /infosys/mailman/<hostname>, which is mounted from my NFS server. I'll mount this directory from both the primary and secondary SMTP server, and thus, they will both see the same list membership data and options, and will be able to append to the same archives.

I'm going to store the alias files and adm.pw (which holds the site admin password) in the same directory, so the secondary SMTP server will have its aliases updated immediately when lists are added or deleted on the primary.

Finally, I'm going to store all the moderation data (messages and sign ups to be moderated) in /infosys/mailman/<hostname>, as well. This so if one of the hosts goes down, I still have access to pending moderation requests.

Both machines should run the web interface. The primary needs it so that we can manage the lists, of course. The secondary needs it so that we can handle messages that need to be moderated.

All list maintenance, aside from moderation, should be done on the primary. Only the primary will be able to list the existing lists in the listinfo and admin views.

First, move the list data from the primary to the NFS directory:

(as root, on the primary)
mkdir -p /infosys/mailman/<hostname>
cd /var/lib/mailman

# archives
rsync -av archives /infosys/mailman/<hostname>
rm -rf archives
ln -s /infosys/mailman/<hostname>/archives .

# list data
rsync -av lists /infosys/mailman/<hostname>
rm -rf lists
ln -s /infosys/mailman/<hostname>/lists .

# moderation data
rsync -av data/ /infosys/mailman/<hostname>/data.primary
rm -rf data
ln -s /infosys/mailman/<hostname>/data.primary ./data

# config dir
rsync -av /etc/mailman/ /infosys/mailman/<hostname>/etc
rm -rf /etc/mailman
ln -s /infosys/mailman/<hostname>/etc /etc/mailman

Now setup the secondary:

(as root, on the secondary)
cd /var/lib/mailman

# archives
rm -rf archives
ln -s /infosys/mailman/<hostname>/archives .

# list data
rm -rf lists
ln -s /infosys/mailman/<hostname>/lists .

# moderation data
rsync -av data/ /infosys/mailman/<hostname>/data.secondary
rm -rf data
ln -s /infosys/mailman/<hostname>/data.secondary ./data

# config dir
rm -rf /etc/mailman
ln -s /infosys/mailman/<hostname>/etc /etc/mailman

/infosys/mailman/<hostname>/etc/mm_cfg.py should be a symlink to /usr/lib/mailman/Mailman/mm_cfg.py, so that each server gets its own version of mm_cfg.py.

Add these lines to mm_cfg.py on the primary, replacing the existing definitions for DEFAULT_URL_HOST and DEFAULT_EMAIL_HOST, and replacing "primary.yourdomain" and "yourdomain" appropriately:

DEFAULT_URL_HOST   = 'primary.yourdomain'
DEFAULT_EMAIL_HOST = 'yourdomain'

DELIVERY_MODULE = 'SMTPDirect'
SMTPHOST = 'localhost'
SMTPPORT = 25
MTA = 'Postfix'

Add these lines to mm_cfg.py on the secondary, replacing the existing definitions for DEFAULT_URL_HOST and DEFAULT_EMAIL_HOST, and replacing "secondary.yourdomain" and "yourdomain" appropriately:

DEFAULT_URL_HOST   = 'secondary.yourdomain'
DEFAULT_EMAIL_HOST = 'yourdomain'

DELIVERY_MODULE = 'SMTPDirect'
SMTPHOST = 'localhost'
SMTPPORT = 25
MTA = 'Postfix'

Mailman mails out monthly emails to people, reminding them that they are subscribed to our mailing lists. It also sends out mails once per day to disabled members, reminding them that they are disabled.

Since the primary already does these things, the secondary doesn't need to. Edit /etc/cron.d/mailman and comment out the lines referring to /usr/lib/mailman/cron/disabled and /usr/lib/mailman/cron/mailpasswds.

Ensure that the alias_maps line in /etc/postfix/main.cf for both your servers contains "hash:/etc/mailman/aliases".

If you're setting up your lists for the first time, do this on the primary:

(on primary, as root)
cd /usr/lib/mailman/bin
./mmsitepass
./newlist mailman

Mailman state that should not be squashed by another mailman instance is:

Extract from horde.sql all the "INSERT INTO horde_prefs" statements into a file called horde_prefs.sql.

We need to adjust the values in the rows of the horde_prefs table. The dump adds extra whitespace to the first three columns that we don't want.

sed "s/'\([a-z0-9_@.]\+\) \+',/'\1',/g" horde_prefs.sql > horde_prefs_fixed.sql

Start up postgresql on the new server, if it's not currently running:

sudo service postgresql start

Create the stub Horde 3 databases:

(as the postgres user)
export PGDATA=/var/lib/pgsql/data
cd /var/www/html/horde/scripts/sql
psql -d template1 -f create.pgsql.sql
psql -qc "ALTER USER horde WITH PASSWORD '<password>';" template1 postgres
psql -d horde -f horde_users.sql
psql -d horde -f horde_prefs.sql
psql -d horde -f horde_datatree.sql

Read in the horde_prefs rows from the old server:

(as the postgres user)
export PGDATA=/var/lib/pgsql/data
psql -d horde -f horde_prefs_fixed.sql

Make sure your servers.php setting for IMP for our IMAP server looks like this:

$servers['imap'] = array(
    'name' => '<domain>',
    'server' => 'localhost',
    'hordeauth' => false,
    'protocol' => 'imap/notls',
    'port' => 413,
    'folders' => '',
    'namespace' => 'INBOX.',
    'maildomain' => '<domain>',
    'smtphost' => 'localhost',
    'smtpport' => 25,
    'realm' => '<domain>',
    'preferred' => true,
    'dotfiles' => false,
    'hierarchies' => array()
);

The "realm" setting is vital, else the Horde prefs engine will look for prefs setting with prefs_uid set to <username> instead of <username>@<domain>, which was the Horde 2 default. This applies to Turba address book entries, too.

Extract from horde.sql all the statements relating to turba_objects: "INSERT INTO turba_object", "CREATE TABLE turba_object", etc.

Read in the turba_objects rows from the old server:

(as the postgres user)
export PGDATA=/var/lib/pgsql/data
psql -d horde -f turba_objects.sql

Now add the extra columns that Turba 2.0 needs:

(as the postgres user)
export PGDATA=/var/lib/pgsql/data
cd /var/www/html/horde/turba/scripts/upgrades
psql -d horde -f 1.2_to_2.0.mysql.sql

Don't fear the "mysql" in the file name — there are only straight SQL statements in that file.

Recent versions of Horde include a postgresql migration script.

The mod_auth_external RPM we use drops a file into /etc/httpd/conf.d with an appropriate LoadModule directive, so we don't have to add that.

If you want to use mod_auth_external in a virtual host, you must include the AddExternalAuth, SetExternalAuthMethod, AddExternalGroup and SetExternalGroupMethod inside the <VirtualHost> block. The <VirtualHost> does not inherit them from the global context.

Add this stanza either in the global context, or in a <VirtualHost> block. If you want to enable this for a virtual host, you must add this inside the <VirtualHost> block.

AddExternalAuth pwauth /usr/bin/pwauth
SetExternalAuthMethod pwauth pipe

Add this stanza either in a <Location>,<File>, or <Directory> block, or in an .htaccess file. If you're using the .htaccess method, ensure that you have "AllowOverride AuthConfig" in httpd.conf for the directory into which you want to put the .htaccess file.

AuthType Basic
AuthName <authname>
AuthExternal pwauth
Require valid-user

If you want to also restrict users by UNIX group, add these lines to the same place you added the "AddExternalAuth" line:

AddExternalGroup groupcheck "/usr/bin/groupcheck -e"
SetExternalAuthMethod groupcheck pipe

If you want to restrict users by AuthGroupFile style group (if, for instance, you have to admit some users who do not have UNIX accounts), add these lines, instead:

AddExternalGroup groupcheck "/usr/bin/groupcheck -f /path/to/group.file"
SetExternalAuthMethod groupcheck pipe

If you want to restrict users by UNIX group or AuthGroupFile style group (if, for instance, you want to admit some users who do not have UNIX accounts, plus some users who are in a standard UNIX group), add these lines, instead:

AddExternalGroup groupcheck "/usr/bin/groupcheck -e -f /path/to/group.file"
SetExternalAuthMethod groupcheck pipe

Then change your authorization block like this:

AuthType Basic
AuthName <authname>
AuthExternal pwauth
GroupExternal groupcheck
Require group <groupname>

This isn't going to work for you out of the box if you need to give access to a site to people both with and without UNIX accounts.

When using AuthExternal, you can't restrict access to a group of people with AuthGroupFile — Apache will refuse to authenticate anyone if you specify an AuthGroupFile line when using AuthExternal. Use GroupExternal instead, as in the examples above.

There is also mod_auth_pam, made (apparently) by the PAM maintaners, and you would think that this is what we would want to use. We don't use it because we use shadow passwords here, and mod_auth_pam can't read /etc/shadow.

/etc/shadow is owned by root, and is mode 0600. PAM is a library, and PAM enabled programs execute the PAM code as the EUID of the program. Since httpd runs as the apache user, it can't read /etc/shadow, and thus can't get the password hash for the user.

If we ever go to LDAP here, mod_auth_pam will become viable again.

Add this stanza either in the global context, or in a <VirtualHost> block. If you want to enable this for a virtual host, you must add this inside the <VirtualHost> block.

AddExternalAuth pwauth /software/bin/pwauth
SetExternalAuthMethod pwauth pipe

Add this stanza either in a <Location>,<File>, or <Directory> block, or in an .htaccess file. If you're using the .htaccess method, ensure that you have "AllowOverride AuthConfig" in httpd.conf for the directory into which you want to put the .htaccess file.

AuthType Basic
AuthName <authname>
AuthExternal pwauth
Require valid-user

If you want to also restrict users by UNIX group, add these lines to the same place you added the "AddExternalAuth" line:

AddExternalGroup groupcheck "/software/bin/groupcheck -e"
SetExternalAuthMethod groupcheck pipe

If you want to restrict users by AuthGroupFile style group (if, for instance, you have to admit some users who do not have UNIX accounts), add these lines, instead:

AddExternalGroup groupcheck "/software/bin/groupcheck -f /path/to/group.file"
SetExternalAuthMethod groupcheck pipe

If you want to restrict users by UNIX group or AuthGroupFile style group (if, for instance, you want to admit some users who do not have UNIX accounts, plus some users who are in a standard UNIX group), add these lines, instead:

AddExternalGroup groupcheck "/software/bin/groupcheck -e -f /path/to/group.file"
SetExternalAuthMethod groupcheck pipe

Then change your authorization block like this:

AuthType Basic
AuthName <authname>
AuthExternal pwauth
GroupExternal groupcheck
Require group <groupname>

This isn't going to work for you out of the box if you need to give access to a site to people both with and without UNIX accounts.

When using AuthExternal, you can't restrict access to a group of people with AuthGroupFile — Apache will refuse to authenticate anyone if you specify an AuthGroupFile line when using AuthExternal. Use GroupExternal instead, as in the examples above.

There is also mod_auth_pam, made (apparently) by the PAM maintaners, and you would think that this is what we would want to use. We don't use it because we use shadow passwords here, and mod_auth_pam can't read /etc/shadow.

/etc/shadow is owned by root, and is mode 0600. PAM is a library, and PAM enabled programs execute the PAM code as the EUID of the program. Since httpd runs as the apache user, it can't read /etc/shadow, and thus can't get the password hash for the user.

If we ever go to LDAP here, mod_auth_pam will become viable again.

First, you need to patch pam_pwdfile to supply a pam_sm_acct_mgmt() callback function — pwauth expects this, and will fail the authentcation if pam_pwdfile doesn't provide it. Here's the patch:

diff -Naur pam_pwdfile-0.99/pam_pwdfile.c pam_pwdfile-0.99.new/pam_pwdfile.c
--- pam_pwdfile-0.99/pam_pwdfile.c      2003-12-20 11:21:19.000000000 -0800
+++ pam_pwdfile-0.99.new/pam_pwdfile.c  2005-06-23 17:27:55.000000000 -0700
@@ -58,6 +58,7 @@
 #include <security/pam_appl.h>

 #define PAM_SM_AUTH
+#define PAM_SM_ACCOUNT
 #include <security/pam_modules.h>

 #include "md5.h"
@@ -414,6 +411,13 @@
     return PAM_SUCCESS;
 }

+/* another expected hook */
+PAM_EXTERN int pam_sm_acct_mgmt(pam_handle_t *pamh, int flags, int argc, const char **argv)
+{
+    return PAM_SUCCESS;
+}
+
+
 #ifdef PAM_STATIC
 struct pam_module _pam_listfile_modstruct = {
     "pam_pwdfile",

Ensure that, when building pwauth, you do not have UNIX_LASTLOG enabled in config.h. If it is enabled, pwauth tries to do some sanity checks on the UID of the user authenticating. Since users in pam_pwdfile's password file don't have UIDs, pwauth always rejects the login.

Make /etc/pam.d/pwauth look like this:

auth       sufficient   pam_pwdfile.so pwdfile $passwordfile
auth       sufficient   pam_stack.so service=system-auth
account    sufficient   pam_pwdfile.so pwdfile $passwordfile
account    sufficient   pam_stack.so service=system-auth

where $passwordfile is the full path to your pam_pwdfile password file.

One solution is to use mod_rewrite. Put these lines either in the global context, outside all <VirtualHost> blocks (for redirecting from the main site to the main site's SSL version), or inside the appropriate non-SSL <VirtualHost> block.

This only seems to work for directories.

RewriteEngine on
RewriteRule /path/to/directory/(.*) https://site.domain/path/to/directory/$1

Here's an example of using Redirect. You must put the Redirect line in a <VirtualHost> block and not in the global context — especially if you're redirecting /. You'll get a redirection loop if you put the Redirect outside a <VirtualHost> block because virtual hosts inherit Redirect lines from the global context.

<VirtualHost 1.2.3.4:80>
    ServerName site.domain
    ServerAdmin webmaster@domain
    Redirect /path https://site.domain/path
</VirtualHost>

<VirtualHost 1.2.3.4:443>
    ServerName site.domain
    ServerAdmin webmaster@domain
    ...
</VirtualHost>

Obtain mod_encoding from http://webdav.todo.gr.jp/download/, compile it up, and put it somewhere Apache can see it.

Add the following code to your httpd.conf file:

LoadModule encoding_module    /path/to/mod_encoding.so
LoadModule headers_module     /path/to/mod_headers.so
LoadModule dav_module         /path/to/libdav.so
LoadModule auth_module        /path/to/mod_auth.so

AddModule mod_encoding.c
AddModule mod_headers.c
AddModule mod_dav.c
AddModule mod_auth.c

#
# Broken WebDAV for Windows XP
#
BrowserMatch "^WebDAVFS/1.[012]" redirect-carefully
BrowserMatch "Microsoft Data Access Internet Publishing Provider" redirect-carefully
BrowserMatch "Microsoft-WebDAV-MiniRedir/5.1.2600" redirect-carefully
BrowserMatch "^WebDrive" redirect-carefully
BrowserMatch "^WebDAVFS" redirect-carefully

<IfModule mod_headers.c>
    Header add MS-Author-Via "DAV"
</IfModule>

<IfModule mod_encoding.c>
    EncodingEngine on
    NormalizeUsername on
</IfModule>

Next, add your DAV directives to httpd.conf. If "directory" is the directory you wish to DAV-ify (just an example — modify to suit your needs):

DAVLockDB /path/to/lockfile/DAVLock
DAVMinTimeout 600

<Directory "directory">
    DAV On
    AuthType Basic
    AuthName test
    AuthUserFile  /path/to/passwd/file
    AuthGroupFile /dev/null
    Require user test
</Directory>

Ensure that the directory into which you're writing your lock file (/path/to/lockfile) is writable by the user which runs the httpd process!

Now populate the passwd file:

htpasswd -c /path/to/passwd/file test

Ensure that the passwd file is readable by the user that runs httpd!

Restart httpd, and you should have a working setup.

See Setting up a Windows XP compatible WebDAV Folder under Apache 2.x and just follow the instructions; they should be directly translatable to Apache 1.3.x.

If you're using a self-signed certificate, and you haven't installed your Certifying Authority's root key in the XP client's certificate store, you'll be presented with a message about it. You won't be able to proceed until you install the certificate into your certificate store.

I describe using AuthBasic here, but I would bet that AuthDigest, et. al. would work as well. The real thing we're doing here is convincing XP that no, this is not a CIFS share.

First, I'm talking about XP Professional, not XP Home. I don't have any XP Home machines.

Second, I'm talking about XP machines with Service Pack 2 installed. I don't have any SP1 or before XP boxes lying around, so your milage may vary if you're using one of those as your DAV client.

Why does connecting via https://host.domain/davdir work when connecting via http://host.domain/davdir fails?

XP uses a different DAV client library for the https:// URLs than for the http:// URLs. The one used for http:// URLs is broken, while the one used for https:// URLs is not.

Below are some Apache logs from my DAV server machine which show this. Accesses are from the same XP client. Note that for the http:// accesses, the client is "Microsoft-WebDAV-MiniRedir/5.1.2600", while for the https:// accesses, the client is "Microsoft Data Access Internet Publishing Provider DAV".

1.2.3.4 - - [13/Apr/2005:16:40:21 -0700] "OPTIONS / HTTP/1.1" 200 - "-" "Microsoft-WebDAV-MiniRedir/5.1.2600"
1.2.3.4 - - [13/Apr/2005:16:40:21 -0700] "PROPFIND /davtest HTTP/1.1" 500 624 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600"
1.2.3.4 - - [13/Apr/2005:16:40:21 -0700] "PROPFIND /davtest HTTP/1.1" 500 624 "-" "Microsoft Data Access Internet Publishing Provider DAV"
1.2.3.4 - - [13/Apr/2005:16:41:19 -0700] "PROPFIND /davtest HTTP/1.1" 401 491 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600"
1.2.3.4 - - [13/Apr/2005:16:41:19 -0700] "PROPFIND /davtest HTTP/1.1" 401 491 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600"
1.2.3.4 - - [13/Apr/2005:16:41:19 -0700] "PROPFIND /davtest HTTP/1.1" 401 491 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600"

1.2.3.4 - - [13/Apr/2005:17:03:48 -0700] "OPTIONS / HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery"
1.2.3.4 - test [13/Apr/2005:17:03:48 -0700] "OPTIONS /davtest HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery" 1.2.3.4 - test [13/Apr/2005:17:03:48 -0700] "PROPFIND /davtest HTTP/1.1" 207 906 "-" "Microsoft Data Access Internet Publishing Provider DAV"
1.2.3.4 - test [13/Apr/2005:17:03:48 -0700] "PROPFIND /davtest HTTP/1.1" 207 964 "-" "Microsoft Data Access Internet Publishing Provider DAV"

Set up a directory for them.

mkdir /infosys/www/corinth.hss.caltech.edu/docroot/$username
chown $username /infosys/www/corinth.hss.caltech.edu/docroot/$username

Add a CGI enabling stanza to apache/httpd.conf/corinth.hss.caltech.edu:

<Directory "/infosys/www/corinth.hss.caltech.edu/docroot/$username/">
    AllowOverride None
    Options +ExecCGI
</Directory>

And publish the changes.

Make a directory for them for the CGI stuff. Probably best not to make a generic cgi-bin directory for them, as they'll feel that they can start dumping things there. Although if they're smart (and this is Caltech, after all, so they're smart) they'll probably figure out that the directory you made for them is equivalent to a cgi-bin directory. So what choo gonna do.

(as them)
mkdir /home/$username/public_html/$directory

Add a CGI enabling stanza to apache/vhost.d/memphis.hss.caltech.edu/www.hss.caltech.edu/www.hss.caltech.edu:

<Directory "/home/$username/public_html/$directory/">
    AllowOverride None
    Options +ExecCGI
</Directory>

And publish the changes.

Tell the person:

This is just a matter of technical and political judgement.

Where should we run this CGI?

You'll need to know the password for the mysql root user for this.

To dump, in general:

mysqldump -u root -p -A --opt > ~/<hostname>.sql

To restore, first set the root password on the database appropriately and then:

mysql -u root -p mysql < <hostname>.sql
mysql -u root -p mysql -e 'flush privileges;'

If you don't do the flush privileges;, all MySQL users other than root will have no privileges — you won't be able to log in as them, or access any databases. Maybe restarting mysql will accomplish the same thing.

su - postgres
pg_dumpall > <hostname>.sql

su - postgres
psql -e template1 < <hostname>.sql

I use /infosys/rpmbuild to hold the user writable build environments for our OSes. In it, there is one build environment for each of the platforms we support.

This directory is on the file server, so we won't lose our data when we rebuild our machines. I used to store all this stuff on my machine's local hard drive, and would have to remember to save and restore my build environment each time I rebuilt my machine.

To make a new build environment for a new OS, do this:

os_version=$(rpm -qf --qf='%{VERSION}' /etc/redhat-release)
os_name=$(if grep -i fedora /etc/redhat-release > /dev/null 2>&1 ; then echo fedora-core; else echo redhat; fi)
topdir="$os_name-$os-version"

cd /infosys/rpmbuild
mkdir $topdir
mkdir $topdir/SPECS
mkdir $topdir/SOURCES
mkdir $topdir/SRPMS
mkdir $topdir/RPMS
mkdir $topdir/BUILD

chgrp -R cf /infosys/rpmbuild/$topdir
chmod -R g+w /infosys/rpmbuild/$topdir
chmod -R g+s /infosys/rpmbuild/$topdir

To use the build environments, add these lines to your "~/.rpmmacros" file:

%fulldistversion %(rpm -qf --qf='%{VERSION}' /etc/redhat-release)
%distname %(if grep -i fedora /etc/redhat-release > /dev/null 2>&1 ; then echo fedora-core; else echo redhat; fi)
%_topdir    /infosys/rpmbuild/%{distname}-%{fulldistversion}

When working on a package, cd to the build directory appropriate to the platform you're building the package for, and work on your RPMs there.