I run a number of jail on my servers and each of these have the same files over and over, the biggest problems are of course /usr/ports and /usr/src, they're a total waste of space and a lot of work running multiple portsnap's etc to keep them all synced.
BJE mentioned he is using nullfs to mount his main /usr/ports into the jails. I did some hunting of information on this and found its a bit thin on the ground, the main reason is that up till 6.0 it's considered broken, though testing shows it works ok, at least for normal use.
FreeBSD 6.0 will have extensions to its rc.conf to make all of this easy.
You can create per-jail fstab files, by default they are stored in /etc/fstab.jailname but you can override the filename using jail_example_fstab. Below is a sample fstab file for one of my jails:
/usr/ports /jails/example/usr/ports nullfs rw 1 1
/usr/src /jails/example/usr/src nullfs ro 1 1
To enable the mounting of these file systems at boot time if you use the rc.conf method of booting your jails simply add a jail_example_mount_enable="YES" to your jail section. A full rc.conf extract to start one example jail below:
jail_enable="YES"
jail_list="example"
jail_socket_unixiproute_only="YES"
jail_sysvipc_allow="NO"
jail_example_rootdir="/jails/example"
jail_example_hostname="example.com"
jail_example_ip="192.168.1.100"
jail_example_exec="/bin/sh /etc/rc"
jail_example_devfs_enable="YES"
jail_example_fdescfs_enable="NO"
jail_example_procfs_enable="YES"
jail_example_devfs_ruleset="devfsrules_jail"
jail_example_mount_enable="YES"
I'll investigate also sharing /bin, /sbin, /lib, /libexec, /usr/sbin, /usr/sbin and a few others between jails but it gets a bit tricky if you want to install other versions of perl from ports since they maintain symlinks in /usr/bin etc. This could however simplify world upgrades a lot.
While researching this I came across ezjail which is a script compatible with FreeBSD 6.0 and later to maintain jails that uses nullfs extensively to share a lot of directories from a base jail install. This gives a big potential security improvement because you can mount the system directories read only to give further protection in the event of a compromise. Will definitely investigate this before I start building my new hosted server once FreeBSD 6.0 is out.
I am the author of ezjail. The project is in its early stages right now and I'd love to here from people trying to solve the same problems I stumbled upon. Feedback is welcome.
/usr/ports /jails/example/usr/ports nullfs rw 1 1
This is highly insecure as it allows programs and users with root priviledges inside the jail to write directly to the filesystem outside of the jail. An attacker who gains root control of the jail can inject malicious code into the ports tree, which would compromise the entire machine.
This negates the entire purpose of jails in the first place. The fix is simple enough: change it to mount readonly and tell the jails to store distfiles and build ports elsewhere. (Wouldn't hurt to do the same for the host as well.)
Also, your blog software doesn't allow newlines for some inane reason.