Typical output is:

Classes included on this node:
        nephilim.ml.org
        common::linux
        <snip>

Resources managed by puppet on this node:
        service{smokeping: }
                defined in common/modules/smokeping/manifests/service.pp:6

        file{/etc/cron.d/mrtg: }
                defined in common/modules/puppet/manifests/init.pp:201
<snip>

It will show all classes and all resources including where in your manifests the resource comes from.  Unfortunately for resources created by defines it shows the define as the source but I guess you can’t have it all.

You can get the code here it’s pretty simple, just pass it a path to your localconfig.yaml file, it supports both YAML and Marshal formats.

The file also has every property of the resources in it etc, so you can easily extend this to print a lot of other information, just use something like pp to dump out the contents of Puppet::TransObject objects to see what’s possible.

Well I accidentally left a machine to defaults, then tried to load a massive dump file into it, a month later I finally killed the process loading the data.  I gave up on it ages ago but it got to the point where it was some curiosity to see just how long it will take.

Anyway, so in my case I have machines all over, one off VPS machines, dom0′s with a subnet routed to them and so forth.  I often have rules that need to match on all my ips, things like allow data into my backup server, allow config retrieval from my puppetmaster etc.  I do not want to maintain my total list of ips 10 times over so how to deal with it?

This is a good fit for ipfw tables, you create a table – essentially an object group like in a Cisco PIX or ASA – and then use it to match source IPs.

In the last week I’ve asked quite a few people how they’d do something similar with iptables but no-one seemed to know, I had people who were happy to maintain the same list many times.  People who would use tools like sed to insert it into their rules and everything in between.  I think I know a better way so I figured I’ll blog about it because it’s obviously something people do not just understand.

Iptables ofcourse use chains, and you can jump to and from chains all you want, this is very simple, so lets create a chain with all my IPs

-A my_ips -s 192.168.1.1 -m comment –comment “box1.com” -j ACCEPT
-A my_ips -s 192.168.2.1 -m comment –comment “box2.com” -j ACCEPT
-A my_ips -s 192.168.3.1 -m comment –comment “box3.com” -j ACCEPT

This creates a chain my_ips that just accepts all traffic from my IP addresses, now lets see how we’d allow all my ip addresses into my webserver?

-A INPUT -p tcp –dport 80 -m tcp -j my_ips

So this is something almost as good as a ipfw table, I can reuse it many times on many machines and my overall configuration is much more simple.  It’s not quite as powerful as a table but for my needs it’s fine.

Combined with a tool like Puppet that manages your configurations you can ensure that this chain is installed on any machine that uses iptables, ready to be used and also trivial to update whenever you need too without having to worry about human error incurred from having to maintain many copies of essentially the same data. 

In my environment when I update this table, I check it into SVN and within 30 minutes every machine in my control has the new table and they’ve all reloaded their iptables rules to activate it.  Testing is very easy since puppet allows you to use environments similar to Rails has and so if I really need to I can easily test firewall changes on a small contained set of machines, distributed object group management with version control and everything rolled into one.

I made a mysqldump that just takes all databases into a single file, already I want to kick myself because I know if I ever need to import it there will be troubles because the target database will already have the mysql database etc.

Really I should have used MySQL Parallel Dump that makes files per tables etc and is much faster but it didn’t exist at the time.

So how to pull lines 8596 to 9613 from this big file?  It’s trivial with sed:

here is a sample file:

$ cat > file.txt
line 1
line 2
line 3
line 4
line 5
^D
$ sed -n '2,4p;4q' file.txt
line 2
line 3
line 4

The sed command just tells it the start to end line and also to quit processing when it hits the end line, really kewl.

Problem is I have since discovered that Lacie UK are the most incompetent people on the planet.  I placed the order with them after their site showed they had the unit in-stock on a 3 days delivery time, after placing my order site said the same so I was confident it was all in order.  Needless to say the device never came.  I emailed their sales lines, no response, I emailed their supports lines, no response.  I called them (after spending about a hour tracking down phone numbers) they didn’t reply to voice mails.

After about 10 calls I eventually spoke to someone who was unhelpful to say the least, I was told next-week, next-week etc a few times, next week came and went and no drive unit so I eventually just canceled my order.  No more Lacie devices in my future ever that is a certainty.

Some searching later I found a few excellent reviews over at SmallNetBuilder for this and other devices, they even have a very awesome tool for comparing different NAS devices for speed etc, I decided based on their review to get the QNAP TS-209 pro.

The TS-209 pro is an attractive yet very well built little system, all the screws and connectors are proper solid bits of kit like you’d expect on real hardware.  It is a Linux box and you can ssh to it:

# uname -a
Linux vault 2.6.12.6-arm1 #2 Thu Nov 1 03:31:14 CST 2007 armv5tejl unknown
# cat /proc/cpuinfo
Processor       : ARM926EJ-Sid(wb) rev 0 (v5l)
BogoMIPS        : 332.59
Features        : swp half thumb fastmult
# cat /proc/mdstat
<snip>md0 : active raid1 sdb3[2] sda3[0]
731423296 blocks [2/1] [U_]
[==>..................]  recovery = 10.6% (78181504/731423296) finish=144.0min speed=75574K/sec

So a proper little box then, I put 2 x Seagate 750GB drives into it for the same amount of storage as I would have had in the Lacie, the total price ended being about GBP50 more or so.

That GBP50 is money really well spent in this case.  The device has hot swap drives – I tested it by yanking one out live without any problems, a few beeps, a few emailed alerts and log entries:

This is a really capable device built on solid technology, so far I am very happy with it and will recommend to anyone.  If anything significant change on my experiences I’ll post more later but I suggest you read the review linked above and seriously consider this for your SOHO NAS needs.

The short of it is that the drives disconnect, file systems go read only and the box needs a reboot:

ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen
ata1.00: (BMDMA stat 0x4)
ata1.00: tag 0 cmd 0xca Emask 0x4 stat 0x40 err 0x0 (timeout)
ata1: soft resetting port
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata1.00: qc timeout (cmd 0xec)
ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
ata1.00: revalidation failed (errno=-5)
sd 0:0:0:0: SCSI error: return code = 0x00040000
end_request: I/O error, dev sda, sector 226813249
Buffer I/O error on device sda3, logical block 27835568
lost page write due to I/O error on sda3
sd 0:0:0:0: SCSI error: return code = 0x00040000
end_request: I/O error, dev sda, sector 227360961
Buffer I/O error on device sda3, logical block 27904032
lost page write due to I/O error on sda3

So in an effort to figure out if this is a CentOS 5 problem – both ISPs certify CentOS 4 on their hardware – I needed to get my application going on CentOS 4. This turned out to be quite a mission involving getting Exim with MySQL and the recently integrate exiscan rather than the patched version.

I looked at the various options and decided to just backport CentOS 5′s Exim package to CentOS 4.

As it turns out I haven’t yet had a machine re-installed with CentOS 4 as I found some posts suggesting some kernel parameters that might fix things, I’ve applied these now to the machines and wait.

My Exim RPMs can be found below:

exim-4.63-3.src.rpm
exim-4.63-3.i386.rpm
exim-mon-4.63-3.i386.rpm
exim-sa-4.63-3.i386.rpm

As with the CentOS 5 ones you’ll need various DB client libraries installed as this supports speaking to Postgres, MySQL, SQLite etc.

This should be useful to anyone who just wants a more recent version of Exim on their CentOS/RedHat 4 machines.

Handle 0x0017
DMI type 16, 15 bytes.
Physical Memory Array
Location: Proprietary Add-on Card
Use: System Memory
Error Correction Type: Multi-bit ECC
Maximum Capacity: 16 GB
Error Information Handle: Not Provided
Number Of Devices: 4
Handle 0x0018
DMI type 17, 21 bytes.
Memory Device
Array Handle: 0x0017
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 512 MB
Form Factor: DIMM
Set: 1
Locator: DIMM1
Bank Locator: Slot 1
Type: DDR
Type Detail: Synchronous
Handle 0x0019
DMI type 17, 21 bytes.
Memory Device
Array Handle: 0x0017
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 512 MB
Form Factor: DIMM
Set: 1
Locator: DIMM2
Bank Locator: Slot 2
Type: DDR
Type Detail: Synchronous
Handle 0x001A
DMI type 17, 21 bytes.
Memory Device
Array Handle: 0x0017
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 512 MB
Form Factor: DIMM
Set: 2
Locator: DIMM3
Bank Locator: Slot 3
Type: DDR
Type Detail: Synchronous
Handle 0x001B
DMI type 17, 21 bytes.
Memory Device
Array Handle: 0x0017
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 512 MB
Form Factor: DIMM
Set: 2
Locator: DIMM4
Bank Locator: Slot 4
Type: DDR
Type Detail: Synchronous

So I have 4 total memory slots, each slot has a 512MB DDR module in it, this means I’ll be throwing it all away and buying new RAM.