R.I.Pienaar

A lifetime ago when I still gave a damn for FreeBSD I wrote about ipfw tables, I still really love ipfw’s simple syntax and really wish there was something similar for Linux rather than Human Error Guaranteed convoluted syntax mess that’s iptables.

Anyway, so in my case I have machines all over, one off VPS machines, dom0′s with a subnet routed to them and so forth.  I often have rules that need to match on all my ips, things like allow data into my backup server, allow config retrieval from my puppetmaster etc.  I do not want to maintain my total list of ips 10 times over so how to deal with it?

This is a good fit for ipfw tables, you create a table – essentially an object group like in a Cisco PIX or ASA – and then use it to match source IPs.

In the last week I’ve asked quite a few people how they’d do something similar with iptables but no-one seemed to know, I had people who were happy to maintain the same list many times.  People who would use tools like sed to insert it into their rules and everything in between.  I think I know a better way so I figured I’ll blog about it because it’s obviously something people do not just understand.

Iptables ofcourse use chains, and you can jump to and from chains all you want, this is very simple, so lets create a chain with all my IPs

-A my_ips -s 192.168.1.1 -m comment –comment “box1.com” -j ACCEPT
-A my_ips -s 192.168.2.1 -m comment –comment “box2.com” -j ACCEPT
-A my_ips -s 192.168.3.1 -m comment –comment “box3.com” -j ACCEPT

This creates a chain my_ips that just accepts all traffic from my IP addresses, now lets see how we’d allow all my ip addresses into my webserver?

-A INPUT -p tcp –dport 80 -m tcp -j my_ips

So this is something almost as good as a ipfw table, I can reuse it many times on many machines and my overall configuration is much more simple.  It’s not quite as powerful as a table but for my needs it’s fine.

Combined with a tool like Puppet that manages your configurations you can ensure that this chain is installed on any machine that uses iptables, ready to be used and also trivial to update whenever you need too without having to worry about human error incurred from having to maintain many copies of essentially the same data. 

In my environment when I update this table, I check it into SVN and within 30 minutes every machine in my control has the new table and they’ve all reloaded their iptables rules to activate it.  Testing is very easy since puppet allows you to use environments similar to Rails has and so if I really need to I can easily test firewall changes on a small contained set of machines, distributed object group management with version control and everything rolled into one.

This is probably old news to most people but I need to remember this so I figured I may as well blog it.

I made a mysqldump that just takes all databases into a single file, already I want to kick myself because I know if I ever need to import it there will be troubles because the target database will already have the mysql database etc.

Really I should have used MySQL Parallel Dump that makes files per tables etc and is much faster but it didn’t exist at the time.

So how to pull lines 8596 to 9613 from this big file?  It’s trivial with sed:

here is a sample file:

$ cat > file.txt
line 1
line 2
line 3
line 4
line 5
^D
$ sed -n '2,4p;4q' file.txt
line 2
line 3
line 4

The sed command just tells it the start to end line and also to quit processing when it hits the end line, really kewl.

I’ve a number of machines that needs memory upgrades, I didn’t want to turn them off to see what is inside in order to plan this. Under windows it’s pretty easy, just download and run CPU-Z and you’ll know all there is to know.
I did a lot of searching etc and eventually came across dmidecode, you just run it as root and it parses through /dev/mem and loads the DMI tables, parses them and prints them in human readable form.
It shows a lot of useful information, on my IBM HS20 Blade it shows model, serial, hardware numbers etc. Here is a sample of the memory section:

Handle 0x0017
DMI type 16, 15 bytes.
Physical Memory Array
Location: Proprietary Add-on Card
Use: System Memory
Error Correction Type: Multi-bit ECC
Maximum Capacity: 16 GB
Error Information Handle: Not Provided
Number Of Devices: 4
Handle 0x0018
DMI type 17, 21 bytes.
Memory Device
Array Handle: 0x0017
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 512 MB
Form Factor: DIMM
Set: 1
Locator: DIMM1
Bank Locator: Slot 1
Type: DDR
Type Detail: Synchronous
Handle 0x0019
DMI type 17, 21 bytes.
Memory Device
Array Handle: 0x0017
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 512 MB
Form Factor: DIMM
Set: 1
Locator: DIMM2
Bank Locator: Slot 2
Type: DDR
Type Detail: Synchronous
Handle 0x001A
DMI type 17, 21 bytes.
Memory Device
Array Handle: 0x0017
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 512 MB
Form Factor: DIMM
Set: 2
Locator: DIMM3
Bank Locator: Slot 3
Type: DDR
Type Detail: Synchronous
Handle 0x001B
DMI type 17, 21 bytes.
Memory Device
Array Handle: 0x0017
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 512 MB
Form Factor: DIMM
Set: 2
Locator: DIMM4
Bank Locator: Slot 4
Type: DDR
Type Detail: Synchronous

So I have 4 total memory slots, each slot has a 512MB DDR module in it, this means I’ll be throwing it all away and buying new RAM.

I’ve previously written that I am trying out lighttpd for serving up my static files, I’ve now been running lighttpd and apache in parallel for a while and must say the results are very good in favour of lighttpd.
First a graphic to show the change: