by R.I. Pienaar | Apr 17, 2008 | Code
UPDATED: The code below is a bit crap – first ever ruby code – try this instead
As this is the first time I am posting about Puppet, maybe a few words about it first. Puppet is a configuration management system that helps you manage large infrastructures. There are ofcourse many similar things, cfengine and lcfg to name just two. I like Puppet as it has a big and active community but it is also a modern approach to the problem being that it has some properties of Object Orientation and so forth.
I’ve used cfengine – actually chose it over Puppet when it was still young – but that is over a year ago now and I if you have many or even 2 of the same nodes to maintain I urge you to check out Puppet.
So the rest is only really useful if you already know Puppet, sorry if this will bore many people ๐
On a site I worked we used to have a file on each machine that defines a few Facter facts, this was very handy and I wanted to do something similar because I define variables like $country=”uk” and so forth in my nodes which then helps my manifests builds correct ntp configs for example. Now doing it with a global variable in the node is ok and all, but not nearly as sexy as letting each node know where they are in via facts.
In the old days hacking in new facts was a right pain and you possibly even needed to maintain your own package of facter, not so anymore, it’s in fact really nice now.
Puppet now lets you drop bits of ruby code into a directory and then sends the code off to clients on demand for integration into Facter, I won’t go into setting it up the Puppet Wiki goes into it at length. Instead I’ll show a simple bit of code to read in /etc/facts.txt on a node and present the values in it as facts.
var = nil
value = nil
facts = {}
if File.exists?(“/etc/facts.txt”)
File.open(“/etc/facts.txt”).each do |line|
var = $1 and value = $2 if line =~ /^(.+)=(.+)$/
if var != nil && value != nil
facts[var] = value
var = nil
value = nil
end
end
facts.each{|var,val|
Facter.add(var) do
setcode do
val
end
end
}
end
This code essentially just reads stuff out of key=val pairs in /etc/facts.txt and adds them to facter. Drop that into your facts directory on the master in a file ending in .rb and that should be it, with facts files on your nodes the values will now be available for use in your manifests.
Maybe not the best Ruby code as it’s my first ever Ruby code and mostly based on a sample I found in the Puppet book, I would have liked to do the Facter.add() call right in the loop that parses the file instead of using a array, but I couldn’t get that going.
Anyway, now I just distribute /etc/facts.txt to all my machines, and they know where they are and I have a simple no-hassle method for teaching machines about new things.
by R.I. Pienaar | Apr 13, 2008 | Usefull Things
Back in 2004 I posted about The Regex Coach, its a great app that I still use today, however it only really works on Windows so I have been looking for some alternatives.
There is a really great resources called Regular-Expressions.info it even has a cheap tool that you can use to do something similar to The Regex Coach. Today Lifehacker mentioned RegExr, its a great web app but also has standalone versions for Windows, Linux and OS X, full of sample regular expressions, good explanations of what a regular expression parses as etc, it is a perfect replacement for The Regex Coach, worth checking it out!
by R.I. Pienaar | Apr 13, 2008 | Code
I use a PHP development framework that I have been building on and improving for the last 4 or 5 years. I used a framework called Roadsend SiteManager years ago but when the amount of non-backward compatible changes they were making got too much I started working on my own.
My framework is a MVC based system that recently had a good refresh to make good use of the improved OO abilities of PHP 5, it isn’t really generic, its more tailor designed to work the way I do and to help me make things I do often easier. As a result of using this framework for years now I have about 15 or so sites developed with it, some for my personal stuff – like a database of Film rolls – other for clients. Till now everything had their own user database which makes changing passwords a right pain in the butt not to mention a huge job to adjust all the sites if/when I want to add additional authentication features.
So I started looking into Single Sign On systems but came up a bit short for complete ones that fits my bill, my needs are more or less:
- Server and Client libraries in PHP but also client libraries in other languages.
- Should store Real Name, Email Address, Time Zone and possibly other bits of information and share it on demand with other sites.
- Work cross domains, so a single SSO server should be able to serve all my sites and possibly those of 3rd parties should they wish to. This basically means the SSO server shouldn’t just go and set a domain cookie.
- Have a good management system where users can manage their identities in a self-service manner.
- Client sites should be registered with the server before they can use the SSO system. When a users identity is accessed he should be shown information about description, contact details etc. it should be completely open to the user when his information gets shared and he should be able to say no.
- The server should keep a log of all uses of the identity, in practice only the first requests do get logged, after that the user can be kept logged in on the clients and so new requests do not get made to the server.
- Authentication should be modular so you can plug any user database into the server. For instance I can share a single database between my imap, pop, smtp, ticketing system and any of my sites using the SSO system.
- Legacy applications and third party applications that rely on standard HTTP Authentication should be able to use it, at least when run under Apache. I want to be able to log into Cacti, Nagios, RT and others with a single login.
This is quite a long list of requirements, none of them are particularly difficult to be honest, the hardest part is the HTTP Authentication plugin, mostly because it relies on writing either a C Apache module or something in mod_perl.
I have written a SSO system that complies to these requirements and will over the course of a number of blog posts detail the design of such a system. My intention is to open source the main server library and client library in PHP but won’t, at least for the foreseeable future, release the management application as that is quite a big deal to release a truly end user ready big web application like this.
I don’t know how many posts it will take to go through it all and I also don’t know how long it will take me to do them all, I guess I will in these posts retrofit a spec onto what I developed and hopefully pick up on bits that I missed along the way. I hope this might be of use/interest to someone out there.
UPDATE: Part 2 is here.
by R.I. Pienaar | Mar 25, 2008 | Code
I have been working on a site that will have potentially quite a few random third parties accessing it and inserting data into a MySQL database. I am thus quite keen on a good solid input filtering method for PHP to prevent things like XSS and SQL Injection.
There are several options out there, of the ones I found Inspekt is about the closest match to my way of working, it essentially imports $_GET, $_POST etc and wraps them in an object which you then use to access variables in a filtered method. It by default then NULLs the original variables so you cannot access them anymore, if backward compatibility is desired it can leave the originals untouched. Not optimal as this gives an unsafe by default result if you want to maintain backwards compatibility.
Another problem with this approach is that it is a lot of work to change existing code, which you might thing is just par for the course but I was convinced I need to find a way to do so more transparently.
I could for example at program start just walk through the $_GET etc arrays and apply some filtering to them using addslashes() and such but this is very restrictive, what if you need to get it unfiltered, especially if you perform destructive filtering? How would you go about filtering some variables for phone numbers, some for email addresses etc?
The answer lies in PHP’s new Standard Programming Library, specifically in its ArrayAccess interface, which if you don’t care for older versions of PHP is the way to go.
The basic advantage of this is that you can expose properties of your objects by using array notation rather than object notation:
$result = $foo->getBar();
compared to:
$result = $foo[‘bar’];
Both statements give access to the private variable $bar just using different syntax. So using this technique we can write a transparent filter for input variables, the basic usage of the final library would be something along these lines:
$_GET = new ArrayArmor($_GET);
print (“Filtered Variable:$_GET[test]<br>\n”);
print (“Unfiltered Variable: ” . $_GET->getRaw(“test”));
A possible output from this script can be seen below:
Filtered Variable: 1234\’;delete from accounts;–
Unfiltered Variable: 1234′;delete from accounts;–
You can see that the default behavior is to protect the input but even for destructive filtering methods the raw unfiltered data would be available if the programmer needed it. You can provide all sorts of extra methods to validate emails, post codes and such.
A quick and dirty example of a class that provides this kind of filtering can be seen below:
<?
class ArrayArmor Implements ArrayAccess {
private $original;
function __construct (&$variable) {
$this->original = $variable;
}
function offsetExists($offset) {
return isset($this->original[$offset]);
}
function offsetGet($offset) {
return addslashes($this->original[$offset]);
}
function offsetSet($offset, $value) {
}
function offsetUnset($offset) {
}
function getRaw($offset) {
return($this->original[$offset]);
}
}
?>
So that’s it, a simple method that is very easy to put into existing code. This is clearly not a full example as addslashes() is hardly the be-all and end-all of input protection, but if you build on this you can get a very easy to use and flexible input filter that is safe by default.
by R.I. Pienaar | Mar 18, 2008 | Code, Front Page
Sometimes you come across things that just make you wonder what is going on in peoples minds.
For years everyone who wrote applications compatible with the standard HTTP Authentication method has used the REMOTE_USER server variable as set by Apache to check the username that was logged in by the webserver, this has worked well for everyone, CGI’s and all would just grab it there and everyone would be happy.
Along comes PHP and they make great big mess of it, PHP suggests that we use $_SERVER[‘PHP_AUTH_USER’] instead, and they give some good reasons for this too, except they have severely crippled this for all but Basic and Digest authentication, the following code from main/main.c
if (auth && auth[0] != ‘\0’ && strncmp(auth, “Basic “, 6) == 0) {
char *pass;
char *user;
user = php_base64_decode(auth + 6, strlen(auth) – 6, NULL);
if (user) {
pass = strchr(user, ‘:’);
if (pass) {
*pass++ = ‘\0’;
SG(request_info).auth_user = user;
SG(request_info).auth_password = estrdup(pass);
ret = 0;
} else {
efree(user);
}
}
}
As you can see above, they only import the user and pass from Apache if the AuthType is Basic, this makes no sense at all. Why not just check with Apache, if it set the username then import it? Surely Apache know if a user has authenticated? Ditto for password. It is so broken in fact that PHP in CGI mode also doesn’t work since those headers don’t get set for that either, countless comments and nasty hacks can be found in the PHP user contributed notes about this, but it is all just sillyness.
The reason this is annoying me is that I have written a Single Singon system in PHP, you can host a identity server on any domain and hook any site in any other domain into the SSO system, its a bit like TypeKey
Of course it’s nice to have a easy to use SSO system in PHP but what is the point if you can’t make legacy apps like Nagios, Cacti, RT etc play along with the SSO? So to solve this I extended Apache::AuthCookie with a new mod_perl module that plugs into Apache and does authentication using my SSO and a small bit of glue that you put on your RT/Cacti/Nagios box.
All’s great, I have SSO to Nagios, RT and countless other things working flawlessly, except of course Cacti because it’s written along the lines of the PHP manual, uses PHP_AUTH_USER instead of REMOTE_USER and so my new fancy AuthType in Apache does not work with Cacti. As it turns out its a quick 2 liner fix in the Cacti code but you would think PHP would be a bit more generic in this regard since as it stands now I think a lot of people who want to do SSO using hardware tokens and such have issues with PHP being silly.