www.devco.net by r.i.pienaar

I wrote a multi threaded server for Adobe Flash Policy requests, some background from Adobe:

Since policy files were first introduced, Flash Player has recognized /crossdomain.xml as a master location for URL policy files. However, prior to version 9,0,115,0, Flash Player did not recognize a fixed master location for socket policy files. Flash Player 9,0,115,0 introduces a concept of socket master policy files, which are served from the fixed TCP port number 843.

If your application currently requires /crossdomain.xml files to work properly while making Socket requests you should pay close attention to this, as of the current latest version of Flash Player your application will STOP working if you do not run a policy server.  You need to read this link and take appropriate action.

The servers that Adobe provides are not full featured so I set out to write one, the resulting server can be found on my wiki at http://www.devco.net/pubwiki/FlashPolicyd I will regularly post updates here.

A quick feature list:

• Serves XML files on port 834 using the Adobe protocol
• Multi Threaded for performance, but limited due to Ruby's green threads, testing did not find this to be a problem though
  
• Supports logging to a log file, debug mode can be toggled on the fly via Unix signals
• Debugging information such as thread lists can be dumped using Unix signals
• Adjustable frequency of status messages showing amount of connections, problematic ones and current connection counts

The tar ball includes the main daemon, a Red Hat compatible init script, a standard Nagios monitoring script and a Puppet module for installation on a Red Hat machine.

You need just the basic Ruby installed, I suggest Ruby newer than 1.8.1 due to bugs in the Logger class on 1.8.1.

Version 2 can be downloaded at http://www.devco.net/code/flashpolicyd-2.0.tgz

Some more about my continuing experiences with ruby, in my last post I said

the language does what you'd expect and as you'll see in my next post spending a week with it on and off is enough to write a capable multi threaded socket server.

As it turns out I quickly lived to regret saying that.  Soon after I hit publish I started running into some problems with the very same socket server.

A bit of background, Adobe has made a change to how things work moving away from their previous crossdomain.xml file served over HTTP for cross domain authorization to a new model that requires you to run a special TCP server on port 834 serving up XML over a special protocol.  I won't go into how brain dead I think this is, suffice to say I needed to run one of those for a client.  Adobe of course does provide a server for this, but it has some issues, I chose the simplest of their examples - Perl under xinetd - and quickly discovered that it has no concept of timeouts, or anything that doesn't speak it's protocol.  The end result is that you just end up with a ever growing number of perl stuff running waiting around for ages.

I took this as a challenge to write something real under Ruby using it as a learning experience as well, so set out to write a multi threaded server for this.  At first glance it looks almost laughably trivial:  The Ruby STL includes GServer - a very nice class that does the hard work of thread management for you, you just inherit from it and supply the logic for your protocol and let it do the rest, awesome.

I wrote this, put in logging, options parsing and all the various bits I needed, tested it locally - 10 concurrent workers doing 200,000 requests and it served it in no time at all with limited CPU impact. I then wrote RC scripts, config files and all that and deployed it at my client.

Real soon after deploying it I noticed the wheels came off a bit.  I, out of curiosity, put in some regular logging that would print lines like:

Jun 23 08:23:37 xmpp1 flashpolicyd[7610]: Had 10042 clients of which 285 were bogus. Uptime 0 days 14 hours 2 min. 23 client(s) connected now.

Note in that line how it claims to have 23 connections at present? That's complete b/s, I added the ability to dump actual created threads and there just weren't enough threads for 23 clients and the TCP stack agreed...Turns out gserver has issues handling bad network connections - my clients are over GPRS, Modems, and all sorts - and it seems threads die without GServer removing them from it's list of active connections.

This is a small problem except that GServer uses the connection count towards figuring out if you've hit its max connections setting.  So while I could just set that to some huge figure, it does indicate theres a memory leak - array grows for ever.  Not to mention it just leaving me with a bad taste in my mouth over the quality of my new and improved solution.

Naturally I gave up on GServer I didn't feel like installing all sorts of Gems on the servers so figured I'll just write my own thread handling.  While it's not trivial its by far not the most complex thing I've ever done.  Happy in this case with a bit of wheel reinventing for the sake of learning. 

I chose to use the Ruby STL Logger Library for logging and even added the ability to alter log level on the fly through sending signals to the process, very nice and I were able to re-use much of the option parsing code etc from my previous attempt so this only took a few hours.

I did the development on my Mac using TextMate - the really kick arse Mac text editor that has nice Ruby support - the Mac is on Ruby 1.8.6.  I intended to run this on RHEL 4 and 5, they have Ruby 1.8.1 and 1.8.5 respectively, so I was really setting myself up for problems all of my own making.

Turn out Logger has a bug, fixed here in revision 6262 without any useful svn log, that only bit me on the RHEL 4 machine.  It would open the initial log correctly with line buffering enabled, but once it rotates the log the new log and subsequent ones wouldn't have line buffering.  Which in my case means I get log lines showing up once every 5 hours!

This sux a lot, and it's unlikely that RedHat will backport such a small little thing, and since RedHat 4 will be here till 2012 I guess I'll just have to patch it myself or move to RedHat 5 on this server, something I planned to do anyway.

So something that should have been fairly trivial has turned into a bit of a pain, not really Ruby's fault that I am using 1.8.1 when much newer versions are out, but not nice regardless.  At the end of it all my flash server is working really well and handling clients perfectly with no leaking or anything bad

I, [2008-06-26T23:02:36.607920 #22532]  INFO -- : -604398464: Had 15611 clients of which 423 were bogus. Uptime 0 days 13 hours 41 min. 0 connection(s) in use now.

Those bogus clients are ones that timeout or just otherwise never complete a request, these were the ones that would trip up GServer in the past.

Once I've done documenting it I'll be releasing the flash server here

I have fallen out of love with Perl some time ago, I cannot point to one specific thing about it that put me off, I think it's just a general un-sexyness about it now.

I have been doing some Java and PHP development that was very OOP heavy and have been doing OO coding since school in the Pascal 6 days, I find it's almost become the natural way of thinking when thinking about problems.  OO though does not generally translate well to scripting tasks which is what I generally use Perl for so it's not been too bad.

I am however a Perl 4 coder, Perl 5's OO syntax really puts me off, I've used it, have written some pluggable systems in the past where plugins were Perl Ojbects, but I've never liked them...with Perl 6 coming (or not) having to relearn a large chunk of Perl is becoming inevitable so I figured I might as well switch.  This is a decision I've made ages ago.

I could however not decide on where to go from here?  There are several contenders out there for scripting languages thesedays and weighing up their merits is a time consuming business for sure. 

So along came Puppet, I am doing massive amount of Puppet work now for 3 clients and my own systems.  Puppet is extendible using Ruby code as it's written in Ruby itself so I think the decision has been made.  I still have no interest in using it on the web, but for scripting I've found my new love.

I wont go into long examples and discussions about the Ruby Language, but I'll say I have learned in the space of a week on the train and have found it incredibly friendly and usable.  The Core and Standard Libraries are rich and modern including out of the box XML, YAML, SOAP, XML-RPC and very powerful networking libraries such as GServer.

Documentation has been generally good but I did buy a copy of Beginning Ruby From Novice to Professional which I believe is a awesome introduction to the language, I followed that up with The Ruby Way 2nd Edition and Ruby Cookbook.

Beginning Ruby takes you on a gentle stroll through a lot of the major features of the language, shows you Strings, Arrays, Numbers, OO, Threads, Sockets and RPC.  It doesn't go too deep into any of these subjects, for instance it does not show much about Mutexes or any locking methods inside threads, but it puts down a very solid framework to go from.

The Ruby Way is more task based and goes into more detail about the tasks, returning to the threads example, The Ruby Way discusses Mutexes and shows some of the concurrency issues you can expect, it has a whole chapter devoted to Threading. 

These two books more than put you in a position to quickly and effectively use any of the online documentation resources so I would greatly recommend this combination.

I have skimmed through the Cookbook, it's your typical O'Reilly Cookbook - I mostly bought it because I have 6 or 7 other Cookbook titles and they are all invaluable, I spent an hour with it so far and it looks to live up to this reputation.

If you've programmed before in any serious way picking up Ruby should be a walk in the park, the syntax is great, the language does what you'd expect and as you'll see in my next post spending a week with it on and off is enough to write a capable multi threaded socket server.

I've not been posting much here, work has been incredibly manic the last while, especially I need to still finish off my SSO posts with the last installment, for now though, some thoughts on my work arrangement and freelancing.
 
I have been working from home as self employed since around December (self employed for longer) and as before when I've done many months of home working it has started to get to me.

In the past I generally had an office to go to but didn't unless I had to, worked from home when I could, home is nearer to the Data Center than the office and generally home has better workstations.

Now as I am self employed I do not have an office to go to and I've also realised that all the reading I have done in the past were generally on the train while commuting, and this is something that I cannot let slip but just do not get around to reading while at home.

So I now have an office in London Soho, it is just a desk in a shared office full of other freelancers and startups but for the last 2 weeks I have been pretty happy, I have learned Ruby (more about that in my next post) and I am finding I really enjoy the ritual of going to an office again, but an office where there aren't project managers and other people waiting to harass you, I think that is the main ingredient.

I will probably stay at this office till the clocks change for winter time, then I'll be working from home again till the summer.  The main thing is I have the freedom to choose now and that more than anything is what I love about self employment.

Previously I blogged a custom fact that reads /etc/facts.txt to build up some custom facts for use in Puppet manifests, well I've since learned a thing or two about Ruby and have improved the code, new code below:

if File.exists?("/etc/facts.txt")
        File.open("/etc/facts.txt").each do |line|
                var = nil
                value = nil

                var = $1 and val = $2 if line =~ /^(.+)=(.+)$/

                if var != nil && val != nil
                        Facter.add(var) do
                                setcode { val }
                        end
                end
        end

end

As I mentioned previously I knew the code was horrible and had a whole redundant loop in it but couldn't get it going otherwise.  The big change is in choice of variable names to use inside the setcode, sees value must be a reserved word or something, so now the code is much cleaner.

This is the 3rd part of my ongoing series of posts about designing a simple Single Signon System for PHP, you should read part 1 and 2 first.

Today we look a bit more about the general information flow between browser, target web site and the SSO Server.  We will use the term Secure Site for the target site, lets say a corporate intranet.

The following diagram shows the flow of information, the information flows via the browser using redirects etc.

First a few words on the requierd bits of information before this exchange can happen.

• The Secure Site has a pre-shared key (PSK) that the SSO Server assigns, this key gets hardcoded in the Secure Site and kept private, it never gets passed between the parties during normal authentication requests.
• The Secure Site has a siteid that is simply a number that uniquely identifies it to the SSO Server.  This too gets assigned by the SSO Server and does not change for the life time of the site.
  
• Encryption gets done using a simple symmetrical algorithm, the PSK is the passphrase.
• The SSO server knows what domain name a site is in, the SSO Server will use this to validate auth requests and generate redirects based on this domain name only.

Now on to the actual information flow, this demonstrates the flow for a first time visit of an unauthenticated user, future visits will be exactly like any cookie based auth system where the user will not interact with the SSO server at all:

1. The user tries to access the Secure Site
2. Secure Site generate a unique single use token, and saves it in a Session
3. The Secure Site redirect the browser to the SSO server with a request that has the siteid unencrypted and a encrypted string containing the URL the browser should go to if he is authenticated correctly by the SSO Server and also the Token generated in step 2.
4. The SSO Server validates the request, checking Next URL against that stored for siteid after using the PSK stored for siteid to decrypt the packet.
5. The user is presented with a login form.  The login form shows detail about the Secure Site such as its URL, a description of the site and who to contact with any support requests about this site.  The user is warned that his personal details will be shared with the site.  The user logs in with a username and password and gets redirected to the Secure Site.
6. The Secure Site gets a authentication packet in the redirect that has - encrypted using the PSK - the username, email address, real name, time zone and the previously generated token.  In unencrypted form is a md5 hash of all the private information concatenated with the PSK, this is signature for the request and could be used as a cryptographic digital signature if the SSO Server is configured to not encrypt the private data.  The Token gets removed from the session and the user gets marked as Logged In using a cookie or existing session.
   
7. At this point the user is logged in and can access the Secure Site, the Secure Site knows his private details and can associate his data with him.  From this point on it's a standard cookie based auth and the Secure Site can decide how long the login session is valid for etc.

I think we'll keep it at that for today, in the next part I'll explain some of the choices made in designing this protocol and what security exploits it tries to prevent (replay attacks), what it is vulnerable too (man in the middle attacks) and how to mitigate those risks.

This is the 2nd part of my ongoing series of posts about designing a simple Single Signon System for PHP, you should read part 1 first.

I am often annoyed about series of blog posts that don't make it clear what the end goal is early on, so you end up wasting time reading through loads of stuff only to realise at the end its a bad fit.  So below you'll see some sample bits of code using my Single Sign On system in PHP and Apache after this you can easily decide to just ignore the rest of the posts or to keep paying attention.

First as I said the authentication should be pluggable, I want to be able to fetch users from LDAP, MySQL, and any number of other things, towards this goal I made the actual code that does the hard work pluggable by using a simple OO module and an interface.  Below is a bit of code to just always allow a user 'john' in with the password 'secret', the values for name etc is hardcoded and you can't change the settings, but you'll get the basic idea!

<?
class StupidAuth implements pSSO_Authenticator {
    public function _authenticate($username, $password) {
        if ($username == "john" && $password == "secret") {
            return(1);
        }

        return(0);
    }

    public function _getEmailAddress() {
        return("john@doe.net");
    }

    public function _getRealName() {
        return("John Doe");
    }

    public function _getUsername() {
        return("john");
    }

    public function _getTimeZone() {
        return("Europe/London");
    }

    public function _setEmailAddress() {
    }

    public function _setRealName() {
    }

    public function _setUsername() {
    }

    public function _setTimeZone() {
    }
}
?>

It doesn't really come simpler than that, within this framework you really should be able to do almost any form of authentication, if PHP can talk to it and auth then so should the SSO system.  This code will live in the server, I won't go much into the server here, it's just a really a system to wrap the code above into a well defined protocol between client and server, more on this some other day.  For now just assume there is a config file on the server and you tell it what Class implements the actual auth - StupidAuth in this case.

Now for a quick client, remember the client can run anywhere on any domain, and I want my clients to be registered with me before they can use the SSO system.  To this end each client has a Pre Shared Key (PSK) and a unique ID.  The PSK is used to encrypt the communications from the SSO server to the SSO client as the reply will have real names, email addresses and such in it, you don't want this to show up in proxy logs and such!

Here's a quick client:

<?
    require("pSSO_Client.class.php");

    $psk = "goG4mUrJeacE7VyidEfd";
    $siteid = 1;
    $ssoServer = "http://sso.yourcompany.com/";
    $thisURL = "http://" . $_SERVER["HTTP_HOST"] . $_SERVER['SCRIPT_NAME'];

    session_start();

    // the SSO server sent us back a token, validate it and set cookies
    if (isSet($_GET['authdata']) && ($_GET['v'])) {
        $psso = new pSSO_Client($_GET['authdata'], $_GET['v'], $psk,
                                     $siteid, $ssoServer);
        
        if ($psso->authenticate()) {
            // The user is logged in, send him back to this same url
            // except without any GET params etc, so he'll be a normal
            // returning logged in user.
            header("Location: " .$thisURL);
            exit;
        } else {
            // login failed, eventhough it shouldn't have, bail out
            Throw new Exception ("Login failed:" . $psso->getError());
        }
    } else {
        // We didn't get a token, either its a guest or he already has cookies
        // from a previous visit
        $psso = new pSSO_Client("", "", $psk, $siteid, $ssoServer);
        if ($psso->isLoggedIn()) {
            print("You are logged in:<br><br>");
            print("Your username is: " . $psso->getUserName() . "<br>");
            print("Your real name is: " . $psso->getRealName() . "<br>");
            print("Your email address is: " . $psso->getEmailAddress() . "<br>");
            print("Your timezone is: " . $psso->getTimeZone() . "<br>");
            exit;
        } else {
            print("Welcome guest, you can login <a href='" . $psso->getAuthURL($thisURL)
                . "'>here</a>");
            exit;
        }
    }
?>

A quick run through the code:

• Define the PSK, SSO Server URL and your unique ID, all of this will be supplied by the SSO server when you register your site.
• Start a session - this is used to associate a one-time-use token that travels between the SSO client and Server, more on this later but it helps prevent session replay attacks.
• Check if we got the data that the SSO server will typically pass us $_GET['authdata'] and $_GET['v'], if we did get it, start up a SSO client and authenticate.  Showing the protected page on success or an error message if it failed.
• If the GET parameters aren't present it is either a first visit or a returning visit, the SSO Client lib gets used to find out if its a returning visit and shows some personal information if the user is logged in, else presents the user with a bit of guest info and a link to the login form.  Again here the SSO client library generate all the needed URLs and encryption and what not.

I think this very simple use case should demonstrate the ease of use, you could now easily decide to either just not have a local user database at all, just perhaps ACL's based on username or some local cached user data that has additional information your app provides in a local database perhaps referenced by username, so relying simply on the SSO for auth.

You'll remember I also wanted to do HTTP Authentication, I have a beta stage Apache mod_perl module to do this, below you'll see a bit of Apache config to protect my Nagios installation using the SSO:

        <Directory "/usr/share/nagios">
               AuthType Apache::Auth_pSSO
               AuthName pSSO
               PerlAuthenHandler Apache::Auth_pSSO->authenticate
               require valid-user
               PerlSetVar pSSO_PSK "goG4mUrJeacE7VyidEfd"
        </Directory>

Again notice the PSK in there, at the moment site id and SSO server URLs are hardcoded, but like I said, its beta code :)

With both of the samples above, instead of a pop-up HTTP Auth dialog, you will simply see the SSO login form if you've not authenticated already, else you will just see the protected resource or nagios no questions asked.

So that's it, a quick run through what can be achieved with the SSO libraries on both server and client, in the next post I'll get into some details of the protocol between server and client.

As this is the first time I am posting about Puppet, maybe a few words about it first.  Puppet is a configuration management system that helps you manage large infrastructures.  There are ofcourse many similar things, cfengine and lcfg to name just two.  I like Puppet as it has a big and active community but it is also a modern approach to the problem being that it has some properties of Object Orientation and so forth.

I've used cfengine - actually chose it over Puppet when it was still young - but that is over a year ago now and I if you have many or even 2 of the same nodes to maintain I urge you to check out Puppet.

So the rest is only really useful if you already know Puppet, sorry if this will bore many people :)

On a site I worked we used to have a file on each machine that defines a few Facter facts, this was very handy and I wanted to do something similar because I define variables like $country="uk" and so forth in my nodes which then helps my manifests builds correct ntp configs for example.  Now doing it with a global variable in the node is ok and all, but not nearly as sexy as letting each node know where they are in via facts.

In the old days hacking in new facts was a right pain and you possibly even needed to maintain your own package of facter, not so anymore, it's in fact really nice now.

Puppet now lets you drop bits of ruby code into a directory and then sends the code off to clients on demand for integration into Facter, I won't go into setting it up the Puppet Wiki goes into it at length.  Instead I'll show a simple bit of code to read in /etc/facts.txt on a node and present the values in it as facts.

var = nil
value = nil
facts = {}

if File.exists?("/etc/facts.txt")
        File.open("/etc/facts.txt").each do |line|
                var = $1 and value = $2 if line =~ /^(.+)=(.+)$/
                if var != nil && value != nil
                        facts[var] = value
                        var = nil
                        value = nil
                end
        end

        facts.each{|var,val|
                Facter.add(var) do
                        setcode do
                                val
                        end
                end
        }
end

This code essentially just reads stuff out of key=val pairs in /etc/facts.txt and adds them to facter.  Drop that into your facts directory on the master in a file ending in .rb and that should be it, with facts files on your nodes the values will now be available for use in your manifests.

Maybe not the best Ruby code as it's my first ever Ruby code and mostly based on a sample I found in the Puppet book, I would have liked to do the Facter.add() call right in the loop that parses the file instead of using a array, but I couldn't get that going.

Anyway, now I just distribute /etc/facts.txt to all my machines, and they know where they are and I have a simple no-hassle method for teaching machines about new things.

Back in 2004 I posted about The Regex Coach, its a great app that I still use today, however it only really works on Windows so I have been looking for some alternatives.

There is a really great resources called Regular-Expressions.info it even has a cheap tool that you can use to do something similar to The Regex Coach.  Today Lifehacker mentioned RegExr, its a great web app but also has standalone versions for Windows, Linux and OS X, full of sample regular expressions, good explanations of what a regular expression parses as etc, it is a perfect replacement for The Regex Coach, worth checking it out!

I use a PHP development framework that I have been building on and improving for the last 4 or 5 years.  I used a framework called Roadsend SiteManager years ago but when the amount of non-backward compatible changes they were making got too much I started working on my own.

My framework is a MVC based system that recently had a good refresh to make good use of the improved OO abilities of PHP 5, it isn't really generic, its more tailor designed to work the way I do and to help me make things I do often easier.  As a result of using this framework for years now I have about 15 or so sites developed with it, some for my personal stuff - like a database of Film rolls - other for clients.  Till now everything had their own user database which makes changing passwords a right pain in the butt not to mention a huge job to adjust all the sites if/when I want to add additional authentication features.

So I started looking into Single Sign On systems but came up a bit short for complete ones that fits my bill, my needs are more or less:

• Server and Client libraries in PHP but also client libraries in other languages.
• Should store Real Name, Email Address, Time Zone and possibly other bits of information and share it on demand with other sites.
  
• Work cross domains, so a single SSO server should be able to serve all my sites and possibly those of 3rd parties should they wish to.  This basically means the SSO server shouldn't just go and set a domain cookie.
  
• Have a good management system where users can manage their identities in a self-service manner.
• Client sites should be registered with the server before they can use the SSO system.  When a users identity is accessed he should be shown information about description, contact details etc. it should be completely open to the user when his information gets shared and he should be able to say no.
  
• The server should keep a log of all uses of the identity, in practice only the first requests do get logged, after that the user can be kept logged in on the clients and so new requests do not get made to the server.
• Authentication should be modular so you can plug any user database into the server.  For instance I can share a single database between my imap, pop, smtp, ticketing system and any of my sites using the SSO system.
• Legacy applications and third party applications that rely on standard HTTP Authentication should be able to use it, at least when run under Apache.  I want to be able to log into Cacti, Nagios, RT and others with a single login.
  

This is quite a long list of requirements, none of them are particularly difficult to be honest, the hardest part is the HTTP Authentication plugin, mostly because it relies on writing either a C Apache module or something in mod_perl.

I have written a SSO system that complies to these requirements and will over the course of a number of blog posts detail the design of such a system.  My intention is to open source the main server library and client library in PHP but won't, at least for the foreseeable future, release the management application as that is quite a big deal to release a truly end user ready big web application like this.

I don't know how many posts it will take to go through it all and I also don't know how long it will take me to do them all, I guess I will in these posts retrofit a spec onto what I developed and hopefully pick up on bits that I missed along the way.  I hope this might be of use/interest to someone out there.

UPDATE: Part 2 is here.