MCollective 0.4.3 Auditing

I just released version 0.4.3 of mcollective which brings a new auditing capability to SimpleRPC. Using the auditing system you can log to a file on each host every request or build a centralized auditing system for all requests on all nodes.

We ship a simple plugin that logs to the local harddrive but there is also a community plugin that creates a centralized logging system running over MCollective as a transport.

This is the kind of log the centralized logger will produce:

01/24/10 18:24:20 dev1.my.net> d53a8306f20e9b3a0f7946adccd6eb5e: 01/24/10 18:24:20 caller=uid=500@ids1.my.net agent=iptables action=block
01/24/10 18:24:20 dev1.my.net> d53a8306f20e9b3a0f7946adccd6eb5e: {:ipaddr=>"114.255.136.120"}
01/24/10 18:24:20 dev2.my.net> d53a8306f20e9b3a0f7946adccd6eb5e: 01/24/10 18:24:20 caller=uid=500@ids1.my.net agent=iptables action=block
01/24/10 18:24:20 dev2.my.net> d53a8306f20e9b3a0f7946adccd6eb5e: {:ipaddr=>"114.255.136.120"}
01/24/10 18:24:20 dev3.my.net> d53a8306f20e9b3a0f7946adccd6eb5e: 01/24/10 18:24:20 caller=uid=500@ids1.my.net agent=iptables action=block
01/24/10 18:24:20 dev3.my.net> d53a8306f20e9b3a0f7946adccd6eb5e: {:ipaddr=>"114.255.136.120"}

Here we see 3 nodes that got a request to add 114.255.136.120 to their local firewall. The request was sent by UID 500 on the machine ids1.my.net. The request is of course the same everywhere so the request id is the same on every node, the log shows agent and all parameters passed.

MCollective 0.4.2 released

Just a quick blog post for those who follow me here to get notified about new releases of MCollective. I just released version 0.4.2 which brings in big improvements for Debian packages, some tweaks to command line and a bug fix in SimpleRPC.

Read all about it at the Release Notes

MCollective Simple RPC

MCollective is a framework for writing RPC style tools that talk to a cloud of servers, till now doing that has been surprisingly hard for non ruby coders. The reason for this is that I was focussing on getting the framework built and feeling my way around the use cases.

I've now spent 2 days working on simplifying actually writing agents and consumers. This code is not released yet - just in SVN trunk - but here's a taster.

First writing an agent should be simple, here's a simple 'echo' server that takes a message as input and returns it back.

class Rpctest<RPC::Agent
    # Basic echo server
    def echo_action(request, reply)
         raise MissingRPCData, "please supply a :msg" unless request.include?(:msg)
 
         reply.data = request[:msg]
    end
end

This creates an echo action, does a quick check that a message was received and sends it back. I want to create a few more validators so you can check easily if the data passed to you is sane and secure if you're doing anything like system() calls with it.

Here's the client code that calls the echo server 4 times:

#!/usr/bin/ruby
 
require 'mcollective'
 
include MCollective::RPC
 
rpctest = rpcclient("rpctest")
 
puts "Normal echo output, non verbose, shouldn't produce any output:"
printrpc rpctest.echo(:msg => "hello world")
 
puts "Flattened echo output, think combined 'mailq' usecase:"
printrpc rpctest.echo(:msg => "hello world"), :flatten => true
 
puts "Forced verbose output, if you always want to see every result"
printrpc rpctest.echo(:msg => "hello world"), :verbose => true
 
puts "Did not specify needed input:"
printrpc rpctest.echo

This client supports full discovery and all the usual stuff, has pretty --help output and everything else you'd expect in the clients I've supplied with the core mcollective. It caches discovery results so above code will do one discovery only and reuse it for the other calls to the collective.

When running you'll see a twirling status indicator, something like:

  - [5 / 10]

This will give you a nice non scrolling indicator of progress and should work well for 100s of machines without spamming you with noise, at the end of the run you'll get the output.

The printrpc helper function tries its best to print output for you in a way that makes sense on large amounts of machines.

• By default it doesn't print stuff that succeeds, you do get a overall progress indicator though
• If anything does go wrong, useful information gets printed but only for hosts that had problems
• If you ran the client with --verbose, or forced it to verbose mode output you'll get a full bit of info of the result from every server.
• It supports flags to modify the output, you can flatten the output so hostnames etc aren't showed, just a concat of the data.

The script above gives the following output when run in non-verbose mode:

$ rpctest.rb --with-class /devel/
 
Normal echo output, non verbose, shouldn't produce any output:
 
Forced verbose output, if you always want to see every result:
dev1.your.com                          : OK
    "hello world"
 
dev2.your.com                          : OK
    "hello world"
 
dev3.your.com                          : OK
    "hello world"
 
Flattened echo output, think combined 'mailq' usecase:
hello world
hello world
hello world
 
Did not specify needed input:
dev1.your.com                          : please supply a :msg
dev2.your.com                          : please supply a :msg
dev3.your.com                          : please supply a :msg

Still some work to do, specifically stats needs a rethink in a scenario where you are making many calls such as in this script.

This will be in mcollective version 0.4 hopefully out early January 2010

MCollective Release 0.2.0

I am pleased to announce the the first actual numbered release of The Marionette Collective, you can grab it from the downloads page.

Till now people wanting to test this had to pull out of SVN directly, I put off doing a release till I had most of the major tick boxes in my mind ticked and till I knew I wouldn't be making any major changes to the various plugins and such. This release is 0.2.x since 0.1.x was the release number I used locally for my own testing.

This being the first release I fully anticipate some problems and weirdness, please send any concerns to the mailing list or ticketing system.

This has been a while coming, I've posted lots on this blog about mcollective, what it is and what it does. For those just joining you want to watch the video on this post for some background.

I am keen to get feedback from some testers, specifically keen to hear thoughts around these points:

• How does the client tools behave on 100s of nodes, I suspect the output format might be useless if it just scrolls and scrolls, I have some ideas about this but need feedback.
• On large amount of hosts, or when doing lots of requests soon after each other, do you notice any replies going missing.
• Feed back about the general design principals, especially how you find the plugin system and what else you might want pluggable. I for example want to make it much easier to add new discovery methods.
• Anything else you can think of

I'll be putting in tickets on the issue system for future features / fixes I am adding so you can track there to get a feel for the milestones toward 0.3.x.

Thanks goes to the countless people who I spoke to in person, on IRC and on Twitter, thanks for all the retweets and general good wishes. Special thanks also to Chris Read who made the debian package code and fixed up the RC script to be LSB compliant.