www.devco.net by r.i.pienaar

MCollective Agent Introspection

With the new SimpleRPC system in MCollective we have a simple interface to creating agents. The way to call an agent would be:

$ mc-rpc service status service=httpd

This is all fine and well and easy enough, however it requires you to know a lot. You need to know there's a status action and you need to know it expects a service argument, not great.

I'm busy adding the ability for an agent to register its metadata and interface so that 3rd party tools can dynamically generate useful interfaces.

A sample registration for service agent is:

register_meta(:name        => "SimpleRPC Service Agent",
              :description => "Agent to manage services using the Puppet service provider",
              :author      => "R.I.Pienaar",
              :license     => "GPLv2",
              :version     => 1.1,
              :url         => "http://mcollective-plugins.googlecode.com/",
              :timeout     => 60)
 
["start", "stop", "restart", "status"].each do |action|
    register_input(:action      => action,
                   :name        => "service",
                   :prompt      => "Service Name",
                   :description => "The service to #{action}",
                   :type        => :string,
                   :validation  => '^[a-zA-Z\-_\d]+$',
                   :maxlength   => 30):

This includes all the meta data, versions, timeouts, validation of inputs, prompts and help text for every input argument.

Using this we can now generate dynamic UI's, and do something like JavaDoc generated documentation. I've recorded a little video demonstrating a proof of concept Text UI that uses this data to generate a UI dynamically. This is ripe for integration into tools like Foreman and Puppet Dashboard.

Please watch the video here, best viewed full screen.

Ruby Plugin Architectures

Most of the applications I write in Ruby are some kind of Framework, ruby-pdns takes plugins, mcollective takes plugins, my nagios notification bot takes plugins etc, yet I have not yet figured out a decent approach to handling plugins.

Google suggests many options, the most suggested one is something along these lines.

class Plugin
    def self.inherited(klass)
        PluginManager << klass.new
    end
end
 
class FooPlugin<Plugin
end

Where PluginManager is some class or module that stores and later allows retrieval, when the FooPlugin class gets created it will trigger the hook in the base class.

This works ok, almost perfectly, except that at the time of the trigger the FooPlugin class is not 100% complete and your constructor will not be called, quite a pain. From what I can tell it calls the constructor on either Class or Object.

I ended up tweaking the pattern a bit and now have something that works well, essentially if you pass a String to the PluginManager it will just store that as a class name and later create you an instance of that class, else if it's not a string it will save it as a fully realized class assuming that you know what you did.

The full class is part of mcollective and you can see the source here but below the short version:

I am quite annoyed that including a module does not also include static methods in Ruby, its quite a huge miss feature in my view and there are discussions about changing that behavior. I had hopes of writing something simple that I can just do include Pluggable and this would set up all the various bits, create the inherited hook etc, but it's proven to be a pain and would be littered with nasty evals etc.

module PluginManager
   @plugins = {}
 
   def self.<<(plugin)
         type = plugin[:type]
         klass = plugin[:class]
 
         raise("Plugin #{type} already loaded") if @plugins.include?(type)
 
         if klass.is_a?(String)
             @plugins[type] = {:loadtime => Time.now, :class => klass, :instance => nil}
         else
             @plugins[type] = {:loadtime => Time.now, :class => klass.class, :instance => klass}
         end
   end
 
   def self.[](plugin)
       raise("No plugin #{plugin} defined") unless @plugins.include?(plugin)
 
       # Create an instance of the class if one hasn't been done before
       if @plugins[plugin][:instance] == nil
            begin
                 klass = @plugins[plugin][:class]
                 @plugins[plugin][:instance] = eval("#{klass}.new")
            rescue Exception => e
                raise("Could not create instance of plugin #{plugin}: #{e}")
            end
       end
 
       @plugins[plugin][:instance]
   end
end
 
class Plugin
   def self.inherited(klass)
      PluginManager << {:type => "facts_plugin", :class => klass.to_s}
   end
end
 
class FooPlugin<Plugin
end

For mcollective I only ever allow one of a specific type of plugin so the code is a bit specific in that regard.

I think late creating the plugin instances is quite an improvement too since often you're loading in plugins that you just don't need like client apps would probably not need a few of the stuff I load in and creating instances is just a waste.

I am not 100% sold on this approach as the right one, I think I'll probably refine it more and would love to hear what other people have done.

This has though removed a whole chunk of grim code from mcollective since I now store all plugins and agents in here and just fetch them as needed. So already this is an improvement to what I had before so I guess it works well and should be easier to refactor for improvements now.