Select Page
NOTE: This is a static archive of an old blog, no interactions like search or categories are current.

This ia a post in a series of posts I am doing about MCollective 2.0 and later.

In my previous post I detailed how you can extend the scope of the information MCollective has available to it about a node using Data Plugins, this was node side plugins today we’ll look at ones that runs on the client.

Background


Using the network as your source of truth works for a certain style of application but as I pointed out in an earlier post there are kinds of application where that is not appropriate. If you want to build a deployer that rolls out the next version of your software you probably want to provide it with a list of nodes rather than have it discover against the network, this way you know when a deploy failed because a node is down rather than it just not being discovered.

These plugins give you the freedom of choice to discover against anything that can give back a list of nodes with mcollective identities. Examples are databases, CMDBs, something like Noah or Zookeeper etc.

To get this to work requires Direct Addressing, I’ll recap an example from the linked post:

c = rpcclient("service")
 
c.discover :nodes => File.readline("hosts.txt").map {|i| i.chomp}
 
printrpc c.restart(:service => "httpd")

In this example MCollective is reading hosts.txt and using that as the source of truth and attempts to communicate only with the hosts discovered against that file. This, as was covered in the previous post, is in stark contrast with MCollective 1.x that had no choice but to use the network as source of truth.

Building on this we’ve built a plugin system that abstracts this away into plugins that you can use on the CLI, web etc – once activated the MCollective usage on the CLI and any existing code can use these plugins without code change.

Using Discovery Plugins


Using these plugins is the same as you’d always do discovery, in fact as of version 2.1.0 if you use mcollective you’re already using this plugin, lets see:

% mco rpc rpcutil ping
Discovering hosts using the mc method for 2 second(s) .... 26
 
 * [============================================================> ] 26 / 26
.
.
---- rpcutil#ping call stats ----
           Nodes: 26 / 26
     Pass / Fail: 26 / 0
      Start Time: Fri Jul 06 09:47:06 +0100 2012
  Discovery Time: 2002.07ms
      Agent Time: 311.14ms
      Total Time: 2313.21ms

Notice the discovery message says it is using the “mc” method, this is the traditional broadcast mode as before, it’s the default mode and will remain the default mode.

Lets look at the generic usage of the hosts.txt above:

% mco rpc rpcutil ping --nodes hosts.txt -v
Discovering hosts using the flatfile method .... 9
 
 * [============================================================> ] 9 / 9
.
.
---- rpcutil#ping call stats ----
           Nodes: 9 / 9
     Pass / Fail: 9 / 0
      Start Time: Fri Jul 06 09:48:15 +0100 2012
  Discovery Time: 0.40ms
      Agent Time: 34.62ms
      Total Time: 35.01ms

Note the change in the discovery message, it is now using the flatfile discovery method and doesn’t have a timeout. Take a look at the Discovery Time statistic, the flatfile example took a fraction of a second vs the usual 2 seconds spent discovering.

There’s a longer form of the above command:

% mco rpc rpcutil ping --disc-method flatfile --disc-option hosts.txt
Discovering hosts using the flatfile method .... 9
.
.

So you can pick a discovery method and they can take options. You can figure out what plugins you have available to you using the plugin application:

% mco plugin doc
Please specify a plugin. Available plugins are:
.
.
Discovery Methods:
  flatfile        Flatfile based discovery for node identities
  mc              MCollective Broadcast based discovery
  mongo           MongoDB based discovery for databases built using registration
  puppetdb        PuppetDB based discovery

And more information about a plugin can be seen:

% mco plugin doc mc
MCollective Broadcast based discovery
 
      Author: R.I.Pienaar <rip@devco.net>
     Version: 0.1
     License: ASL 2.0
     Timeout: 2
   Home Page: http://marionette-collective.org/
 
DISCOVERY METHOD CAPABILITIES:
      Filter based on configuration management classes
      Filter based on system facts
      Filter based on mcollective identity
      Filter based on mcollective agents
      Compound filters combining classes and facts

The discovery methods have capabilities that declare what they can do. The flatfile one for example has no idea about classes, facts etc so it’s capabilities would only be identity filters.

If you decide to always use a different plugin than mc as your discovery source you can set it in client.cfg:

default_discovery_method = mongo

The RPC api obviously can also choose method and supply options, below code forces the flatfile mode:

c = rpcclient("service")
 
c.discovery_method = "flatfile"
c.discovery_options << "hosts.txt"
 
printrpc c.restart(:service => "httpd")

This has the same effect as mco rpc service restart service=httpd –dm=flatfile –do=hosts.txt

Writing a Plugin


We’ll look at the simplest plugin which is the flatfile one, this plugin ships with MCollective but it’s a good example.

This plugin will let you issue commands like:

% mco service restart httpd
% mco service restart httpd -I some.host
% mco service restart httpd -I /domain/ -I /otherdomain/

So your basic identity filters with regular expression support or just all hosts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
module MCollective
  class Discovery
    class Flatfile
      def self.discover(filter, timeout, limit=0, client=nil)
        unless client.options[:discovery_options].empty?
          file = client.options[:discovery_options].first
        else
          raise "The flatfile discovery method needs a path to a text file"
        end
 
        raise "Cannot read the file %s specified as discovery source" % file unless File.readable?(file)
 
        discovered = []
        hosts = File.readlines(file).map{|l| l.chomp}
 
        unless filter["identity"].empty?
          filter["identity"].each do |identity|
            identity = Regexp.new(identity.gsub("\/", "")) if identity.match("^/")
 
            if identity.is_a?(Regexp)
              discovered = hosts.grep(identity)
            elsif hosts.include?(identity)
              discovered << identity
            end
          end
        else
          discovered = hosts
        end
 
        discovered
      end
    end
  end
end

Past the basic boiler plate in lines 5 to 11 we deal with the discovery options, you’ll notice discovery options is an array so users can call –disc-option many times and each call just gets appended to this array. We’ll just take one flat file and raise if you didn’t pass a file or if the file can’t be read.

Lines 13 and 14 sets up a empty array where the selected nodes will go into and reads all the hosts found in the file.

Lines 16 and 17 checks if we got anything in the identity filter, if it was not we set the discovered list to all hosts in the file in line 27. The filters are arrays so in the case of multiple -I passed you will have multiple entries here, line 17 loops all the filters. You do not need to worry about someone accidentally setting a Class filter as MCollective will know from the DDL that you are incapable of doing class filters and will just not call your plugin with those.

The body of the loop in lines 18 to 25 just does regular expression matching or exact matching over the list and if anything is found it gets added to the discovered list.

In the end we just return the list of discovered nodes, you do not need to worry about duplicates in the list or sorting it or anything.

As there were automatic documentation generated and input validation done you need to create a DDL file that describes the plugin and the data it can accept and return, here’s the DDL for this plugin:

1
2
3
4
5
6
7
8
9
10
11
metadata    :name        => "flatfile",
            :description => "Flatfile based discovery for node identities",
            :author      => "R.I.Pienaar <rip@devco.net>",
            :license     => "ASL 2.0",
            :version     => "0.1",
            :url         => "http://marionette-collective.org/",
            :timeout     => 0
 
discovery do
    capabilities :identity
end

The meta block is familiar – set timeout to 0 if there’s no timeout and then MCollective will not inform the user about a timeout in the discovery message. Lines 9 to 11 declares the capabilities, possible capabilities are :classes, :facts, :identity, :agents, :compound. Technically :compound isn’t usable by your plugins as MCollective will force the mc plugin when you use any -S filters as those might contain references to data plugins that has to be done using the nodes as source of truth.

Finally store this in a directory like below and you can package it into a RPM or a Deb:

% tree flatfile
flatfile
โ””โ”€โ”€ discovery
    โ”œโ”€โ”€ flatfile.ddl
    โ””โ”€โ”€ flatfile.rb
% cd flatfile
% mco plugin package
Created package mcollective-flatfile-discovery
% ls -l *rpm
-rw-rw-r-- 1 rip rip 2893 Jul  6 10:20 mcollective-flatfile-discovery-0.1-1.noarch.rpm

Install this plugin to all your clients and it will be available to use, if you do not want to use the packages just dump the files in $libdir/discovery/.

Available Plugins


There are a few plugins available now, you saw the mc and flatfile ones here.

If you use the MongoDB based discovery system there is a fully capable discovery plugin that can work against a local MongoDB instance. This plugin has all the capabilities possible with full regular expression support and full sub collective support. I use this as my default discovery method now.

We’re also working on a PuppetDB one, it is not quite ready to publish as I am waiting for PuppetDB to get wildcard support. And finally there is a community plugin that discovers using Elastic Search.

Conclusion


These plugins conclude the big rework done on MCollective discovery. You can now mix and match any source of truth you like even ones we as MCollective developers are not aware of as you can write your own plugin.

Use the network when appropriate, use databases or flat files when appropriate and you can switch freely between modes during the life of a single application.

Using these plugins is fun as they can be extremely fast. The short 1 minute video embedded below (click here if its not shown) shows the mco, puppetdb and mongodb plugins in action.

Version 2.1.0 made these plugins available, we’re looking to bump the Production branch to support these soon.