Puppet : spare 3 minutes per provisionning for your ruby installation.


Info: /Stage[main]/Ruby/Rvm_system_ruby[ruby-1.9.3-p362]: Evaluated in 261.84 seconds


Info: /Stage[main]/Ruby/Exec[rvm_system_ruby_mount-1.9.3-p374]: Evaluated in 69.66 seconds

Thanks to @mark_olson and this puppet snippet :

exec { "rvm_system_ruby_mount-${ruby_full_version}":
     command =>  "/usr/local/rvm/bin/rvm mount -r http://mycorp.com/binaries/debian/${operatingsystemrelease}/x86_64/ruby-${ruby_full_version}.tar.bz2 --verify-downloads 1",
     creates => "/usr/local/rvm/wrappers/ruby-${ruby_full_version}",
     require => [

Leave a comment

Vag’rant 1.2.x or the missing upgrade manual

In this article, I will show you the issues you will encounter to upgrade vagrant from 1.0.x to the 1.2.x and how to fix them.

You will need to
– update vagrant it-self
– update the Vagrantfile
– migrate your base boxes to the new layout

Upgrade Vagrant it-self

First the distribution of the software is no more handled by rubygems, so first uninstall all previous vagrant gem versions.

gem uninstall vagrant

also remove any reference to vagrant from your Gemfile.

-gem 'vagrant' 

Then download the new version os-package depending on your distribution and install it.
For my ubuntu laptop it looks like this :

wget http://files.vagrantup.com/packages/7e400d00a3c5a0fdf2809c8b5001a035415a607b/vagrant_1.2.2_x86_64.deb
sudo dpkg -i vagrant_1.2.2_x86_64.deb
rm vagrant_1.2.2_x86_64.deb

Upgrade the Vagrantfile

Vagrant 2.0 final will have a stable Vagrantfile format that will remain backwards compatible, just as 1.0 is considered stable.

vagrant up… and then the trouble begins :

A Vagrant 1.0.x state file was found for this environment. Vagrant has
gone ahead and auto-upgraded this to the latest format. Everything
should continue working as normal. Beware, however, that older versions
of Vagrant may no longer be used with this environment.

However, in case anything went wrong, the old dotfile was backed up
to the location below. If everything is okay, it is safe to remove
this backup.

Backup: .../.vagrant.v1.1372160849
.... Vagrantfile:3:in `local_cache': uninitialized constant Vagrant::Environment::DEFAULT_HOME (NameError)

No luck for the backup is polluted with a .vagrant, my Vagrantfile is empty. Thanks to git I can restore it

git checkout -- Gemfile

uninitialized constant Vagrant::Environment::DEFAULT_HOME (NameError)

Now lets fix the first errors, as I was using this gist to cache on my laptop the apt artifacts.
We should replace the DEFAULT_HOME with current directory

   File.expand_path(Vagrant::Environment::DEFAULT_HOME) to              

Don’t forget to add the cache directory to your .gitignore

Shared folders

The apt cache solution is working with a share folder between my laptop, and it’s also affected by a dsl change in vagrant :

config.vm.share_folder "v-cache", "/var/cache/apt/archives/", cache_dir 

is now called synced folder :

config.vm.synced_folder cache_dir, '/var/cache/apt/archives/', id: 'v-cache', owner: 'vagrant', group: 'www-data' 

Forwarded port


config.vm.forward_port 80, 8080

with the upgraded dsl :

config.vm.network :forwarded_port, host: 8080, guest: 80  

Network hostonly

A small config changes from

config.vm.network :hostonly, "" 


config.vm.network :private_network, ip: ""

Customize natdns

Previously I was enabling memory and natdns properties via

config.vm.customize(["modifyvm", :id, "--name", "dev-dev606", "--memory", "1024",

These customizations are now specific to the virtualbox provider :

config.vm.provider 'virtualbox' do |v|
      'modifyvm', :id,
      '--name', 'dev-dev606',
      '--memory', '1024',
      '--natdnshostresolver1', 'on',
      '--natdnsproxy1', 'on'

Boot with ui

Sometimes you need to boot the vm with a gui (to diagnose boot issue, clear bad firewall rules)
Previously it was :

  config.vm.boot_mode = :gui

Not this is done via the provider

config.vm.provider "virtualbox" do |v|
  v.gui = true

System call

I used to have a call to librarian-puppet. But now it’s complaining that he can’t find his own gem. Thats because we are launching it in a vagrant with sandboxed ruby that has ownly access to vagrant dependencies. My work around for now is to prefix the call with a rvm use.

 system "/bin/bash -c -l 'rvm use ruby-2.0.0-p247 && cd #{File.dirname(__FILE__)} && bundle exec librarian-puppet install'"

If somebody know a better way to escape the sandbox… leave a comment.
Did someoned knows if librarian-puppet-vagrant is vagrant 1.2 compatible ?

Update your existing base box

The basebox layout has changed (to support multiple provider), it seems that the code that is supposed to upgrade it isn’t working for me.

So here you have the manual steps to migrate them :

cd ~/.vagrant.d/debian607
mkdir virtualbox
mv box-disk1.vmdk box.ovf Vagrantfile virtualbox
cd ../working-box

Then add a metadata.json in ~/.vagrant.d/debian607

cat << EOF > metadata.json
  "provider": "virtualbox"


This upgrade wasn’t really transparent for me. It makes me sad when I think about all these obsolete/broken articles on the net, all these books that will need a second edition. I hope it will not discourage the vagrant adoption too much.

You can use the new plugins :

vagrant plugin install vagrant-vbguest

My final Vagrantfile is available here.

1 Comment

How do you puppet ?

Infrastructure as Code : lessons learned.

Raise your #Reproducibility, #Maintainability, #Testability, #Reusability up to 98%.

Prepared and presented for the Tech Talk #1 @ ICAB thanks to 8thcolor.

, , ,


Improved jenkins-github navigation

At work, we are using git feature branches extensively, we have a jenkins job configured to build all appearing branches origin/feature/* but it’s hard to know which commit/branch is linked to the build. So I will show you how we use the Groovy Postbuild Plugin to add github link and the branch that was built.

+ +
= cheap and useful navigation links

Continuous build

Add a groovy post build action :

def matcher = manager.getLogMatcher(".*Commencing build of Revision (.*) (.*)\$")
if(matcher?.matches()) {
    branch = matcher.group(2).substring(8,matcher.group(2).length()-1)
    commit = matcher.group(1).substring(0,6)
    githuburl = manager.build.getParent().getProperty("com.coravy.hudson.plugins.github.GithubProjectProperty").getProjectUrl().commitId(matcher.group(1))
    description = "<a href='${githuburl}'>${commit}</a>"+" - "+branch 

It assumes that you have configured the GitHub project url in the job configuration page from the github plugin.

Don’t forget to install the Extra columns plugin and configure your main view to display the build description.


Deployment pipeline

For deployment job, inspired by GitHub, let’s say that you have an url on your website returning the current sha like /site/sha. You have a jenkins job that tracks commit on origin/develop and trigger a deployment.

Let’s add a shell script step in your job :

DEPLOYED_SHA="`wget --no-check-certificate -qO- https://github.com/site/sha`" 

Than postbuild groovy script that will show the deployed sha and the github difference between the previously deployed version :

def matcher = manager.getLogMatcher(".*commit (.*)\$")
if(matcher?.matches()) {
    branch = 'develop'
    commit = matcher.group(1).substring(0,6)
    projectUrl = manager.build.getParent().getProperty("com.coravy.hudson.plugins.github.GithubProjectProperty").getProjectUrl()
    githuburl = projectUrl.commitId(matcher.group(1))
    def matcher_currently_depoyed = manager.getLogMatcher(".*CURRENTLY_DEPLOYED_SHA (.*)\$")
    commit_from = matcher_currently_depoyed.group(1).substring(0,6)
    description = "<a href='${githuburl}'>${commit}</a> - ${branch} - <a href='${projectUrl.baseUrl}compare/${commit_from}...${commit}'>diff</a>"

Where the diff links gives you something like diff.

If you have other hack around GitHub and jenkins, keep me posted !

, ,

Leave a comment

The puppet-lint –fix effect

Learning puppet by my self, I found useful to avoid common mistakes in my modules/manifests. No body knows puppet in my current position, so it’s hard get reviews of my work. I started looking at automated codereview/lint and stumble upon puppet-lint. In his last pre release, the tool implemented autofix of common errors. Let’s see how to measure my progress and learn from my mistakes 😉

First step, to gain visibility, I’ve plugged puppet-lint in our jenkins instance. Setting up jenkins to collect puppet-lint warning. Small modification for me, I don’t abort the build to enable warnings collection with this modified rake task.

Install the pre-release of puppet-lint.

gem install --pre puppet-lint -v 0.4.0.pre1

Uninstall the previous version

gem uninstall puppet-lint

Select gem to uninstall:
 1. puppet-lint-0.3.2
 2. puppet-lint-0.4.0.pre1
 3. All versions
> 1
Successfully uninstalled puppet-lint-0.3.2

Launch puppet-lint with auto-fix options.

ERROR: two-space soft tabs not used on line 15
ERROR: two-space soft tabs not used on line 19
FIXED: unquoted resource title on line 14
WARNING: line has more than 80 characters on line 5
WARNING: line has more than 80 characters on line 6

Double check the changes, launch rspecs, and a vagrant provision than commit

autofix with puppet-lint
 8 files changed, 82 insertions(+), 63 deletions(-)

Lets see the jenkins statistics.

puppet-lint effect

Thanks puppet-lint ! Now it’s time to fix the trivial one (two-space soft tabs) and less trivial one (“foo::bar not documented” , “class inheriting from params class” ,“define defined inside a class”,… ) 😉

What I’ve learned with this experiment:
— if you don’t show your errors, you are not really pushed to fix them
— it takes time to setup these quality tools, but it worth it.
— you are overloaded by warnings, fixing the stupid one automatically and fixing the easy one make you more optimist/confident to attack the harder one.
— it’s easier to fix when there are fresh in your mind

, ,


Through the eyes of sonar : equals & hashCode.

Equals and hashCode



Indicates whether some other object is “equal to” this one.

The equals method implements an equivalence relation on non-null object references:

— It is reflexive: for any non-null reference value x, x.equals(x) should return true.
— It is symmetric: for any non-null reference values x and y, x.equals(y) should return true if and only if y.equals(x) returns true.
— It is transitive: for any non-null reference values x, y, and z, if x.equals(y) returns true and y.equals(z) returns true, then x.equals(z) should return true.
— It is consistent: for any non-null reference values x and y, multiple invocations of x.equals(y) consistently return true or consistently return false, provided no information used in equals comparisons on the objects is modified.
— For any non-null reference value x, x.equals(null) should return false.

The equals method for class Object implements the most discriminating possible equivalence relation on objects; that is, for any non-null reference values x and y, this method returns true if and only if x and y refer to the same object (x == y has the value true).

Note that it is generally necessary to override the hashCode method whenever this method is overridden, so as to maintain the general contract for the hashCode method, which states that equal objects must have equal hash codes.


Returns a hash code value for the object. This method is supported for the benefit of hashtables such as those provided by java.util.HashMap or java.util.HashSet.

The general contract of hashCode is:

— Whenever it is invoked on the same object more than once during an execution of a Java application, the hashCode method must consistently return the same integer, provided no information used in equals comparisons on the object is modified. This integer need not remain consistent from one execution of an application to another execution of the same application.
— If two objects are equal according to the equals(Object) method, then calling the hashCode method on each of the two objects must produce the same integer result.
— It is not required that if two objects are unequal according to the equals(java.lang.Object) method, then calling the hashCode method on each of the two objects must produce distinct integer results. However, the programmer should be aware that producing distinct integer results for unequal objects may improve the performance of hashtables.

As much as is reasonably practical, the hashCode method defined by class Object does return distinct integers for distinct objects. (This is typically implemented by converting the internal address of the object into an integer, but this implementation technique is not required by the JavaTM programming language.)


Use the commons lang classes to help your code to be “nullsafe” and easier to implement :


If you are using hibernate :
— use the getter to avoid “proxy issues”
— avoid using the id as equality (which isn’t always filled)
— read the faq

If you implement them… unit test your equals/hashCode
— There’s a EqualsHashCodeTestCase that you can extends in the junit-addons library
— note that there are also constraint for serialisation/clone
— don’t call hashCode() but toHashCode() on hashCodeBuilder 😉
— avoid reflection based equals/hashCode.

Unit testing

the EqualsHashCodeTestCase is extended in order to test a class’s functional compliance with the equals and hashCode contract.
Override my createInstance and createNotEqualInstance methods to provide me with objects to test against. Both methods should return objects that are of the same class.
will test :
— whether equals holds up against a new Object (should always be false).
— whether equals holds up against null.
— whether equals holds up against objects that should not compare equal.
— whether equals is consistent.
— whether equals is reflexive.
— whether equals is symmetric and transitive.
— the hashCode contract.
— the consistency of hashCode Across Invocations .

Unit Tests and Serialization and serialization

If you adopt a unit-testing methodology, then any serializable class should pass the following three tests:
— If it implements readObject(), it should implement writeObject(), and vice-versa.
— It is equal (using the equals() method) to a serialized copy of itself.
— It has the same hashcode as a serialized copy of itself.

Similar constraints hold for classes that implement the Externalizable interface.

Sonar rules

Warn – eclipse generated equals check class

	public boolean equals(Object obj) {
		if (this == obj)
			return true;
		if (obj == null)
			return false;
		if (getClass() != obj.getClass()) // BAD IDEA IF USING HIBERNATE
			return false;

Prefer instanceof check, because the “obj” may be a HibernateProxy dynamically extending MyObject

	public boolean equals(Object obj) {
		if (this == obj){
			return true;
		if (obj == null) {
			return false;
		if (!(obj instanceof MyObject)) {
			return false;

For the fields comparisons… prefer the EqualsBuilder

see http://tomaszdziurko.pl/2009/05/eclipse-and-problem-with-auto-generated-equals/

Bad practice – Class defines equals() and uses Object.hashCode()

This rule checks that classes that override equals() also need to override hashCode(). If you implement equals… you should implement hashCode accordingly.

Bad practice – equals() method does not check for null argument

As stated in the theory… “whether equals holds up against null”.
A simple check with instanceof… could do the trick.

 public boolean equals(final Object object) {
   if (object.getClass().equals(this.getClass())) {
 	DecisionDetail decisionDetail = (DecisionDetail) object;
    	if (getId() == null || decisionDetail.getId() == null) {
    		return object == this;	

See openpojo to test your model against such issue.

Correctness – equals method always returns false / Correctness – equals method always returns true

Having an equals method like this isn’t a good idea and doesn’t follow the contract on these methods

 public boolean equals(Object object) {		
   return true; // or false			

If you want a good default implementation

 public boolean equals(Object object) {		
   return this==object;

Performance – hashCode return a constant

Returning a constant for the hashCode isn’t good for performance, as the hash is use to “compartiment/bucketing” data in structure like HashSet/hashMap.
Returning a constant will put all the element in the same bucket transforming a lookup of O(1) to O(n) function.

If you don’t have a clean equals criteria fallback to default implementation :

  public int hashCode() {
 	return System.identityHashCode(this);
  public boolean equals(Object object) {		
   return this==object;

Correctness – equals() used to compare array and nonarray

Comparing apples and oranges, an array of X won’t be equals to an object instance X



  public class PersonTypeCode extends Code {
      public static final PersonTypeCode[] ALL = { PROSPECT, MEMBER, PROSPECT_OR_MEMBER, PROSPECT_FAMILY_MEMBER };
      public static final PersonTypeCode PROSPECT = new PersonTypeCode("PROSPECT");

Equals vs == :Literal Equality

Literal Strings should be compared using equals(), not ‘!=’.

 String text = ...
 if (text != "") { //BAD

Ideally… you should use StringUtils to compare in a null safeway.
Note that you have StringUtils.isEmpty(...) or StringUtils.isBlank(...)

 String text = ...
 if (!StringUtils.isEmmpty(text)) {

Class defines equal(Object); should it be equals(Object)?

Surely a typo. If it’s not a typo choose a better name.

Class defines hashcode(); should it be hashCode()?

Surely a typo. If it’s not a typo choose a better name.

Class defines tostring(); should it be toString()?

Surely a typo. If it’s not a typo choose a better name.

Dodgy – Potentially dangerous use of non-short-circuit logic

 return equalsRoot(other) & new EqualsBuilder().append(this.sourceId, other.getSourceId()).isEquals();

This code seems to be using non-short-circuit logic (e.g., & or |) rather than short-circuit logic (&& or ||). In addition, it seem possible that, depending on the value of the left hand side, you might not want to evaluate the right hand side (because it would have side effects (hibernate initialization), could cause an exception (NullPointerException) or could be expensive (hibernate initialization).

 String s = null;   
 if (s == null || s.trim().length() == 0) {  
     // String s is really empty. Do some stuff.  
 if (s != null && s.trim().length() != 0) {  
    // String s is not empty. Do some stuff.  

Both examples would throw a NullPointerException when s is null and you use | and & respectively. (Prefer StringUtils for this kind of check)

Non-short-circuit logic causes both sides of the expression to be evaluated even when the result can be inferred from knowing the left-hand side. This can be less efficient and can result in errors if the left-hand side guards cases when evaluating the right-hand side can generate an error.

, , ,

Leave a comment

Puppet – Vagrant : Smarter / Better / Stronger

After a few weeks of devops ‘light’ practice @8thcolor, I thought I should share my findings. The presentation of Nathen Harvey pushed me a bit further to share them. I will assume you just read deploying rails or a similar book. Started to play with vagrant/puppet/capistrano, and discovered that the real world is not like the sample in the book.

I’d like to live in theory
In theory everything works fine.


Create a single file for your project to describe the type of machine you want, the software that needs to be installed, and the way you want to access the machine (ssh, port forwarding, shared drive, …). Store this file with your project code.

Run a single command — “vagrant up” — and sit back as Vagrant puts together your complete development environment. Say goodbye to the “works on my machine” excuse as Vagrant creates identical development environments for everyone on your team.

If you are not using it yet … you should ! Developed by Mitchell Hashimoto, Vagrant is your new IDE. You won’t test on beta, pre-production, … simply on your workstation. With vagrant, you messed up your manifests ? No problem vagrant destroy && vagrant up, you are back on road.

Basebox with veewee : because time is money

Don’t waste your time with vagrant sample boxes… often not up to date, not sure about the content, older puppet version, obsolete Virtualbox Guest Additions… Build your own with veewee. 30 minutes download included, just too good to be true. Thanks to Patrick Debois. Don’t forget to adapt the mirror in the preseed.cfg file of your vm configuration.

In my case, I had to tweak a bit veewee to fix dns resolution conflicts with my ubuntu laptop :
— adapt the network settings in lib/veewee/provider/virtualbox/box/helper/create.rb
— add at the end of the bootorder line : --natdnsproxy1 on --natdnshostresolver1 on

Reuse : be lazy and clever

Reusing existing modules is always a good idea, people smarter than you (or at least me) already worked on the installation process for most open source components

git submodule add https://github.com/puppetlabs/puppet-postgresql.git modules/postgresql
git submodule add https://github.com/puppetlabs/puppetlabs-stdlib.git modules/stdlib
git submodule add https://github.com/blt04/puppet-rvm.git modules/rvm
git submodule add https://github.com/rodjek/puppet-logrotate modules/logrotate
git submodule add https://github.com/blt/puppet-ssh.git modules/ssh

Another (better) option is to use librarian-puppet to avoid the hassle of managing git submodules (specially rm). See Carlos Sanchez post for the benefits.

If you build, build to be maintainable

If you don’t find an existing module in the forge don’t hesitate to create one or improve an existing one. But don’t do it blindly, some patterns are emerging amongst puppeteers. A good layout of your modules, that allows parametrization/separation of concerns/optional concerns like monitoring, firewall, … is documented in the Example42 foo module. Quality matters, puppeteers push the adoption of rspec for unit testing modules specially if you plan to support multiple distributions.

You can also check your manifests with puppet-lint and see if they conforms to style guide :

sudo gem install puppet-lint
cd /tmp/vagrant-puppet/
puppet-lint --with-filename .

To fix some issues like tabs, trailing whitespaces, All strings that do not contain variables or escape characters like \n or \t should be enclosed in single quotes see some of the bash scripts from bashrc_puppet. Note that the next version of puppet-lint can fix some of them with –fix option. The next step is to enforce puppet-lint reviews with your continuous build : a jenkins-ci sample. During fosdem2013, Bryan Berry demonstrated how to use test-kitchen for multi-node integration tests. The demo is for chef but possible to port to puppet 😉

Performance Vagrant and puppet

If you are using vagrant and provisioning/destroying/reprovisioning/… you don’t want to waste your time so we can speed up the installation by sharing a folder for your package manager, here’s an example for debian

For puppet, adding –evaltrace will enable performance logs :

Info: /Stage[main]/Ruby/Rvm_system_ruby[ruby-1.9.3-p362]: Evaluated in 261.84 seconds
Info: /Stage[main]/Ruby/Rvm_gem[ruby-1.9.3-p362@rails3.2.11/passenger]: Evaluated in 18.14 seconds
Info: /Stage[main]/Rvm::Passenger::Apache::Ubuntu::Post/Exec[passenger-install-apache2-module]: Evaluated in 43.45 seconds
Info: /Stage[main]/Ruby/Rvm_gem[ruby-1.9.3-p362@rails3.2.11/rails]: Evaluated in 89.32 seconds

You can parse them with a script similar to puppet-profiler. Again a Tim Sharpe (rodjek) project, thanks Tim for puppet-lint and puppet-profiler.

In our case we can still spare a few minutes if we can speed up the rvm/ruby installation.
This can be done by skipping the ruby compilation with prebuilt binaries and a corporate repository.

Production environment

You perhaps don’t have the need (or the infrastructure) to install puppet-master, puppet-agent, puppet-dashboard, … so you can start with a master-less setup. Once you have received a new server (from your hosting provider), you can bootstrap the installation of puppet via capistrano and let librarian-puppet take care of the puppet manifest/modules and apply.

Debugging tricks

And finally some debugging tricks that can help you diagnose some issues.

Adding debugging notice

notice("Installing ruby      : ${ruby_rails_version} ")

Re-applying puppet manifests without vagrant reload or up/destroy

vagrant ssh
sudo su
cd /tmp/vagrant-puppet/manifests && puppet apply --pluginsync --verbose --modulepath '/tmp/vagrant-puppet/modules-0' /tmp/vagrant-puppet/manifests/default.pp

Debugging cycle or missing dependency

As you probably know, the order isn’t guaranteed (if you forgot a dependency) or you accidentally creates cycles in the puppet catalog. It’s possible to graph these dependencies to ease the debugging.

Add the graph options --graph and --graphdir '/tmp/vagrant-puppet/modules-0/graphs'

root@host:/tmp/vagrant-puppet/manifests# cd /tmp/vagrant-puppet/manifests && puppet apply --graph --graphdir '/tmp/vagrant-puppet/modules-0/graphs' --pluginsync --verbose --modulepath '/tmp/vagrant-puppet/modules-0' /tmp/vagrant-puppet/manifests/default.pp

— download Gephi
— launch ./bin/gephi
— open the expanded_relationship.dot
— layout, navigate, filter, … and finally discover the cycle or the missing dependency

, , , , , , ,