Skip to content

Latest commit

 

History

History
88 lines (72 loc) · 4.93 KB

README.md

File metadata and controls

88 lines (72 loc) · 4.93 KB
Raw

Nitra

Nitra is a multi-process, optionally multi-server rspec and cucumber runner that uses forking to reduce memory usage and IPC to distribute builds amongst available CPUs efficiently.

Philosophy

  • Nitra attempts to do the simplest thing possible
  • Nitra (ab)uses unix primitives where possible
  • Nitra doesn't do thing that unix already does better (eg. rsync)
  • IPC is accomplished via pipes and select
  • Forking is used heavily for several reasons
  • Running nitra locally should be easy
  • Running nitra on a cluster should be easy too (though verbose)
  • Config files are a nuisance and should be stuffed into rake files (deals with the verbosity)

Usage

  nitra [options] [spec_filename [...]]
      -c, --cpus NUMBER                Specify the number of CPUs to use on the host, or if specified after a --slave, on the slave
          --cucumber                   Add full cucumber run, causes any files you list manually to be ignored
          --debug                      Print debug output
      -p, --print-failures             Print failures immediately when they occur
      -q, --quiet                      Quiet; don't display progress bar
          --rake-after-runner task:1,task:2,task:3
                                       The list of rake tasks to run, once per runner, in the runner's environment, just before the runner exits
          --rake-before-runner task:1,task:2,task:3
                                       The list of rake tasks to run, once per runner, in the runner's environment, after the runner starts
          --rake-before-worker task:1,task:2,task:3
                                       The list of rake tasks to run, once per worker, in the worker's environment, before the worker starts
          --reset                      r
                                       Reset database, equivalent to --rake-before-worker db:reset
          --slave-mode                 Run in slave mode; ignores all other command-line options
          --slave CONNECTION_COMMAND   Provide a command that executes "nitra --slave-mode" on another host
          --rspec                      Add full rspec run, causes any files you list manually to be ignored
      -e, --environment ENV            Set the RAILS_ENV to load
      -h, --help                       Show this message

Getting started

First things first add nitra to your Gemfile:

gem 'nitra'

Then run your specs locally across your cpu's cores:

bundle exec nitra --rspec

This will just run all your specs using the default number of cpu's reported by your system. Hyperthreaded intels will report a high number which might not be a good fit, you can tune this with the -c option.

Clustered commands run slightly differently. Effectively nitra will fork and exec a command that it expects will be a nitra slave, this means we can use ssh as our tunnel of choice. It looks something like this:

bundle exec nitra --rspec --slave "ssh your.server.name 'cd your/project && bundle exec nitra --slave-mode'"

When nitra --slave command it forks, execs it, and assumes it's another process that's running "nitra --slave-mode".

Running a build cluster

Nitra doesn't prescribe how you get your code onto the other machines in your cluster. For example you can run a git checkout on the boxes you want to build on if it's the fastest way for you. For our part - we've had the most success with rsync.

Our build is run via rake tasks so we end up with a bunch of generated code to rsync files back and forth - here's a basic version that might help get you up and running:

namespace :nitra do
  task :config do
      @servers = [
        {:name => "server1", :port => "66666", :c => "4"},
        {:name => "server2", :port => "99999", :c => "2"},
        {:name => "server3", :port => "77777", :c => "8"},
        {:name => "server4", :port => "88888", :c => "4"}
      ]
  end

  desc "Sync the local directory and install the gems onto the remote servers"
  task :prep => :config do
    @servers.each do |server|
      fork do
        system "ssh -p #{server[:port]} #{server[:name]} 'mkdir -p nitra/projects/your_project'"
        exec %(rsync -aze 'ssh -q -p #{server[:port]}' --exclude .git --delete ./ #{server[:name]}:nitra/projects/your_project/ && ssh -q -t -p #{server[:port]} #{server[:name]} 'cd nitra/projects/your_project && bundle install --quiet --path ~/gems')
      end
    end
    Process.waitall
  end

  task :all => :config do
    cmd = %(bundle exec nitra -p -q --rspec --cucumber --rake-before-worker db:reset)
    @servers.each do |server|
      cmd << %( --slave "ssh -p #{server[:port]} #{server[:name]} 'cd nitra/projects/your_project && bundle exec nitra --slave-mode'" -c#{server[:c]})
    end
    system cmd
  end
end

Copyright

Copyright 2012-2013 Roger Nesbitt, Powershop Limited, YouDo Limited. MIT licence.