Simple health check of Rails edge apps for use with Pingdom, NewRelic, EngineYard or uptime.openacs.org etc.
The basic goal is to quickly check that rails is up and running and that it has access to correctly configured resources (database, email gateway)
Use gem versions for stable releases, or github branch / commits for development versions:
-
gem versions ~> 3.0 for Rails 5.x, or the rails5 development branch
-
gem versions ~> 2.0 for Rails 4.x, or the rails4 development branch;
-
gem versions ~> 1.7 for Rails 3.x, or the rails3 development branch;
-
gem versions ~> 1.6.1 for Rails 2.3, or the rails2.3 development branch;
-
master branch for edge development;
Note: it is best to pin to a specific commit if using a development branch as sometimes tests break.
health_check provides various monitoring URIs, for example:
curl localhost:3000/health_check
success
curl localhost:3000/health_check/all.json
{"healthy":true,"message":"success"}
curl localhost:3000/health_check/database_cache_migration.xml
<?xml version="1.0" encoding="UTF-8"?>
<hash>
<healthy type="boolean">true</healthy>
<message>success</message>
</hash>
You may also issue POST calls instead of GET to these urls.
On failure (detected by health_check) a 500 http status is returned with a simple explanation of the failure
curl localhost:3000/health_check/fail health_check failed: invalid argument to health_test.
The health_check controller disables sessions for versions that eagerly load sessions.
-
standard (default) - site, database and migrations checks are run plus email if ActionMailer is defined and it is not using the default configuration
-
all / full - all checks are run (can be overriden in config block)
-
cache - checks that a value can be written to the cache
-
custom - runs checks added via config.add_custom_check
-
database - checks that the current migration level can be read from the database
-
email - basic check of email - :test returns true, :sendmail checks file is present and executable, :smtp sends HELO command to server and checks response
-
migration - checks that the database migration level matches that in db/migrations
-
redis / redis-if-present - checks Redis connectivity
-
resque-redis / resque-redis-if-present - checks Resque connectivity to Redis
-
s3 / s3-if-present - checks proper permissions to s3 buckets
-
sidekiq-redis / sidekiq-redis-if-present - checks Sidekiq connectivity to Redis
-
site - checks rails is running sufficiently to render text
Some checks have a *-if-present form, which only runs the check if the corresponding library has been required.
The email gateway is not checked unless the smtp settings have been changed. Specify full or include email in the list of checks to verify the smtp settings (eg use 127.0.0.1 instead of localhost).
Note: rails 4.0 also checks migrations by default in development mode and throws an ActiveRecord::PendingMigrationError exception (http error 500) if there is an error
Add the following line to Gemfile
gem "health_check"
And then execute
bundle
Or install it yourself as:
gem install health_check
To change the configuration of health_check, create a file ‘config/initializers/health_check.rb` and add a configuration block like:
HealthCheck.setup do |config| # uri prefix (no leading slash) config.uri = 'health_check' # Text output upon success config.success = 'success' # Timeout in seconds used when checking smtp server config.smtp_timeout = 30.0 # http status code used when plain text error message is output # Set to 200 if you want your want to distinguish between partial (text does not include success) and # total failure of rails application (http status of 500 etc) config.http_status_for_error_text = 500 # http status code used when an error object is output (json or xml) # Set to 200 if you want your want to distinguish between partial (healthy property == false) and # total failure of rails application (http status of 500 etc) config.http_status_for_error_object = 500 # bucket names to test connectivity - required only if s3 check used, access permissions can be mixed config.buckets = {'bucket_name' => [:R, :W, :D]} # You can customize which checks happen on a standard health check, eg to set an explicit list use: config.standard_checks = [ 'database', 'migrations', 'custom' ] # Or to exclude one check: config.standard_checks -= [ 'emailconf' ] # You can set what tests are run with the 'full' or 'all' parameter config.full_checks = ['database', 'migrations', 'custom', 'email', 'cache', 'redis', 'resque-redis', 'sidekiq-redis', 's3'] # Add one or more custom checks that return a blank string if ok, or an error message if there is an error config.add_custom_check do CustomHealthCheck.perform_check # any code that returns blank on success and non blank string upon failure end # Add another custom check with a name, so you can call just specific custom checks. This can also be run using # the standard 'custom' check. # You can define multiple tests under the same name - they will be run one after the other. config.add_custom_check('sometest') do CustomHealthCheck.perform_another_check # any code that returns blank on success and non blank string upon failure end # max-age of response in seconds # cache-control is public when max_age > 1 and basic_auth_username is not set # You can force private without authentication for longer max_age by # setting basic_auth_username but not basic_auth_password config.max_age = 1 # Protect health endpoints with basic auth # These default to nil and the endpoint is not protected config.basic_auth_username = 'my_username' config.basic_auth_password = 'my_password' # Whitelist requesting IPs # Defaults to blank and allows any IP config.origin_ip_whitelist = %w(123.123.123.123) # http status code used when the ip is not allowed for the request config.http_status_for_ip_whitelist_error = 403 # When redis url is non-standard config.redis_url = 'redis_url' # Disable the error message to prevent /health_check from leaking # sensitive information config.include_error_in_response_body = false end
You may call add_custom_check multiple times with different tests. These tests will be included in the default list (“standard”).
If you have a catchall route then add the following line above the catch all route (in ‘config/routes.rb`):
health_check_routes
Install health_check as middleware if you want to sometimes ignore exceptions from later parts of the Rails middleware stack, eg DB connection errors from QueryCache. The “middleware” check will fail if you have not installed health_check as middleware.
To install health_check as middleware add the following line to the config/application.rb:
config.middleware.insert_after Rails::Rack::Logger, HealthCheck::MiddlewareHealthcheck
Note: health_check is installed as a full rails engine even if it has been installed as middleware. This is so the remaining checks continue to run through the complete rails stack.
You can also adjust what checks are run from middleware, eg if you want to exclude the checking of the database etc, then set
config.middleware_checks = ['middleware', 'standard', 'custom'] config.standard_checks = ['middleware', 'custom']
Middleware checks are run first, and then full stack checks. When installed as middleware, exceptions thrown when running the full stack tests are formatted in the standard way.
Use a website monitoring service to check the url regularly for the word “success” (without the quotes) rather than just a 200 http status so that any substitution of a different server or generic information page should also be reported as an error.
If an error is encounted, the text “health_check failed: some error message/s” will be returned and the http status will be 500.
See
-
Pingdom Website Monitoring - www.pingdom.com
-
NewRelic Availability Monitoring - newrelic.com/docs/features/availability-monitoring-faq
-
Uptime by OpenACS - uptime.openacs.org/uptime/
-
Engine Yard’s guide - support.cloud.engineyard.com/entries/20996821-monitor-application-uptime (although the guide is based on fitter_happier plugin it will also work with this gem)
-
Nagios check_http (with -s success) - www.nagios-plugins.org/doc/man/check_http.html
-
Any other montoring service that can be set to check for the word success in the text returned from a url
Health_check will respond with an encoded hash object if json or xml is requested. Either set the HTTP Accept header or append .json or .xml to the url.
The hash contains two keys:
-
healthy - true if requested checks pass (boolean)
-
message - text message (“success” or error message)
The following commands
curl -v localhost:3000/health_check.json curl -v localhost:3000/health_check/email.json curl -v -H "Accept: application/json" localhost:3000/health_check
Will return a result with Content-Type: application/json and body like:
{"healthy":true,"message":"success"}
These following commands
curl -v localhost:3000/health_check.xml curl -v localhost:3000/health_check/migration_cache.xml curl -v -H "Accept: text/xml" localhost:3000/health_check/cache
Will return a result with Content-Type: application/xml and body like:
<?xml version="1.0" encoding="UTF-8"?> <hash> <healthy type="boolean">true</healthy> <message>success</message> </hash>
See github.com/ianheggie/health_check/wiki/Ajax-Example for an Ajax example
It is recomended that you use silencer, lograge or one of the other log filtering gems.
For example, with lograge use the following to exclude health_check from being logged:
config.lograge.ignore_actions = ["HealthCheck::HealthCheckController#index"]
Likewise you will probably want to exclude health_check from monitoring systems like newrelic.
Cache-control is set with
-
public if max_age is > 1 and basic_auth_username is not set (otherwise private)
-
no-cache
-
must-revalidate
-
max-age (default 1)
Last-modified is set to the current time (rounded down to a multiple of max_age when max_age > 1)
Feedback welcome! Especially with suggested replacement code and corresponding tests
-
Fork it
-
Create your feature branch (
git checkout -b my-new-feature) -
Commit your changes (
git commit -am 'Add some feature') -
Push to the branch (
git push origin my-new-feature) -
Create new Pull Request.
-
No inline documentation for methods
-
rvm gemsets breaks the test - specifically
rvm use 1.9.3works butrvm gemset use ruby-1.9.3-p385@health_check --createtriggers a “Could not find gem ‘coffee-rails (~> 3.2.1) ruby’ in the gems available on this machine.” error in the last call to bundle (installing health_check as a gem via a path into the temp railsapp)
-
fitter_happier plugin by atmos - plugin with similar goals, but not compatible with uptime, and does not check email gateway
-
<img src=“https://badge.fury.io/rb/health_check.png” alt=“Gem Version” /> - Latest Gem
-
<img src=“https://travis-ci.org/ianheggie/health_check.png”> - Travis CI
-
<img src=“https://codeclimate.com/github/ianheggie/health_check.png” /> - Code quality
-
<img src=“https://gemnasium.com/ianheggie/health_check.png”> - Gem dependencies
The instructions have been changed to using a vagrant virtual box for consistant results.
Install vagrant 1.9.7 or later and virtual_box or other local virtual machine providor.
Create a temp directory for throw away testing, and clone the health_check gem into it
mkdir -p ~/tmp cd ~/tmp git clone https://github.com/ianheggie/health_check.git ~/tmp/health_check
The Vagrantfile includes provisioning rules to install chruby (ruby version control), ruby-build will also be installed and run to build various rubies under /opt/rubies.
Use vagrant ssh to connect to the virtual box and run tests.
The test script will package up and install the gem under a temporary path, create a dummy rails app configured for sqlite, install the gem, and then run up tests against the server. This will require TCP port 3456 to be free.
Cd to the checked out health_check directory and then run the test as follows:
cd ~/tmp/health_check
vagrant up # this will also run vagrant provision and take some time
# chruby and various ruby versions will be installed
vagrant ssh
cd /vagrant # the current directory on your host is mounted here on the virtual machine
chruby 2.2.2 # or some other ruby version (run chruby with no arguments to see the current list)
test/test_with_railsapp
exit # from virtual machine when finished
The script will first call ‘test/setup_railsapp` to setup a rails app with health_check installed and then run up the rails server and perform veraious tests.
The script ‘test/setup_railsapp` will prompt you for which gemfile under test you wish to use to install the appropriate rails version, and then setup tmp/railsapp accordingly.
The command ‘rake test` will also launch these tests, except it cannot install the bundler and rake gems if they are missing first (unlike test/test_with_railsapp)
Copyright © 2010-2017 Ian Heggie, released under the MIT license. See MIT-LICENSE for details.
Thanks go to the various people who have given feedback and suggestions via the issues list and pull requests.
-
Fork it
-
Create your feature branch (‘git checkout -b my-new-feature`)
-
Commit your changes (‘git commit -am ’Add some feature’‘)
-
Push to the branch (‘git push origin my-new-feature`)
-
Create new Pull Request (Code with BDD tests are favoured)