Oxidized is a network device configuration backup tool. It's a RANCID replacement!
- automatically adds/removes threads to meet configured retrieval interval
- restful API to move node immediately to head-of-queue (GET/POST /node/next/[NODE])
- syslog udp+file example to catch config change event (ios/junos) and trigger config fetch
- will signal ios/junos user who made change, which output module can (git does) use (via POST)
- 'git blame' will show for each line who and when the change was made
- restful API to reload list of nodes (GET /reload)
- restful API to fetch configurations (/node/fetch/[NODE] or /node/fetch/group/[NODE])
- restful API to show list of nodes (GET /nodes)
- restful API to show list of version for a node (/node/version[NODE]) and diffs
Youtube Video: Oxidized TREX 2014 presentation
- Supported OS Types
- Installation
- Debian
- [CentOS, Oracle Linux, Red Hat Linux version 6](#centos-oracle-linux-red-hat-linux-version 6)
- Initial Configuration
- Installing Ruby 2.1.2 using RVM
- Running with Docker
- Cookbook
- Ruby API
- A10 Networks
- ACOS
- Alcatel-Lucent
- ISAM
- AOS
- AOS7
- Wireless
- TiMOS
- Arista
- EOS
- Arris
- C4CMTS
- Aruba
- AOSW
- Brocade
- FabricOS
- Ironware
- NOS (Network Operating System)
- Vyatta
- Cisco
- AireOS
- ASA
- IOS
- IOSXR
- NXOS
- SMB (Nikola series)
- Cumulus
- Linux
- DELL
- PowerConnect
- AOSW
- Extreme Networks
- XOS
- Force10
- FTOS
- Force10
- DNOS
- FortiGate
- FortiOS
- HP
- Comware (HP A-series, H3C, 3Com)
- Procurve
- Huawei
- VRP
- Juniper
- JunOS
- ScreenOS (Netscreen)
- Mikrotik
- RouterOS
- MRV
- MasterOS
- Ubiquiti
- AirOS
- Edgeos
- Palo Alto
- PANOS
- Zyxel
- ZyNOS
Install all required packages and gems.
apt-get install ruby ruby-dev libsqlite3-dev libssl-dev pkg-config cmake
gem install oxidized
gem install oxidized-script oxidized-web # if you don't install oxidized-web, make sure you remove "rest" from your configInstall Ruby 1.9.3 or greater (for Ruby 2.1.2 installation instructions see "Installing Ruby 2.1.2 using RVM"), then install Oxidized dependencies
yum install cmake sqlite-devel openssl-develNow lets install oxidized via Rubygems:
gem install oxidized
gem install oxidized-script oxidized-webOxidized configuration is in YAML format. Configuration files are subsequently sourced from /etc/oxidized/config then ~/.config/oxidized/config. The hashes will be merged, this might be useful for storing source information in a system wide file and user specific configuration in the home directory (to only include a staff specific username and password). Eg. if many users are using oxs, see Oxidized::Script.
To initialize a default configuration in your home directory ~/.config/oxidized/config, simply run oxidized once. If you don't further configure anything from the output and source sections, it'll extend the examples on a subsequent oxidized execution. This is useful to see what options for a specific source or output backend are available.
Oxidized supports CSV, SQLite and HTTP as source backends. The CSV backend reads nodes from a rancid compatible router.db file. The SQLite backend will fire queries against a database and map certain fields to model items. The HTTP backend will fire queries against a http/https url. Take a look at the Cookbook for more details.
Possible outputs are either file or git. The file backend takes a destination directory as argument and will keep a file per device, with most recent running version of a device. The GIT backend (recommended) will initialize an empty GIT repository in the specified path and create a new commit on every configuration change. Take a look at the Cookbook for more details.
Maps define how to map a model's fields to model model fields. Most of the settings should be self explanatory, log is ignored if Syslog::Logger exists (>=2.0) and syslog is used instead.
First create the directory where the CSV output is going to store device configs and start Oxidized once.
mkdir ~/.config/oxidized/configs
oxidized
Now tell Oxidized where it finds a list of network devices to backup configuration from. You can either use CSV or SQLite as source. To create a CSV source add the following snippet:
source:
default: csv
csv:
file: ~/.config/oxidized/router.db
delimiter: !ruby/regexp /:/
map:
name: 0
model: 1
Now lets create a file based device database (you might want to switch to SQLite later on). Put your routers in ~/.config/oxidized/router.db (file format is compatible with rancid). Simply add an item per line:
router01.example.com:ios
switch01.example.com:procurve
router02.example.com:ios
Run oxidized again to take the first backups.
Install Ruby 2.1.2 build dependencies
yum install curl gcc-c++ patch readline readline-devel zlib zlib-devel
yum install libyaml-devel libffi-devel openssl-devel make cmake
yum install bzip2 autoconf automake libtool bison iconv-devel
Install RVM
curl -L get.rvm.io | bash -s stable
Setup RVM environment and compile and install Ruby 2.1.2 and set it as default
source /etc/profile.d/rvm.sh
rvm install 2.1.2
rvm use --default 2.1.2
- clone git repo:
root@bla:~# git clone https://github.com/ytti/oxidized
- build container locally:
root@bla:~# docker build -q -t oxidized/oxidized:latest oxidized/
- create config directory in main system:
root@bla~:# mkdir /etc/oxidized
- run container the first time:
root@bla:~# docker run -v /etc/oxidized:/root/.config/oxidized -p 8888:8888/tcp -t oxidized/oxidized:latest oxidized
- add 'router.db' to /etc/oxidized:
root@bla:~# vim /etc/oxidized/router.db
[ ... ]
root@bla:~#
- run container again:
root@bla:~# docker run -v /etc/oxidized:/root/.config/oxidized -p 8888:8888/tcp -t oxidized/oxidized:latest oxidized
oxidized[1]: Oxidized starting, running as pid 1
oxidized[1]: Loaded 1 nodes
Puma 2.13.4 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://0.0.0.0:8888
^C
root@bla:~#
In case a model plugin doesn't work correctly (ios, procurve, etc.), you can enable live debugging of SSH/Telnet sessions. Just add a debug option, specifying a log file destination to the input section.
The following example will log an active ssh session to /home/fisakytt/.config/oxidized/log_input-ssh and telnet to log_input-telnet. The file will be truncated on each consecutive ssh/telnet session, so you need to put a tailf or tail -f on that file!
input:
default: ssh, telnet
debug: /tmp/oxidized_log_input
ssh:
secure: false
To start privileged mode before pulling the configuration, Oxidized needs to send the enable command. You can globally enable this, by adding the following snippet to the global section of the configuration file.
vars:
enable: S3cre7
One line per device, colon seperated.
source:
default: csv
csv:
file: /var/lib/oxidized/router.db
delimiter: !ruby/regexp /:/
map:
name: 0
model: 1
username: 2
password: 3
vars_map:
enable: 4
One row per device, filtered by hostname.
source:
default: sql
sql:
adapter: sqlite
database: "/var/lib/oxidized/devices.db"
table: devices
map:
name: fqdn
model: model
username: username
password: password
vars_map:
enable: enable
One object per device.
source:
default: http
http:
url: https://url/api
scheme: https
delimiter: !ruby/regexp /:/
map:
name: hostname
model: os
username: username
password: password
vars_map:
enable: enable
headers:
X-Auth-Token: 'somerandomstring'
Parent directory needs to be created manually, one file per device, with most recent running config.
output:
file:
directory: /var/lib/oxidized/configs
output:
default: git
git:
user: Oxidized
email: [email protected]
repo: "/var/lib/oxidized/devices.git"
If you prefer to have different outputs in different files and/or directories, you can easily do this by modifying the corresponding model. To change the behaviour for IOS, you would edit lib/oxidized/model/ios.rb.
For example, let's say you want to split out show version and show inventory into separate files in a directory called nodiff which your tools will not send automated diffstats for. You can apply a patch along the lines of
- cmd 'show version' do |cfg|
- comment cfg.lines.first
+ cmd 'show version' do |state|
+ state.type = 'nodiff'
+ state
- cmd 'show inventory' do |cfg|
- comment cfg
+ cmd 'show inventory' do |state|
+ state.type = 'nodiff'
+ state
+ end
- cmd 'show running-config' do |cfg|
- cfg = cfg.each_line.to_a[3..-1].join
- cfg.gsub! /^Current configuration : [^\n]*\n/, ''
- cfg.sub! /^(ntp clock-period).*/, '! \1'
- cfg.gsub! /^\ tunnel\ mpls\ traffic-eng\ bandwidth[^\n]*\n*(
+ cmd 'show running-config' do |state|
+ state = state.each_line.to_a[3..-1].join
+ state.gsub! /^Current configuration : [^\n]*\n/, ''
+ state.sub! /^(ntp clock-period).*/, '! \1'
+ state.gsub! /^\ tunnel\ mpls\ traffic-eng\ bandwidth[^\n]*\n*(
(?:\ [^\n]*\n*)*
tunnel\ mpls\ traffic-eng\ auto-bw)/mx, '\1'
- cfg
+ state = Oxidized::String.new state
+ state.type = 'nodiff'
+ state
which will result in the following layout
diff/$FQDN--show_running_config
nodiff/$FQDN--show_version
nodiff/$FQDN--show_inventory
The RESTful API and Web Interface is enabled by configuring the rest: parameter in the config file. This parameter can optionally contain a relative URI.
# Listen on http://127.0.0.1:8888/
rest: 127.0.0.1:8888
# Listen on http://10.0.0.1:8000/oxidized/
rest: 10.0.0.1:8000/oxidized
Below is an advanced example configuration. You will be able to (optinally) override options per device. The router.db format used is hostname:model:username:password:enable_password. Hostname and model will be the only required options, all others override the global configuration sections.
---
username: oxidized
password: S3cr3tx
model: junos
interval: 3600
log: ~/.config/oxidized/log
debug: false
threads: 30
timeout: 20
retries: 3
prompt: !ruby/regexp /^([\w.@-]+[#>]\s?)$/
vars:
enable: S3cr3tx
groups: {}
rest: 127.0.0.1:8888
input:
default: ssh, telnet
debug: false
ssh:
secure: false
output:
default: git
git:
user: Oxidized
email: [email protected]
repo: "~/.config/oxidized/oxidized.git"
source:
default: csv
csv:
file: ~/.config/oxidized/router.db
delimiter: !ruby/regexp /:/
map:
name: 0
model: 1
username: 2
password: 3
vars_map:
enable: 4
model_map:
cisco: ios
juniper: junos
You can define arbitrary number of hooks that subscribe different events. The hook system is modular and different kind of hook types can be enabled.
Following configuration keys need to be defined for all hooks:
events: which events to subscribe. Needs to be an array. See below for the list of available events.type: what hook class to use. See below for the list of available hook types.
node_success: triggered when configuration is succesfully pulled from a node and right before storing the configuration.node_fail: triggered afterretriesamount of failed node pulls.post_store: triggered after node configuration is stored.
The exec hook type allows users to run an arbitrary shell command or a binary when triggered.
The command is executed on a separate child process either in synchronous or asynchronous fashion. Non-zero exit values cause errors to be logged. STDOUT and STDERR are currently not collected.
Command is executed with the following environment:
OX_EVENT
OX_NODE_NAME
OX_NODE_FROM
OX_NODE_MSG
OX_NODE_GROUP
OX_JOB_STATUS
OX_JOB_TIME
Exec hook recognizes following configuration keys:
timeout: hard timeout for the command execution. SIGTERM will be sent to the child process after the timeout has elapsed. Default: 60async: influences whether main thread will wait for the command execution. Set this true for long running commands so node pull is not blocked. Default: falsecmd: command to run.
hooks:
name_for_example_hook1:
type: exec
events: [node_success]
cmd: 'echo "Node success $OX_NODE_NAME" >> /tmp/ox_node_success.log'
name_for_example_hook2:
type: exec
events: [post_store, node_fail]
cmd: 'echo "Doing long running stuff for $OX_NODE_NAME" >> /tmp/ox_node_stuff.log; sleep 60'
async: true
timeout: 120
The following objects exist in Oxidized.
- gets config from nodes
- must implement 'connect', 'get', 'cmd'
- 'ssh' and 'telnet' implemented
- stores config
- must implement 'store' (may implement 'fetch')
- 'git' and 'file' (store as flat ascii) implemented
- gets list of nodes to poll
- must implement 'load'
- source can have 'name', 'model', 'group', 'username', 'password', 'input', 'output', 'prompt'
- name - name of the devices
- model - model to use ios/junos/xyz, model is loaded dynamically when needed (Also default in config file)
- input - method to acquire config, loaded dynamically as needed (Also default in config file)
- output - method to store config, loaded dynamically as needed (Also default in config file)
- prompt - prompt used for node (Also default in config file, can be specified in model too)
- 'sql', 'csv' and 'http' (supports any format with single entry per line, like router.db)
- lists commands to gather from given device model
- can use 'cmd', 'prompt', 'comment', 'cfg'
- cfg is executed in input/output/source context
- cmd is executed in instance of model
- 'junos', 'ios', 'ironware' and 'powerconnect' implemented
Copyright 2013-2015 Saku Ytti [email protected] 2013-2015 Samer Abdel-Hafez [email protected]
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.