Luna is a baremetal provisioning tool that uses an image based approach. It delivers full images of operating systems and not a 'recipe' on how to configure one. It also dramatically speeds up installation time, and reduces administrative efforts.
Luna uses the BitTorrent protocol to provision nodes. As such, every booting node helps the others to boot.
Once a node is fully booted it stops being a torrent seeder and other nodes can no longer use it to download the image. The torrent client only acts in the initrd environment.
Luna does not require any additional services to run on a node. By default it changes very a limited number of files on provisioned nodes.
It us usually limited to /etc/hostname and /etc/sysconfig/network-scripts/ifcfg-* files.
| Number of nodes | Time for cold boot, min | xCAT cold boot, min |
|---|---|---|
| 1 | 3 | 9 |
| 36 | 4 | 26 |
| 72 | 4 | 53 |
Image size is 1GB. Provisioning node is equiped with a 1Gb ethernet interface.
In a cluster of 300 nodes. Boot time using luna has been measured to be aproximately 5 minutes. This includes BIOS POST procedures and all starting systemd services.
Let's assume you have a server using the IP address 10.30.255.254 to provision the cluster
yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum -y install wget python-docutils gcc-c++ rb_libtorrent-devel boost-devel make rpm-build redhat-rpm-config
git clone https://github.com/clustervision/luna
cd luna
make rpm
Note. Instead of building luna from scratch you can easily configure repository and install everything from it:
curl https://updates.clustervision.com/luna/1.2/centos/luna-1.2.repo > /etc/yum.repos.d/luna-1.2.repo
yum install luna
Source code is available here
wget https://www.nsc.liu.se/~kent/python-hostlist/python-hostlist-1.17.tar.gz
rpmbuild -ta python-hostlist-1.17.tar.gz
yum -y install python-hostlist-1.17-1.noarch.rpm
yum -y install rpm/RPMS/x86_64/luna-[0-9]*rpm
vim /etc/luna.conf
[ -f /root/.ssh/id_rsa ] || ssh-keygen -t rsa -f /root/.ssh/id_rsa -N ''
# Disable SELINUX
sed -i -e 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
setenforce 0
You can also choose not to disable selinux but install luna-selinux package
# Configure xinetd
mkdir /tftpboot
sed -e 's/^\(\W\+disable\W\+\=\W\)yes/\1no/g' -i /etc/xinetd.d/tftp
sed -e 's|^\(\W\+server_args\W\+\=\W-s\W\)/var/lib/tftpboot|\1/tftpboot|g' -i /etc/xinetd.d/tftp
cp /usr/share/ipxe/undionly.kpxe /tftpboot/luna_undionly.kpxe
# Configure nginx and named
cp /usr/share/luna/nginx-luna.conf /etc/nginx/conf.d/luna.conf
echo 'include "/etc/named.luna.zones";' >> /etc/named.conf
touch /etc/named.luna.zones
# Enable and start services
systemctl enable nginx
systemctl enable mongod
systemctl enable named
systemctl enable xinetd
systemctl restart xinetd
systemctl restart mongod
systemctl restart nginx
systemctl restart named
export OSIMAGE_PATH=/opt/luna/os/compute
mkdir -p ${OSIMAGE_PATH}/var/lib/rpm
rpm --root ${OSIMAGE_PATH} --initdb
yum -y install yum-utils
yumdownloader centos-release
rpm --root ${OSIMAGE_PATH} -ivh centos-release\*.rpm
yum --installroot=${OSIMAGE_PATH} -y groupinstall Base
yum --installroot=${OSIMAGE_PATH} -y install kernel
yum --installroot=${OSIMAGE_PATH} -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum --installroot=${OSIMAGE_PATH} -y install luna-client*.rpm
mkdir ${OSIMAGE_PATH}/root/.ssh
chmod 700 ${OSIMAGE_PATH}/root/.ssh
mount -t devtmpfs devtmpfs ${OSIMAGE_PATH}/dev/
chroot ${OSIMAGE_PATH}
ssh-keygen -f /etc/ssh/ssh_host_ecdsa_key -N '' -t ecdsa
abrt-auto-reporting enabled
passwd
exit
umount ${OSIMAGE_PATH}/dev/
cat /root/.ssh/id_rsa.pub >> ${OSIMAGE_PATH}/root/.ssh/authorized_keys
chmod 600 ${OSIMAGE_PATH}/root/.ssh/authorized_keys
luna cluster init --frontend_address 10.30.255.254
luna network add -n cluster -N 10.30.0.0 -P 16
luna cluster makedhcp --network cluster --start_ip 10.30.128.1 --end_ip 10.30.140.255
systemctl start lweb ltorrent
systemctl enable lweb ltorrent
luna osimage add -n compute -p ${OSIMAGE_PATH}
luna osimage pack compute
luna bmcsetup add -n base
luna network add -n ipmi -N 10.31.0.0 -P 16
luna switch add -n switch01 --oid .1.3.6.1.2.1.17.7.1.2.2.1.2 --network ipmi --ip 10.31.253.21
luna group add -n compute -o compute -N cluster
luna group change compute -b base
luna group change -n compute --bmcnetwork --setnet ipmi
luna group change compute -i BMC -A
luna group change compute -i BMC --setnet ipmi
luna node add -g compute
luna cluster makedns
Please note that group and corresponding node has interface named BOOTIF. This is special placeholder for interface connected to provision network. If interface is know it can be renamed or recreated. Another placeholde is BMC. It is used in ipmitool commands to set up BMC interface.
In service mode you can perform an inventory of the interfaces, local disks, BMC features
luna node change -n node001 --service y
You can boot the nodes in diskless mode, or write your own partitioning script using:
luna group change -n compute --partscript -e
Sample partitioning script for a device called /dev/sda:
parted /dev/sda -s 'mklabel msdos'
parted /dev/sda -s 'rm 1; rm 2'
parted /dev/sda -s 'mkpart p ext2 1 256m'
parted /dev/sda -s 'mkpart p ext3 256m 100%'
parted /dev/sda -s 'set 1 boot on'
mkfs.ext2 /dev/sda1
mkfs.ext4 /dev/sda2
mount /dev/sda2 /sysroot
mkdir /sysroot/boot
mount /dev/sda1 /sysroot/boot
cat << EOF | luna group change -n compute --post -e
mount -o bind /proc /sysroot/proc
mount -o bind /dev /sysroot/dev
chroot /sysroot /bin/bash -c "/usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg; /usr/sbin/grub2-install /dev/sda"
umount /sysroot/dev
umount /sysroot/proc
EOF
luna node add -g compute
A node name will be automatically generated using the default nodeXXX format
luna node change -n node001 -s switch01
luna node change -n node001 -p 1
systemctl ltorrent start
systemctl lweb start
curl "http://10.30.255.254:7050/luna?step=boot"
wget "http://10.30.255.254:7050/boot/compute-vmlinuz-3.10.0-327.10.1.el7.x86_64"
curl "http://10.30.255.254:7050/luna?step=install&node=node001"
Also it is possible to fetch install and boot scripts for the node usin luna CLI:
luna node show node001 --script boot
luna node show node001 --script install
luna cluster makedhcp -N cluster -s 10.30.128.1 -e 10.30.140.255
luna cluster makedns
Luna supports multiple modes of booting a node:
- Booting from localdisk:
luna node change -n node001 --localboot y
- Booting into service mode for diagnostics:
luna node change -n node001 --service y
- Configure the BMC when booting:
luna node change -n node001 --setupbmc y