diff --git a/docs/advanced_article.md b/docs/advanced_article.md new file mode 100644 index 0000000..5da7279 --- /dev/null +++ b/docs/advanced_article.md @@ -0,0 +1,77 @@ +Table of Contents +================= + + * [Table of Contents](#table-of-contents) + * [Configure HA](#configure-ha) + * [Keepalived configuration and start](#keepalived-configuration-and-start) + * [Step1 LVS](#step1-lvs) + * [Step2 Compile Keepalived.conf](#step2-compile-keepalivedconf) + * [Step3 Start keepalived](#step3-start-keepalived) + +# Configure HA + +## Keepalived configuration and start + +Keepalived is a routing software written in C. The main goal of this project is to provide simple and robust facilities for loadbalancing and high-availability to Linux system and Linux based infrastructures. + +In the following steps, keepalived is installed by default. If not, you can refer to [Install](http://www.keepalived.org/doc/installing_keepalived.html) for configuration + +For learning more news, please see its [official website](http://www.keepalived.org/). + +**Note**: All of the operation is under root. + +### Step1 LVS + +LVS（Linux Virtual Server）is load balancing software for Linux kernel–based operating systems. + +A group of servers are connected to each other via a high-speed LAN(Local Area Network) or a geographically distributed wide area network. At their front end there is a Load Balancer which seamlessly dispatches network requests to real servers. + +Therefore, the structure of the server cluster is transparent to the user. The user accesses the network service provided by the cluster system just as if accessing a high performance and highly available server. + +Here are some specific operations : + +``` +$ sudo su - + +# vip=${{YOUR-VIP}} + +# /sbin/ifconfig lo down; + +# /sbin/ifconfig lo up; + +# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore; + +# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce; +# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore; + +# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce; + +# /sbin/ifconfig lo:0 ${vip} broadcast ${vip} netmask 255.255.255.255 up; + +# /sbin/route add -host ${vip} dev lo:0; + +# MySQL_port=${{YOUR-MYSQL-PORT}} + +# M_MAC=${{YOUR-MASTER-MAC}} +# iptables -t mangle -I PREROUTING -d ${vip} -p tcp -m tcp --dport ${MySQL_port} -m mac ! --mac-source ${M_MAC} -j MARK --set-mark 0x1; + +# S_MAC=${{YOUR-SLAVE-MAC}} +# iptables -t mangle -I PREROUTING -d ${vip} -p tcp -m tcp --dport ${MySQL_port} -m mac ! --mac-source ${S_MAC} -j MARK --set-mark 0x1; + +# N_MAC=${{YOUR-NORMAL-MAC}} +# iptables -t mangle -I PREROUTING -d ${vip} -p tcp -m tcp --dport ${MySQL_port} -m mac ! --mac-source ${N_MAC} -j MARK --set-mark 0x1; +``` + +### Step2 Compile Keepalived.conf + +If you want to see a simple configuration, there is a [template](config/192.168.0.11_keepalived.md). If you want to know more, there are a lot of [keepalived configuration introduction](http://www.keepalived.org/doc/configuration_synopsis.html). + +### Step3 Start keepalived + +``` +# /etc/init.d/keepalived start +``` + +After done these, `ipvsadm -ln` can help us check the configure right or wrong. + +Now you can refer to this [demo](config_template.md) to do the corresponding test. diff --git a/docs/config_template.md b/docs/config_template.md new file mode 100644 index 0000000..54103ad --- /dev/null +++ b/docs/config_template.md @@ -0,0 +1,187 @@ +Table of Contents +================= + + * [Xenon Confige Template](#xenon-confige-template) + * [An easy example : Xenon starts with mysql](#an-easy-example--xenon-starts-with-mysql) + * [Step1 Machine Condition](#step1-machine-condition) + * [Step2 Mutual Trust](#step2-mutual-trust) + * [Step3 Start Mysqld](#step3-start-mysqld) + * [Step4 Start Xenon](#step4-start-xenon) + * [Step5 Start Keepalived](#step5-start-keepalived) + +# Xenon Confige Template + +## An easy example : Xenon starts with mysql + +**Note**: Following is a synopsis of command line samples. For simplicity, we assume `xenon` is in your path. If not, replace `xenon` with `/path/to/xenon`. And the operating system user is root. + +### Step1 Machine Condition + +First create three machines (the default version is Ubuntu16.04). They all have mysqld service + +| HostName | IP | LVS-Role | MAC | +| ------------------ | ------------ | -------- | ----------------- | +| i-lf9g3f5n(Master) | 192.168.0.11 | Master | 52:54:39:8c:d1:e3 | +| i-0dc5giev(Slave) | 192.168.0.2 | Slave | 52:54:01:67:c2:82 | +| i-arb90jhc(Normal) | 192.168.0.3 | Normal | 52:54:4f:f7:26:82 | + +### Step2 Mutual Trust + +Set up the trust of the three machines configured to reduce the possibility of bugs behind + +- On i-lf9g3f5n(M): + +``` +# vi /etc/hosts + add these at last: + 192.168.0.2 i-0dc5giev + 192.168.0.3 i-arb90jhc +# su - ubuntu +$ ssh-keygen +$ ssh-copy-id ubuntu@i-0dc5giev +$ ssh-copy-id ubuntu@i-arb90jhc +``` + +- On i-0dc5giev(S1): + +``` +# vi /etc/hosts + add these at last: + 192.168.0.3 i-arb90jhc + 192.168.0.11 i-lf9g3f5n +# su - ubuntu +$ ssh-keygen +$ ssh-copy-id ubuntu@i-arb90jhc +$ ssh-copy-id ubuntu@i-lf9g3f5n +``` + +- On i-arb90jhc(S2): + +``` +# vi /etc/hosts + add these at last: + 192.168.0.2 i-0dc5giev + 192.168.0.11 i-lf9g3f5n +# su - ubuntu +$ ssh-keygen +$ ssh-copy-id ubuntu@i-0dc5giev +$ ssh-copy-id ubuntu@i-lf9g3f5n +``` + +### Step3 Start Mysqld + +Start mysqld on each machine. + +If you want to get my configure, please click [my.cnf](config/MySQL.md) + +``` +# su - ubuntu +$ mysqld_safe --defaults-file=/etc/mysql/mysqld.conf.d/mysqld.conf & +``` + +### Step4 Start Xenon + +**Note :** Before starting xenon make sure the mysqld service is up and running + +Start xenon on each machine. The three nodes add the other two node `ip:port` to each other. + +If you want to get my configure, please click [192.168.0.11_xenon](config/192.168.0.11_xenon.md), [192.168.0.2_xenon](config/192.168.0.2_xenon.md) and [192.168.0.3_xenon](config/192.168.0.3_xenon.md). + +For more information on start xenon please refer to `Step3` and `Step4`. + +- On each node + +``` +# mkdir -p /etc/xenon/ + +# mkdir -p /data/raft + +# mkdir -p /data/mysql + +# mkdir -p /opt/xtrabackup/ + +# mkdir -p /data/log + +# touch /etc/xenon/xenon.json + +# su - ubuntu + +# chown ubuntu:ubuntu /data/ -R + +$ ./xenon -c /etc/xenon/xenon.json > /data/log/xenon.log 2>&1 & +``` + +- On Master(192.168.0.11) + +``` +$ ./xenoncli cluster add 192.168.0.2:8801,192.168.0.3:8801 +``` + +- On Slave1(192.168.0.2) + +``` +$ ./xenoncli cluster add 192.168.0.11:8801,192.168.0.3:8801 +``` + +- On Slave2 (192.168.0.3) + +``` +$ ./xenoncli cluster add 192.168.0.11:8801,192.168.0.2:8801 +``` + +### Step5 Start Keepalived + +**Note :** I just configured the keepalived service on `Master` and `Slave`. You can follow my configuration to operate, you can also follow your train of thought(for more detail about config and start Keepalived, refer to `Step5`). + +If you want to get my configure, please click [192.168.0.11_keepalived](config/192.168.0.11_keepalived.md) and [192.168.0.2_keepalived](config/192.168.0.2_keepalived.md). + +For more information on start xenon please refer to [Keepalived-Configuration](keepalived.md) + +- On each node + +``` +# /sbin/ifconfig lo down; + +# /sbin/ifconfig lo up; + +# echo 1 >/proc/sys/net/ipv4/conf/lo/arp_ignore; + +# echo 2 >/proc/sys/net/ipv4/conf/lo/arp_announce; + +# echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore; + +# echo 2 >/proc/sys/net/ipv4/conf/all/arp_announc; + +# /sbin/ifconfig lo:0 192.168.0.252 broadcast 192.168.0.252 netmask 255.255.255.255 up; + +# /sbin/route add -host 192.168.0.252 dev lo:0; +``` + +- On Master(192.168.0.11) + +``` +# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:39:8c:d1:e3 -j MARK --set-mark 0x1 + +# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:01:67:c2:82 -j MARK --set-mark 0x1 + +# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:4f:f7:26:82 -j MARK --set-mark 0x1 + +# ipvsadm --set 5 4 120 + +# /etc/init.d/keepalived start +``` + +- On Slave(192.168.0.2) + +``` +# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:39:8c:d1:e3 -j MARK --set-mark 0x1 + +# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:01:67:c2:82 -j MARK --set-mark 0x1 + +# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:4f:f7:26:82 -j MARK --set-mark 0x1 + +# ipvsadm --set 5 4 120 + +# /etc/init.d/keepalived start +``` + diff --git a/docs/how_to_build_and_run_xenon.md b/docs/how_to_build_and_run_xenon.md index f01ac8a..e3c1d13 100644 --- a/docs/how_to_build_and_run_xenon.md +++ b/docs/how_to_build_and_run_xenon.md @@ -13,17 +13,6 @@ Table of Contents * [Step3.2 Configuration instructions](#step32-configuration-instructions) * [Step3.3 Account Description](#step33-account-description) * [Step4 Start xenon](#step4-start-xenon) - * [Step5 Keepalived configuration and start](#step5-keepalived-configuration-and-start) - * [Step5.1 LVS](#step51-lvs) - * [Step5.2 Compile Keepalived.conf](#step52-compile-keepalivedconf) - * [Step5.3 Start keepalived](#step53-start-keepalived) - * [Step6 An easy example : Xenon starts with mysql](#step6-an-easy-example--xenon-starts-with-mysql) - * [Step6.1 Machine Condition](#step61-machine-condition) - * [Step6.2 Mutual Trust](#step62-mutual-trust) - * [Step6.3 Start Mysqld](#step63-start-mysqld) - * [Step6.4 Start Xenon](#step64-start-xenon) - * [Step6.5 Start Keepalived](#step65-start-keepalived) - # How to build and run xenon @@ -116,12 +105,12 @@ Suppose you have already installed mysqld, if not, please reference [[MySQL 5.7 ``` $ sudo cp xenon/conf/xenon-sample.conf.json /etc/xenon/xenon.json ``` -* Make the following changes to the "${YOUR -....}" section: +* Make the following changes to the "${YOUR -....}" section. Here's a [simple template](config/xenon-simple.conf.json) for your reference.: ``` $ sudo vi /etc/xenon/xenon.json ``` -``` +```json { "server": { @@ -171,8 +160,6 @@ $ sudo vi /etc/xenon/xenon.json } } ``` -Here's a [simple template](config/xenon-simple.conf.json) for your reference. - ### Step3.2 Configuration instructions All of the above Fields marked with ${YOUR -...} needs to be replaced with your own parameters before starting. @@ -239,243 +226,4 @@ If the configuration is no problem, xenon will do after boot: * Detect mysqld, if the process does not exist then start * Waiting for the mysql can serve to detect the existence of duplicate accounts, or create -Now xenon has started successfully, the final step is keepalived configuration. - -## Step5 Keepalived configuration and start - -Keepalived is a routing software written in C. The main goal of this project is to provide simple and robust facilities for loadbalancing and high-availability to Linux system and Linux based infrastructures. - -In the following steps, keepalived is installed by default. If not, you can refer to [Install](http://www.keepalived.org/doc/installing_keepalived.html) for configuration - -For learning more news, please see its [official website](http://www.keepalived.org/). - -**Note**: All of the operation is under root. - -### Step5.1 LVS - -LVS（Linux Virtual Server）is load balancing software for Linux kernel–based operating systems. - -A group of servers are connected to each other via a high-speed LAN(Local Area Network) or a geographically distributed wide area network. At their front end there is a Load Balancer which seamlessly dispatches network requests to real servers. - -Therefore, the structure of the server cluster is transparent to the user. The user accesses the network service provided by the cluster system just as if accessing a high performance and highly available server. - - -Here are some specific operations : -``` -$ sudo su - - -# vip=${{YOUR-VIP}} - -# /sbin/ifconfig lo down; - -# /sbin/ifconfig lo up; - -# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore; - -# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce; -# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore; - -# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce; - -# /sbin/ifconfig lo:0 ${vip} broadcast ${vip} netmask 255.255.255.255 up; - -# /sbin/route add -host ${vip} dev lo:0; - -# MySQL_port=${{YOUR-MYSQL-PORT}} - -# M_MAC=${{YOUR-MASTER-MAC}} -# iptables -t mangle -I PREROUTING -d ${vip} -p tcp -m tcp --dport ${MySQL_port} -m mac ! --mac-source ${M_MAC} -j MARK --set-mark 0x1; - -# S_MAC=${{YOUR-SLAVE-MAC}} -# iptables -t mangle -I PREROUTING -d ${vip} -p tcp -m tcp --dport ${MySQL_port} -m mac ! --mac-source ${S_MAC} -j MARK --set-mark 0x1; - -# N_MAC=${{YOUR-NORMAL-MAC}} -# iptables -t mangle -I PREROUTING -d ${vip} -p tcp -m tcp --dport ${MySQL_port} -m mac ! --mac-source ${N_MAC} -j MARK --set-mark 0x1; -``` -### Step5.2 Compile Keepalived.conf - -If you want to see a simple configuration, there is a [template](config/192.168.0.11_keepalived.md). If you want to know more, there are a lot of [keepalived configuration introduction](http://www.keepalived.org/doc/configuration_synopsis.html). - -### Step5.3 Start keepalived - -``` -# /etc/init.d/keepalived start -``` - -After done these, `ipvsadm -ln` can help us check the configure right or wrong. - - -## Step6 An easy example : Xenon starts with mysql - -**Note**: Following is a synopsis of command line samples. For simplicity, we assume `xenon` is in your path. If not, replace `xenon` with `/path/to/xenon`. And the operating system user is root. - -### Step6.1 Machine Condition - -First create three machines (the default version is Ubuntu16.04). They all have mysqld service - -| HostName | IP | LVS-Role | MAC | -| ------------------ | ------------ | ------ | ----------------- | -| i-lf9g3f5n(Master) | 192.168.0.11 | Master | 52:54:39:8c:d1:e3 | -| i-0dc5giev(Slave) | 192.168.0.2 | Slave | 52:54:01:67:c2:82 | -| i-arb90jhc(Normal) | 192.168.0.3 | Normal | 52:54:4f:f7:26:82 | - -### Step6.2 Mutual Trust - -Set up the trust of the three machines configured to reduce the possibility of bugs behind - -* On i-lf9g3f5n(M): - -``` -# vi /etc/hosts - add these at last: - 192.168.0.2 i-0dc5giev - 192.168.0.3 i-arb90jhc -# su - ubuntu -$ ssh-keygen -$ ssh-copy-id ubuntu@i-0dc5giev -$ ssh-copy-id ubuntu@i-arb90jhc -``` - -* On i-0dc5giev(S1): - -``` -# vi /etc/hosts - add these at last: - 192.168.0.3 i-arb90jhc - 192.168.0.11 i-lf9g3f5n -# su - ubuntu -$ ssh-keygen -$ ssh-copy-id ubuntu@i-arb90jhc -$ ssh-copy-id ubuntu@i-lf9g3f5n -``` - -* On i-arb90jhc(S2): - -``` -# vi /etc/hosts - add these at last: - 192.168.0.2 i-0dc5giev - 192.168.0.11 i-lf9g3f5n -# su - ubuntu -$ ssh-keygen -$ ssh-copy-id ubuntu@i-0dc5giev -$ ssh-copy-id ubuntu@i-lf9g3f5n -``` - -### Step6.3 Start Mysqld - -Start mysqld on each machine. - -If you want to get my configure, please click [my.cnf](config/MySQL.md) - -``` -# su - ubuntu -$ mysqld_safe --defaults-file=/etc/mysql/mysqld.conf.d/mysqld.conf & -``` - -### Step6.4 Start Xenon - -**Note :** Before starting xenon make sure the mysqld service is up and running - -Start xenon on each machine. The three nodes add the other two node `ip:port` to each other. - -If you want to get my configure, please click [192.168.0.11_xenon](config/192.168.0.11_xenon.md), [192.168.0.2_xenon](config/192.168.0.2_xenon.md) and [192.168.0.3_xenon](config/192.168.0.3_xenon.md). - -For more information on start xenon please refer to `Step3` and `Step4`. - -* On each node - -``` -# mkdir -p /etc/xenon/ - -# mkdir -p /data/raft - -# mkdir -p /data/mysql - -# mkdir -p /opt/xtrabackup/ - -# mkdir -p /data/log - -# touch /etc/xenon/xenon.json - -# su - ubuntu - -# chown ubuntu:ubuntu /data/ -R - -$ ./xenon -c /etc/xenon/xenon.json > /data/log/xenon.log 2>&1 & -``` - -* On Master(192.168.0.11) - -``` -$ ./xenoncli cluster add 192.168.0.2:8801,192.168.0.3:8801 -``` - -* On Slave1(192.168.0.2) - -``` -$ ./xenoncli cluster add 192.168.0.11:8801,192.168.0.3:8801 -``` - -* On Slave2 (192.168.0.3) - -``` -$ ./xenoncli cluster add 192.168.0.11:8801,192.168.0.2:8801 -``` - -### Step6.5 Start Keepalived - -**Note :** I just configured the keepalived service on `Master` and `Slave`. You can follow my configuration to operate, you can also follow your train of thought(for more detail about config and start Keepalived, refer to `Step5`). - -If you want to get my configure, please click [192.168.0.11_keepalived](config/192.168.0.11_keepalived.md) and [192.168.0.2_keepalived](config/192.168.0.2_keepalived.md). - -For more information on start xenon please refer to [Keepalived-Configuration](keepalived.md) - -* On each node - -``` -# /sbin/ifconfig lo down; - -# /sbin/ifconfig lo up; - -# echo 1 >/proc/sys/net/ipv4/conf/lo/arp_ignore; - -# echo 2 >/proc/sys/net/ipv4/conf/lo/arp_announce; - -# echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore; - -# echo 2 >/proc/sys/net/ipv4/conf/all/arp_announc; - -# /sbin/ifconfig lo:0 192.168.0.252 broadcast 192.168.0.252 netmask 255.255.255.255 up; - -# /sbin/route add -host 192.168.0.252 dev lo:0; -``` - -* On Master(192.168.0.11) - -``` -# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:39:8c:d1:e3 -j MARK --set-mark 0x1 - -# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:01:67:c2:82 -j MARK --set-mark 0x1 - -# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:4f:f7:26:82 -j MARK --set-mark 0x1 - -# ipvsadm --set 5 4 120 - -# /etc/init.d/keepalived start -``` - -* On Slave(192.168.0.2) - -``` -# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:39:8c:d1:e3 -j MARK --set-mark 0x1 - -# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:01:67:c2:82 -j MARK --set-mark 0x1 - -# iptables -t mangle -I PREROUTING -d 192.168.0.252 -p tcp -m tcp --dport 3306 -m mac ! --mac-source 52:54:4f:f7:26:82 -j MARK --set-mark 0x1 - -# ipvsadm --set 5 4 120 - -# /etc/init.d/keepalived start -``` - +Now xenon started successfully. Continue to refer to the [advanced documentation](advanced_article.md) to configure the highly available cluster.