Create a BOINC server (cookbook)
This is a cookbook for creating a BOINC server on Ubuntu 22.04.
You can use Xubuntu (which uses less memory) instead of Ubuntu; everything should work. You can also use other versions of Ubuntu, or other Linux distributions, but you may need to resolve some glitches.
Alternatively to this you can watch the video tutorial:
We suggest setting up the BOINC server inside a Virtualbox VM. This way
- you can create snapshots, and roll back changes if something goes wrong;
- you can move the VM image to a different machine if needed.
All these instructions should work on bare metal too, so you can skip the Virtualbox portion if you'd like.
We're going to run Ubuntu 22.04 in a Virtualbox virtual machine. You can do this on a Windows, Mac, or Linux desktop system.
You can also run Ubuntu on a cloud node or on bare hardware; in that case, skip the following.
- Install VirtualBox 7.0 from virtualbox.org, and download the Guest Additions.
- Download the Ubuntu 22.04 .iso from ubuntu.com
- Create a VM; instructions are here. Name it 'BOINC' or whatever you want. NOTE: only alphanumerics, - and . are allowed in the hostname. Give it at least 4 GB RAM, 1 CPU, and 25 GB disk. In the 'Unattended Guest OS Install Setup' dialog, create an account named 'boincadm', with a password of your choice, and select Guest Additions.
- Start the VM (this takes a few minutes)
In the Virtualbox Manager:
- Right-click on the VM and click Settings
- Click Network
- Next to Attached to:, select 'Bridged Adapter'.
- Click OK
Close the VM window ('X' in upper right corner), selecting Power Off.
In the Virtualbox Manager, start the VM.
In the VM: log in, open a terminal window (click on the icon at lower left, click Terminal), and type
ip address
This will print something like
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 78:2b:cb:55:e1:30 brd ff:ff:ff:ff:ff:ff
inet 192.168.42.17/24 brd 192.168.42.255 scope global dynamic noprefixroute enp0s3
valid_lft 86345sec preferred_lft 86345sec
...
The 192.168.42.17 is the IP address of the VM;
it's accessible from the guest system,
the host system, and other hosts on the same LAN.
Log in as boincadm. The account should already have sudo permission. If not:
su
adduser boincadm sudo
su boincadm
Set up the boincadm account as you like. I append this to .bashrc to prevent accidents:
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
and I put this in .vimrc to facilitate editing indented files with vim:
set ai
set ts=4
set expandtab
You can use any other editor (like nano) instead.
Go to Settings / Power and set Screen Blank to Never to avoid annoying screen blanking.
Open Firefox and navigate to this page. That will let you copy and paste the commands below.
Update apt:
sudo apt update && sudo apt upgrade -y
sudo snap refresh
then install these packages:
sudo apt-get -y install git
sudo apt-get -y install mysql-server
sudo apt-get -y install libmysqlclient-dev
sudo apt-get -y install apache2
sudo apt-get -y install autoconf
sudo apt-get -y install pkg-config
sudo apt-get -y install libtool
sudo apt-get -y install libcurl4-openssl-dev
sudo apt-get -y install python3-pip
sudo apt-get -y install python-is-python3
sudo apt-get -y install libapache2-mod-php
sudo apt-get -y install php-mysql
sudo apt-get -y install php-cli
sudo apt-get -y install php-gd
sudo apt-get -y install php-xml
Install the Python MySQL interface:
sudo pip install -U pip
pip install mysqlclient
Run a script to make MySQL secure:
sudo mysql_secure_installation
validate passwords: yes
(use min password strength 0)
set password to foobar99
remove anon users: yes
disallow remote root: yes
remove test database: yes
reload: yes
Then create a MySQL identity for the boincadm account:
sudo mysql
CREATE USER 'boincadm'@'localhost' IDENTIFIED BY 'foobar99';
GRANT ALL ON *.* TO 'boincadm'@'localhost';
^D
See more info on this.
Check that MySQL is working:
mysql -u boincadm -p
(type foobar99)
show databases;
^D
The BOINC server software has two parts. Some of it runs in the web server, under the Apache account (www-data), The rest of it is daemon programs and command-line scripts, which run as boincadm. To allow these two parts to share files, we put both users in the same group, and all files are created with group RW access.
Put www-data in the boincadm group:
sudo usermod -a -G boincadm www-data
Enable CGI:
sudo a2enmod cgi
sudo systemctl restart apache2
Check that Apache is working by opening Firefox (in the VM)
and visitinghttp://localhost.
Download the BOINC software from github:
cd
git clone https://github.com/BOINC/boinc.git
Build it:
cd boinc
./_autosetup
./configure --disable-client --disable-manager --enable-apps
make
Note that we're building just the server, not the client. The compile will produce lots of compiler warnings; ignore them.
To create a project:
cd ~/boinc/tools
./make_project --db_passwd foobar99 --url_base http://192.168.42.17 test
where 192.168.42.17 is the IP address of the VM (see above).
This will show something like this:
Creating project 'test' (short name 'test'):
PROJECT_ROOT = /home/boincadm/projects/test/
PROJECT_HOST = BOINC
URL_BASE = http://192.168.42.17/
HTML_USER_URL = http://192.168.42.17/test/
HTML_OPS_URL = http://192.168.42.17/test_ops/
KEY_DIR = /home/boincadm/projects/test/keys/
DB_NAME = test
DB_HOST =
Continue? [Y/n]
('BOINC' is the name of the host, specified when you created the VM.)
Enter 'y'.
This does two things:
- It creates a database (named 'test' in this case).
- It creates a tree of files rooted at
~/projects/test. This is called the 'project directory'.
The project directory will contain a file test.readme
telling you what to do next.
Look at it:
cd ~/projects/test
more test.readme
The items include:
- Change the Apache configuration to enable the project's web pages:
sudo cp test.httpd.conf /etc/apache2/sites-enabled
sudo apache2ctl restart
- Add a crontab entry to run a maintenance script periodically:
crontab -e
add the line:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * cd /home/boincadm/projects/test; /home/boincadm/projects/test/bin/start --cron
- Initialize the project's database:
bin/xadd
- Enable the 'admin web interface'.
In addition to the public web site, the project has a web site for use by project administrators. To enable it, set a password:
cd ~/projects/test/html/ops
htpasswd -c .htpasswd boincadm
Check this by going to http://192.168.42.17/test_ops in Firefox.
You'll need to enter user name 'boincadm' and the password.
- Add web page content.
There are various ways that you can add your own content to the public web site. For starters, change the project name and copyright holder.
cd ~/projects/test/html/project
vi project.inc
... replace text for PROJECT and COPYRIGHT_HOLDER
Check this by going to http://192.168.42.17/test in Firefox.
Finally, start the project by typing
cd ~/projects/test
bin/start
(see the Project Control section below)
Point Firefox to your project's web site at http://192.168.42.17/test/.
Click 'Join'.
Enter a name, email address, and password.
sudo add-apt-repository --remove ppa:costamagnagianfranco/boinc
sudo apt-get install boinc
Due to bugs in the BOINC client package, running the client requires the following:
sudo chown -R boincadm /var/lib/boinc-client- Open a terminal window and enter
cd
boinc
- In the original terminal, enter
boincmgr.
(When we fix these, you'll be able to run BOINC by double clicking its icon).
Anyway: the BOINC Manager will bring up the Add Project dialog.
- Enter
http://192.168.42.17/test/. - Enter the email address and password of the account you just created.
Select View / Advanced View. The project will appear in the Projects tab. Open the Event Log (Tools / Event Log). you'll see that the client asks the project for jobs but doesn't get any.
We'll create an application called 'uppercase'.
It reads a file named in, converts its text to uppercase,
and writes the result to a file named out.
It also computes for 20 seconds.
This application is supplied as part of BOINC,
in ````/boinc/apps/uppercase.cpp```.
In Firefox, go to the admin web page: http://192.168.42.17/test_ops.
Click 'Manage applications'.
Add an application with name 'uppercase' and description 'Uppercase'
(or whatever you want).
Input and output templates are XML files
that describe an app's input and output files.
They're stored in the project's templates directory.
Install the templates for the uppercase app:
cd ~/projects/test
cp ~/boinc/apps/uppercase_in templates
cp ~/boinc/apps/uppercase_out templates
Each app has a 'validator' (which checks the correctness of output files) and an 'assimilator' (which processes correct output files). These are daemons, which means that they run all the time.
In this case we'll use a validator that accepts all results as correct,
and an assimilator that moves the output files to
a directory test/sample_results.
Add the following to ~/projects/test/config.xml,
in the <daemons> section:
<daemon>
<cmd>sample_trivial_validator --app uppercase</cmd>
</daemon>
<daemon>
<cmd>sample_assimilator --app uppercase</cmd>
</daemon>
An app can have versions for different platforms. Here we'll just create a version for Linux on Intel 64-bit.
Create the following directory structure:
projects/
test/
apps/
uppercase/
1.0/
x86_64-pc-linux-gnu/
You can do this with:
cd ~/projects/test/apps
mkdir -p uppercase/1.0/x86_64-pc-linux-gnu
Then
cd ~/projects/test/apps/uppercase/1.0/x86_64-pc-linux-gnu
cp ~/boinc/apps/uppercase .
cp ~/boinc/apps/version.xml .
Create the app version:
cd ~/projects/test
bin/update_versions
This will give some warnings related to code signing; ignore them and enter Y.
Then stop and restart the project to pick up these changes:
cd ~/projects/test
bin/stop
bin/start
BOINC projects generally process thousands or millions of jobs a day, and these jobs are created by 'work generator' programs. In this demo we'll create jobs one at a time using a command-line script.
cd ~/projects/test
vi infile
(add some mixed-case text)
bin/submit_job uppercase infile
(this prints a job name)
Go to BOINC Manager. In the Projects tab, select the test project and click Update. Go to the Tasks tab and verify that a job is running. Wait for it to finish.
In the terminal window:
bin/query_job <jobname>
It should say that the job is done, and show you its output.
A BOINC project has a number of daemon processes.
Each one writes a log file.
These files are stored in projects/test/log_BOINC
(BOINC is the host name).
The set of log files might be:
feeder.log
file_upload_handler.log
sample_trivial_validator.log
transitioner.log
file_deleter.log
sample_assimilator.log
scheduler.log
scheduler.log is the log file for the job dispatcher,
which runs as a CGI program.
These files contain low-level information, which won't make any sense. But if they contain messages containing 'Error' or 'warning', there may be a problem you'll need to fix.
Errors generated by BOINC's PHP web code are written to Apache's log files. To look at these:
sudo ls /var/log/apache2/
sudo vi /var/log/apache2/error.log
A project includes a number of daemon processes that handle jobs. To start these processes:
cd ~/projects/test
bin/start
To stop the project:
bin/stop
When a project is stopped, its web site is still available but it won't send or receive jobs.
To check the status of the project:
bin/status
To delete a project:
mysql -u boincadm -p
drop database test;
^D
cd ~/projects
rm -r test
sudo rm /etc/apache2/sites-enabled/test.httpd.conf
crontab -e
(remove line for test)
To upgrade a project to the latest BOINC code:
cd ~/boinc
git pull
./configure --disable-client --disable-manager
make
cd tools
upgrade test
You can create multiple BOINC projects on the same server. They must have different names. Each one will have its own database, URL, and website.
If you run into problems,
the best way to get help is by sending email
to the [email protected] email list.
You can subscribe to this list here:
https://groups.google.com/a/ssl.berkeley.edu/g/boinc_projects
You can also report bugs in BOINC by creating an issue on the BOINC github repo: https://github.com/BOINC/boinc
Congratulations! You've created a BOINC project and used it to process a job. To make a real project, that volunteers can participate in, there are several next steps:
-
Use a server machine that's visible on the Internet, has a domain name, and uses Secure Socket Layer (SSL).
-
Port your applications to BOINC, and make app versions for Windows and Mac as well as Linux.
-
Develop your own work generators, validators and assimilators for these apps.
-
Recruit volunteers: publicize your project, or register it on Science United.