Puli contains code for the dispatcher (server), worker (client) as well as some utilities to submit jobs and inspect what is happening in the network.
Currently, Puli is only supported on Linux (32 or 64 bits). The packages listed below are for Red Hat based distributions. Bot the central manager and the rendernode require Python 2.x
The manager is the main server component, it holds the following subsystems:
- a central db
- a dispatcher daemon
- python
- python-tornado (<= 3.0.2)
- MySQL-python
- python-sqlobject
- python-requests
- mysql-server
A render node is installed on each machine that can start a rendering process. It holds the following subsystems:
- a worker daemon
- an optional commandwatcher process when a process is currently rendering
- python-tornado (<= 3.0.2)
- python-requests
- python-psutil
Several tools come along with OpenRenderManagement that have specific requirements. They can be installed on a separate computer to operate the system:
- puliquery: command line request tools
- puliexec: simplify submission of scripts and command line
- pulistats: set of script to trace graphs based on server statistics
- pulistatsviewer: GUI program to trace and display graphs
- python-tornado (<= 3.0.2)
- numpy
- python-pygal
- PyQt4
The required packages can be installed on the shell:
yum install mysql mysql-server python python-devel python-tornado MySQL-python python-sqlobject python-requestsalternatively you can also use a python virtual env:
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
pip install virtualenv
virtualenv puli_venv
cd puli_venv
source bin/activate
pip install tornado==3.0.2 sqlobject MySQL-python requests psutil formencode simplejsonConnect to your database:
service mysqld start
mysql -u root -pExecute the following mysql commands to create the database and the dedicated user:
create database pulidb character set utf8;
create database pulistatdb character set utf8;
create user puliuser identified by 'yourpasswd';
grant all privileges on pulidb.*
to 'puliuser'@'localhost'
identified by 'yourpasswd' with grant option;
grant all privileges on pulistatdb.*
to 'puliuser'@'localhost'
identified by 'yourpasswd' with grant option;
You can install Puli to wherever you like (for example /opt/puli) just make sure this dir is in your PYTHONPATH and all the settings are being adjusted.
cd
git checkout https://github.com/mikrosimage/OpenRenderManagement.git
mkdir /opt/puli
mkdir /opt/puli/conf
mkdir /opt/puli/logs
cp -R ~/OpenRenderManagement/Puli/src/octopus/ /opt/puli/
cp -R ~/OpenRenderManagement/Puli/scripts/ /opt/puli/
cp -R ~/OpenRenderManagement/Puli/etc/puli /opt/puli/conf/
export PYTHONPATH=/opt/puli:$PYTHONPATHThe settings are located in the file /opt/puli/octopus/dispatcher/settings.py.
Please check the following settings and adjust:
CONFDIR = /opt/puli/conf
LOGDIR = /opt/puli/logs
POOLS_BACKEND_TYPE = "file"
DB_ENABLE = True
DB_CLEAN_DATA = True
DB_URL = "mysql://puliuser:[email protected]/pulidb"
STAT_DB_URL = "mysql://puliuser:[email protected]/pulistatdb"To launch the dispatcher, execute the following command in a shell:
python /opt/puli/scripts/dispatcherd.py --debug --console
Upon the first execution, Puli will create the appropriate tables in the database.
For the following executions, change these parameters in the settings file to:
POOLS_BACKEND_TYPE = "db"
DB_ENABLE = True
DB_CLEAN_DATA = FalseThese will tell Puli to init itself with the database that was previously created, and it will enable the persistence of the jobs.
If you want to erase the database in order to have a clean start, reset these parameters to the values in the installation section.
While the dispatcher manages the joblist, the actual job is executed by the worker. You need at least one worker running. Workers are installed on every computer that you want to use for renderfarm needs.
Installing the worker is pretty similar to the dispatcher:
yum install python-tornado python-requests python-psutilLaunch:
export PYTHONPATH=/opt/puli:$PYTHONPATH
python /opt/puli/scripts/workerd.py --debug --console --server 192.168.1.2 --serverport 8004
#RELEASE NOTES None
#KNOWN ISSUES
The model needs a bit of a refactoring. It has been designed several years ago to address some use-cases that are no longer relevant. It can be more simple and straightforward than it is now.
Several aspects of the worker can be optimized (like the handling of status and completion update, and the interaction between the cmdwatcher and the process)
#LICENSING
Puli is distributed using the modified BSD license. Please read the "LICENSE" file for the legal wording.
Long story short, Puli is free, as well as freely modifiable and redistributable.
You may use part or all of it in your own applications, whether proprietary or open, free or commercial or not.