graphite-project · piotr1212 · Jan 7, 2019 · Jan 18, 2019 · Jan 31, 2019 · Jan 31, 2019
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -7,6 +7,6 @@ include conf/*.example
 include webapp/graphite/local_settings.py.example
 recursive-include distro/ *
 recursive-include webapp/graphite/ *.html
-recursive-include webapp/content/ *
+recursive-include webapp/graphite/static/ *
 exclude webapp/graphite/local_settings.py
 exclude conf/*.conf
diff --git a/conf/graphite.wsgi.example b/conf/graphite.wsgi.example
@@ -2,6 +2,5 @@ import sys
 # In case of multi-instance graphite, uncomment and set appropriate name
 # import os
 # os.environ['GRAPHITE_SETTINGS_MODULE'] = 'graphite.local_settings'
-sys.path.append('/opt/graphite/webapp')
 
 from graphite.wsgi import application
diff --git a/docs/ceres.rst → docs/Ceres.rst b/docs/ceres.rst → docs/Ceres.rst
@@ -61,12 +61,12 @@ large but allowed gap occurs it has to get filled in, which means instead of a s
 new file we could end up doing an ``(8 * MAX_SLICE_GAP)``-byte write to the latest slice.
 
 
-Rollup Aggregation
-------------------
+Roll-up Aggregation
+-------------------
 Expected features such as roll-up aggregation and data expiration are not provided by Ceres itself, but
 instead are implemented as maintenance plugins.
 
-Such a rollup plugin exists in Ceres that aggregates data points in a way that is similar behavior of
+Such a roll-up plugin exists in Ceres that aggregates data points in a way that is similar behavior of
 Whisper archives. Where multiple data points are collapsed and written to a lower precision slice, and
 data points outside of the set slice retentions are trimmed. By default, an average function is used,
 however alternative methods can be chosen by changing the metadata.

diff --git a/docs/faq.rst → docs/FAQ.rst b/docs/faq.rst → docs/FAQ.rst
@@ -3,7 +3,7 @@ FAQ
 
 What is Graphite?
 -----------------
-Graphite is a highly scalable real-time graphing system. As a user, you write an application that collects numeric time-series data that you are interested in graphing, and send it to Graphite's processing backend, :doc:`carbon </carbon-daemons>`, which stores the data in Graphite's specialized database. The data can then be visualized through graphite's web interfaces.
+Graphite is a highly scalable real-time graphing system. As a user, you write an application that collects numeric time-series data that you are interested in graphing, and send it to Graphite's processing backend, :doc:`carbon </carbon-daemons>`, which stores the data in Graphite's specialized database. The data can then be visualized through Graphite's web interfaces.
 
 
 Who should use Graphite?
@@ -35,7 +35,7 @@ Graphite was internally developed by `Orbitz`_ where it is used to visualize a v
 
 What is Graphite written in?
 ----------------------------
-Python2. The Graphite webapp is built on the `Django`_ web framework and uses the ExtJS javascript GUI toolkit. The graph rendering is done using the Cairo graphics library. The backend and database are written in pure Python.
+Python. The Graphite webapp is built on the `Django`_ web framework and uses the ExtJS javascript GUI toolkit. The graph rendering is done using the Cairo graphics library. The backend and database are written in pure Python.
 
 
 Who writes and maintains Graphite?
@@ -52,7 +52,7 @@ The `Apache 2.0 License`_.
 
 Does Graphite use RRDtool?
 --------------------------
-No, not since Graphite was first written in 2006 at least. Graphite has its own specialized database library called :doc:`whisper </whisper>`, which is very similar in design to RRD, but has a subtle yet fundamentally important difference that Graphite requires. There are two reasons whisper was created. The first reason is that RRD is designed under the assumption that data will always be inserted into the database on a regular basis, and this assumption causes RRD behave undesirably when given irregularly occurring data. Graphite was built to facilitate visualization of various application metrics that do not always occur regularly, like when an uncommon request is handled and the latency is measured and sent to Graphite. Using RRD, the data gets put into a temporary area inside the database where it is not accessible until the current time interval has passed *and* another value is inserted into the database for the following interval. If that does not happen within an allotted period of time, the original data point will get overwritten and is lost. Now for some metrics, the lack of a value can be correctly interpreted as a value of zero, however this is not the case for metrics like latency because a zero indicates that work was done in zero time, which is different than saying no work was done. Assuming a zero value for latency also screws up analysis like calculating the average latency, etc.
+No, not since Graphite was first written in 2006 at least. Graphite has its own specialized database library called :doc:`whisper </Whisper>`, which is very similar in design to RRD, but has a subtle yet fundamentally important difference that Graphite requires. There are two reasons whisper was created. The first reason is that RRD is designed under the assumption that data will always be inserted into the database on a regular basis, and this assumption causes RRD behave undesirably when given irregularly occurring data. Graphite was built to facilitate visualization of various application metrics that do not always occur regularly, like when an uncommon request is handled and the latency is measured and sent to Graphite. Using RRD, the data gets put into a temporary area inside the database where it is not accessible until the current time interval has passed *and* another value is inserted into the database for the following interval. If that does not happen within an allotted period of time, the original data point will get overwritten and is lost. Now for some metrics, the lack of a value can be correctly interpreted as a value of zero, however this is not the case for metrics like latency because a zero indicates that work was done in zero time, which is different than saying no work was done. Assuming a zero value for latency also screws up analysis like calculating the average latency, etc.
 
 The second reason whisper was written is performance. RRDtool is very fast; in fact it is much faster than whisper. But the problem with RRD (at the time whisper was written) was that RRD only allowed you to insert a single value into a database at a time, while whisper was written to allow the insertion of multiple data points at once, compacting them into a single write operation. This improves performance drastically under high load because Graphite operates on many many files, and with such small operations being done (write a few bytes here, a few over there, etc) the bottleneck is caused by the *number of I/O operations*. Consider the scenario where Graphite is receiving 100,000 distinct metric values each minute; in order to sustain that load Graphite must be able to write that many data points to disk each minute. But assume that your underlying storage can only handle 20,000 I/O operations per minute. With RRD (at the time whisper was written), there was no chance of keeping up. But with whisper, we can keep caching the incoming data until we accumulate say 10 minutes worth of data for a given metric, then instead of doing 10 I/O operations to write those 10 data points, whisper can do it in one operation. The reason I have kept mentioning "at the time whisper was written" is that RRD now supports this behavior. However Graphite will continue to use whisper as long as the first issue still exists.
 
@@ -76,7 +76,7 @@ Is there a diagram of Graphite's architecture?
 ----------------------------------------------
 There sure is! Here it is:
 
-.. image:: ../webapp/content/img/overview.png
+.. image:: ../webapp/graphite/static/img/overview.png
 
 
 .. _Django: http://www.djangoproject.com/

diff --git a/docs/whisper.rst → docs/Whisper.rst b/docs/whisper.rst → docs/Whisper.rst
@@ -39,8 +39,8 @@ That is, a pair of archives with retentions of 1 month and 1 year will not provi
 as may be guessed. Instead, it will provide 1 year of storage - the length of its longest archive.
 
 
-Rollup Aggregation
-------------------
+Roll-up Aggregation
+-------------------
 Whisper databases with more than a single archive need a strategy to collapse multiple data points for
 when the data rolls up a lower precision archive. By default, an average function is used.
 Available aggregation methods are:
@@ -56,7 +56,7 @@ Multi-Archive Storage and Retrieval Behavior
 --------------------------------------------
 When Whisper writes to a database with multiple archives, the incoming data point is written to all
 archives at once. The data point will be written to the highest resolution archive as-is, and will be
-aggregated by the configured aggregation method (see `Rollup Aggregation`_) and placed into each
+aggregated by the configured aggregation method (see `Roll-up Aggregation`_) and placed into each
 of the higher-retention archives. If you are in need for aggregation of the highest resolution points,
 please consider using :doc:`carbon-aggregator </carbon-daemons>` for that purpose.
 

diff --git a/docs/admin-carbon.rst b/docs/admin-carbon.rst
diff --git a/docs/admin-webapp.rst b/docs/admin-webapp.rst
diff --git a/docs/carbon-daemons.rst b/docs/carbon-daemons.rst
@@ -13,6 +13,32 @@ the data once they receive it. This document gives a brief overview of what each
 does and how you can use them to build a more sophisticated storage backend.
 
 
+Starting and stopping Carbon
+----------------------------
+
+To start the Carbon daemons run:
+
+.. code-block:: bash
+
+   carbon-cache.py start
+   carbon-relay.py start
+   carbon-aggregator.py start
+   carbon-aggregator-cache.py start
+
+
+.. note:: If you are using Virtualenv you must specify the full path
+   ``/opt/graphite/bin/carbon-cache.py start``
+
+To stop the daemons replace ``start`` with ``stop``.
+
+
+Logs
+----
+
+If installed in the :ref:`default location <default-installation-layout>` then the
+Carbon daemons write their logs to ``/opt/graphite/storage/log/``.
+
+
 carbon-cache.py
 ---------------
 

diff --git a/docs/composer.rst b/docs/composer.rst
diff --git a/docs/config-database-setup.rst → docs/config-webapp-setup.rst b/docs/config-database-setup.rst → docs/config-webapp-setup.rst
@@ -1,14 +1,15 @@
-Webapp Database Setup
-=====================
+Webapp Setup
+============
+
+Before Graphite-web can be started the database needs to be initialized and the static files need to be collected.
+
+Database Setup
+--------------
+
 You must tell Django to create the database tables used by the graphite webapp. This is very straight forward, especially if you are using the default SQLite setup.
 
 The following configures the Django database settings. Graphite uses the database for storing user profiles, dashboards, and for the Events functionality. Graphite uses an SQLite database file located at ``STORAGE_DIR/graphite.db`` by default. If running multiple Graphite-web instances, a database such as PostgreSQL or MySQL is required so that all instances may share the same data source.
 
-.. note ::
-  As of Django 1.2, the database configuration is specified by the DATABASES
-  dictionary instead of the old ``DATABASE_*`` format. Users must use the new
-  specification to have a working database.
-
 See the
 `Django documentation <https://docs.djangoproject.com/en/dev/ref/settings/#databases>`_
 for full documentation of the DATABASES setting.
@@ -20,21 +21,27 @@ To set up a new database and create the initial schema, run:
 
 .. code-block:: none
 
-  PYTHONPATH=$GRAPHITE_ROOT/webapp django-admin.py migrate --settings=graphite.settings
+  django-admin.py migrate --settings=graphite.settings
+
+.. note::
+
+   If you get a ``Could not import settings 'graphite.settings'`` error message add your ``PYTHONPATH`` in front of the command:
+   ``PYTHONPATH=$GRAPHITE_ROOT/webapp django-admin.py migrate --settings=graphite.settings``
 
 .. note ::
   Graphite-Web 1.0 and earlier had some models without migrations, and with Django 1.9 or later, the ``--run-syncdb`` option was needed for migrate to create tables for these models. (Django 1.8 and earlier did not have this option, but always exhibited this behavior.) In Graphite-Web 1.1 and later all models have migrations, so ``--run-syncdb`` is no longer needed. If upgrading a database created by Graphite-Web 1.0 or earlier, you need to use the ``--fake-initial`` option for migrate: it considers an initial migration to already be applied if the tables it creates already exist.
 
+
 If you are experiencing problems, uncomment the following line in /opt/graphite/webapp/graphite/local_settings.py:
 
 .. code-block:: none
-  
+
   # DEBUG = True
 
 and review your webapp logs. If you're using the default graphite-example-vhost.conf, your logs will be found in /opt/graphite/storage/log/webapp/.
 
 If you're using the default SQLite database, your webserver will need permissions to read and write to the database file. So, for example, if your webapp is running in Apache as the 'nobody' user, you will need to fix the permissions like this:
 
 .. code-block:: none
-  
+
   sudo chown nobody:nobody /opt/graphite/storage/graphite.db
diff --git a/docs/config-webapp.rst b/docs/config-webapp.rst
@@ -3,13 +3,13 @@ Configuring The Webapp
 
 There are multiple ways to expose the Graphite webapp. The following stack configurations exist:
 
-* nginx + gunicorn
+* nginx + Gunicorn
 * Apache + mod_wsgi
 * nginx + uWSGI
 
 Depending on the configuration you choose, the webapp configuration is slightly different.
 
-nginx + gunicorn
+nginx + Gunicorn
 ----------------
 
 In this setup, nginx will proxy requests for Gunicorn, which will itself listen locally on port 8080 and serve the webapp (Django application).
@@ -31,6 +31,53 @@ On Debian-based systems, run:
 
    sudo apt install gunicorn
 
+Next, you will have to create a configuration to start Graphite.
+If your system is using Systemd create following unit files:
+
+``/etc/systemd/system/graphite-web.socket``:
+
+.. code-block:: none
+
+    [Unit]
+    Description=Graphite-web socket
+
+    [Socket]
+    ListenStream=/run/graphite-web.sock
+    ListenStream=127.0.0.1:8080
+
+    [Install]
+    WantedBy=sockets.target
+
+``/etc/systemd/system/graphite-web.service``:
+
+.. code-block:: none
+
+    [Unit]
+    Description=Graphite-web service
+    Requires=graphite-web.socket
+
+    [Service]
+    ExecStart=/usr/bin/gunicorn -w2 graphite.wsgi -b 127.0.0.1:8080
+    # If you are using virtualenv specify the path to gunicorn in virtualenv.
+    # ExecStart=/opt/graphite/bin/gunicorn -w2 graphite.wsgi -b 127.0.0.1:8080
+    Restart=on-failure
+    #User=graphite
+    #Group=graphite
+    ExecReload=/bin/kill -s HUP $MAINPID
+    ExecStop=/bin/kill -s TERM $MAINPID
+    PrivateTmp=true
+
+    [Install]
+    WantedBy=multi-user.target
+
+
+Reload the systemd configuration, the service is socket activated so no need to start it manually.
+
+.. code-block:: none
+
+   systemctl daemon-reload
+
+
 Install nginx
 ^^^^^^^^^^^^^
 
@@ -40,9 +87,6 @@ On Debian-based systems, run:
 
    sudo apt install nginx
 
-Configure nginx
-^^^^^^^^^^^^^^^
-
 We will use dedicated log files for nginx when serving Graphite:
 
 .. code-block:: none
@@ -74,12 +118,6 @@ Write the following configuration in ``/etc/nginx/sites-available/graphite``:
             return 204;
         }
 
-        # serve static content from the "content" directory
-        location /static {
-            alias /opt/graphite/webapp/content;
-            expires max;
-        }
-
         location / {
             try_files $uri @graphite;
         }
@@ -131,7 +169,6 @@ Then create the graphite.wsgi. (You can find example of graphite.wsgi file on th
     # /opt/graphite/conf/graphite.wsgi
 
     import sys
-    sys.path.append('/opt/graphite/webapp')
     from graphite.wsgi import application
 
 Finally, configure the apache vhost. (You can find example of Graphite vhost configuration in the `contrib`_ directory of source ditribution):
@@ -144,6 +181,9 @@ Finally, configure the apache vhost. (You can find example of Graphite vhost con
 
     LoadModule wsgi_module modules/mod_wsgi.so
 
+    # Set WSGIPythonHome when using virtualenv
+    # WSGIPythonHome /opt/graphite
+
     WSGISocketPrefix /var/run/wsgi
 
     Listen 80
@@ -161,18 +201,6 @@ Finally, configure the apache vhost. (You can find example of Graphite vhost con
 
         WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi
 
-        Alias /static/ /opt/graphite/static/
-
-        <Directory /opt/graphite/static/>
-                <IfVersion < 2.4>
-                        Order deny,allow
-                        Allow from all
-                </IfVersion>
-                <IfVersion >= 2.4>
-                        Require all granted
-                </IfVersion>
-        </Directory>
-
         <Directory /opt/graphite/conf/>
                 <IfVersion < 2.4>
                         Order deny,allow
@@ -184,6 +212,9 @@ Finally, configure the apache vhost. (You can find example of Graphite vhost con
         </Directory>
     </VirtualHost>
 
+
+.. note:: You have to configure ``WSGIPythonHome`` if you are using Virtualenv
+
 Adapt the mod_wsgi configuration to your requirements.
 
 See the `mod_wsgi QuickConfigurationGuide`_ for an overview of configurations and `mod_wsgi ConfigurationDirectives`_ to see all configuration directives
@@ -234,7 +265,6 @@ Then create the file ``wsgi.py``:
     # /opt/graphite/conf/wsgi.py
 
     import sys
-    sys.path.append('/opt/graphite/webapp')
     from graphite.wsgi import application
 
 Enable ``graphite.ini`` and restart uWSGI:
@@ -271,8 +301,8 @@ Enable the vhost and restart nginx:
     $ service nginx restart
 
 
-Acnowlegments
-------------_
+Acknowledgments
+---------------
 
 Portions of that manual are based on `Graphite-API deployment manual`_.
 

diff --git a/docs/development.rst b/docs/development.rst
@@ -5,8 +5,8 @@ Graphite-web accepts contributions on `GitHub
 <https://github.com/graphite-project/graphite-web>`_, in the form of issues or
 pull requests. If you're comfortable with Python, here is how to get started.
 
-First, keep in mind that Graphite-web supports Python versions **2.6 to 2.7**
-and Django versions **1.4 and above**.
+First, keep in mind that Graphite-web supports Python versions **2.7 and above**
+and Django versions **1.8 and above**.
 
 Setting up a development environment
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

diff --git a/docs/feeding-carbon.rst b/docs/feeding-carbon.rst
@@ -101,7 +101,7 @@ When using a tagged naming scheme it is much easier to add or alter individual t
 Step 2 - Configure your Data Retention
 --------------------------------------
 
-Graphite is built on fixed-size databases (see :doc:`Whisper. </whisper>`) so we have to configure in advance how much data we intend to store and at what level of precision. For instance you could store your data with 1-minute precision (meaning you will have one data point for each minute) for say 2 hours. Additionally you could store your data with 10-minute precision for 2 weeks, etc. The idea is that the storage cost is determined by the number of data points you want to store, the less fine your precision, the more time you can cover with fewer points.
+Graphite is built on fixed-size databases (see :doc:`Whisper. </Whisper>`) so we have to configure in advance how much data we intend to store and at what level of precision. For instance you could store your data with 1-minute precision (meaning you will have one data point for each minute) for say 2 hours. Additionally you could store your data with 10-minute precision for 2 weeks, etc. The idea is that the storage cost is determined by the number of data points you want to store, the less fine your precision, the more time you can cover with fewer points.
 To determine the best retention configuration, you must answer all of the following questions.
 
 1. How often can you produce your data?