Docs: Add upgrade instructions

Fix some spelling

docs/ceres.rst → docs/Ceres.rst

@@ -6,7 +6,7 @@ for Graphite. In contrast with Whisper, Ceres is not a fixed-size database and i
 better support sparse data of arbitrary fixed-size resolutions. This allows Graphite to distribute
 individual time-series across multiple servers or mounts.
 
-Ceres is not actively developped at the moment. For alternatives to whisper look at :doc:`alternative storage backends </tools>`.
+Ceres is not actively developed at the moment. For alternatives to whisper look at :doc:`alternative storage backends </tools>`.
 
 Storage Overview
 ----------------
@@ -61,12 +61,12 @@ large but allowed gap occurs it has to get filled in, which means instead of a s
 new file we could end up doing an ``(8 * MAX_SLICE_GAP)``-byte write to the latest slice.
 
 
-Rollup Aggregation
-------------------
+Roll-up Aggregation
+-------------------
 Expected features such as roll-up aggregation and data expiration are not provided by Ceres itself, but
 instead are implemented as maintenance plugins.
 
-Such a rollup plugin exists in Ceres that aggregates data points in a way that is similar behavior of
+Such a roll-up plugin exists in Ceres that aggregates data points in a way that is similar behavior of
 Whisper archives. Where multiple data points are collapsed and written to a lower precision slice, and
 data points outside of the set slice retentions are trimmed. By default, an average function is used,
 however alternative methods can be chosen by changing the metadata.

docs/faq.rst → docs/FAQ.rst

@@ -52,7 +52,7 @@ The `Apache 2.0 License`_.
 
 Does Graphite use RRDtool?
 --------------------------
-No, not since Graphite was first written in 2006 at least. Graphite has its own specialized database library called :doc:`whisper </whisper>`, which is very similar in design to RRD, but has a subtle yet fundamentally important difference that Graphite requires. There are two reasons whisper was created. The first reason is that RRD is designed under the assumption that data will always be inserted into the database on a regular basis, and this assumption causes RRD behave undesirably when given irregularly occurring data. Graphite was built to facilitate visualization of various application metrics that do not always occur regularly, like when an uncommon request is handled and the latency is measured and sent to Graphite. Using RRD, the data gets put into a temporary area inside the database where it is not accessible until the current time interval has passed *and* another value is inserted into the database for the following interval. If that does not happen within an allotted period of time, the original data point will get overwritten and is lost. Now for some metrics, the lack of a value can be correctly interpreted as a value of zero, however this is not the case for metrics like latency because a zero indicates that work was done in zero time, which is different than saying no work was done. Assuming a zero value for latency also screws up analysis like calculating the average latency, etc.
+No, not since Graphite was first written in 2006 at least. Graphite has its own specialized database library called :doc:`whisper </Whisper>`, which is very similar in design to RRD, but has a subtle yet fundamentally important difference that Graphite requires. There are two reasons whisper was created. The first reason is that RRD is designed under the assumption that data will always be inserted into the database on a regular basis, and this assumption causes RRD behave undesirably when given irregularly occurring data. Graphite was built to facilitate visualization of various application metrics that do not always occur regularly, like when an uncommon request is handled and the latency is measured and sent to Graphite. Using RRD, the data gets put into a temporary area inside the database where it is not accessible until the current time interval has passed *and* another value is inserted into the database for the following interval. If that does not happen within an allotted period of time, the original data point will get overwritten and is lost. Now for some metrics, the lack of a value can be correctly interpreted as a value of zero, however this is not the case for metrics like latency because a zero indicates that work was done in zero time, which is different than saying no work was done. Assuming a zero value for latency also screws up analysis like calculating the average latency, etc.
 
 The second reason whisper was written is performance. RRDtool is very fast; in fact it is much faster than whisper. But the problem with RRD (at the time whisper was written) was that RRD only allowed you to insert a single value into a database at a time, while whisper was written to allow the insertion of multiple data points at once, compacting them into a single write operation. This improves performance drastically under high load because Graphite operates on many many files, and with such small operations being done (write a few bytes here, a few over there, etc) the bottleneck is caused by the *number of I/O operations*. Consider the scenario where Graphite is receiving 100,000 distinct metric values each minute; in order to sustain that load Graphite must be able to write that many data points to disk each minute. But assume that your underlying storage can only handle 20,000 I/O operations per minute. With RRD (at the time whisper was written), there was no chance of keeping up. But with whisper, we can keep caching the incoming data until we accumulate say 10 minutes worth of data for a given metric, then instead of doing 10 I/O operations to write those 10 data points, whisper can do it in one operation. The reason I have kept mentioning "at the time whisper was written" is that RRD now supports this behavior. However Graphite will continue to use whisper as long as the first issue still exists.
 

docs/whisper.rst → docs/Whisper.rst

@@ -39,8 +39,8 @@ That is, a pair of archives with retentions of 1 month and 1 year will not provi
 as may be guessed. Instead, it will provide 1 year of storage - the length of its longest archive.
 
 
-Rollup Aggregation
-------------------
+Roll-up Aggregation
+-------------------
 Whisper databases with more than a single archive need a strategy to collapse multiple data points for
 when the data rolls up a lower precision archive. By default, an average function is used.
 Available aggregation methods are:
@@ -56,7 +56,7 @@ Multi-Archive Storage and Retrieval Behavior
 --------------------------------------------
 When Whisper writes to a database with multiple archives, the incoming data point is written to all
 archives at once. The data point will be written to the highest resolution archive as-is, and will be
-aggregated by the configured aggregation method (see `Rollup Aggregation`_) and placed into each
+aggregated by the configured aggregation method (see `Roll-up Aggregation`_) and placed into each
 of the higher-retention archives. If you are in need for aggregation of the highest resolution points,
 please consider using :doc:`carbon-aggregator </carbon-daemons>` for that purpose.
 

docs/conf.py

@@ -80,16 +80,16 @@ def setup(app):
 
 # General information about the project.
 project = u'Graphite'
-copyright = u'2008-2012, Chris Davis; 2011-2017 The Graphite Project'
+copyright = u'2008-2012, Chris Davis; 2011-2019 The Graphite Project'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
-version = '1.1.5'
+version = '1.2.0'
 # The full version, including alpha/beta/rc tags.
-release = '1.1.5'
+release = '1.2.0-pre'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/config-webapp.rst

@@ -3,13 +3,13 @@ Configuring The Webapp
 
 There are multiple ways to expose the Graphite webapp. The following stack configurations exist:
 
-* nginx + gunicorn
+* nginx + Gunicorn
 * Apache + mod_wsgi
 * nginx + uWSGI
 
 Depending on the configuration you choose, the webapp configuration is slightly different.
 
-nginx + gunicorn
+nginx + Gunicorn
 ----------------
 
 In this setup, nginx will proxy requests for Gunicorn, which will itself listen locally on port 8080 and serve the webapp (Django application).
@@ -169,7 +169,6 @@ Then create the graphite.wsgi. (You can find example of graphite.wsgi file on th
     # /opt/graphite/conf/graphite.wsgi
 
     import sys
-    sys.path.append('/opt/graphite/webapp')
     from graphite.wsgi import application
 
 Finally, configure the apache vhost. (You can find example of Graphite vhost configuration in the `contrib`_ directory of source ditribution):
@@ -266,7 +265,6 @@ Then create the file ``wsgi.py``:
     # /opt/graphite/conf/wsgi.py
 
     import sys
-    sys.path.append('/opt/graphite/webapp')
     from graphite.wsgi import application
 
 Enable ``graphite.ini`` and restart uWSGI:
@@ -303,8 +301,8 @@ Enable the vhost and restart nginx:
     $ service nginx restart
 
 
-Acnowlegments
--------------
+Acknowledgments
+---------------
 
 Portions of that manual are based on `Graphite-API deployment manual`_.
 

docs/feeding-carbon.rst

@@ -101,7 +101,7 @@ When using a tagged naming scheme it is much easier to add or alter individual t
 Step 2 - Configure your Data Retention
 --------------------------------------
 
-Graphite is built on fixed-size databases (see :doc:`Whisper. </whisper>`) so we have to configure in advance how much data we intend to store and at what level of precision. For instance you could store your data with 1-minute precision (meaning you will have one data point for each minute) for say 2 hours. Additionally you could store your data with 10-minute precision for 2 weeks, etc. The idea is that the storage cost is determined by the number of data points you want to store, the less fine your precision, the more time you can cover with fewer points.
+Graphite is built on fixed-size databases (see :doc:`Whisper. </Whisper>`) so we have to configure in advance how much data we intend to store and at what level of precision. For instance you could store your data with 1-minute precision (meaning you will have one data point for each minute) for say 2 hours. Additionally you could store your data with 10-minute precision for 2 weeks, etc. The idea is that the storage cost is determined by the number of data points you want to store, the less fine your precision, the more time you can cover with fewer points.
 To determine the best retention configuration, you must answer all of the following questions.
 
 1. How often can you produce your data?

docs/index.rst

@@ -6,19 +6,18 @@ Graphite Documentation
    :maxdepth: 1
 
    overview
-   faq
+   FAQ
    install
    carbon-daemons
    config-carbon
    feeding-carbon
    config-local-settings
    config-webapp
-   composer
    render_api
    functions
    dashboard
-   whisper
-   ceres
+   Whisper
+   Ceres
    storage-backends
    events
    tags

docs/install.rst

@@ -3,7 +3,7 @@ Installing Graphite
 
 There are several ways to install Graphite. Docker is the easiest way and is recommended for new
 users. If you need to install Graphite directly on your system (not using Docker) then using
-`Virtualenv`_ with `pip`_ would be recommended. With virtualenv all dependencies are installed
+`Virtualenv`_ with `pip`_ would be recommended. With Virtualenv all dependencies are installed
 isolated so they will not interfere with your system Python. Installation from source should
 only be used in specific cases (e.g. installing a development release).
 
@@ -48,7 +48,7 @@ Basic Graphite requirements:
 * `fontconfig`_ and at least one font package (a system package usually)
 * A WSGI server and web server. Popular choices are:
 
-  - `gunicorn`_ with `nginx`_
+  - `Gunicorn`_ with `nginx`_
 
   - `Apache`_ with `mod_wsgi`_
 
@@ -91,20 +91,31 @@ For RHEL based distributions enable the `EPEL`_ repository.
    yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    yum install dejavu-sans-fonts dejavu-serif-fonts python-pip python-virtualenv cairo
 
-Debian-based distributions
-^^^^^^^^^^^^^^^^^^^^^^^^^^
+Debian
+^^^^^^
 .. code-block:: bash
 
    apt-get install python-dev libcairo2-dev libffi-dev build-essential
 
+.. note:: This might be outdated
+
+Ubuntu 18.04
+^^^^^^^^^^^^
+.. code-block:: bash
+
+   apt-get install python-pip libcairo2
+
 .. _default-installation-layout :
 
 Default Installation Layout
 ---------------------------
 
 Graphite traditionally installed all components in ``/opt/graphite``, since release 1.2.0
 Graphite's installation location depends on your `Python prefix`_. If you are using `Virtualenv`_
-the ``prefix`` is set to the root of your virtualenv. You can find your ``prefix`` by running:
+the ``prefix`` is set to the root of your virtualenv. For more information on custom prefixes
+see `Custom Installation Locations`_. See the release notes for :doc:`upgrading </releases/1_2_0>`
+
+You can find your ``prefix`` by running:
 
 .. code-block:: bash
 
@@ -170,9 +181,14 @@ To install Graphite execute:
 Installing From Source
 ----------------------
 
-The latest source tarballs for Graphite-web, Carbon, and Whisper may be fetched from the
-Graphite project `download page`_ or the latest development branches may be cloned from
-the `Github project page`_:
+The latest source releases for Graphite-web, Carbon, and Whisper may be fetched from the
+GitHub release pages:
+
+* `Whisper releases <https://github.com/graphite-project/whisper/releases>`_
+* `Carbon releases <https://github.com/graphite-project/carbon/releases>`_
+* `Graphite-web releases <https://github.com/graphite-project/graphite-web/releases>`_
+
+The latest development branches may be cloned from the `GitHub project page`_:
 
 .. code-block:: bash
 
@@ -229,10 +245,6 @@ Post-Install Tasks
     With data getting into carbon, you probably want to look at graphs of it.
     So now we turn our attention to the webapp.
 
-:doc:`Using the Composer </composer>`
-    Now that the webapp is running, you probably want to learn how to use it.
-
-
 Windows Users
 -------------
 Unfortunately, native Graphite on Windows is unsupported, but you can run Graphite on Windows in `Docker`_.
@@ -263,7 +275,7 @@ Unfortunately, native Graphite on Windows is unsupported, but you can run Graphi
 .. _Docker: https://www.docker.com/
 .. _docker repo: https://github.com/graphite-project/docker-graphite-statsd
 .. _Virtualenv: http://virtualenv.org/
-.. _Github project page: http://github.com/graphite-project
-.. _download page: https://launchpad.net/graphite/+download
+.. _GitHub project page: http://github.com/graphite-project
 .. _pyhash: https://pypi.org/project/pyhash/
 .. _Python prefix: https://docs.python.org/3/library/sys.html#sys.prefix
+.. _Custom Installation Locations: https://setuptools.readthedocs.io/en/latest/easy_install.html#custom-installation-locations

docs/releases.rst

@@ -5,6 +5,7 @@ Release Notes
    :maxdepth: 1
    :glob:
 
+   releases/1_2_0
    releases/1_1_5
    releases/1_1_4
    releases/1_1_3

docs/releases/1_2_0.rst

@@ -0,0 +1,132 @@
+.. _1-2-0:
+
+1.2.0-pre
+===========================
+*02/03/2018*
+
+Graphite 1.2.0-pre is now available for usage. This release changes the installation procedure.
+Graphite does not install in ``/opt/graphite`` by default anymore. See the
+:ref:`upgrading <install-pip>` section for migration instructions.
+
+Main features
+-------------
+* Removed hardcoded ``/opt/graphite installation`` prefix
+* Static files are served by whitenoise by default, no need webserver configuration and ``collectstatic`` anymore
+* TODO:
+
+Thanks a lot for all Graphite contributors and users! You are the best!
+
+Source bundles are available from GitHub:
+
+* https://github.com/graphite-project/graphite-web/archive/1.1.5.tar.gz
+* https://github.com/graphite-project/carbon/archive/1.1.5.tar.gz
+* https://github.com/graphite-project/whisper/archive/1.1.5.tar.gz
+* https://github.com/graphite-project/carbonate/archive/1.1.5.tar.gz
+
+Graphite can also be installed from `PyPI <http://pypi.python.org/>`_ via
+`pip <http://www.pip-installer.org/en/latest/index.html>`_. PyPI bundles are here:
+
+* http://pypi.python.org/pypi/graphite-web/
+* http://pypi.python.org/pypi/carbon/
+* http://pypi.python.org/pypi/whisper/
+* http://pypi.python.org/pypi/carbonate/
+
+You can also use docker image from https://hub.docker.com/r/graphiteapp/graphite-statsd/
+
+.. _upgrading :
+
+Upgrading
+---------
+Create a virtualenv (skip if you already are using one), activate it and install the packages.
+
+.. code-block:: bash
+
+   virtualenv /opt/graphite
+   source /opt/graphite/bin/activate
+
+   pip install --upgrade whisper
+   pip install --upgrade carbon
+   pip install --upgrade graphite-web
+
+Then copy your local_settings.py to the new directory. If you don't have a local_settings.py
+then you can copy the local_settings.py.example or skip this step.
+
+.. code-block:: bash
+
+   cp /opt/graphite/webapp/graphite/local_settings.py /opt/graphite/lib/python2.7/site-packages/graphite/local_settings.py
+
+
+Backup and remove old files:
+
+.. code-block:: bash
+
+   cd /opt/graphite
+   mkdir backup
+   mv lib/carbon* lib/twisted/ webapp/ backup/
+
+Depending on your webserver you will have to configure your wsgi.py and or webserver config.
+Configuration for static files can be removed as they is now served from the application
+directly.
+
+Gunicorn
+^^^^^^^^
+Install gunicorn in the virtualenv ``pip install gunicorn`` with the virtualenv activated.
+Gunicorn then automatically picks up the virtualenv when run from ``/opt/graphite/bin/gunicorn``
+(no need to activate the virtualenv.
+
+Apache+mod_wsgi
+^^^^^^^^^^^^^^^
+Add ``WSGIPythonHome /opt/graphite`` to your apache httpd config. You can remove
+``sys.path.append('/opt/graphite/webapp')`` from your wsgi.conf.
+
+uWSGI
+^^^^^
+Add ``virtualenv = /opt/graphite`` to your ``graphite.ini`` config file. You can remove
+``sys.path.append('/opt/graphite/webapp')`` from your wsgi.conf.
+
+Incompatible changes
+--------------------
+Default installation prefix is removed, be careful with upgrading, check if the correct libraries
+and configurations are being loaded.
+
+Security Notes
+--------------
+None
+
+New features
+------------
+
+Graphite-Web
+^^^^^^^^^^^^
+* TODO:
+
+Carbon
+^^^^^^
+* TODO:
+
+Whisper
+^^^^^^^
+* TODO:
+
+Carbonate
+^^^^^^^^^
+* None
+
+Bug Fixes
+---------
+
+Graphite-Web
+^^^^^^^^^^^^
+* TODO
+
+Carbon
+^^^^^^
+* TODO
+
+Whisper
+^^^^^^^
+* TODO
+
+Carbonate
+^^^^^^^^^
+kA None

webapp/graphite/local_settings.py.example

@@ -99,7 +99,7 @@ DEFAULT_XFILES_FACTOR = 0
 # Static Files #
 #####################################
 #
-# Graphite servers static files (.html, .css, .js) directly from the
+# Graphite serves static files (.html, .css, .js) directly from the
 # application by using whitenoise. If you want to serve the files from a
 # webserver instead:
 #  - Uncomment following lines