If you want to become active as developer, we provide all important information here to make the start as easy as possible. The code you produce should be seamlessly integrable into PyDynamic by aligning your work with the established workflows. This guide should work on all platforms and provide everything needed to start developing for PyDynamic. Please open an issue or ideally contribute to this guide as a start, if problems or questions arise.
The PyDynamic development process is based on the following guiding principles:
- support all major Python versions supported upstream .
- actively maintain, ensuring security vulnerabilities or other issues are resolved in a timely manner
- employ state-of-the-art development practices and tools, specifically
- follow semantic versioning
- use conventional commit messages
- consider the PEP8 style guide, wherever feasible
For collaboration, we recommend forking the repository as described here. Simply apply the changes to your fork and open a Pull Request on GitHub as described here. For small changes it will be sufficient to just apply your changes on GitHub and send the PR right away. For more comprehensive work, you should clone your fork and read on carefully.
This guide assumes you already have a valid runtime environment for PyDynamic as described in the installation section of the docs. To start developing, install the known to work dependencies' versions for your specific Python version.
Afterwards you should always install PyDynamic itself in "development mode" into your virtual environment to be able to properly test against the shipped version:
(PyDynamic_venv) $ python -m pip install -e .
For the testing refer to the according testing section in this guide.
We use black to implement our coding style, Sphinx for automated generation of our documentation on ReadTheDocs. We use pytest managed by tox as testing framework backed by hypothesis and coverage. For automated releases we use python-semantic-release in our pipeline on CircleCI . All requirements for contributions are derived from this. If you followed the steps for the initial development setup you have everything at your hands.
As long as the readability of mathematical formulations is not impaired, our code should follow PEP8. For automating this uniform formatting task we use the Python package black. It is easy to handle and integrable into most common IDEs, such that it is automatically applied.
PyDynamic commit messages follow some conventions to be easily human and machine-readable.
Conventional commit messages are required for the following:
- Releasing automatically according to semantic versioning
- Generating a changelog automatically
Parts of the commit messages and links appear in the changelogs of subsequent releases as a result. We use the following types:
- feat: for commits that introduce new features (this correlates with MINOR in semantic versioning)
- docs: for commits that contribute significantly to documentation
- fix: commits in which bugs are fixed (this correlates with PATCH in semantic versioning)
- build: Commits that affect packaging
- ci: Commits that affect the CI pipeline
- test: Commits that apply significant changes to tests
- chore: Commits that affect other non-PyDynamic components (e.g., ReadTheDocs, Git, ... )
- revert: commits, which undo previous commits using
git revert
- refactor: commits that merely reformulate, rename or similar
- style: commits, which essentially make changes to line breaks and whitespace
- wip: Commits which are not recognizable as one of the above-mentioned types until later, usually during a PR merge. The merge commit is then marked as the corresponding type.
Of the types mentioned above, the following appear in separate sections of the changelog:
- Feature: feat
- Documentation: docs
- Fix: fix
- Test: test
Based on conventional commit messages, the so-called should complete the following sentence:
If this commit is applied, it will...
The first line of a commit message should not exceed 100 characters.
If a commit changes parts of PyDynamic's public interface so that previously written code may no longer be executable, it is considered a BREAKING CHANGE (correlating with MAJOR in semantic versioning). This has to be marked by putting BREAKING CHANGE in the footer of the message as described in the conventional commit messages . The introduced change and especially how to convert breaking code to further work should follow without a blank line and will be included in the changelog in full length.
For examples please check out the Git Log.
We strive to increase our code coverage with every change introduced. This requires that every new feature and every change to existing features is accompanied by appropriate pytest testing. We test the basic components for correctness and, if necessary, the integration into the big picture. It is usually sufficient to create appropriately named methods in one of the existing modules in the subfolder test. If necessary, add a new module that is appropriately named.
To execute the whole of the PyDynamic test suite simply run:
(PyDynamic_venv) $ python -m pytest
Further details, e.g. to execute only one specific test, can be found in the official pytest docs.
In case you add a new feature you generally follow the pattern:
- read through and follow this contribution advices and tips, especially regarding the advised tool set and coding style
- open an according issue to submit a feature request and get in touch with other PyDynamic developers and users
- fork the repository or update the main branch of your fork and create an arbitrary named feature branch from main
- decide which package and module your feature should be integrated into
- if there is no suitable package or module, create a new one and a corresponding module in the test subdirectory with the same name prefixed by test_
- after adding your functionality add it to all higher-level
__all__
variables in the module itself and in the higher-level__init__.py
s - if new dependencies are introduced, add them to setup.py or dev-requirements.in
- during development write tests in alignment with existing test modules, for example test_interpolate or test_propagate_filter
- write docstrings in the NumPy docstring format for all public functions you add
- as early as possible create a draft pull request onto the upstream's main branch
- once you think your changes are ready to merge, request a review from the PTB-M4D/pydynamic-devs (you will find them in the according drop-down) and mark your PR as ready for review
- at the latest now you will have the opportunity to review the documentation automatically generated from the docstrings on ReadTheDocs after your reviewers will set up everything
- resolve the conversations and have your pull request merged
The documentation of PyDynamic consists of three parts. Every adaptation of an existing feature and every new feature requires adjustments on all three levels:
- user documentation on ReadTheDocs
- examples in the form of Jupyter notebooks for extensive features and Python scripts for features which can be comprehensively described with few lines of commented code
- developer documentation in the form of comments in the code
To locally generate a preview of what ReadTheDocs will generate from your docstrings, you can simply execute after activating your virtual environment:
(PyDynamic_venv) $ sphinx-build docs/ docs/_build
Sphinx v3.1.1 in Verwendung
making output directory...
[...]
build abgeschlossen.
The HTML pages are in docs/_build.
After that, you can open the file ./docs/_build/index.html relative to the project's root with your favourite browser. Simply re-execute the above command after each change to the docstrings to update your local version of the documentation.
We want to provide extensive sample material for all PyDynamic features in order to simplify the use or even make it possible in the first place. We collect the examples in a separate repository PyDynamic_tutorials separate from PyDynamic to better distinguish the core functionality from additional material. Please refer to the corresponding README for more information about the setup and create a pull request accompanying the pull request in PyDynamic according to the same procedure we describe here.
Regarding comments in the code we recommend investing 45 minutes for the PyCon DE 2019 Talk of Stefan Schwarzer, a 20+-years Python developer: Commenting code - beyond common wisdom.
As stated in the README and above we use
pip-tools for dependency management. The
requirements' subdirectory contains a requirements.txt and a dev-requirements.txt
for all supported Python versions, with a suffix naming the version, for example
requirements-py36.txt
To keep them up to date semi-automatically we use the bash script
requirements/upgrade_dependencies.sh.
It contains extensive comments on its use. pip-tools' command pip-compile
finds
the right versions from the dependencies listed in setup.py and the
dev-requirements.in.
All contributions are released under PyDynamic's GNU Lesser General Public License v3.0.