From c3a03b99eb698f7a048adb2b4bb902fea7b1d1b9 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Mon, 30 Dec 2024 16:22:08 +0100 Subject: [PATCH 01/18] Update desgin document with known issues section --- doc/design.rst | 191 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 191 insertions(+) diff --git a/doc/design.rst b/doc/design.rst index 05dd48c7a..443cfbed3 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -209,3 +209,194 @@ _________________ - Description * - python-environment - Sets up an appropriate poetry based python environment + + +Known Issues +------------ + +This section documents flaws, sins, and known issues with the current design and/or its current implementation that were either known upfront or surfaced through the course of implementing it. Additionally, it attempts to explain why certain choices were made at the time, so one can better understand whether it may be reasonable to make changes now or in the future. + +Passing files as individual arguments on the CLI +++++++++++++++++++++++++++++++++++++++++++++++++ + +**Description:** + +As of today selection of python files for litting formatting etc. is done by passing all relevant python files as individual argument(s) +to the tools used/invoked by the python toolbox. + +**Downsides:** + +- Most shells have limitations on the number of arguments and their length. +- Noisey output, making it hard to decipher the actual command. +- Not ideal for all use cases. + +**Rationale/History:** + +- The current method of passing files as individual arguments by default offers ease in collection and filtering. It also allows users to simply provide or replace the selection mechanism fairly easily. + +- Every tool used by the toolbox (e.g., `black`, `isort`) used to support passing files by argument. However, not all of them provided the same mechanism for selection or deselection patterns (e.g., "glob"). + +**Ideas/Solutions:** + +- Develop a wrapper that allows for different selection mechanisms + + +Inconsistent Naming ++++++++++++++++++++ + +**Description:** + +The naming is not consistent across the project name (python-toolbox) and the PyPI package name (exasol-toolbox). + +**Downsides:** + +- Misalignment between the PyPI package name and the project name causes confusion when discussing or referring to the project/package. + +**Rationale/History:** + +- Initially, this was a proof of concept (POC) to verify a few ideas, and the naming was not well thought out at the time. +- Later, when publishing the first package for distribution, the project name was unavailable on PyPI, resulting in a different name being used on PyPI. + +**Ideas/Solutions:** + +- `Issue-325 `_ + +Cluttered Configuration ++++++++++++++++++++++++ + +**Description:** + +**Downsides:** + +- Multiple and scattered configuration points make management difficult. +- Tool leakage where configurations overlap or conflict. + +**Rationale/History:** + +- Initial decisions aimed to simplify individual adjustments in the projects until a better understanding of what is needed could be achieved. +- Configuration scattered across various files and tools was a quick decision to expedite development and accommodate various tools. + +while we needed to commonolize code we also needed to be somewhat flexible in the individual projects while also there was only one person +working on the toolbox on the side at the time it also wasn't possible to imidealty act on a need of the individual project(s). +THerfore the it was built with less restrictions to provide various kinds of flexibliity + +**Ideas/Possible Solutions:** + +Over time the flexibility should to be reduced to: + +- Centralize all configurations in the toolbox config file (`noxconfig.py`), considering renaming it to reflect its purpose better. +- Implement layered configurations: + 1. Config file + 2. Plugin/extension points + 3. Custom overloads (properly documented inputs and outputs) + +Note: +Already today there is prefered ways to do things but nothing is enforced yet. + + +Nox Task Runner ++++++++++++++++ + +**Downsides:** + +- Imports over top-level modules are problematic as all are imported. + +**Rationale:** + +- Nox serves as a task runner or means to define tasks. + +**History:** + +- Use of Nox needed for task assignment. However, it presented issues with handling module imports at the top level. + +**Ideas/Possible Solutions:** + +- Investigate other task runners that might address these import issues more efficiently. +- Consider modularization of tasks to handle top-level imports better. + + +Poetry for Project Management ++++++++++++++++++++++++++++++ + +**Downsides:** + +- There's a potential for another tool that addresses the issues better in the future. + +**Rationale:** + +- Poetry was chosen for project management; however, adjustments and reevaluations might be necessary. + +**History:** + +- Initial choice for ease of dependency management and project configuration. + +**Ideas/Possible Solutions:** + +- Continuously evaluate alternative tools that might better serve the project's needs. +- Stay updated on the development and new features of Poetry and its competitors. + + +Black & Isort for Code Formatting ++++++++++++++++++++++++++++++++++ + +**Downsides:** + +- Potential for performance enhancements and consolidation under one tool. + +**Rationale:** + +- Black and Isort are currently used but may be supplanted by a more efficient tool. + +**History:** + +- Initially chosen for their effectiveness and simplicity. + +**Ideas/Possible Solutions:** + +- Transition to Ruff for performance improvements: + - One tool does it all + - More widely adopted and has better performance metrics. + + +Pylint Instead of Ruff ++++++++++++++++++++++++ + +**Downsides:** + +- Pylint slower and less usable in pre-commit hooks. + +**Rationale:** + +- Transitioning to Ruff provides better usability and speed for linting processes. + +**History:** + +- Pylint was used, but Ruff's emergence offers superior performance in many areas. + +**Ideas/Possible Solutions:** + +- Tentatively use Ruff for linting: + - Evaluate impacts on scoring and migration strategy adjustments. + - Continue using Pylint for evaluation/rating while Ruff is integrated for linting. + + +Workflows Dependency Structure +++++++++++++++++++++++++++++++ + +**Downsides:** + +- Lack of clear documentation and structure for workflow dependencies. + +**Rationale:** + +- Proper documentation will streamline workflow management and dependency tracing. + +**History:** + +- Workflow dependencies were initially structured without thorough documentation. + +**Ideas/Possible Solutions:** + +- Work out and document dependencies and structure thoroughly. +- Develop a visual or diagrammatic representation to aid comprehension. +- Regularly review and update the documentation to stay current with project evolution. From 6dba4a8307eede3a05bcca09c7677bf28b75b002 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Thu, 2 Jan 2025 09:06:54 +0100 Subject: [PATCH 02/18] Update project configuration --- doc/design.rst | 42 ++++++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/design.rst b/doc/design.rst index 443cfbed3..a17a4a655 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -211,8 +211,8 @@ _________________ - Sets up an appropriate poetry based python environment -Known Issues ------------- +Known Issues (Sins) +-------------------- This section documents flaws, sins, and known issues with the current design and/or its current implementation that were either known upfront or surfaced through the course of implementing it. Additionally, it attempts to explain why certain choices were made at the time, so one can better understand whether it may be reasonable to make changes now or in the future. @@ -261,37 +261,39 @@ The naming is not consistent across the project name (python-toolbox) and the Py - `Issue-325 `_ -Cluttered Configuration -+++++++++++++++++++++++ +Project Configuration ++++++++++++++++++++++ **Description:** +Currently, the documentation regarding the configuration of projects using the toolbox has various gaps and does not follow a clear configuration hierarchy or structure. **Downsides:** -- Multiple and scattered configuration points make management difficult. -- Tool leakage where configurations overlap or conflict. +- Multiple scattered configuration points make management and understanding difficult. +- Configurations overlap or conflict with unclear priorities. +- Tool leakage (e.g., the ``[isort]`` section in ``pyproject.toml``). + (If everything were done via toolbox config file(s), backing tools could be swapped more easily). **Rationale/History:** -- Initial decisions aimed to simplify individual adjustments in the projects until a better understanding of what is needed could be achieved. -- Configuration scattered across various files and tools was a quick decision to expedite development and accommodate various tools. +- Initial decisions aimed to simplify individual adjustments in the projects until we had a better understanding of what needed to be configured. +- Scattering configuration across various files and tools was a hasty decision to expedite development and accommodate various tools. -while we needed to commonolize code we also needed to be somewhat flexible in the individual projects while also there was only one person -working on the toolbox on the side at the time it also wasn't possible to imidealty act on a need of the individual project(s). -THerfore the it was built with less restrictions to provide various kinds of flexibliity +**Ideas/Solutions:** -**Ideas/Possible Solutions:** +Currently used methods to configure toolbox-based projects: -Over time the flexibility should to be reduced to: +#. Project configuration: ``noxconfig.py`` +#. Tool-specific configuration files or sections in ``pyproject.toml`` +#. Implementing plugin extension points +#. Overwriting nox tasks with custom implementations +#. Replacing with customized workflows of the same name (only applicable for action/workflows) -- Centralize all configurations in the toolbox config file (`noxconfig.py`), considering renaming it to reflect its purpose better. -- Implement layered configurations: - 1. Config file - 2. Plugin/extension points - 3. Custom overloads (properly documented inputs and outputs) +Refinement: -Note: -Already today there is prefered ways to do things but nothing is enforced yet. +- Centralize all toolbox based configurations in a toolbox config file (``noxconfig.py``). +- Rename the toolbox config file from ``noxconfig.py`` to a more appropriate name that reflects its purpose. +- Document configuration hierarchy and usage. Nox Task Runner From 72bb3c1e283fd55480d200e924e010cb0f478ed9 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Thu, 2 Jan 2025 09:56:23 +0100 Subject: [PATCH 03/18] Add kown issue(s) regarding nox task runner --- doc/design.rst | 39 +++++++++++++++++++++++++++++---------- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/doc/design.rst b/doc/design.rst index a17a4a655..d001bea61 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -211,8 +211,8 @@ _________________ - Sets up an appropriate poetry based python environment -Known Issues (Sins) --------------------- +Known Issues +------------ This section documents flaws, sins, and known issues with the current design and/or its current implementation that were either known upfront or surfaced through the course of implementing it. Additionally, it attempts to explain why certain choices were made at the time, so one can better understand whether it may be reasonable to make changes now or in the future. @@ -299,22 +299,41 @@ Refinement: Nox Task Runner +++++++++++++++ +**Description:** +While Nox isn't a perfect fit, it still meets most of our requirements for a task runner. + **Downsides:** -- Imports over top-level modules are problematic as all are imported. +- Imports over top-level modules are problematic as all contained tasks are imported. +- Passing and receiving additional arguments to a task is clunky. +- The default behavior of creating a venv for tasks is undesirable. +- Nox does not support grouping. -**Rationale:** +**Rationale/History:** -- Nox serves as a task runner or means to define tasks. +Why Nox was choosen: -**History:** +- No Additional Language(s) Required: There was no need to introduce extra programming languages or binaries, simplifying the development process. +- Python-based: Being Python-based, Nox can be extended and understood by Python developers. +- Python code: As Nox tasks are defined via Python code, existing scripts can be reused and code can be shared easily. +- Simplicity: Nox is relatively "small" in functionality, making it somewhat simple to use and understand. + +**Ideas/Solutions:** -- Use of Nox needed for task assignment. However, it presented issues with handling module imports at the top level. +Grouping: -**Ideas/Possible Solutions:** +Since Nox doesn't natively support task grouping, we need a strategy to group commands. +Therefore, a naming convention to indicate grouping should be adopted. + + Suggestion: Groups will be separated using a :code:`:` (colon) because :code:`-` (dash) might already be used within task names. + +Imports: + +Consider modularizing tasks to handle top-level imports better. + +Others Issues: -- Investigate other task runners that might address these import issues more efficiently. -- Consider modularization of tasks to handle top-level imports better. +Generally, one may consider addressing the other issues by choosing another task runner or creating a small set of CLI tools and extension points manually provided by the toolbox. Poetry for Project Management From d7ebafde482f49247d95f2602922cd32d41cb0e7 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Thu, 2 Jan 2025 15:34:38 +0100 Subject: [PATCH 04/18] Add code formatting to known issues --- doc/design.rst | 58 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 23 deletions(-) diff --git a/doc/design.rst b/doc/design.rst index d001bea61..5b36bb0b3 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -339,59 +339,72 @@ Generally, one may consider addressing the other issues by choosing another task Poetry for Project Management +++++++++++++++++++++++++++++ +**Description:** +The commonly used + **Downsides:** - There's a potential for another tool that addresses the issues better in the future. +- poetry slightly diverges from stardanrd poetproject.toml e.g. [popetry...] +- no workspaces support -**Rationale:** +**Rationale/History:** - Poetry was chosen for project management; however, adjustments and reevaluations might be necessary. - -**History:** - - Initial choice for ease of dependency management and project configuration. **Ideas/Possible Solutions:** +- uv, workspaces, scripts ... - Continuously evaluate alternative tools that might better serve the project's needs. - Stay updated on the development and new features of Poetry and its competitors. -Black & Isort for Code Formatting -+++++++++++++++++++++++++++++++++ +Code Formatting ++++++++++++++++ + +**Description:** + +Currently we use Black and Isort for code formatting, though running them on a larger code base as pre-commit hooks or such can take quite a bit of time. **Downsides:** -- Potential for performance enhancements and consolidation under one tool. +- Two tools and an aligned configuration of them are required to cleanly and correctly format the codebase. +- Code needs to be processed at least twice as we apply two individual tools. +- The performance of Black and Isort is okay but not great compared to other tools. -**Rationale:** +**Rationale/History:** -- Black and Isort are currently used but may be supplanted by a more efficient tool. +- Black and Isort have been used because they are battle-tested and widely used +- When we opted for Black and Isort, ``ruff`` wasn't "a thing" yet and at best in its early stages. +- Black and Isort already have been known by most python devs when we where selecting the tools -**History:** +**Ideas/Solutions:** -- Initially chosen for their effectiveness and simplicity. +As `Ruff `_ is fairly stable and also tested and used by many Python projects +we should consider transitioning to it. -**Ideas/Possible Solutions:** +Advantages: -- Transition to Ruff for performance improvements: - - One tool does it all - - More widely adopted and has better performance metrics. +- Well-tested +- Widely used +- Excellent performance +- Single tool for imports and formatting the codebase +- Simplifies adopting ruff for linting Pylint Instead of Ruff +++++++++++++++++++++++ +**Description:** + **Downsides:** - Pylint slower and less usable in pre-commit hooks. -**Rationale:** +**Rationale/History:** - Transitioning to Ruff provides better usability and speed for linting processes. - -**History:** - - Pylint was used, but Ruff's emergence offers superior performance in many areas. **Ideas/Possible Solutions:** @@ -404,16 +417,15 @@ Pylint Instead of Ruff Workflows Dependency Structure ++++++++++++++++++++++++++++++ +**Description:** + **Downsides:** - Lack of clear documentation and structure for workflow dependencies. -**Rationale:** +**Rationale/History:** - Proper documentation will streamline workflow management and dependency tracing. - -**History:** - - Workflow dependencies were initially structured without thorough documentation. **Ideas/Possible Solutions:** From 96a117df8d9080c2e631223c031cf0886537d8a4 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Fri, 3 Jan 2025 09:16:51 +0100 Subject: [PATCH 05/18] Add details about poetry and uv to known issues --- doc/design.rst | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/doc/design.rst b/doc/design.rst index 5b36bb0b3..c918cd044 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -339,25 +339,10 @@ Generally, one may consider addressing the other issues by choosing another task Poetry for Project Management +++++++++++++++++++++++++++++ -**Description:** -The commonly used - -**Downsides:** - -- There's a potential for another tool that addresses the issues better in the future. -- poetry slightly diverges from stardanrd poetproject.toml e.g. [popetry...] -- no workspaces support - -**Rationale/History:** - -- Poetry was chosen for project management; however, adjustments and reevaluations might be necessary. -- Initial choice for ease of dependency management and project configuration. - -**Ideas/Possible Solutions:** - -- uv, workspaces, scripts ... -- Continuously evaluate alternative tools that might better serve the project's needs. -- Stay updated on the development and new features of Poetry and its competitors. +While poetry was and is a good choice for exasol project, dependency, build tool etc. "most recently" +`uv `_ has surffeced and made big advanced. Looking at uv it addresses additional itches with +our projects and therefore in the long run it may be a good idea to migrate our project setups to it. +Use poetry for project, build and depenency management. Code Formatting From 4ec6905ffc023760247c97836c2b8e330e2f30c3 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Fri, 3 Jan 2025 14:06:43 +0100 Subject: [PATCH 06/18] Add section about pylint to known issues in design document --- doc/design.rst | 42 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/doc/design.rst b/doc/design.rst index c918cd044..4f91e5672 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -216,6 +216,7 @@ Known Issues This section documents flaws, sins, and known issues with the current design and/or its current implementation that were either known upfront or surfaced through the course of implementing it. Additionally, it attempts to explain why certain choices were made at the time, so one can better understand whether it may be reasonable to make changes now or in the future. + Passing files as individual arguments on the CLI ++++++++++++++++++++++++++++++++++++++++++++++++ @@ -378,31 +379,56 @@ Advantages: - Simplifies adopting ruff for linting -Pylint Instead of Ruff -+++++++++++++++++++++++ +Pylint +++++++ **Description:** +We are currently using Pylint instead of Ruff. **Downsides:** -- Pylint slower and less usable in pre-commit hooks. +- Pylint is slower and less usable in pre-commit hooks +- It is an additional tool, therefore at leas one more processing run of the code is required +- No LSP (e.g. compare to `ruff lsp`) **Rationale/History:** -- Transitioning to Ruff provides better usability and speed for linting processes. -- Pylint was used, but Ruff's emergence offers superior performance in many areas. +- Well known +- Pylint provides built-in project score/rating +- Project score is good for improving legacy code bases which haven't been linted previously +- Plugin support **Ideas/Possible Solutions:** -- Tentatively use Ruff for linting: - - Evaluate impacts on scoring and migration strategy adjustments. - - Continue using Pylint for evaluation/rating while Ruff is integrated for linting. +Replacing Pylint with Ruff for linting would provide significant performance improvement. Additionally, Ruff offers an LSP and IDE integrations and is widely used these days. Additionaly there would be an additional synergy if we adopt ruff for formatting the code base. + +Transitioning to Ruff requires us to adjust the migration and improvement strategies for our projects: + +- Currently, our codebase improvements are guided by scores. However, with Ruff, a new approach is necessary. For example, we could incrementally introduce specific linting rules, fix the related issues, and then enforce these rules. + +- The project rating and scoring system will also need modification. One possiblity would be to run Ruff and Pylint in parallel, utilizing Pylint solely for rating and issue resolution while Ruff is incorporated for linting tasks. + + +Security Linter ++++++++++++++++ +Currently the secuexit zero + + +Rationale: +- simplify adaption into projects +- rating makes it still visiable +- Workflows Dependency Structure ++++++++++++++++++++++++++++++ **Description:** +The common Workflows have been undergooging common and frequent changes, as their full +required features are still discovered. Therfore no clear interface and reqruirements +on the workflow "interfaces" have been documented. Currently it is with most workflows +either get all of them or understand what they do and exchange before one can easily +customize the workflows. **Downsides:** From 70c79bf1c2584653ee0eae65dc826886fe831ba8 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Fri, 3 Jan 2025 14:36:47 +0100 Subject: [PATCH 07/18] Add section regarding security linter --- doc/design.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/doc/design.rst b/doc/design.rst index 4f91e5672..54f0044b9 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -411,13 +411,19 @@ Transitioning to Ruff requires us to adjust the migration and improvement strate Security Linter +++++++++++++++ -Currently the secuexit zero +**Description:** +As of today, the security linter does not fail if it has findings. This was intentionally done to simplify integration and adoption of the tool. Developers can still use the results to improve and find issues within the codebase, and additionally, a rating will be generated to provide some guidance on which projects need attention. + +**Downsides:** +- No enforced safeguard on introducing potential security issues -Rationale: -- simplify adaption into projects -- rating makes it still visiable -- +**Rationale/History:** +- Simplify adoption into projects +- First step to introduce tooling and make the current state/rating visible + +**Ideas/Possible Solutions:** +Define a strategy to address potential security issues in projects. Once this has been done, enforce the immediate addressing of potential security issues in the codebase upon introduction. Workflows Dependency Structure From 137b20dd8ddc71b9f70af1742dcc0a580fe1dd0d Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Fri, 3 Jan 2025 14:57:27 +0100 Subject: [PATCH 08/18] Add section regarding workflows --- doc/design.rst | 19 ++++++------------- 1 file changed, 6 insertions(+), 13 deletions(-) diff --git a/doc/design.rst b/doc/design.rst index 54f0044b9..3ceb54071 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -430,23 +430,16 @@ Workflows Dependency Structure ++++++++++++++++++++++++++++++ **Description:** -The common Workflows have been undergooging common and frequent changes, as their full -required features are still discovered. Therfore no clear interface and reqruirements -on the workflow "interfaces" have been documented. Currently it is with most workflows -either get all of them or understand what they do and exchange before one can easily -customize the workflows. +Undocumented workflow interdependencies and structure **Downsides:** - -- Lack of clear documentation and structure for workflow dependencies. +- Hard to customize if one does not understand the overall setup and dependencies **Rationale/History:** - -- Proper documentation will streamline workflow management and dependency tracing. -- Workflow dependencies were initially structured without thorough documentation. +- Simplify development during the discovery phase (what is needed, how to implement, adjust to discovered needs) +- Ideally, all workflows will be integrated and use a standard setup (part of the customization can also be done in the called nox tasks) **Ideas/Possible Solutions:** -- Work out and document dependencies and structure thoroughly. -- Develop a visual or diagrammatic representation to aid comprehension. -- Regularly review and update the documentation to stay current with project evolution. +- Define clear requirements and interfaces +- Document those requirements and interfaces From 97d254bcf953a116162a49ceb4f0db060362cd03 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:14:02 +0100 Subject: [PATCH 09/18] Update doc/design.rst Co-authored-by: Torsten Kilias --- doc/design.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/design.rst b/doc/design.rst index 3ceb54071..01d92ec4f 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -388,7 +388,7 @@ We are currently using Pylint instead of Ruff. **Downsides:** - Pylint is slower and less usable in pre-commit hooks -- It is an additional tool, therefore at leas one more processing run of the code is required +- It is an additional tool, therefore at least one more processing run of the code is required - No LSP (e.g. compare to `ruff lsp`) **Rationale/History:** From a1a4b2b064d62c93b473f3e68bbd4c931e71f15b Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:29:15 +0100 Subject: [PATCH 10/18] Update doc/design.rst Co-authored-by: Christoph Pirkl <4711730+kaklakariada@users.noreply.github.com> --- doc/design.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/design.rst b/doc/design.rst index 01d92ec4f..248de980d 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -228,7 +228,7 @@ to the tools used/invoked by the python toolbox. **Downsides:** - Most shells have limitations on the number of arguments and their length. -- Noisey output, making it hard to decipher the actual command. +- Noisy output, making it hard to decipher the actual command. - Not ideal for all use cases. **Rationale/History:** From 4cd8e45f4b458cde8886a00c43ff06cd40156ce5 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:29:23 +0100 Subject: [PATCH 11/18] Update doc/design.rst Co-authored-by: Christoph Pirkl <4711730+kaklakariada@users.noreply.github.com> --- doc/design.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/design.rst b/doc/design.rst index 248de980d..46f2d5a9b 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -222,7 +222,7 @@ Passing files as individual arguments on the CLI **Description:** -As of today selection of python files for litting formatting etc. is done by passing all relevant python files as individual argument(s) +As of today selection of python files for linting, formatting etc. is done by passing all relevant python files as individual argument(s) to the tools used/invoked by the python toolbox. **Downsides:** From 8c15519078838986db794679e8e60ee491743778 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:29:38 +0100 Subject: [PATCH 12/18] Update doc/design.rst Co-authored-by: Christoph Pirkl <4711730+kaklakariada@users.noreply.github.com> --- doc/design.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/design.rst b/doc/design.rst index 46f2d5a9b..c9c489ee3 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -235,7 +235,7 @@ to the tools used/invoked by the python toolbox. - The current method of passing files as individual arguments by default offers ease in collection and filtering. It also allows users to simply provide or replace the selection mechanism fairly easily. -- Every tool used by the toolbox (e.g., `black`, `isort`) used to support passing files by argument. However, not all of them provided the same mechanism for selection or deselection patterns (e.g., "glob"). +- Every tool used by the toolbox (e.g., `black`, `isort`) used to support passing files by argument. However, not all of them provided the same mechanism for selection or deselection patterns (e.g. "glob"). **Ideas/Solutions:** From 151f149d4aec0d36577d9430e859aa7f4bc226df Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:30:00 +0100 Subject: [PATCH 13/18] Update doc/design.rst Co-authored-by: Christoph Pirkl <4711730+kaklakariada@users.noreply.github.com> --- doc/design.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/design.rst b/doc/design.rst index c9c489ee3..5b4475def 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -260,7 +260,7 @@ The naming is not consistent across the project name (python-toolbox) and the Py **Ideas/Solutions:** -- `Issue-325 `_ +- Consistently rename project to ``exasol-python-toolbox``: `Issue-325 `_ Project Configuration +++++++++++++++++++++ From 06c6f77d8eb8d646c7566e5174b36ca77f02de4d Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:30:16 +0100 Subject: [PATCH 14/18] Update doc/design.rst Co-authored-by: Christoph Pirkl <4711730+kaklakariada@users.noreply.github.com> --- doc/design.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/design.rst b/doc/design.rst index 5b4475def..26f20dc5a 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -314,7 +314,7 @@ While Nox isn't a perfect fit, it still meets most of our requirements for a tas Why Nox was choosen: -- No Additional Language(s) Required: There was no need to introduce extra programming languages or binaries, simplifying the development process. +- No additional language(s) required: There was no need to introduce extra programming languages or binaries, simplifying the development process. - Python-based: Being Python-based, Nox can be extended and understood by Python developers. - Python code: As Nox tasks are defined via Python code, existing scripts can be reused and code can be shared easily. - Simplicity: Nox is relatively "small" in functionality, making it somewhat simple to use and understand. From 0836f642e6da3ec5452258901a7364b5a6c3492e Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:30:40 +0100 Subject: [PATCH 15/18] Update doc/design.rst Co-authored-by: Christoph Pirkl <4711730+kaklakariada@users.noreply.github.com> --- doc/design.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/design.rst b/doc/design.rst index 26f20dc5a..f449adb1b 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -340,10 +340,10 @@ Generally, one may consider addressing the other issues by choosing another task Poetry for Project Management +++++++++++++++++++++++++++++ -While poetry was and is a good choice for exasol project, dependency, build tool etc. "most recently" -`uv `_ has surffeced and made big advanced. Looking at uv it addresses additional itches with +While poetry was and is a good choice for Exasol project, dependency, build tool etc. "most recently" +`uv `_ has surfaced and made big advanced. Looking at uv it addresses additional itches with our projects and therefore in the long run it may be a good idea to migrate our project setups to it. -Use poetry for project, build and depenency management. +Use poetry for project, build and dependency management. Code Formatting From dffc376b96b875970e4d23627eeb623a6b66d6fe Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:31:07 +0100 Subject: [PATCH 16/18] Update doc/design.rst Co-authored-by: Christoph Pirkl <4711730+kaklakariada@users.noreply.github.com> --- doc/design.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/design.rst b/doc/design.rst index f449adb1b..9b58d927c 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -389,7 +389,7 @@ We are currently using Pylint instead of Ruff. - Pylint is slower and less usable in pre-commit hooks - It is an additional tool, therefore at least one more processing run of the code is required -- No LSP (e.g. compare to `ruff lsp`) +- No support for Language Server Protocol (LSP, e.g. compare to `ruff lsp`) **Rationale/History:** From 9415b8db745640467a445b9d22f23afd4b4992b5 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Tue, 7 Jan 2025 09:31:34 +0100 Subject: [PATCH 17/18] Update doc/design.rst Co-authored-by: Christoph Pirkl <4711730+kaklakariada@users.noreply.github.com> --- doc/design.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/design.rst b/doc/design.rst index 9b58d927c..0c7b55e6f 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -423,7 +423,8 @@ As of today, the security linter does not fail if it has findings. This was inte - First step to introduce tooling and make the current state/rating visible **Ideas/Possible Solutions:** -Define a strategy to address potential security issues in projects. Once this has been done, enforce the immediate addressing of potential security issues in the codebase upon introduction. +* Define a strategy to address potential security issues in projects. Once this has been done, enforce the immediate addressing of potential security issues in the codebase upon introduction. +* Allow excluding individual findings in projects until they are fixed. Workflows Dependency Structure From 46ef3255542af3aebd0ea1d37f383bc3ac9e7769 Mon Sep 17 00:00:00 2001 From: Nicola Coretti Date: Wed, 8 Jan 2025 09:15:50 +0100 Subject: [PATCH 18/18] Update changelog --- doc/changes/unreleased.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/doc/changes/unreleased.md b/doc/changes/unreleased.md index cdfc2ea3a..aee57f1c0 100644 --- a/doc/changes/unreleased.md +++ b/doc/changes/unreleased.md @@ -9,7 +9,10 @@ 🚨 Attention: Due to these changes, the workflows will no longer be executed if the PR comes from a branch not located in this repository. As third-party contributions from outside forks are rare to nearly non-existent, this downside was considered a reasonable trade-off at this time. - + +## 📚 Documentation +* Updated design doc (Added known Issues) + ## 🔩 Internal * Relocked dependencies -* Update referenced github actions \ No newline at end of file +* Update referenced github actions