Merge branch 'master' into pro-1316-ui-feriennet

.pre-commit-config.yaml

@@ -15,7 +15,7 @@ repos:
         exclude: .pre-commit-config.yaml
       - id: pt_structure
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.8.2
+    rev: v0.8.4
     hooks:
       - id: ruff
         args: [ "--fix" ]
@@ -32,7 +32,7 @@ repos:
     - id: sass-lint
       files: '^src/.*\.scss'
   - repo: https://github.com/pre-commit/mirrors-eslint
-    rev: v9.16.0
+    rev: v9.17.0
     hooks:
     - id: eslint
       files: '^src/.*\.jsx?$'

CHANGES.md

@@ -1,8 +1,143 @@
 # Changes
 
+## 2024.60
+
+`2024-12-19` | [5925b224b9...947c48974f](https://github.com/OneGov/onegov-cloud/compare/5925b224b9^...947c48974f)
+
+**Upgrade hints**
+- On the off-chance that we have some duplicate `Payment` associations this upgrade task will fail, in which case we would need to write another migration to remove duplicates for all links on `Payment`.
+### Agency
+
+##### Import Staka LU: Skip label 'Telefon' but keep 'Telefonist'
+
+`Feature` | [OGC-1954](https://linear.app/onegovcloud/issue/OGC-1954) | [5925b224b9](https://github.com/onegov/onegov-cloud/commit/5925b224b9354ea45f2ab9fc7bd384e4bf624449)
+
+##### Allows ticket notifications to target parent organisations
+
+`Feature` | [OGC-1658](https://linear.app/onegovcloud/issue/OGC-1658) | [0250a72e1e](https://github.com/onegov/onegov-cloud/commit/0250a72e1eeedf91ab2fbb1e2a99f558226d776c)
+
+##### Prevent cutting content in person card
+
+`Bugfix` | [OGC-1963](https://linear.app/onegovcloud/issue/OGC-1963) | [d296c64877](https://github.com/onegov/onegov-cloud/commit/d296c64877697d606fa4653e749164e5a5ea4a8f)
+
+### Api
+
+##### Return a 403 for missing authorization when requesting a JWT token
+
+`Bugfix` | [7d33b93711](https://github.com/onegov/onegov-cloud/commit/7d33b93711a3804f9fc9941dd790598dfd7df7bf)
+
+##### Makes sure invisible endpoint items are inaccessible
+
+`Bugfix` | [OGC-1992](https://linear.app/onegovcloud/issue/OGC-1992) | [9e317877f1](https://github.com/onegov/onegov-cloud/commit/9e317877f1d21f11c2658415fde14c4c187fe65d)
+
+### Core
+
+##### Adds unique constraint to association tables
+
+`Bugfix` | [OGC-1969](https://linear.app/onegovcloud/issue/OGC-1969) | [4adcb66f82](https://github.com/onegov/onegov-cloud/commit/4adcb66f825d3be0a67c27ef4cda190d18f9711d)
+
+### Election Day
+
+##### Election results shown as pending as long as election not completed/final
+
+Note: No model or api adaption done
+
+`Feature|` | [OGC-1939](https://linear.app/onegovcloud/issue/OGC-1939) | [7663a133d9](https://github.com/onegov/onegov-cloud/commit/7663a133d90f35221e3b3fce660babdbd64872a8)
+
+### Electionday
+
+##### Adds additional date metadata to subscribers
+
+`Feature` | [OGC-1882](https://linear.app/onegovcloud/issue/OGC-1882) | [1770d45927](https://github.com/onegov/onegov-cloud/commit/1770d45927b012e16ff7e99b6a56a6e869bd2158)
+
+##### Include specific URLs for sub-results in SMS when possible
+
+`Feature` | [OGC-1881](https://linear.app/onegovcloud/issue/OGC-1881) | [01eb7d8b03](https://github.com/onegov/onegov-cloud/commit/01eb7d8b034861d80a24166838f95974db679776)
+
+### Feriennet
+
+##### Make it possible to change attendee info without active period
+
+`Bugfix` | [PRO-1341](https://linear.app/projuventute/issue/PRO-1341) | [8693110cd8](https://github.com/onegov/onegov-cloud/commit/8693110cd8edf4d4d93b58efadf790f82f3e9de5)
+
+### File
+
+##### Apply EXIF orientation to image when stripping EXIF metadata
+
+`Bugfix` | [OGC-1993](https://linear.app/onegovcloud/issue/OGC-1993) | [5859fb2455](https://github.com/onegov/onegov-cloud/commit/5859fb2455bd1c3b6c68c0248b1ee2be526ee945)
+
+### Files
+
+##### Sort sidebar documents alphabetically (org and town6)
+
+`Feature` | [OGC-1797](https://linear.app/onegovcloud/issue/OGC-1797) | [0e99c5e925](https://github.com/onegov/onegov-cloud/commit/0e99c5e92511685250f051c15dcdf952d182c2c9)
+
+### Org
+
+##### Avoid generating redundant file links for linked general files
+
+This also updates WebTest to the newest version, since we need it in
+order to test multi-file uploads.
+
+`Bugfix` | [OGC-1967](https://linear.app/onegovcloud/issue/OGC-1967) | [27c3e25ed5](https://github.com/onegov/onegov-cloud/commit/27c3e25ed579219df1454d5fcfa36dd6708b640d)
+
+##### Re-add submit-button for directory entry suggestion button
+
+`Bugfix` | [OGC-1986](https://linear.app/onegovcloud/issue/OGC-1986) | [7e4076face](https://github.com/onegov/onegov-cloud/commit/7e4076face9c972078d73bedd4b2cdef53bd511b)
+
+##### Submits individual uploads in dropzone sequentially
+
+Submitting them in parallel sometimes results in nginx producing 503
+errors and there's no significant speed benefit to starting the uploads
+in parallel, since we will still be limited by our bandwidth.
+
+`Bugfix` | [OGC-1994](https://linear.app/onegovcloud/issue/OGC-1994) | [e12dd0bb88](https://github.com/onegov/onegov-cloud/commit/e12dd0bb884315e8a299549e219554254ac8f3c2)
+
+### Pas
+
+##### Generating various exports.
+
+`Feature` | [OGC-1878](https://linear.app/onegovcloud/issue/OGC-1878) | [96f09cfc20](https://github.com/onegov/onegov-cloud/commit/96f09cfc200a20d153fa81ab5b4dccbaecba0f9b)
+
+### Town6
+
+##### Sidebar make person function as wide as document names and map
+
+`Feature` | [NONE](#NONE) | [13d6a10ad3](https://github.com/onegov/onegov-cloud/commit/13d6a10ad3fbd27e9e26440acfdb47917ac6b467)
+
+##### Icon Links
+
+Correct margin and padding if there is no text
+
+`Feature` | [OGC-1943](https://linear.app/onegovcloud/issue/OGC-1943) | [471d46565c](https://github.com/onegov/onegov-cloud/commit/471d46565c7158a752b41722125a600bdaffea13)
+
+##### Make the context-specific-function multi line capable.
+
+`Feature` | [OGC-1955](https://linear.app/onegovcloud/issue/OGC-1955) | [597a6cc034](https://github.com/onegov/onegov-cloud/commit/597a6cc034b87d7b3e727c9aa83e3cf38aafd4a4)
+
+##### Lead is no longer a mandatory field in ExternalLinkForm
+
+`Feature` | [OGC-1941](https://linear.app/onegovcloud/issue/OGC-1941) | [5523e08e31](https://github.com/onegov/onegov-cloud/commit/5523e08e31cc2b0e4ee865b3e3126f41aabe1cde)
+
+##### Main image in news
+
+Enable the option to hide main images on news
+
+`Bugfix` | [OGC-1903](https://linear.app/onegovcloud/issue/OGC-1903) | [53135169e2](https://github.com/onegov/onegov-cloud/commit/53135169e2b654ca6782f448e19446ae2c989680)
+
+### User
+
+##### Adds scope setting to OIDC authentication provider
+
+`Feature` | [OGC-1767](https://linear.app/onegovcloud/issue/OGC-1767) | [dc2ebc87dc](https://github.com/onegov/onegov-cloud/commit/dc2ebc87dcc8d8e699c59f2e21eb53492e899d87)
+
+## test
+
+`2024-12-17` | [9c3aee5da3...b6de332b06](https://github.com/OneGov/onegov-cloud/compare/9c3aee5da3^...b6de332b06)
+
 ## 2024.59
 
-`2024-12-12` | [0e99c5e925...04ab5ae76d](https://github.com/OneGov/onegov-cloud/compare/0e99c5e925^...04ab5ae76d)
+`2024-12-12` | [0e99c5e925...5ae8a276f3](https://github.com/OneGov/onegov-cloud/compare/0e99c5e925^...5ae8a276f3)
 
 ### Files
 
@@ -2098,29 +2233,3 @@ Some files in the FileLinkExtension could not be deleted, they will return to th
 
 `2024-03-08` | [39058784ad...c181f4f875](https://github.com/OneGov/onegov-cloud/compare/39058784ad^...c181f4f875)
 
-## 2024.13
-
-`2024-03-08` | [63430a80b0...1b15796fc5](https://github.com/OneGov/onegov-cloud/compare/63430a80b0^...1b15796fc5)
-
-## 2024.12
-
-`2024-03-08` | [1d64c923a1...c9d492e36a](https://github.com/OneGov/onegov-cloud/compare/1d64c923a1^...c9d492e36a)
-
-### Election Day
-
-##### Improve relationships of elections.
-
-Uses lazy relationships for election models.
-
-`Feature` | [OGC-1478](https://linear.app/onegovcloud/issue/OGC-1478) | [d0766448db](https://github.com/onegov/onegov-cloud/commit/d0766448db4eeb1117f47879ebaaa0d1c8344044)
-
-##### Add experimental eCH-0252 import for elections.
-
-`Feature` | [OGC-1172](https://linear.app/onegovcloud/issue/OGC-1172) | [047da42ae9](https://github.com/onegov/onegov-cloud/commit/047da42ae9b8a0bea2c977dadedf22a725b950b7)
-
-### Org
-
-##### Per page option to switch off showing files in the sidebar
-
-`Feature` | [OGC-1477](https://linear.app/onegovcloud/issue/OGC-1477) | [1d64c923a1](https://github.com/onegov/onegov-cloud/commit/1d64c923a12c9236447887c8c5511f9f309c51a1)
-

do/release

@@ -106,7 +106,7 @@ function configure-sentry() {
 }
 
 function current-version() {
-    git tag --sort=-version:refname --merged master | head -n 1
+    git tag --sort=version:refname --merged master | tail -n 1
 }
 
 function announce-release-on-sentry-io() {

src/onegov/api/models.py

@@ -88,7 +88,10 @@ def __init__(
     ):
         self.message = (
             exception.message
-            if exception and hasattr(exception, 'message') else message
+            if exception and hasattr(exception, 'message') else
+            exception.title
+            if exception and hasattr(exception, 'title') else
+            message
         )
         self.status_code = (
             exception.status_code

src/onegov/api/utils.py

@@ -14,6 +14,8 @@
     from morepath.request import Response
 
 
+# TODO: Do we allow this to update request.identity, so we can elevate
+#       privileges for what a given API token is allowed to see?
 def authenticate(request: 'CoreRequest') -> ApiKey:
     if request.authorization is None:
         raise HTTPUnauthorized()

src/onegov/api/views.py

@@ -6,7 +6,7 @@
 from onegov.api.utils import authenticate, check_rate_limit
 from onegov.core.security import Public
 from onegov.form.fields import HoneyPotField
-from webob.exc import HTTPMethodNotAllowed, HTTPUnauthorized
+from webob.exc import HTTPMethodNotAllowed, HTTPNotFound, HTTPUnauthorized
 from wtforms import HiddenField
 
 
@@ -156,6 +156,12 @@ def add_headers(response: 'Response') -> None:
         assert endpoint is not None
         links = self.links or {}
         data = self.data or {}
+
+        # make sure we are actually supposed to be able to see this
+        # the API shouldn't include invisible items either (for now)
+        if (item := self.item) and not request.is_visible(item):
+            raise HTTPNotFound()
+
         payload: dict[str, JSONObject] = {
             'collection': {
                 'version': '1.0',
@@ -222,6 +228,11 @@ def edit_api_endpoint_item(
             if api_key.read_only:
                 raise HTTPUnauthorized()
 
+        # make sure we are actually supposed to be able to see this
+        # the API shouldn't include invisible items either (for now)
+        if (item := self.item) and not request.is_visible(item):
+            raise HTTPNotFound()
+
         def walk_errors(
             errors: 'Sequence[str] | _FormErrors',
             prefix: str | None

src/onegov/core/__init__.py

@@ -5,7 +5,7 @@
 # version is to create release-dependent urls, artifacts and caches. During
 # development these dependencies do not need to be updated in lock-step.
 #
-__version__ = '2024.59'
+__version__ = '2024.60'
 
 # The module levels used for dependency tests and to have a well defined
 # onegov core upgrade order.

src/onegov/core/csv.py

@@ -755,7 +755,7 @@ def has_duplicates(a_list: 'Sequence[Any]') -> bool:
 
 def list_duplicates_index(a: 'Sequence[Any]') -> list[int]:
     """
-    returns a list of indexes of duplicates in a list.
+    Returns a list of indexes of duplicates in a list.
     for example::
 
         a = [1, 2, 3, 2, 1, 5, 6, 5, 5, 5]

src/onegov/election_day/locale/de_CH/LC_MESSAGES/onegov.election_day.po

@@ -1138,6 +1138,9 @@ msgstr "Ja"
 msgid "No"
 msgstr "Nein"
 
+msgid "Pending"
+msgstr "Pendent"
+
 msgid "All candidates"
 msgstr "Alle Kandidierenden"
 

src/onegov/election_day/locale/fr_CH/LC_MESSAGES/onegov.election_day.po

@@ -1139,6 +1139,9 @@ msgstr "Oui"
 msgid "No"
 msgstr "Non"
 
+msgid "Pending"
+msgstr "On cours"
+
 msgid "All candidates"
 msgstr "Tous les candidats"
 

src/onegov/election_day/locale/it_CH/LC_MESSAGES/onegov.election_day.po

@@ -1133,6 +1133,9 @@ msgstr "Si"
 msgid "No"
 msgstr "No"
 
+msgid "Pending"
+msgstr "In attesa"
+
 msgid "All candidates"
 msgstr "Tutti i candidati"
 

src/onegov/election_day/locale/rm_CH/LC_MESSAGES/onegov.election_day.po

@@ -1146,6 +1146,9 @@ msgstr "Gea"
 msgid "No"
 msgstr "Na"
 
+msgid "Pending"
+msgstr "Pendent"
+
 msgid "All candidates"
 msgstr "Tut las candidatas e candidats"
 

src/onegov/election_day/path.py

@@ -320,7 +320,7 @@ def get_data_source_item(
 
 @ElectionDayApp.path(
     model=ArchivedResultCollection,
-    path='/archive/{date}'  # noqa: RUF027
+    path='/archive/{date}'
 )
 def get_archive_by_year(
     app: ElectionDayApp,

src/onegov/election_day/templates/macros.pt

@@ -767,8 +767,9 @@
                     <td tal:condition="layout.majorz" class="column-party right-aligned">${candidate.party}</td>
                     <td tal:condition="layout.proporz" class="column-list right-aligned">${candidate.list_name}</td>
                     <td tal:condition="layout.majorz and show_percentage" class="column-votes-percentage right-aligned">${layout.format_number(candidate.percentage, 1)}%</td>
-                    <td class="column-elected answer accepted right-aligned" tal:condition="candidate.elected is True" i18n:translate="">Yes</td>
-                    <td class="column-elected answer rejected right-aligned" tal:condition="candidate.elected is False" i18n:translate="">No</td>
+                    <td class="column-elected answer accepted right-aligned" tal:condition="election.completed and candidate.elected is True" i18n:translate="">Yes</td>
+                    <td class="column-elected answer rejected right-aligned" tal:condition="election.completed and candidate.elected is False" i18n:translate="">No</td>
+                    <td class="column-elected answer right-aligned" tal:condition="not election.completed" i18n:translate="">Pending</td>
                     <td class="column-elected answer right-aligned" tal:condition="candidate.elected is None" i18n:translate=""></td>
                 </tr>
             </tal:block>

src/onegov/file/attachments.py

@@ -11,7 +11,7 @@
 from onegov.file.utils import get_svg_size
 from onegov.file.utils import IMAGE_MIME_TYPES
 from onegov.file.utils import word_count
-from PIL import Image, UnidentifiedImageError
+from PIL import Image, ImageOps, UnidentifiedImageError
 from tempfile import SpooledTemporaryFile
 from onegov.pdf.utils import extract_pdf_info
 
@@ -74,6 +74,9 @@ def strip_exif_and_limit_and_store_image_size(
         params: _ImageSaveOptionalParams = {}
 
         if has_exif:
+            # bake EXIF orientation into the image
+            ImageOps.exif_transpose(image, in_place=True)
+
             # replace EXIF section with an empty one
             params['exif'] = Image.Exif()
 

src/onegov/file/filters.py

@@ -10,7 +10,7 @@
 from onegov.core.utils import module_path
 from onegov.file.utils import IMAGE_MIME_TYPES, get_image_size
 from pathlib import Path
-from PIL import Image
+from PIL import Image, ImageOps
 from tempfile import TemporaryDirectory
 
 
@@ -93,6 +93,7 @@ def generate_thumbnail(
         output = BytesIO()
 
         thumbnail: Image.Image = Image.open(fp)
+        ImageOps.exif_transpose(thumbnail, in_place=True)
         thumbnail.thumbnail(self.size, Image.Resampling.LANCZOS)
         thumbnail = thumbnail.convert('RGBA')
 

src/onegov/form/fields.py

@@ -16,7 +16,7 @@
 from onegov.form import log, _
 from onegov.form.utils import path_to_filename
 from onegov.form.validators import ValidPhoneNumber
-from onegov.form.widgets import ChosenSelectWidget
+from onegov.form.widgets import ChosenSelectWidget, LinkPanelWidget
 from onegov.form.widgets import HoneyPotWidget
 from onegov.form.widgets import IconWidget
 from onegov.form.widgets import MultiCheckboxWidget
@@ -676,6 +676,11 @@ def populate_obj(self, obj: object, name: str) -> None:
         pass
 
 
+class URLPanelField(PanelField):
+
+    widget = LinkPanelWidget()
+
+
 class DateTimeLocalField(DateTimeLocalFieldBase):
     """ A custom implementation of the DateTimeLocalField to fix issues with
     the format and the datetimepicker plugin.