Tracking pull request to merge release-2.14.0 to master (#2737)

* update for 2.14.0

* update pr number for 2.14.0

* fix: fix ui category display mismatch for approved transfers.

* fix: update the ui tests for the CreditTransferSigningHistory component

* crunchy related update in pipeline

* fix: attorny address if null/empty

* fix: updated trade effective date to be set upon approval

* chore: Update functions.js, remove unneeded comments

* Bump jinja2 from 3.1.2 to 3.1.3 in /backend

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* Fix: Effective Date fix for HDE entries-2769

* Feat: Merge January Candidate - 240117 (#2807)

This merge represents a significant effort to align the TFRS application with the latest legislative requirements, improve the user experience, and enhance the overall reliability and functionality of the system. Here is a high-level summary:

1. **User Interface and Experience Improvements:**
   - Enhanced error messages for credit transfer agreement dates and other functionalities to improve clarity and user guidance.
   - Adjustments in the UI, particularly in the Excel export feature and the 'Edit Organization' page, for better data readability and consistency.

2. **Legislative Compliance Updates:**
   - Removal of outdated options (e.g., the year 2023) from various dropdown menus to align with current legislation.
   - Introduction of compliance unit concepts in compliance reports, replacing previous credits and debits systems.
   - Adjustments to the compliance report model and related queries to improve loading times and data management.
   - Various bug fixes and updates related to compliance report generation and submission, including handling negative balances and reserve compliance units.

3. **System Enhancements and Bug Fixes:**
   - Removal of redundant caching logic to ensure updated details are displayed.
   - Significant backend updates, including Django version bump from 3.2.20 to 3.2.23.
   - Addition of new features like the maxCredits service method and more flexible filtering for organization status.
   - A multitude of bug fixes, including UI corrections, compliance unit calculations, and test case enhancements.

4. **Development and Deployment Process Improvements:**
   - Implementation of new pipelines and build processes for both frontend and backend, enhancing the deployment workflow.
   - Addition of unit tests and continuous integration steps to ensure code stability and reliability.

5. **Label and Terminology Updates:**
   - Comprehensive updates to labels and terminology across various views and functionalities, aligning with the new act.
   - Specific focus on compliance reports, transaction types, and dashboard displays to ensure clarity and legislative compliance.

6. **Database and Data Handling Enhancements:**
   - Database migrations to support new features and legislative changes.
   - Optimization of code and data handling processes, including compliance report spreadsheet updates and handling of different scenarios in compliance unit calculations.

* add backendKeycloakCertsUrl

* fix: fix incorrect button label in draft initiative agreement issuance (#2811)

* fix: fix unresponsive filter input in id column for transactions table (#2815)

* Fix: TFRS - Compliance summary rounding discrepancy in supplemental reporting #2817 (#2819)

* fix: update act citations in transaction history (#2820)

* Fix: TFRS - Date/time stamp records twice in report history box when hitting "recommend for acceptance" #2814 (#2818)

* Fix: Date/time stamp records twice in report history box when hitting "recommend for acceptance" #2814

* fix: minor fix

* Fix: TFRS - Issues with the effective date for transactions entered using the Historical Data Entry feature #2821 (#2822)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Hamed Valiollahi Bayeki <[email protected]>
Co-authored-by: Alex Zorkin <[email protected]>
Co-authored-by: Kevin Hashimoto <[email protected]>
Co-authored-by: Your Name <[email protected]>
Co-authored-by: kevin-hashimoto <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Prashanth Venkateshappa <[email protected]>
Co-authored-by: Prashanth <[email protected]>

.github/readme.md

@@ -40,8 +40,17 @@
 * tfrs-release.yaml (TFRS release-2.10.0): the pipeline builds the release and deploys on Test and Prod, it needs to be manually triggered
 * create-release.yaml (Create Release after merging to master): tag and create the release after merging release branch to master. The description of the tracking pull request becomes release notes
 
+* dev-jan-release.yaml (TFRS Dev Jan Release): the pipeline build Jan 2024 release and deploy on dev for every commit
+* dev-release.yaml (TFRS Dev release-2.9.0): the pipeline is automatically triggered when there is a commit to the release branch
+* tfrs-release.yaml (TFRS release-2.9.0): the pipelin builds the release and deploy on Test and Prod, it needs to be manually triggered
+
 ## Other Pipelines
 
+* branch-deploy-template.yaml (Branch Deploy Template): a pipeline template to deploy a branch
+* build-template.yaml (Build Template): a pipeline template to build branch or pull request
 * cleanup-cron-workflow-runs.yaml (Scheduled cleanup old workflow runs): a cron job to cleanup the old workflows
 * cleanup-workflow-runs.yaml  (Cleanup old workflow runs): manually cleanup teh workflow runs
-
+* pr-dev-cicd.yaml (TFRS Dev Jan PR CICD): the pipeline builds Jan 2024 pull requests and deploy on dev if the pull request title ends with build-on-dev
+* pr-dev-database-template.yaml (PR Dev Database Template): the template to create database for pull request build
+* pr-deploy-template (PR Dev Deploy Template): the template deploys pull request build to dev
+* pr-teardown.yaml (TFRS Dev Jan PR Teardown): tear down the Jan 2024 pull request builds from dev

.github/workflows/branch-deploy-template.yaml

@@ -0,0 +1,142 @@
+name: Branch Deploy Template
+
+on:
+  workflow_call:
+    inputs:
+      branch-name: # sample value: release-2.9.0 or main-release-jan-2024
+        required: true
+        type: string
+      # suffix is in format of -dev, -test, -dev-jan, test-jan, -dev-1923, dev-jan-1923
+      suffix:
+        required: true
+        type: string   
+      # env-name is in format of dev, test     
+      env-name:
+        required: true
+        type: string   
+      # database-service-host-name, sample tfrs-spilo, tfrs-spilo-jan, tfrs-spilo-dev-1988
+      database-service-host-name:
+        required: true
+        type: string
+      # this virtual host name, sample  tfrs-jan-vhost
+      rabbitmq-vhost:
+        required: true
+        type: string           
+    secrets:
+      tools-namespace:
+        required: true
+      namespace:
+        required: true
+      openshift-server:
+         required: true 
+      openshift-token:   
+         required: true       
+
+jobs:
+
+  deploy:
+
+    name: Deploy tfrs
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+
+      - name: Check out repository
+        uses: actions/checkout@v3
+        with:
+          ref: ${{ inputs.branch-name }}
+
+      - name: Log in to Openshift
+        uses: redhat-actions/[email protected]
+        with:
+          openshift_server_url: ${{ secrets.openshift-server }}
+          openshift_token: ${{ secrets.openshift-token }}
+          insecure_skip_tls_verify: true
+          namespace: ${{ secrets.tools-namespace }}  
+
+      - name: Deploy tfrs-frontend
+        shell: bash {0}
+        run: |
+          oc tag ${{ secrets.tools-namespace }}/tfrs-frontend:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-frontend:${{ inputs.env-name }}-${{ inputs.branch-name }}
+          cd charts/tfrs-apps/charts/tfrs-frontend
+          helm status -n ${{ secrets.namespace }} tfrs-frontend${{ inputs.suffix }}
+          helm upgrade --install \
+            --set frontendImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \
+            --set suffix=${{ inputs.suffix }} \
+            --set namespace=${{ secrets.namespace }} \
+            --set envName=${{ inputs.env-name }} \
+            -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-frontend${{ inputs.suffix }} .
+          
+      - name: Deploy tfrs-backend
+        shell: bash {0}
+        run: |
+          oc tag ${{ secrets.tools-namespace }}/tfrs-backend:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-backend:${{ inputs.env-name }}-${{ inputs.branch-name }}
+          cd charts/tfrs-apps/charts/tfrs-backend
+          helm status -n ${{ secrets.namespace }} tfrs-backend${{ inputs.suffix }}
+          helm upgrade --install \
+            --set backendImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \
+            --set suffix=${{ inputs.suffix }} \
+            --set namespace=${{ secrets.namespace }} \
+            --set envName=${{ inputs.env-Name }} \
+            --set databaseServiceHostName=${{ inputs.database-service-host-name }} \
+            --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \
+            -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-backend${{ inputs.suffix }} .
+          
+      - name: Deploy tfrs-celery
+        shell: bash {0}
+        run: |
+          oc tag ${{ secrets.tools-namespace }}/tfrs-celery:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-celery:${{ inputs.env-name }}-${{ inputs.branch-name }}
+          cd charts/tfrs-apps/charts/tfrs-celery
+          helm status -n ${{ secrets.namespace }} tfrs-celery${{ inputs.suffix }}
+          helm upgrade --install \
+            --set celeryImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \
+            --set suffix=${{ inputs.suffix }} \
+            --set namespace=${{ secrets.namespace }} \
+            --set envName=${{ inputs.env-Name }} \
+            --set databaseServiceHostName=${{ inputs.database-service-host-name }} \
+            --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \
+            -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-celery${{ inputs.suffix }} .
+
+      - name: Deploy tfrs-scan-handler
+        shell: bash {0}
+        run: |
+          oc tag ${{ secrets.tools-namespace }}/tfrs-scan-handler:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-scan-handler:${{ inputs.env-name }}-${{ inputs.branch-name }}
+          cd charts/tfrs-apps/charts/tfrs-scan-handler
+          helm status -n ${{ secrets.namespace }} tfrs-scan-handler${{ inputs.suffix }}
+          helm upgrade --install \
+            --set scanHandlerImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \
+            --set suffix=${{ inputs.suffix }} \
+            --set namespace=${{ secrets.namespace }} \
+            --set envName=${{ inputs.env-Name }} \
+            --set databaseServiceHostName=${{ inputs.database-service-host-name }} \
+            --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \
+            -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-scan-handler${{ inputs.suffix }} .
+
+      - name: Deploy tfrs-scan-coordinator
+        shell: bash {0}
+        run: |
+          oc tag ${{ secrets.tools-namespace }}/tfrs-scan-coordinator:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-scan-coordinator:${{ inputs.env-name }}-${{ inputs.branch-name }}
+          cd charts/tfrs-apps/charts/tfrs-scan-coordinator
+          helm status -n ${{ secrets.namespace }} tfrs-scan-coordinator${{ inputs.suffix }}
+          helm upgrade --install \
+            --set scanCoordinatorImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \
+            --set suffix=${{ inputs.suffix }} \
+            --set namespace=${{ secrets.namespace }} \
+            --set envName=${{ inputs.env-Name }} \
+            --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \
+            -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-scan-coordinator${{ inputs.suffix }} .      
+
+      - name: Deploy tfrs-notification-server
+        shell: bash {0}
+        run: |
+          oc tag ${{ secrets.tools-namespace }}/tfrs-notification-server:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-notification-server:${{ inputs.env-name }}-${{ inputs.branch-name }}
+          cd charts/tfrs-apps/charts/tfrs-notification-server
+          helm status -n ${{ secrets.namespace }} tfrs-notification-server${{ inputs.suffix }}
+          helm upgrade --install \
+            --set notificationServerImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \
+            --set suffix=${{ inputs.suffix }} \
+            --set namespace=${{ secrets.namespace }} \
+            --set envName=${{ inputs.env-Name }} \
+            --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \
+            -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-notification-server${{ inputs.suffix }} .          

.github/workflows/build-template.yaml

@@ -0,0 +1,233 @@
+
+# This template supports both pr build and branch build
+name: Build Template
+
+on:
+  workflow_call:
+    inputs:
+      # when build branch, the sample value is -main-release-jan-2024
+      # when build pull request, the sample value is -jan-2024
+      suffix:
+        required: true
+        type: string
+      # when build branch, the sample value is main-release-jan-2024
+      # when build pull request, the sample value is refs/pull/2024/head
+      checkout-ref:
+        required: true
+        type: string
+    secrets:
+      tools-namespace:
+        required: true    
+      openshift-server:
+         required: true 
+      openshift-token:   
+         required: true       
+
+env:
+  GIT_URL: https://github.com/bcgov/tfrs.git 
+
+jobs:
+
+  build-backend:
+
+    name: Build TFRS Backend on Openshift
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+
+      - name: Check out repository
+        uses: actions/[email protected]
+        with:
+          ref: ${{ inputs.checkout-ref }}
+
+      - name: Log in to Openshift
+        uses: redhat-actions/[email protected]
+        with:
+          openshift_server_url: ${{ secrets.openshift-server }}
+          openshift_token: ${{ secrets.openshift-token }}
+          insecure_skip_tls_verify: true
+          namespace: ${{ secrets.tools-namespace }}
+
+      - name: Build tfrs Backend
+        run: |
+          cd openshift-v4/templates/backend
+          oc process -f ./backend-bc.yaml NAME=tfrs \
+            SUFFIX=-build${{ inputs.suffix}} \
+            VERSION=build${{ inputs.suffix }} \
+            GIT_URL=${{ env.GIT_URL }} \
+            GIT_REF=${{ inputs.checkout-ref }} \
+            | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }}
+          oc cancel-build bc/tfrs-backend-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true
+          oc start-build --wait=true tfrs-backend-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }}
+
+  build-frontend:
+
+    name: Build TFRS Frontend on Openshift
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+
+      - name: Check out repository
+        uses: actions/[email protected]
+        with:
+          ref: ${{ inputs.checkout-ref }}
+
+      - name: Log in to Openshift
+        uses: redhat-actions/[email protected]
+        with:
+          openshift_server_url: ${{ secrets.openshift-server }}
+          openshift_token: ${{ secrets.openshift-token }}
+          insecure_skip_tls_verify: true
+          namespace: ${{ secrets.tools-namespace }}
+
+      - name: Build TFRS Frontend
+        run: |
+          cd openshift-v4/templates/frontend
+          oc process -f ./frontend-bc-docker.yaml NAME=tfrs \
+            SUFFIX=-build${{ inputs.suffix}} \
+            VERSION=build${{ inputs.suffix }} \
+            GIT_URL=${{ env.GIT_URL }} \
+            GIT_REF=${{ inputs.checkout-ref }} \
+            | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }}
+          oc cancel-build bc/tfrs-frontend-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true
+          oc start-build --wait=true tfrs-frontend-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }}
+
+  build-celery:
+
+    name: Build TFRS Celery on Openshift
+    needs: [build-frontend, build-backend]
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+
+      - name: Check out repository
+        uses: actions/[email protected]
+        with:
+          ref: ${{ inputs.checkout-ref }}
+
+      - name: Log in to Openshift
+        uses: redhat-actions/[email protected]
+        with:
+          openshift_server_url: ${{ secrets.openshift-server }}
+          openshift_token: ${{ secrets.openshift-token }}
+          insecure_skip_tls_verify: true
+          namespace: ${{ secrets.tools-namespace }}
+
+      - name: Build TFRS Celery
+        run: |
+          cd openshift-v4/templates/celery
+          pwd
+          ls -l
+          oc process -f ./celery-bc-docker.yaml NAME=tfrs \
+            SUFFIX=-build${{ inputs.suffix}} \
+            VERSION=build${{ inputs.suffix }} \
+            GIT_URL=${{ env.GIT_URL }} \
+            GIT_REF=${{ inputs.checkout-ref }} \
+            | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }}
+          oc cancel-build bc/tfrs-celery-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true
+          oc start-build --wait=true tfrs-celery-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }}
+
+  build-scan-coordinator:
+
+    name: Build TFRS Scan Coordinator on Openshift
+    needs: [build-frontend, build-backend]
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+
+      - name: Check out repository
+        uses: actions/[email protected]
+        with:
+          ref: ${{ inputs.checkout-ref }}
+
+      - name: Log in to Openshift
+        uses: redhat-actions/[email protected]
+        with:
+          openshift_server_url: ${{ secrets.openshift-server }}
+          openshift_token: ${{ secrets.openshift-token }}
+          insecure_skip_tls_verify: true
+          namespace: ${{ secrets.tools-namespace }}
+
+      - name: Build TFRS Scan Coordinator
+        run: |
+          cd openshift-v4/templates/scan-coordinator
+          oc process -f ./scan-coordinator-bc.yaml NAME=tfrs \
+            SUFFIX=-build${{ inputs.suffix}} \
+            VERSION=build${{ inputs.suffix }} \
+            GIT_URL=${{ env.GIT_URL }} \
+            GIT_REF=${{ inputs.checkout-ref }} \
+            | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }}
+          oc cancel-build bc/tfrs-scan-coordinator-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true
+          oc start-build --wait=true tfrs-scan-coordinator-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }}          
+
+  build-scan-handler:
+
+    name: Build TFRS Scan Handler on Openshift
+    needs: [build-scan-coordinator, build-celery]
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+
+      - name: Check out repository
+        uses: actions/[email protected]
+        with:
+          ref: ${{ inputs.checkout-ref }}
+
+      - name: Log in to Openshift
+        uses: redhat-actions/[email protected]
+        with:
+          openshift_server_url: ${{ secrets.openshift-server }}
+          openshift_token: ${{ secrets.openshift-token }}
+          insecure_skip_tls_verify: true
+          namespace: ${{ secrets.tools-namespace }}
+
+      - name: Build TFRS Scan-Handler
+        run: |
+          cd openshift-v4/templates/scan-handler
+          oc process -f ./scan-handler-bc-docker.yaml NAME=tfrs \
+            SUFFIX=-build${{ inputs.suffix}} \
+            VERSION=build${{ inputs.suffix }} \
+            GIT_URL=${{ env.GIT_URL }} \
+            GIT_REF=${{ inputs.checkout-ref }} \
+            | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }}
+          oc cancel-build bc/tfrs-scan-handler-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true
+          oc start-build --wait=true tfrs-scan-handler-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }}   
+
+  build-notification-server:
+
+    name: Build TFRS Notification Server on Openshift
+    needs: [build-scan-coordinator, build-celery]
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+
+      - name: Check out repository
+        uses: actions/[email protected]
+        with:
+          ref: ${{ inputs.checkout-ref }}
+
+      - name: Log in to Openshift
+        uses: redhat-actions/[email protected]
+        with:
+          openshift_server_url: ${{ secrets.openshift-server }}
+          openshift_token: ${{ secrets.openshift-token }}
+          insecure_skip_tls_verify: true
+          namespace: ${{ secrets.tools-namespace }}
+
+      - name: Build TFRS Notification Server
+        run: |
+          cd openshift-v4/templates/notification
+          oc process -f ./notification-server-bc.yaml NAME=tfrs \
+            SUFFIX=-build${{ inputs.suffix}} \
+            VERSION=build${{ inputs.suffix }} \
+            GIT_URL=${{ env.GIT_URL }} \
+            GIT_REF=${{ inputs.checkout-ref }} \
+            | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }}
+          oc cancel-build bc/tfrs-notification-server-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true
+          oc start-build --wait=true tfrs-notification-server-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }}