SIMSBIOHUB-424: Support Artifact Intake + Misc Enhancements

[NickPhura]

Links to Jira Tickets

https://apps.nrs.gov.bc.ca/int/jira/browse/SIMSBIOHUB-424

Description of Changes

Added a call to clamav to app.ts, alongside the multer middleware, so that we shouldn't have to call it in the endpoint anymore. Endpoint code should just be able to assume any files getting that far have already been checked and are safe.

Update submission/intake endpoint/service/repo

• Updates to better traverse the submission json structure, inserting each submission feature.
  • Now also correctly assigns parent_submission_feature_id when inserting submission features

Update submission search indexing

• Previously, checked all feature properties against all incoming features.
• Now, only checks feature properties registered to a given feature type, against incoming features of the same type.
• Some room for optimizing the code here a bit, as right now it calls .find repeatedly, when it probably should just pre-parse all of the codes ahead of time, and then do direct object de-referencing. It was doing this before, but the codes response object got more complicated and I just skipped optimizing it for now in order to get this PR out quicker.

Update codes endpoint/service/repo

• Updated api types, and modified endpoint response
• Updated app types accordingly

Update artifact/intake endpoint/service/repo

Add required_value column to feature_type_property table to indicate if properties are required or optional when running validation.

Added additionalProperties: false to most endpoints, and fixed response validation issues.

Update get submission features endpoint to replace the original data object with one populated from search_* rows.

• Added related service/repo code

Update submission patch function to also unpublish all other submissions for the same uuid on publish=true

Testing Notes

If running both SIMS and BioHub branches, should be able to publish a survey that has 0 or more attachments. Attachments should have their corresponding submission feature records created, and matching search_string s3_key records added, and the file should be uploaded the S3.

Related PR for the SIMS changes: bcgov/biohubbc#1193

[github-actions]

Openshift URLs for the PR Deployment:

• App Route: https://biohub-platform-app-230-a0ec71-dev.apps.silver.devops.gov.bc.ca/
• Api Route: https://biohub-platform-api-230-a0ec71-dev.apps.silver.devops.gov.bc.ca/
• Pods: https://console.apps.silver.devops.gov.bc.ca/k8s/ns/a0ec71-dev/pods?name=230

[codecov]

Codecov Report

Attention: 46 lines in your changes are missing coverage. Please review.

Comparison is base (9ef67c6) 62.29% compared to head (9448c5c) 62.82%.

Files	Patch %	Lines
api/src/repositories/submission-repository.ts	70.37%	3 Missing and 5 partials ⚠️
api/src/app.ts	0.00%	6 Missing ⚠️
api/src/repositories/search-index-respository.ts	40.00%	6 Missing ⚠️
api/src/services/search-index-service.ts	76.00%	4 Missing and 2 partials ⚠️
api/src/services/submission-service.ts	89.18%	3 Missing and 1 partial ⚠️
api/src/services/validation-service.ts	73.33%	2 Missing and 2 partials ⚠️
...ubmissionDataGrid/useSubmissionDataGridColumns.tsx	40.00%	3 Missing ⚠️
api/src/services/code-service.ts	87.50%	2 Missing ⚠️
api/src/zod-schema/json.ts	66.66%	2 Missing ⚠️
...ures/submissions/components/SubmissionDataGrid.tsx	0.00%	2 Missing ⚠️
... and 3 more

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #230      +/-   ##
==========================================
+ Coverage   62.29%   62.82%   +0.52%     
==========================================
  Files         265      265              
  Lines        7320     7402      +82     
  Branches     1034     1039       +5     
==========================================
+ Hits         4560     4650      +90     
+ Misses       2514     2496      -18     
- Partials      246      256      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Openshift URLs for the PR Deployment:

• App Route: https://biohub-platform-app-230-a0ec71-dev.apps.silver.devops.gov.bc.ca/
• Api Route: https://biohub-platform-api-230-a0ec71-dev.apps.silver.devops.gov.bc.ca/
• Pods: https://console.apps.silver.devops.gov.bc.ca/k8s/ns/a0ec71-dev/pods?name=230

[NickPhura]

Working on unit tests:

artifact-service.ts (Nick - done)
artifact-repository.ts

code-repository.ts (Curtis - done)

submission-service.ts (Nick - done)
submission-repository.ts (Nick - done)

search-index-repository.ts (Curtis - done)

artifact/intake.ts (Nick - done)

[curtisupshall]

Code looks good to me!

[sonarqubecloud]

Quality Gate failed

Failed conditions

3.0% Duplication on New Code (required ≤ 3%)

See analysis details on SonarCloud

[al-rosenthal]

ran this locally and things are working as expected 🌯