Skip to content

v2.0.0

Latest
Latest
Compare
Choose a tag to compare
@ThorodanBrom ThorodanBrom released this 14 Nov 02:04
v2.0.0
5f9232b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Version Summary

Version Number : v2.0.0 [tag: v2.0.0]
Date: 14-November-2024

Projects / Repos Included in this release: OGC Resource Server, DX Deployment and installation, DX Documentation

Release Summary :

Centre of Data for Public Good (CDPG) | OGC Resource Server v2.0.0 is released with enhanced features.

Highlights:

  • Added support for vector tilesets
  • Implemented OGC coverages Core conformance
  • Implemented STAC Items and Items Search
  • Added the following processes
    • append dynamic feature data to an existing OGC feature collection
    • add tiles metadata to the database for an existing tileset in S3
    • generate a pre-signed URL to upload data into S3
  • Incorporated auditing flows for Tiles API
  • Added support for provider-delegate authorization for process execution
  • Refactored authentication to use handlers and changed token header to Authorization: Bearer ...
  • Updated the client SDK for OGC and STAC
  • Added HTML versions of API docs
  • Added detailed server documentation
  • Enhanced Software Testing with RESTAssured

Test Reports:

Please find the release test details and reports here

API Docs

The api docs can be found here

How to use the Release:

Prerequisites

  • Make a config file based on the template in example-config/config-example.json
  • Set up AWS S3 for serving tiles and STAC assets
  • Set up PostGIS for storing information related to geo spatial data
  • Set up RabbitMQ for publishing the auditing data
  • Set up the database using Flyway

AWS S3 setup

  • AWS S3 is used to serve map tiles as well as STAC asset files
  • An S3 bucket can be set up by following the AWS S3 documentation, after which the S3 bucket name, region name, access key and secret key can be added to the config.

PostGIS

  • PostGIS is an extension for PostgreSQL that adds support for geographic objects, allowing users to store and query spatial data.
  • To setup PostgreSQL refer setup and installation instructions available here
Table Name Purpose
collections_details To store metadata about collections, including title, description, and bounding box
processes_table To store details of processes including input, output, and execution modes
ri_details To store information related to resource instance (RI) details, including role and access type
roles To store user roles, such as provider, consumer, or delegate
jobs_table To store job details, including status, type, progress, and timestamps, related to different processes
collection_type To store types associated with collections, based on the type column from collections_details
tilematrixset_metadata To store metadata for tile matrix sets, including scale, cell size, and matrix dimensions
tilematrixsets_relation To store the relation between collections and tile matrix sets
collection_supported_crs To store the Coordinate Reference Systems (CRS) supported by specific collections
crs_to_srid To map Coordinate Reference Systems (CRS) to Spatial Reference Identifiers (SRID)
stac_collections_assets To store assets linked to collections, such as thumbnails, data, and metadata, including their size, type, and role
collection_coverage To store the coverage schema and associated hrefs related to collections, helping define the spatial/temporal extent of collections

Auditing

  • Auditing is done using the DX Auditing Server which uses Postgres for storing the audit logs for OGC Resource Server
  • The schema for auditing table in PostgreSQL is present here - postgres auditing table schema
Table Name Purpose DB
auditing_ogc To store audit logs for operations in the OGC Resource Server PostgreSQL

RabbitMQ

  • RabbitMQ is used to push the logs which is consumed by the auditing server
  • To setup RabbitMQ refer the setup and installation instructions available here
  • After deployment of RabbitMQ, we need to ensure that there are certain prerequisites met. Incase if it is not met, please login to RabbitMQ management portal using the RabbitMQ management UI and create a the following
Create vHost
Type Name Details
vHost IUDX-INTERNAL Create a vHost in RabbitMQ
Create Exchange
Exchange Name Type of exchange features Details
auditing direct durable Create an exchange in vHost IUDX-INTERNAL to allow audit information to be published
Create Queue and Bind to Exchange
Exchange Name Queue Name vHost routing key Details
auditing direct durable # Create a queue in vHost IUDX-INTERNAL to allow audit information to be consumed. Ensure that the queue is binded to the auditing exchange

Database Migration using Flyway

  • Database flyway migrations help in updating the schema, permissions, grants, triggers etc., with the latest version
  • Each flyway schema file is versioned with the format V<majorVersion>_<minorVersion>__<description>.sql, ex : V1_1__init-tables.sql to manage the database schema and handle migrations
  • The migration files are located at src/main/resources/db/migrations. The following pre-requisites are needed before running flyway:
    1. An admin user - a database user who has created schema/table privileges for the database. It can be the super user.
    2. A normal user - this is the database user that will be configured to make queries from the server
      (e.g. CREATE USER ogc WITH PASSWORD 'randompassword';)

flyway.conf must be updated with the required data.

  • flyway.url - the database connection URL
  • flyway.user - the username of the admin user
  • flyway.password - the password of the admin user
  • flyway.schemas - the name of the schema under which the tables are created
  • flyway.placeholders.ogcUser - the username of the server user

Please refer here for more information about Flyway config parameters.

After this, the info command can be run to test the config. Then, the migrate command can be run to set up the database. At the /ogc-resource-server directory, run

mvn flyway:info -Dflyway.configFiles=flyway.conf
mvn flyway:migrate -Dflyway.configFiles=flyway.conf

Installation Steps

Maven based execution

  1. Install Java 11 and maven
  2. Set Environment variables
export LOG_LEVEL=INFO
  1. Use the maven exec plugin based starter to start the server
    mvn clean compile exec:java@ogc-resource-server
  2. The server will be up on port 8080. To change the port, add httpPort:<desired_port_number> to the config in the ApiServerVerticle module. See configs/config-example.json for an example.

JAR based execution

  1. Install Java 11 and maven
  2. Set Environment variables
export LOG_LEVEL=INFO
  1. Use maven to package the application as a JAR
    mvn clean package -Dmaven.test.skip=true
  2. 2 JAR files would be generated in the target/ directory
    • ogc-resource-server-dev-0.0.1-SNAPSHOT-fat.jar - non-clustered vert.x and does not contain micrometer metrics

Docker based execution

  1. Install docker and docker-compose
  2. Clone this repo
  3. Build the images
    ./docker/build.sh
  4. Modify the docker-compose.yml file to map the config file you just created
  5. Start the server in production (prod) or development (dev) mode using docker-compose
    docker-compose up prod
  6. The server will be up on port 8080. To change the port, add httpPort:<desired_port_number> to the config in the ApiServerVerticle module. See example-config/config-example.json for an example

License

See LICENSE

Issues and Suggestions

Any Release related issues/suggestions, please raise here
For all other issues/suggestions related specific projects, please raise them to the respective project repo/issues.

What's Changed

New Contributors

Full Changelog: v1.0.1...v2.0.0

Contributors

ThorodanBrom, code-akki, and 4 other contributors
Assets 2
Loading