This is an implementation of an International Data Spaces (IDS) Metadata Broker extended to handle metadata of a Data App, which is a registry for IDS Connector self-description documents and Metadata of the Data App. It is currently under development and intends to act as a central Metadata Registry for members of Platoon project.
The Open-source Metadata Registry has been funded by the PLATOON H2020 project funded by the EU commission..
The goal of this implementation is to show how the concepts introduced in the Handshake Document (currently restricted to IDSA members) can be turned into an actual application. It, therefore, demonstrates the usage of the IDS Information Model for core communication tasks. More specifically, it shows:
- Implementation of the messaging interfaces for IDS infrastructure-level communication,
- Information flow of typical interactions with the Metadata Registry.
Security is currently supported in terms of TLS via a reverse proxy.
broker-core: The main Maven Artifact to start with.
open-broker-common: Shared code which broker-core requires.
open-index-common: Shared functionalities not only for open-broker-common but also for further IDS index services (for instance ParIS).
docker: Docker (installation guide) and DockerCompose (installation guide) files to deploy the IDS Metadata Registry.
In this section, we will provide some guidance as to recommendations for the number of resources that should be available to smoothly operate the IDS Metadata Registry. The actual number of resources required heavily depends on the load. In case of very little traffic, fewer resources than listed below might be required.
2GB of disk space is required for operating the Metadata Registry, though we recommend providing at least 20GB of free disk storage to avoid running out of disk space with increasing number of registered items. To provide enough processing power for all Docker containers, we recommend using a 64bit quad core processor or better.
- OS: We recommend using a Linux based operating system. However, any operating system with a Docker installation can be used (tested on Ubuntu 20.04 and Windows 10). More strict hardware requirements than listed above might apply if a non-Linux operating system is used.
- Docker: version 20.10.7 or later
- Docker Compose: version 1.29.1 or later
- OpenSSL: Version 1.1.1k or later. A valid X.509 certificate, signed by a trusted certification authority, is strongly recommended to avoid warnings about insecure HTTPS connections. Docker must be installed on the target machine.
- Java: Java 11 or later should be installed in your local environment to build the docker image.
- Maven: Maven 3.6.3 or later should be installed in your local environment to build the docker image (execute
mvn -versionto check the successful installation).
This part aims to aid IT administrators or developers in the installation of the Metadata Registry. Metadata Registry is still actively maintained by Fraunhofer IAIS. If any problem arises while following the installation guide, please get in touch with the email provided at the end of this file.
For the SSL certificate, you need to have these two files:
- server.crt: an x509 certificate, either self-signed or from an official CA
- server.key: the private key for the certificate.
The certificate needs to be of .crt format and must have the name server.crt and the file for private key should have the name server.key. Here is a sample command to create a self-signed certificate, which requires OpenSSL to be installed:
openssl req -newkey rsa:4096 \
-x509 \
-sha256 \
-days 3650 \
-nodes \
-out server.crt \In case your certificate is in .pem format, it can be converted with the following commands:
openssl x509 -in mycert.pem -out server.crt
openssl rsa -in mykey.pem -out server.key
mkdir cert
mv server.crt cert/
mv server.key cert/
To run the Metadata Registry you can either make use of docker images provided by us as shown in Section 4.2.1 or build your customized docker as shown in Section 4.2.2.
If you want to run the Metadata Registry with the provided image please follow the following steps:
Step 1: Clone the repository
git clone https://github.com/PLATOONProject/Metadata-Registry.git
Step 2: Configure the docker-compose file
Once the repository is cloned, the docker-compose file will be found in this path:
`./docker/composefiles/broker-localhost/docker-compose.yml`
Step 2.1 Please put the SSL certificate (server.key and server.crt) and the DAPS certificate, for example "isst-broker.jks"; under the same folder which will be mounted in container.
Step 2.2 Provide the correct location of the X.509 certificate created above in the "broker-reverseproxy" and "broker-core" services.
For Linux users: if the location of the certificate is “/home/ids/cert”, the corresponding configuration in the yml file is:
services: broker-reverseproxy:
image: registry.gitlab.cc-asp.fraunhofer.de/eis-ids/metadata-registry/reverseproxy:1.0.2
volumes:
- /home/ids/cert:/etc/cert/
[…]
For Windows users: if the location of the certificate is “c:/etc/ids/cert”, the corresponding configuration in the yml file is:
services: broker-reverseproxy:
image: registry.gitlab.cc-asp.fraunhofer.de/eis-ids/metadata-registry/reverseproxy:1.0.2
volumes:
- c:/etc/ids/cert:/etc/cert/
[…]
Do the same for "broker-core" service.
Step 2.3 After successful completion of the previous steps, the location of the certificates is mounted to "/etc/cert" of the container. If the name of your DAPS certificate is "isstbroker-keystore.jks", please change the following line in the docker-compose file:
environment:
[…]
- IDENTITY_JAVAKEYSTORE=/etc/cert/isstbroker-keystore.jks
Please note: only adapt the name of your certificate in the line.
Step 3: Download the docker images
All the IDS Metadata Registry Docker images are hosted at the GitLab of Fraunhofer IAIS. No credentials needed to download the images. The following command is for pulling all docker images:
docker-compose pull
Note that this command should be executed in the same path of docker-compose.yml file.
Step 4: Start up the IDS Metadata Registry
To start up the IDS Metadata Registry, run the following command inside the directory of the docker-compose.yml file:
docker-compose up –d
This process can take several minutes to complete. You can test whether the IDS Metadata Registry has successfully started by opening https://localhost. The result should be a JSON document, providing some general metadata about the IDS Metadata Registry.
Furthermore, the docker-compose logs command can be used to access the logs for a docker-compose.yml file, see here.
Step 5: Stop the IDS Metadata Registry
To stop the Metadata Registry, run the following in the terminal in the same path as the docker-compose.yml file:
docker-compose down
Step 6: Update the IDS Metadata Registry
To update an existing installation of the IDS Metadata Registry, first repeat the steps explained in Step 3. Containers can be either hot updated or restarted to apply the changes. To hot update a container, run the following command:
docker-compose up -d --no-deps --build <container name>
Alternatively, one can restart the entire service by running:
docker-compose down
docker-compose up –d
- Maven - Dependency Management
- Spring Boot - Application Framework
- Apache Jena - Parsing and serializing RDF and Fuseki as triple store for meta data
- Fraunhofer IAIS: Najmehsadat Mousavinezhad, Tasneem Tazeen Rashid
- or create an issue