O2K-Connector is a lightweight connector, built in Python, that enables context data sharing from Orion Context Broker towards Apache Kafka. In more detail, O2K-Connector subscribes to Orion Context Broker and publishes the received context-data to a built-in kafka topic.
 quay.io |
🐳 Docker Hub |
|---|
O2K-Connector can be fully customized and configured by the means of environment variables listed on o2k.env file.
| Environment variables | Description |
|---|---|
| LOG_LEVEL | Level of O2K Debug. Possible values: INFO, DEBUG, ERROR |
| LOGS_PATH | Path of O2K-Connector logs |
| O2K_HOST | Listener for O2K-Connector |
| O2K_PORT | Port of O2K-Connector |
| SCHEMA_URL | Kafka Schema url for avro messages |
| BOOTSTRAP_SERVERS | Kafka Boostrap Servers |
| KAFKA_TOPIC | User-specified Kafka Topic, if left blank, it will be auto-generated on the fly taking into account FIWARE_SERVICE and FIWARE_SERVICEPATH environments for NGSI V2 and entity id for NGSI-LD |
| ORION_HOST | Hostname of Orion Context Broker |
| ORION_PORT | Port of Orion Context Broker |
| FIWARE_SERVICE | Fiware Service for NGSI V2 |
| FIWARE_SERVICEPATH | Fiware ServicePath for NGSI V2s |
| MQTT_HOST | Hostname of MQTT Server, if used |
| MQTT_PORT | Port of MQTT Server, if used |
| MQTT_TOPIC | Topic to which publish on MQTT Server, if used |
| SUBSCRIPTION_JSON_PATH | Path of subscription configuration files |
| SUBSCRIPTION_JSON_FILENAME | Filename of subscription configuration file for NGSI V2 subscription |
| SUBSCRIPTION_JSON_FILENAME_LD | Filename of subscription configuration file for NGSI-LD subscription |
| SUBSCRIPTION_JSON_FILENAME_MQTT | Filename of subscription configuration file for NGSI-V2 subscription and MQTT underlying subscription engine |
| SUBSCRIPTION_SCHEMA_FILE_PATH | Filename of subscription schema file for NGSI V2 subscription |
| SUBSCRIPTION_SCHEMA_FILE_PATH_LD | Filename of subscription schema file for NGSI-LD subscription |
| SUBSCRIPTION_SCHEMA_FILE_PATH_MQTT | Filename of subscription schema file for NGSI-V2 subscription and MQTT underlying subscription engine |
| ORION_VERSION | Orion Context Broker version. Possible values: V2, LD |
| ORION_SUBSCRIPTION | Orion Context Broker subscription engine. Possible values: http, mqtt |
| MULTIPLE_SUBSCRIPTIONS | For NGSI V2 only: set to "false" to handle a unique subscription with multiple attributes |
| KAFKA_ENABLE_SSL | Set to "true" if want to use SSL connection to Kafka |
| KAFKA_SSL_CA | Certificate Authority key (path file inside the container) to use for SSL connection to Kafka |
| KAFKA_SSL_KEY | Client key (path file inside the container) to use for SSL connection to Kafka |
| KAFKA_SSL_CERTIFICATE | Certificate key (path file inside the container) to use for SSL connection to Kafka |
To instantiate O2K-Connector you can use docker-compose which will take care of the creation of the container running python. Alternatively you can locally run the connector by running the python script.
N.B:
The connector will create a Kafka Topic structured as follows:
{FIWARE_SERVICE}_{FIWARE_SERVICEPATH*}_{NGSI_ENTITY_ID}_{NGSI_ENTITY_TYPE}
*FIWARE_SERVICEPATH is deprived of '/' character
- Configure subscription payload configuration 'subscription.json' inside "conf" folder.
- Configure your environment variables in the 'docker-compose.yml' file
- Run docker-compose command into project root folder:
docker-compose up| HTTP Method | Service | Description |
|---|---|---|
| POST | /notify | V2 Context data ingestion route |
| POST | /notifyld?subscriptionId=urn:ngsi-ld:Type:Entity | LD Context data ingestion route |
To test O2K-Connector, few services are required, in particular:
- Orion Context Broker
- A fully working Kafka Environment
At startup O2K-Connector subscribes to Orion Context Broker which is required to be running. Context-data will start being published to a given Kafka Topic (generate on-the-fly) as soon as possible.
To text context-data consumpion, run kafka-console-consumer command:
kafka-console-consumer --bootstrap-server broker-1:29092,broker-2:29093,broker-3:29094 --topic {FIWARE_SERVICE}_{FIWARE_SERVICEPATH*}_{NGSI_ENTITY_ID}_{NGSI_ENTITY_TYPE}
*FIWARE_SERVICEPATH is deprived of '/' character
O2K-Connector is licensed under
GNU Affero General Public License v3.0