Installation guide
Purpose
This page is intended for audiences who wants to run OS2iot on their own machine to test it out, or deploy to a server for a production environment.
Running locally using docker-compose
Prerequisites
docker version version 19.03.13 (or above)
docker-compose version 1.27.4 (or above)
Configuration
If the application needs to be accessable from non-localhost machines then modify the
environment.prod.ts
file inOS2IoT-frontend/src/environments
folder.This changes the location the frontend accesses the backend on, so this URL should be the way another computer would reach the computer running docker-compose.
If callbacks from Sigfox is required then the backend needs to know what URL to configure Sigfox to callback to.
This is done by setting the environment variable
BACKEND_BASEURL
in the runtime environment for the backend.In docker-compose this can be set in the
docker-compose.yml
file.I.e.
BACKEND_BASEURL: "https://os2iot.dk"
Steps
Clone the three repositories into the same folder, such that the structure is as follows:
OS2IoT
├── OS2IoT-backend (https://github.com/OS2iot/OS2IoT-backend)
├── OS2IoT-docker (https://github.com/OS2iot/OS2IoT-docker)
├── OS2IoT-frontend (https://github.com/OS2iot/OS2IoT-frontend)
Make sure that Chirpstacks configuration files for initializing the postgres database is in unix format, such that the docker container can run it
This can be done with dos2unix or another tool
dos2unix OS2IoT-docker/configuration/postgresChirpstackV4/initdb/001-init-chirpstack.sh OS2IoT-docker/configuration/postgresChirpstackV4/initdb/002-chirpstack_extensions.sh
Build the docker containers using docker-compose
navigate to the OS2IoT-docker folder in your terminal
Run
docker-compose build
Run the docker containers using docker-compose
Run
docker-compose run
Use the
-d
flag to start it in the background.
Access the frontend on http://localhost:8081/auth’
If this instance of OS2IoT should be accessible from other machines, then naturally the relevant ports should be opened in the firewall(s).
Relevant ports:
Frontend: 8081
Backend: 3000
Chirpstack: 8080
Chirpstack Gateway (UDP from gateways to Chirpstack): 1700
Troubleshooting
Error: Issue connecting to Chirpstacks PostgreSQL.
Fix: Goto os2iot-docker in your a terminal
Run:
docker volume ls
Located the image name os2iot-docker_postgresqldata and delete it by running:
docker-compose down
docker volume rm os2iot-docker_postgresqldata
Add os2iot-docker to docker files share (Described in the quick setup). Once the path is added run:
docker-compose up
More docker related troubleshooting can be found at: https://github.com/OS2iot/OS2IoT-docker#troubleshooting-faq
Security
OS2IoT only supports tls 1.2+. It is however recommended to only use tls 1.3.
Running in Kubernetes
During the development of OS2IoT, Azure was used for hosting the solution.
Helm was used for configuration, the Helm charts used can be found in the OS2IoT-docker/helm
folder.
Disclaimer
This configuration is not intended for a production setup. There is no persistent volumes for the database(s), Redis, Kafka etc. so data will not be persisted when using this configuration. There is no log gathering in this setup. These should at least be resolved before deploying OS2iot in a production setup.
Setting up an environment
During development the following was used:
Azure AKS to run Kubernetes.
Azure ACR to hold our docker images.
Azure DNS to manage DNS.
Azure Load balancer to load balancing the traffic to Azure AKS.
Both TCP (HTTP) traffic for web-browsers, and HTTP callbacks.
… and UDP traffic to chirpstack-gateway-bridge on port 1700 in a separate loadbalancer.
Azure VM to host Jenkins.
The exact steps will depend on the requirements for the exact deployment, and therefore it is left as an exercise for the reader.
Deployment
Jenkins was used for deployment, the deploy used the following shell script to perform the deploy. Sensitive information have been redacted.
#!/bin/sh
set -xe
az login --service-principal --username redacted --password redacted --tenant redacted
az acr login --name os2iot
# Build containers
sed -i "s/baseUrl: 'http:\/\/localhost:3000\/api\/v1\/'/baseUrl: 'https:\/\/${namespace}-os2iot-backend.os2iot.dk\/api\/v1\/'/" OS2IoT-frontend/src/environments/environment.prod.ts
# Replace BACKEND_BASEURL for backend:
sed -i "s/'https:\/\/test-os2iot-backend.os2iot.dk'/'https:\/\/${namespace}-os2iot-backend.os2iot.dk'/" OS2IoT-docker/helm/charts/os2iot-backend/templates/deployment.yaml
if $USE_DOCKER_BUILD_CACHE; then export OPTIONAL_ARGS=""; else export OPTIONAL_ARGS="--no-cache"; fi
docker build $OPTIONAL_ARGS -t os2iot-backend:${BUILD_NUMBER} ./OS2IoT-backend
docker build $OPTIONAL_ARGS -t os2iot-frontend:${BUILD_NUMBER} -f ./OS2IoT-frontend/Dockerfile-prod ./OS2IoT-frontend
# Tag and push to ACR
docker tag os2iot-backend:${BUILD_NUMBER} os2iot.azurecr.io/os2iot-backend:${BUILD_NUMBER}
docker push os2iot.azurecr.io/os2iot-backend:${BUILD_NUMBER}
docker tag os2iot-frontend:${BUILD_NUMBER} os2iot.azurecr.io/os2iot-frontend:${BUILD_NUMBER}
docker push os2iot.azurecr.io/os2iot-frontend:${BUILD_NUMBER}
# Setup right private key for KOMBIT
if [ "${namespace}" = "test" ]; then export PRIVATEKEY="-----BEGIN PRIVATE KEY-----\nM-REDACTEDoP\n-----END PRIVATE KEY-----"; fi
if [ "${namespace}" = "demo" ]; then export PRIVATEKEY="-----BEGIN PRIVATE KEY-----\nM-REDACTEDoP\n-----END PRIVATE KEY-----"; fi
if [ "${namespace}" = "test" ]; then export ENTRYPOINT="https://adgangsstyring.eksterntest-stoettesystemerne.dk/runtime/saml2/issue.idp"; fi
if [ "${namespace}" = "demo" ]; then export ENTRYPOINT="https://adgangsstyring.stoettesystemerne.dk/runtime/saml2/issue.idp"; fi
# Create namespace or not
NOT_EXISTS=`kubectl get po -n ${namespace} 2>&1 | grep "No resources" | wc -l`
if [ "$NOT_EXISTS" = "1" ]; then kubectl create namespace ${namespace}; fi
# Helm deploy
cat <<EOT >> OS2IoT-docker/helm/values.yaml
os2iot-backend:
DOCKER_IMAGE_TAG: $BUILD_NUMBER
KOMBIT_CERTIFICATEPRIVATEKEY: '$PRIVATEKEY'
KOMBIT_ENTRYPOINT: '$ENTRYPOINT'
os2iot-frontend:
DOCKER_IMAGE_TAG: $BUILD_NUMBER
EOT
helm upgrade --install os2iot ./OS2IoT-docker/helm --namespace ${namespace}
Configuration
OS2IoT-backend
OS2IoT-backend takes several environment variables as configuration, if these are not set a default will be used.
Environment variable |
Purpose |
Default value |
---|---|---|
PORT |
Port to run the backend on. |
|
DATABASE_HOSTNAME |
Hostname to connect to Postgresql on |
|
DATABASE_PORT |
Port to connect to Postgresql on |
|
DATABASE_USERNAME |
Username for Postgresql |
|
DATABASE_PASSWORD |
Password for Postgresql |
|
DATABASE_ENABLE_SSL |
Enable SSL for database connection |
|
JWT_SECRET |
Secret value to sign JWT (THIS SHOULD BE CHANGED!) |
|
JWT_EXPIRESIN |
Time to expiry for the JWT tokens used |
|
BACKEND_BASEURL |
URL for external services to connect to the backend (THIS SHOULD BE CHANGED!) |
|
FRONTEND_BASEURL |
URL for the frontend, used when building email links (THIS SHOULD BE CHANGED!) |
|
EMAIL_HOST |
URL for the SMTP server, used when sending emails (THIS SHOULD BE CHANGED!) |
|
EMAIL_PORT |
Port for the email server, used when sending emails (THIS SHOULD BE CHANGED!) |
|
EMAIL_USER |
Username for email server authentification, used when sending emails (THIS SHOULD BE CHANGED!) |
|
EMAIL_PASS |
Password for email server authentification, used when sending emails (THIS SHOULD BE CHANGED!) |
|
EMAIL_FROM |
Email sender address. Either a plain address or a display name and the address (CHANGE IT!) |
|
KOMBIT_ENTRYPOINT |
The context broker URL for KOMBIT adgangsstyring |
|
KOMBIT_CERTIFICATEPRIVATEKEY |
The certificate private key for KOMBIT adgangsstyring |
|
KOMBIT_CERTIFICATEPUBLICKEY |
Public certificate from the KOMBIT idp for verifying SAML response |
|
KOMBIT_ROLE_NAME |
This string must be a substring of the brugersystemrolle you grant users for them to be given access |
|
LOG_LEVEL |
Minimum Log Level. Levels ordered from high to low are: ‘log’, ‘error’, ‘warn’, ‘debug’, ‘verbose’ |
|
METADATA_SAVED_COUNT |
Maximum number of message metadata to store from an IoT device |
|
MQTT_BROKER_HOSTNAME |
The hostname of the MQTT broker. |
|
MQTT_SUPER_USER_PASSWORD |
The password for the internal MQTT listener. |
|
ENCRYPTION_SYMMETRIC_KEY |
A symmetric key that is used for encrypting |
|
CA_KEY_PASSWORD |
The password for the Certificate Authority key. |
|
CHIRPSTACK_API_KEY |
The API key (Bearer token) created in Chirpstack. |
|
CHIRPSTACK_HOSTNAME |
Hostname for Chirpstack |
|
CHIRPSTACK_PORT |
Chirpstack port |
|
Logs levels
Specifying a LOG_LEVEL makes sure that only logs with that level or higher are included. Using ‘debug’ or ‘verbose’ LOG_LEVEL in a production environment is not recommended.
OS2IoT-frontend
The frontend can also be configured using environment variables. If these are not set a default will be used.
Defaults are set in OS2IoT-frontend/src/environments/environment.ts
Environment variable |
Purpose |
Default value |
---|---|---|
PRODUCTION |
If true, then Angular is set in production mode, disabling debugging features |
|
BASE_URL |
The Url which users will connect to the backend from. This must be changed for the system to work externally |
|
TABLE_PAGE_SIZE |
Default page size of tables |
|
DAF_USERNAME |
Username to https://datafordeler.dk |
|
DAF_PASSWORD |
Password to https://datafordeler.dk |
|
OS2IoT-Mosquitto broker
To get the mosquitto broker working, you have to create some certificates and update some values. These following steps is done with Windows. If you use linux, then write sudo
before the commands.
Prerequisites: openssl installed and accesible from path
Generate files:
Open the command prompt in administrator mode.
Create a certificate authority(CA) key with this command:
openssl genrsa -des3 -out ca.key 2048
. You will be prompted to enter a password. It’s very important that you save this password, since it will be used later.Create the CA certificate with this command:
openssl req -new -x509 -days 1826 -key ca.key -out ca.crt
. You will be asked to enter the password from the step before. After this, you will be prompted to enter informations. These values are not important, except one: “Common name”. Common name HAS to be the ip/hostname of your broker.Create the server key (for the broker) with the command:
openssl genrsa -out server.key 2048
Create the server signing request with the command:
openssl req -new -out server.csr -key server.key
. You will be prompted to enter some informations. These values are not important, except one: “Common name”. Common name HAS to be the ip/hostname of your broker. The rest of the values should not be exact the same as in step 3.Create the server certificate (that is signed by the CA) with this command:
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 360
. You will be prompted to enter the password from step 3.
If you want to get docker container with mosquitto running, then follow these steps:
Place the generated files, ca.key, ca.crt, server.key and server.crt from the above steps in the folder “OS2IoT-docker/configuration/mosquitto-broker-os2iot”. You don’t need the server.csr.
Open the mosquitto-os2iot.conf file placed in OS2IoT-docker/configuration/mosquitto-broker-os2iot in a text editor and update the values to match your database.
Copy the files ca.crt and ca.key and place them in OS2IoT-backend/resources.
Update the
MQTT_BROKER_HOSTNAME
with the ip/hostname that you used for step 3 and 5, andCA_KEY_PASSWORD
with the password that you entered in step 2 in the docker-compose.yml file placed in OS2IoT-docker.
If you want to use kubernetes to host mosquitto then you need some futher steps.
Prerequisites: kubectl installed and accesible from path
Open a command prompt in administrator mode.
Create a secret with the server.key and server.crt with the command:
kubectl create secret generic server-keys --from-file=server.key=path/to/server.key --from-file=server.crt=path/to/server.crt
. Replace path/to/ with the path to your server.key and server.crt, created in the steps above.Create a secret with the ca.crt and ca.key with the command:
kubectl create secret generic ca-keys --from-file=ca.crt=path/to/ca.crt --from-file=ca.key=path/to/ca.key
. Replace path/to/ with the path to your server.key and server.crt, created in the steps above.Update the empty values in OS2IoT-docker/helm/charts/mosquitto-os2iot/values.yaml
Update the
MQTT_BROKER_HOSTNAME
with the ip/hostname that you used for step 3 and 5 in the steps above, andCA_KEY_PASSWORD
with the password that you entered in step 2 in the steps above, in the file “OS2IoT-docker/helm/charts/os2iot-backend/deployment.yaml”.