This repository presents the reference implementation of a product data server as part of the product data network buildingenvelopedata.org. Before deploying this repository, machine can be used to set up the machine.
The API specification of the product data servers is available in the repository api. There is also a visualization of the API of a product data server.
This repository is deployed as the product data server of TestLab Solar Facades of Fraunhofer ISE.
If you have a question for which you don't find the answer in this repository, please raise a new issue and add the tag question
! All ways to contribute are presented by CONTRIBUTING.md. The basis for our collaboration is decribed by our Code of Conduct.
- Open your favorite shell, for example, good old
Bourne Again SHell, aka,
bash
, the somewhat newer Z shell, aka,zsh
, or shiny newfish
. - Install Git by running
sudo apt install git-all
on Debian-based distributions like Ubuntu, orsudo dnf install git
on Fedora and closely-related RPM-Package-Manager-based distributions like CentOS. For further information see Installing Git. - Clone the source code by running
git clone git@github.com:building-envelope-data/database.git
and navigate into the new directorydatabase
by runningcd database
. - Initialize, fetch, and checkout possibly-nested submodules by running
git submodule update --init --recursive
. An alternative would have been passing--recurse-submodules
togit clone
above. - Prepare your environment by running
cp .env.sample .env
,cp frontend/.env.local.sample frontend/.env.local
, and adding the line127.0.0.1 local.solarbuildingenvelopes.com
to your/etc/hosts
file. - Install Docker Desktop, and GNU Make.
- List all GNU Make targets by running
make help
. - Generate and trust a self-signed certificate authority and SSL certificates
by running
make ssl
. If you are locally working the the metabase and the database and if you need them to communicate over HTTPS, then instead of runningmake ssl
, make theCERTIFICATE_AUTHORITY_*
variable values in the.env
file match the ones from the metabase, copy the certificate authority files from the directories./ssl
,./backend/ssl
, and./frontend/ssl
of the metabase project into the respective directories in the database project, and run the commandmake generate-ssl-certificate
. - Generate JSON Web Token (JWT) encryption and signing certificates by running
make jwt-certificates
. - Start all services and follow their logs by running
make up logs
. - To see the web frontend navigate to
https://local.solarbuildingenvelopes.com:5051
in your web browser, to see the GraphQL API navigate tohttps://local.solarbuildingenvelopes.com:5051/graphql/
, and to see sent emails navigate tohttps://local.solarbuildingenvelopes.com:5051/email/
.
In another shell
- Drop into
bash
with the working directory/app
, which is mounted to the host's./backend
directory, inside a fresh Docker container based on./backend/Dockerfile
by runningmake shellb
. If necessary, the Docker image is (re)built automatically, which takes a while the first time. - List all backend GNU Make targets by running
make help
. - For example, update packages and tools by running
make update
. - Drop out of the container by running
exit
or pressingCtrl-D
.
The same works for frontend containers by running make shellf
.
For information on using Docker in production see Configure and troubleshoot the Docker daemon and the pages following it.
- Use the sibling project machine and its instructions for the first stage of the set-up.
- Enter a shell on the production machine using
ssh
. - Change into the directory
/app
by runningcd /app
. - Clone the repository twice by running
for environment in staging production ; do git clone git@github.com:building-envelope-data/database.git ./${environment} done
- For each of the two environments staging and production referred to by
${environment}
below:- Change into the clone
${environment}
by runningcd /app/${environment}
. - Prepare the environment by running
cp .env.${environment}.sample .env
,cp frontend/.env.local.sample frontend/.env.local
, and by replacing dummy passwords in the copies by newly generated ones, where random passwords may be generated runningopenssl rand -base64 32
. - Prepare PostgreSQL by generating new password files by running
make --file Makefile.production postgres_passwords
and creating the database by runningmake --file Makefile.production createdb
.
- Change into the clone
- Draft a new release with a new version according to
Semantic Versioning by running the GitHub action
Draft a new release
which, creates a new branch named
release/v*.*.*
, creates a corresponding pull request, updates the Changelog, and bumps the version inpackage.json
, where*.*.*
is the version. Note that this is not the same as "Draft a new release" on Releases. - Fetch the release branch by running
git fetch
and check it out by runninggit checkout release/v*.*.*
, where*.*.*
is the version. - Prepare the release by running
make prepare-release
in your shell, review, add, commit, and push the changes. In particular, migration and rollback SQL files are created in./backend/src/Migrations/
which need to be reviewed --- see Migrations Overview and following pages for details. - Publish the new release
by merging the release branch into
main
whereby a new pull request frommain
intodevelop
is created that you need to merge to finish off.
- Enter a shell on the production machine using
ssh
. - Change to the staging envrionment by running
cd /app/staging
. - Deploy the new release in the staging environment by running
make --file Makefile.production deploy
. - If it fails after the database backup was made, rollback to the previous
state by running
make --file Makefile.production rollback
, figure out what went wrong, apply the necessary fixes to the codebase, create a new release, and try to deploy that release instead. - If it succeeds, deploy the new reverse proxy that handles sub-domains by
running
cd /app/machine && make deploy
and test whether everything works as expected and if that is the case, repeat all stages but this one in the directory/app/production
(instead of/app/staging
). Note that in the staging environment sent emails can be viewed in the web browser underhttps://staging.solarbuildingenvelopes.com/email/
and emails to addresses in the variableRELAY_ALLOWED_EMAILS
in the.env
file are delivered to the respective inboxes (the variable's value is a comma separated list of email addresses).