How to Install

The user interface allows to run and edit pipelines. It is used through the browser, but doesn’t need to be installed on a web server. This can all happen locally. The code for the scripts, however, is not edited through the UI but rather with a standard text editor such as VS Code or RStudio.

Video tutorial

Deploying BON in a Box locally

BON in a Box can be installed and ran on a local computer. While personal computers have less computing power than servers, it is more convenient for pipeline development, or when working with small examples. Users adding scripts and creating pipelines on their local computer can directly run them through BON in a Box.

Prerequisites:

  • A GitHub account, and git installed on your computer

  • At least 6GB of free space (RAM requirements depend on the script that is being run)

  • A linux shell

    • This can be Terminal on a Mac or Git Bash on a PC

Step 1: Install Docker

  • Install Docker here

    • Note: It is not necessary to make an account
    • To test whether docker is added to the path of your terminal, run docker run --rm hello-world
    • For Mac users: Make sure to check the chip of your machine and download the correct version.
    • For linux users: Make sure to add your user to the docker group

Step 2: Generate SSH Key

  1. First, make sure that you do not already have an SSH key registered to your computer by running ls -al ~/.ssh in your terminal.

    • If you already have a key it will be called one of the following:

      • id_rsa.pub

      • id__ecdsa.pub

      • id_ed25519.pub

  2. If you do not already have a key, generate one by running ssh-keygen ed25519 -C "your_email@example.com" replacing the email with the email address associated with your GitHub account.

    • This will prompt you to make a password. We recommend skipping this because it becomes a hassle. You can keep pressing enter to skip these steps.

  3. To check if you successfully generated the SSH key, you can run ls -al ~/.ssh in the terminal. Your SSH key should be there now.

  4. Now, open the file with your SSH key by running cat ~/.ssh/<name of SSH key> e.g. cat ~/.ssh/id_ed25519.pub and copy your SSH key.

  5. Now, add your SSH key to your GitHub account by going to GitHub settings > SSH and GPG keys and “New SSH key”. You can name this whatever you want and paste your SSH key into the “Key” field.

Step 3: Clone repository

  1. Clone repository (Windows users: do not clone the repo in a folder under OneDrive.) This can be done in terminal using the following code: git clone git@github.com:GEO-BON/bon-in-a-box-pipelines.git or in GitHub desktop.

Step 4: Set environment file

  1. Provide the environment variables: - Open the newly cloned repository on your computer

    1. Find the file called runner-sample.env

    2. Duplicate the file and rename the copy to runner.env.

    3. Fill the properties depending on what you intend to run. Include any API keys that you need to access data (e.g. GBIF or IUCN)

    4. Adjust any server option as you see fit.

  2. Using a linux terminal (terminal on Mac or Git Bash), navigate to the folder of the cloned repository.

  3. In the linux terminal, type ./server-up.sh

    • Make sure the server is started from a linux terminal that is in the same folder as the cloned repository so it can access the files.

    • Make sure you have docker open and running on your computer.

    • The first execution will be long, in order to download the micro-services. The next ones will be shorter or immediate, depending on the changes.

    • Network problems may cause the process to fail. First try running the command again. Intermediate states are saved so not everything will be redone even when there is a failure.

    • Windows users may need to turn on virtualization and other tools for Docker Desktop to work and update wsl (“wsl –update”, see https://docs.docker.com/desktop/troubleshoot/topics/#virtualization. Access to the BIOS may be required to enable virtualization)

Step 5: Connect to server

  1. Type http://localhost/ to open BON in a Box

  2. Run ./server-down.sh in a terminal to stop the server when done

  3. On Windows, to completely stop the processes, you might have to run wsl --shutdown

When modifying scripts in the /scripts folder, servers do not need to be restarted: - When modifying an existing script, simply re-run the script from the UI and the new version will be executed. - When adding or renaming scripts, refresh the browser page.

When modifying pipelines in the /pipelines folder, servers do not need to be restarted: - In the pipeline editor, click save, paste the file to your file in the pipeline folder and run it from the “pipeline run” page. - When adding or renaming pipelines, refresh the browser page.

Deploying without pulling the new server

If the server has already been deployed at least once, and low bandwith does not allow to download the latest version (this can be a few gigabytes), the server can be started with option ./server-up.sh --offline. Use with caution, since this may create errors if elements of the repository depend on new server features.

Deploying BON in a Box on a server

Installation steps on a server are the same as installing on a Linux computer, but some additional configurations need to be changed:

  • In runner.env, choose the appropriate HTTP_ADDRESS and HTTP_PORT for your server configuration. In a typical installation, use 0.0.0.0 as an HTTP address and default port (80).

  • In runner.env, select the “partial” cache cleaning mode. This will allow for calculated results to be long-lived on the server, as long as they are not run again with a new version of the script. The results in the output folder are always accessible by URL.

  • In the case where a server is used for demonstration purpose only, set environment variable BLOCK_RUNS=true

  • The outputs are precious, while the server machine can be re-generated anytime. Consider mounting an external backed-up drive to the output folder of your BON in a Box installation.

  • By default, BON in a Box listens for http. To enable https, we hide it behind a reverse proxy and activate https with certbot. How to do this is out of scope for this readme.

Running a script or pipeline

You now have an instance of BON in a Box deployed, either locally or remotely, and you want to run your first script or pipeline.

There is one page to run scripts, and one to run pipelines. Once there, select the script or pipelines from the dropdown and fill the form.

The form might ask you for a file. In order to provide a file that you own locally, upload or copy it to the userdata folder. You can then refer to it with a url as such: /userdata/myFile.shp, or /userdata/myFolder/myFile.shp if there are subfolders.