3. Administration

The BI Explorer is a python web application that runs against a PostgreSQL server. The database connection parameters are set in the explorer configuration file.

3.1. Explorer Configuration

The configuration of the main Explorer application is stored in a YAML file. Separately from this file, there is a directory with filters, presets, projections and reports.

3.1.1. Sessions

Server side caches are tied to users using Web sessions. Users receive a user specific http-cookie that facilitates quick continuation of a session without additional login. SECRET_KEY is an arbitrary secret that is used to encode the session data. It should be set to a new value for every explorer installation.

3.1.2. Program caches

The explorer uses program caches to store information that is used often.

The short_term cache is used to cache distinctlists, which are used to enumerate possible filter values found in the explorer table.

The platform_api cache is used to cache shared preset results typically used for dashboards. Using this cache data can already be calculated and stored before a user requests it.

3.1.3. User security

User passwords are salted, peppered and hashed using sha512 to the database. The salt is randomly generated and stored per user. The SECRET_KEY is used to encode data in the cookie before sending it to the user.

Make sure SECURITY_PEPPER, SECRET_KEY are set differently per explorer installation.

The JSON Web Token settings (SECURITY_JWT_*) are part of the Platform API as used by the MGRID Dashboard, so these should match on both sides. The SECURITY_PLATFORM_USER should match the subject in the JWT.

User roles are obtained during login (e.g., using OAuth2). A role determines thw following access:

  • SECURITY_PII_ROLES: which roles are allowed to see personally identifiable information (PII).

  • SECURITY_ADMIN_ROLES: which roles are allowed to change configuration using the configadmin page.

  • SECURITY_GROUP_MANAGEMENT_ROLES: which roles are allowed to create and manage group presets.

  • SECURITY_PRESET_MANAGEMENT_ROLES: which roles are allowed to create and manage public presets.

3.2. SSL configuration

The nginx software is used to expose the web application externally. By default the Python webcontainer is configured to only serve on Additional configuration in /etc/nginx/nginx.conf:

user              nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;

    keepalive_timeout  65;

    include /etc/nginx/conf.d/*.conf;

    ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;

With /etc/nginx/conf.d/explorer.conf:

upstream explorer-site {

server {
    listen 443 ssl;
    server_name  explorer.outsidename.nl;
    ssl_certificate /etc/ssl/explorer/server.crt;
    ssl_certificate_key /etc/ssl/explorer/server.key;

    location / {
        proxy_set_header        Host $http_host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;

        client_max_body_size    10m;
        client_body_buffer_size 128k;
        proxy_connect_timeout   60s;
        proxy_send_timeout      90s;
        proxy_read_timeout      90s;
        proxy_buffering         off;
        proxy_temp_file_write_size 64k;
        proxy_pass http://explorer-site;
        proxy_redirect          off;

3.3. User Management

User management is done using the command line tool useradmin.py. This tool determines database location using a supplied configuration file, and can be instructed to add, remove and list users. There is a facility to change a stored password for a user, and to reset the locked views for a user to the views as set in config/views.py.

$ python useradmin.py -h
usage: useradmin.py [-h] (-c CONFIG | -d host port name user)
                    (-i | -s schema | -a user group schema password | -r user schema | -p user schema password | -l | -x schema | -G user schema role | -R user schema role)

Administration tool for Explorer user table

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        explorer configuration file
  -d host port name user, --database host port name user
                        explicit database settings
  -i, --init            create and initialize explorer tables
  -s schema, --schema schema
                        create and initialize user schema
  -a user group schema password, --add user group schema password
                        add a user
  -r user schema, --remove user schema
                        remove a user
  -p user schema password, --password user schema password
                        change a stored password
  -l, --list            show the known users
  -x schema, --resetpreset schema
                        reset user presets
  -G user schema role, --grant user schema role
                        grant role to a user
  -R user schema role, --revoke user schema role
                        revoke role from a user