Skip to main content

Testiny Server

Testiny is built entirely on Docker and is designed to be deployed in your own infrastructure. Testiny Server is available for on-premises use.

info

If you're interested in trying out Testiny Server, please contact us via email: [email protected].

Setup​

Testiny Server is shipped in a single docker image, which contains all necessary components.

  1. Pull the image from the provided container registry.
  2. Launch the image as described in Running Testiny Server.
  3. Open the Testiny Server web interface and follow the setup instructions.
  4. Apply your license key to unlock all features.
info

Please note, that with Testiny Server you are solely responsible for creating backups and storing them safely.

System requirements​

We recommend the following minimum system requirements for your Testiny Server installation for 50-100 users:

  • reasonable modern CPU with dual core
  • 4GB Memory

Environment variables​

Testiny Server is configured using environment variables that are passed to the Docker container.
Alternatively, you can put the same settings as KEY=value lines into the file testiny.conf in the storage volume (/testiny/storage/testiny.conf). Values in testiny.conf take precedence over environment variables. Only CLUSTER_ROLE, SECRETS_FILE, TESTINY_API_PROCESSES and NODE_EXTRA_CA_CERTS are read from the environment only.
Testiny Server does not start without the required settings. All optional settings fall back to their defaults.

Important: SERVER_URL has to be set correctly; otherwise, Testiny Server will not work correctly!

Required​

VariableDescriptionExample
SERVER_URLPublic URL for Testiny Serverhttps://testiny.mycompany.org
DBpostgres or sqlite (cluster deployments always use postgres)postgres

Optional​

A - in the "Defaults to" column means that there is no default value.

Show all Testiny Server environment variables
VariableDescriptionDefaults to
PG_HOSTHost of postgres DB127.0.0.1
PG_PORTPort of postgres DB5432
PG_USERUser of postgres DBtestiny
PG_PASSWORDPassword of postgres DB-
PG_MAX_POOLMaximum size of the postgres connection pool per database and process — each API process and the job worker have their own pools20
PG_MAX_POOL_CENTRALMaximum size of the postgres connection pool for the central database per process (from v1.42.2)30
PG_IDLE_TIMEOUTPostgres idle timeout in milliseconds60000
PG_USE_SSLPostgres connection uses SSLfalse
PG_DB_PREFIXAn optional prefix added to database name (should be alphanumeric + underscore, should end in underscore), for example staging_-
SMTP_HOSTHost of SMTP Mailserver127.0.0.1
SMTP_PORTPort of SMTP Mailserver587
SMTP_USERUser of SMTP Mailserver-
SMTP_PASSWORDPassword of SMTP Mailserver-
SMTP_AUTH_SECUREAlways connect with TLS instead of STARTTLS (disable for ports 25 and 587)false
SMTP_REQUIRE_TLSRequire a TLS connection (or STARTTLS)false
SMTP_IGNORE_TLSDon't use STARTTLS, even if the server offers itfalse
TESTINY_MAIL_FROMSender mail address[email protected]
TESTINY_LOG_LEVELdebug,info,warn,errorinfo
TESTINY_LOG_FILE_ENABLEDWhether to write log filestrue
TESTINY_LOG_FILEThe log file to use, relative to /testiny — the %DATE% placeholder for daily file rolling is added if missing. Must be writable on cluster deployments without a mounted storage volume. "Download server logs" only finds log files under storage/logstorage/log/testiny-app-%DATE%.log
TESTINY_LOG_GELF_ENABLEDWhether to also send logs to a GELF-compatible UDP endpoint (e.g. Graylog)false
TESTINY_LOG_GELF_SERVERHost of the GELF endpoint127.0.0.1
TESTINY_LOG_GELF_PORTUDP port of the GELF endpoint12201
TESTINY_API_PROCESSESNumber of API processes in the container, 1 to 8 (at most 2 with sqlite) — read from the environment only (from v1.42.2). Each API process and the job worker open their own database pools, so a container can use up to (TESTINY_API_PROCESSES + 1) × PG_MAX_POOL connections per database (PG_MAX_POOL_CENTRAL for the central database) — make sure PostgreSQL allows enough connections2
TESTINY_JOB_CONCURRENCYNumber of background jobs the job worker runs at the same time (1 or more, from v1.42.2)4
BLOB_MAX_SIZEMax attachment size (in bytes) for file and S3 storage — larger values only apply if your license allows them104857600 (100MiB)
JSON_MAX_REQUEST_SIZEMax request size (in bytes)8388608 (8MiB)
BLOBSTORAGE_S3Set to primary to switch from file to S3 blob storage (set BLOBSTORAGE_FILE to disabled if not needed), cluster deployments always use S3disabled
BLOBSTORAGE_S3_ENDPOINTThe S3 endpoint URL to usehttps://my-s3-service.corp.internal/
BLOBSTORAGE_S3_REGIONS3 region to use (can be empty)
BLOBSTORAGE_S3_BUCKETThe name of the bucket to use (Note that some storage systems require the bucket to be also specified in the endpoint URL)
BLOBSTORAGE_S3_ACCESSKEYThe access key for the S3 bucketABCDEFDUMMY
BLOBSTORAGE_S3_SECRETThe secret for the S3 bucketyoursecretkeyhere
BLOBSTORAGE_S3_ENCRYPTION_KEYAn optional encryption key for SSE-C encryption (with AES256), 32 bytes hex, defaults to off - DATA IS UNRECOVERABLE IF KEY IS LOST!00112233445566778899AABBCCDDEEFF00112233445566778899AABBCCDDEEFF
BLOBSTORAGE_FILESet to fallback to enable file storage in parallel to S3 blob storage (if files already exist), or disabled to turn off — not available for cluster deploymentsprimary
BLOBSTORAGE_S3_USEPATHSTYLEUse path style for buckets instead of host name labels (e.g. s3-storage.internal/bucket vs bucket.s3-storage.internal)false
BLOBSTORAGE_S3_USE_PROXYConnect to an HTTPS S3 endpoint through the outbound proxy (HTTPS_PROXY), see Setting up S3 Storagefalse
BLOBSTORAGE_S3_REQUEST_CHECKSUMEnable request checksum calculation (true=when_supported / false=when_required)true
ENABLE_JIRA_DCEnable Jira DC integration accesstrue
ENABLE_JIRA_CLOUDEnable Jira Cloud Connect integration accessfalse
JIRA_CONNECT_META_KEYJira cloud installation key e.g. a1b2c3d4e5f6 (for installing the app, this is not the same as the configuration token). Your install URL will be https://testiny.mycompany.org/api/v1/jira-integration/connect-server/<JIRA_CONNECT_META_KEY>. See the setup guide.-
JIRA_OAUTH_CLIENT_IDClient ID of the OAuth 2.0 app used as the default provider for the Jira Cloud integration. See the setup guide.-
JIRA_OAUTH_CLIENT_SECRETClient secret of the OAuth 2.0 app used as the default provider for the Jira Cloud integration. See the setup guide.-
HTTPS_PROXYProxy URL (supports credentials, e.g. http://proxy.corp.internal:3128) for outbound HTTPS connections: issue trackers like Jira, SSO, notification webhooks, importers and the update check (also set up the extra CA certificates file if necessary). Plain HTTP connections, SMTP, PostgreSQL and NATS never use the proxy; S3 only with BLOBSTORAGE_S3_USE_PROXY-
HTTPS_NO_PROXYA comma-separated list of host names or IPv4 addresses (company.net) and domain suffixes (.company.net, matches all subdomains but not the domain itself) that are connected to without the proxy. Wildcards (*), ports, CIDR ranges and IPv6 addresses are not supported.company.net,.company.cloudprovider.net
ALLOW_LOCALHOST_TARGETSWhether integrations, test case importers (TestRail, Xray, Zephyr), SSO and OAuth requests to identity providers and webhook notification channels may connect to localhost/loopback, link-local (169.254.0.0/16, fe80::/10), 0.0.0.0/8 and multicast addresses (from v1.42.2). Private network addresses, including the Testiny Server host's own, are always reachable. HTTPS connections through HTTPS_PROXY are resolved by the proxy, so for them only localhost names and IP literals are refusedfalse
CORS_ORIGINSAllowed origins (in addition to SERVER_URL)https://testiny.corp.internal,https://testiny-app.local
ENABLE_IP_ALLOWLISTWhether organizations can restrict client IPs with an IP allowlist, see Security Policiestrue
TRUST_PROXYWhether to trust the X-Forwarded-For header of your reverse proxy for client IP detection (if true, also set TRUSTED_PROXIES)true
TRUSTED_PROXIESA comma-separated list of proxy IPs or CIDR ranges to trust for client IP detection, e.g. 10.244.0.0/16 (only used if TRUST_PROXY is true). If empty, all proxies are trusted and clients can set their own IP — set it to the network of your reverse proxy / ingress controller-
ACCOUNT_LOCKOUT_ENABLEDWhether to temporarily lock accounts with increasing delays after repeated failed loginstrue
TESTINY_RATE_LIMITWhether to rate-limit the login, signup and OAuth endpoints (per client IP, or per user when signed in)true

PostgreSQL Configuration​

Testiny Server officially supports PostgreSQL version 14, 15 and 16.

Testiny Server creates three databases automatically for you if the specified PG_USER in the environment variables has permissions to create databases. The following three databases are created:

  • central - containing user information
  • tenant_main - containing data (tests, runs, ...) of the main organization
  • tenant_testing - containing data of the test organization (to use as a playground, should not contain any production data)
info

If the specified PG_USER does not have permission to create databases, you could temporarily give the user more permissions during setup and then reduce their permissions to create/drop tables, CRUD operations, etc. Otherwise, you need to create these three databases yourself. Learn how to setup the database yourself in this database initialization guide.

Also make sure, that the following configuration is set:

  • set ENCODING to UTF8 (default value for PostgreSQL)
  • set TABLESPACE to pg_default (default value for PostgreSQL)
  • If you use replication in your PostgreSQL database, set synchronous_commit to remote_apply.
  • Collation can be configured freely, as long as it's compatible with UTF8 encoding.
  • set autovacuum_analyze_scale_factor to 0 or, in most cases, <0.01 might be sufficient for on-premise installation
caution

Please note that the optimization parameters may vary depending on your system resources and expected performance requirements. Please ensure that your database is correctly configured and follows best practices for tuning and maintenance.

Running Testiny Server​

You can run Testiny Server using Docker Compose or Docker CLI, as shown with the examples below.

Testiny Server stores blobs (attachments, images, etc...) in a volume and this volume needs to be mounted at /testiny/storage.

Please note, that you need to use and maintain your own PostgreSQL or SQLite database. In the Docker Compose example, a PostgreSQL database is pre-configured so that you can easily get started with Testiny Server.

Once Testiny Server is running, open the Testiny Server UI in your browser using the SERVER_URL that you've configured and continue with the setup.

info

You need to have access to our registry to pull the image. Please contact us via email to get access: [email protected].

info

Please note, that Testiny Server does not provide HTTPS. You need to set up your own HTTPS reverse proxy/gateway.

Using Docker Compose​

docker-compose.yml​

  • Copy the following code into a docker-compose.yml file. In this file, a PostgreSQL database is pre-configured to get started easily. This compose file may not be suitable for production use and you will need to modify it to suit your infrastructure and needs.

  • Adjust the environment variables according to your needs (especially SERVER_URL and YOUR_SECRET_DB_PASSWORD). In this example, SERVER_URL is set to http://localhost:7700.

    services:
      testiny_postgres:
        image: postgres:14
        container_name: testiny_postgres
        networks:
          - testiny_net
        environment:
          POSTGRES_USER: admin
          POSTGRES_PASSWORD: YOUR_SECRET_DB_PASSWORD
        volumes:
          - testiny_postgres_data:/var/lib/postgresql/data
        restart: unless-stopped

      testiny_server:
        image: registry.gitlab.com/testiny/server:stable
        container_name: testiny_server
        depends_on:
          testiny_postgres:
            condition: service_started
        networks:
          - testiny_net
        ports:
          - "7700:7700"
        environment:
          SERVER_URL: "http://localhost:7700"
          DB: postgres
          PG_HOST: testiny_postgres
          PG_USER: admin
          PG_PASSWORD: YOUR_SECRET_DB_PASSWORD
        volumes:
          - testiny_storage:/testiny/storage
        restart: unless-stopped

    volumes:
      testiny_postgres_data:
        name: testiny_postgres_data
        external: false
      testiny_storage:
        name: testiny_storage
        external: false

    networks:
      testiny_net:
        name: testiny_net

  • Start Testiny Server with docker compose up -d

Using Docker CLI​

docker run -it -p 7700:7700 \
    -v testiny_storage:/testiny/storage \
    -e DB='postgres' \
    -e PG_HOST='127.0.0.1' \
    -e PG_PORT='5432' \
    -e PG_USER='postgres' \
    -e PG_PASSWORD='mysecretpassword' \
    -e SERVER_URL='http://localhost:7700' \
    --name testiny-server \
    registry.gitlab.com/testiny/server:stable

Finishing the Setup​

Once Testiny Server is running, open the Testiny Server UI in your browser using the SERVER_URL that you've configured and follow the setup instructions shown. We have also summarized the steps here:

  1. Fill in all required fields and click 'Complete setup' to create a new instance and admin account.

  2. Testiny Server generates secrets to access the database. Please make sure you save or copy the secrets to a secured location. You will need them to create backups or to recover data. The following tokens and keys will be created:

    • Backup export token and the private key passphrase (testiny_server_backup_secrets.txt)
    • Public key .pem file (testiny_server_backup_public_key.pem)
    • Private key .pem file (testiny_server_backup_private_key.pem)
    danger

    Make sure to save or copy the generated secrets to a secure location. You (nor anyone else, including Testiny) do not have access to those secrets afterward. Also, ensure that your container storage volume (/testiny/storage) is mounted to a persistent volume with backups.

  3. You have now successfully created your instance and admin account. You can now continue with applying a license key to activate Testiny Server.

Activating Testiny Server​

A valid license key is required to activate and use Testiny Server. Please contact us at [email protected] to purchase Testiny Server. Follow the steps below to activate a license:

  1. Login with the administrator account (created during the initial setup)
  2. Navigate to Settings > Organization
  3. Click on 'Apply License' and insert the license provided by the Testiny team.

Upgrading Testiny Server​

To upgrade Testiny Server, pull the latest stable version from the container registry and recreate the container. All required upgrade steps will be automatically executed upon startup.

info

Downgrading to a previous version is not supported.

Best Practices
  • Please ensure that backups are available before upgrading to a new version.
  • We recommend testing the upgrade to a new Testiny Server version in your testing/staging environment. Although we thoroughly test every version and deploy it in Testiny Cloud before releasing a new stable version of Testiny Server, we cannot test Testiny Server within your local infrastructure.

Stable Versions​

We recommend to install and stay on the stable version (server:stable).

If you would like to upgrade to a newer version (server:vX.X.X), we recommend upgrading to the latest bugfix/patch version of a minor version. All available versions including the change log can be found in the Release Notes.

In Testiny Server, you can see if your current version is up to date with the stable version when hovering on the info icon in the toolbar:

Testiny Server Version Info

A notice is shown when a new version is available.

Docker compose​

Just pull a new image and restart the server.

  • docker compose pull
  • docker compose up -d

Docker​

  • docker stop testiny-server
  • docker rm testiny-server
  • docker pull registry.gitlab.com/testiny/server:stable
  • Execute the same command as shown under Using Docker
Last updated on