Secrets

Passwords, tokens, and keys are given to programs as Secret<T> values, most often Secret<string>. A secret holds where its value comes from rather than the value itself, and only the elements that need the value can reveal it, at the moment they use it. Its value never appears in parameters, logs, debug files, or reports, and never in command arguments.

Locating secrets

A secret parameter takes a locator, <scheme>:<path>, telling where the value is read from when revealed:

Programs only get the environment variables whose name starts with MELODIUM_SECRET_: the other ones, such as those configuring Mélodium itself, stay out of their reach. Whoever runs a program exposes a variable to it on purpose, for example MELODIUM_SECRET_GITHUB_TOKEN: ${{ github.token }} in a GitHub workflow.

Locators are written as string literals, for defaults as well as for arguments:

use sql::SqlPool
use std/ops/option::|wrap
 
treatment database(const password: Secret<string> = "env:MELODIUM_SECRET_DB_PASSWORD")
  model pool: SqlPool(url="postgresql://app@db.local/app", password=|wrap<Secret<string>>(password))
{
}

On the command line, the same parameter takes a locator, never a value:

$ melodium run app.mel --password file:/run/secrets/db_password

--check-secrets resolves the secrets given as parameters before running, so that a missing variable or file fails right away rather than when first used.

Using secrets

Elements receiving secrets reveal them only when they use them:

• sql::SqlPool puts its password into the connection URL when connecting, and refuses query bindings holding secrets.
• http/client::HttpClient adds its secret_headers to each request.
• process environments give secret variables and a secret standard input to commands, never through their arguments.
• ml remote providers (RemoteLlm, RemoteStt, RemoteTts) and the HuggingFace Hub take their API key or token as a secret.
• work/distant::DistantEngine and container pull secrets are revealed to send the worker request.
• work/access::|new_access takes the authentication keys of a distribution engine as secrets, revealed by distrib::start to authenticate the connection.

Incoming HTTP requests give their Authorization and Cookie headers as secrets of the @HttpRequest context, and leave them out of the request headers. A server can list other headers to receive only as secrets, such as the token of GitLab webhooks: HttpServer(..., secret_headers="X-Gitlab-Token").

The crypto package checks such values without revealing them to the program: crypto/token::check compares a received token with a secret in constant time, and crypto/signature/hmac::verify checks the HMAC signature of a body with a key given as a secret. It also verifies Ed25519 signatures and JSON Web Tokens, signs JSON Web Tokens, and generates random tokens as secrets. The Signed Webhooks example puts these together.

Secrets can be built from other secrets without being revealed. The functions of std/secret, such as |format, |base64, and |url_encode, compute their result only when it is revealed:

use http/client::HttpClient
use std/data/map::Map
use std/data/map::|map as |secret_map
use std/data/map::|entry as |secret_entry
use std/data/string_map::|map
use std/ops/option::|wrap
use std/secret::|format
 
treatment api(const token: Secret<string> = "env:MELODIUM_SECRET_API_TOKEN")
  model client: HttpClient(
    base_url=|wrap<string>("https://api.example.com"),
    headers=|map([]),
    secret_headers=|wrap<Map>(|secret_map([
        |secret_entry<Secret<string>>("Authorization", |format(
            "Bearer {token}",
            |secret_map([|secret_entry<Secret<string>>("token", token)]),
            "authorization"
        ))
    ]))
  )
{
}

Some secrets are only known at runtime, such as a token given by an API response, made a secret by the std/secret::conceal treatment, or a secret read by vault::get from a path built at runtime. They arrive on inputs, and the treatments of std/secret/block, process/environment/block, and http/client::requestWithSecretHeaders handle them the same way, without revealing them before they are needed.

Policies

Each secret has a policy, set when it is created, and that can only be narrowed afterwards:

• transmission, where the secret may go when the program is distributed: local (never leaves the engine, the default), reference (only the locator is sent, and the distant engine resolves it with its own sources), or value (the value is sent, over TLS only);
• reveal, which elements may reveal it: any of them by default, or only those listed;
• plain reveal, whether std/secret::reveal may turn it into ordinary data: not by default, and never for secrets from env: and file:.

use std/secret::|from_environment
use std/secret::|locate
use std/secret::|reveal_only_by
 
// Sent by value to distant engines, never plainly revealed.
|locate<string>("vault:kv/data/ci#token", "ci_token", "value", false)
 
// Only revealed by `SqlPool`.
|reveal_only_by<string>(|from_environment("MELODIUM_SECRET_DB_PASSWORD", "db_password"), ["sql::SqlPool"])

A secret built from others gets the most restrictive combination of their policies.

Audit and masking

Every reveal, and every secret sent to a distant engine, is recorded as a debug event naming the element and its label. Refused and failed reveals are also written to the log.

Values revealed during a run are replaced by <secret "name"> in log messages, including their base64 and URL-encoded forms, as long as they are at least 8 bytes long.

Plaintext held by Mélodium is overwritten once not needed anymore, and before the process exits, including on interruption. This is best effort: memory written to swap or core dumps, and copies made by libraries or commands given the value, are out of its reach.

Distribution

When a treatment runs on a distant engine, its parameters and the data sent to it follow the transmission policy of each secret they hold:

• local secrets are refused, with an explicit error;
• reference secrets are resolved by the distant engine, with its own environment, files, and sources;
• value secrets are resolved by the sending engine and sent over TLS, never over plain TCP.

The distant engine keeps the policy of the secrets it receives, and enforces it.