Signed Webhooks

Source: tutorial/11_signed_webhooks See in Playground

An HTTP server receiving webhooks from GitHub and GitLab, accepting only the requests that prove they come from them. GitHub signs each body with a shared secret and sends the HMAC-SHA256 signature in X-Hub-Signature-256; GitLab sends a shared token in X-Gitlab-Token. Accepted requests get 200 accepted and their event name is logged; any other request gets 403 rejected.

This step introduces secrets: both shared values are Secret<string> parameters, and the program checks requests against them without ever holding their value in plain data.

Note

Requires a free TCP port (8080 by default), plus curl and openssl to send signed requests.

Running


cd tutorial/11_signed_webhooks
MELODIUM_SECRET_GITHUB_WEBHOOK_SECRET=gh-shared MELODIUM_SECRET_GITLAB_WEBHOOK_TOKEN=gl-token \
    melodium run Compo.toml

In a second terminal:


BODY='{"ref":"main"}'
SIGNATURE=$(printf %s "$BODY" | openssl dgst -sha256 -hmac gh-shared | awk '{print $NF}')
 
curl -X POST http://127.0.0.1:8080/github -H "X-GitHub-Event: push" -H "X-Hub-Signature-256: sha256=$SIGNATURE" -d "$BODY"
# accepted
curl -X POST http://127.0.0.1:8080/github -H "X-GitHub-Event: push" -H "X-Hub-Signature-256: sha256=00ff" -d "$BODY"
# rejected
 
curl -X POST http://127.0.0.1:8080/gitlab -H "X-Gitlab-Event: Push Hook" -H "X-Gitlab-Token: gl-token" -d "$BODY"
# accepted
curl -X POST http://127.0.0.1:8080/gitlab -H "X-Gitlab-Event: Push Hook" -H "X-Gitlab-Token: wrong" -d "$BODY"
# rejected

The server log shows:


info: github: push
warning: github: invalid signature, request rejected
info: gitlab: Push Hook
warning: gitlab: invalid token, request rejected

To receive real webhooks, the server has to be reachable from GitHub or GitLab (behind a reverse proxy or a tunnel), with the same secret configured in the webhook settings of the repository.

Optional: add --api-report and an API token (MELODIUM_API_TOKEN) to see this run’s full trace on Cadence.CI .

How it works

main takes the two shared values as secrets, and declares a single HttpServer for both routes:


treatment main(
    const github_secret: Secret<string> = "env:MELODIUM_SECRET_GITHUB_WEBHOOK_SECRET",
    const gitlab_token:  Secret<string> = "env:MELODIUM_SECRET_GITLAB_WEBHOOK_TOKEN",
    const port:          u16            = 8080
)
  model server: HttpServer(host=|from_ipv4(|localhost_ipv4()), port=port, secret_headers="X-Gitlab-Token")
{
    startup()
    start[http_server=server]()
    startup.trigger -> start.trigger
 
    github[http_server=server](secret=github_secret)
    gitlab[http_server=server](token=gitlab_token)
}

github_secret and gitlab_token default to locators, env:MELODIUM_SECRET_GITHUB_WEBHOOK_SECRET and env:MELODIUM_SECRET_GITLAB_WEBHOOK_TOKEN. Programs only get the environment variables whose name starts with MELODIUM_SECRET_, and the values are revealed only inside the crypto treatments that check them, so they appear in no parameter, log, or report. Another locator can be given on the command line instead, such as --github_secret file:/run/secrets/github_webhook.

The two routes follow the same shape:

Verifying an HMAC signature

POST /github gives the signature header and the body to crypto/signature/hmac::verify, keyed with the shared secret:


treatment github[http_server: HttpServer](const secret: Secret<string>)
{
    connection[http_server=http_server](method=|post(), route="/github")
 
    signature: getHeader(key="X-Hub-Signature-256")
    verify(key=secret, prefix="sha256=")
    connection.headers -> signature.map,value -> verify.signature
    connection.data ----------------------------> verify.data
 
    respond()
    rejected: one<void>()
    verify.valid ------------------> respond.accepted
    verify.invalid -> rejected.a
    verify.failed --> rejected.b,value -> respond.rejected
    respond.status --> connection.status
    respond.headers -> connection.headers
    respond.data ----> connection.data
 
    event: getHeader(key="X-GitHub-Event")
    eventName: unwrapOr<string>(default="unknown")
    onceValid: releaseBlock<string>()
    logEvent: logInfo(label="github")
    connection.headers -> event.map,value -> eventName.option,value -> onceValid.data,released -> logEvent.message
    verify.valid -------------------------------------------------> onceValid.leverage
 
    /* … warning and error logs … */
}

verify computes the HMAC of the body as it arrives, and compares it in constant time with the signature once the body ends. A missing header, a header without the sha256= prefix, or a signature that is not valid hexadecimal all count as invalid. The status is only decided once the verification ends, so no request is ever answered 200 before its signature is known to be good.

invalid and failed both lead to 403, merged by one. failed means the secret itself could not be revealed (such as an unset environment variable), and its reason goes to the error log, never to the client.

Checking a token received as a secret header

GitLab sends its token in a header, and a token is as sensitive on the receiving side as on the sending one. HttpServer(..., secret_headers="X-Gitlab-Token") lists it as a secret header: it is left out of the plain request headers, and given only in the secret_headers map of @HttpRequest, as a Secret<string>. Authorization and Cookie are always handled this way.


treatment receivedToken()
  require @HttpRequest
  input  trigger: Block<void>
  output token:   Block<Option<Secret<string>>>
{
    emitToken: emit<Option<Secret<string>>>(value=|get<Secret<string>>(@HttpRequest[secret_headers], "X-Gitlab-Token"))
    Self.trigger -> emitToken.trigger,emit -> Self.token
}

crypto/token::check then compares the received token with the configured one in constant time, without revealing either of them to the program:


receivedToken()
checkToken(token=token)
connection.started -> receivedToken.trigger,token -> checkToken.received

A token received this way cannot be logged or sent back by mistake: it is never ordinary data.

Logging only accepted events

The event name is read from X-GitHub-Event or X-Gitlab-Event as soon as the request arrives, but releaseBlock holds it until valid is emitted. The log line only exists for accepted requests: when a request is rejected, valid never comes and the held value is dropped.

One response treatment for every route

respond takes the two Block<void> outcomes and drives the three outputs connection needs (status, headers, data), with one<T>() merging the two possible values of each:


treatment respond()
  input  accepted: Block<void>
  input  rejected: Block<void>
  output status:   Block<HttpStatus>
  output headers:  Block<StringMap>
  output data:     Stream<byte>
{
    okStatus:        emit<HttpStatus>(value=|ok())
    forbiddenStatus: emit<HttpStatus>(value=|forbidden())
    status: one<HttpStatus>()
    Self.accepted -> okStatus.trigger,emit -----> status.a
    Self.rejected -> forbiddenStatus.trigger,emit -> status.b,value -> Self.status
 
    answered: check<HttpStatus>()
    emitHeaders: emit<StringMap>(value=|map([]))
    status.value -> answered.value,check -> emitHeaders.trigger,emit -> Self.headers
 
    acceptedText: emit<string>(value="accepted\n")
    rejectedText: emit<string>(value="rejected\n")
    text: one<string>()
    asStream: stream<string>()
    encode()
    Self.accepted -> acceptedText.trigger,emit -> text.a
    Self.rejected -> rejectedText.trigger,emit -> text.b,value -> asStream.block,stream -> encode.text,data -> Self.data
}

Dependencies


[dependencies]
std      = "0.11.0"  # core flows, logging, data structures
http     = "0.11.0"  # HTTP server and secret headers
net      = "0.11.0"  # IP address helpers
encoding = "0.11.0"  # UTF-8 encode / decode
crypto   = "0.11.0"  # HMAC signatures and token checks