JavaScript Transform

Source: tutorial/08_javascript_transform See in Playground

Reads one JSON object per line, computes a letter grade from its score field in JavaScript, and writes the graded records back out. The sample data, students.jsonl, uses names with full UTF-8 characters (Amélie, Øystein, Carmiña, Łukasz), read and written back without any special-casing.

Running


cd tutorial/08_javascript_transform
melodium run Compo.toml --input_file students.jsonl


{"grade":"A","name":"Amélie","score":92}
{"grade":"E","name":"Øystein","score":58}
{"grade":"C","name":"Carmiña","score":74}
{"grade":"FX","name":"Łukasz","score":45}

Optional: add --api-report and an API token (MELODIUM_API_TOKEN) to see this run’s full trace on Cadence.CI .

How it works

The Grader model loads a grade() JavaScript function once at startup, and that same compiled function is reused for every line processed:


model Grader() : JavaScriptEngine {
    code = ${{function grade(input) {
        var g = 'F';
        if (input.score >= 90) g = 'A';
        else if (input.score >= 80) g = 'B';
        else if (input.score >= 70) g = 'C';
        else if (input.score >= 60) g = 'D';
        else if (input.score >= 50) g = 'E';
        else if (input.score >= 45) g = 'FX';
        return { name: input.name, score: input.score, grade: g };
    }
}}
}

The ${{...}} raw block string is the standard way to embed multiline JavaScript source without escaping newlines or quotes.

Lines are extracted the same way as in earlier text-processing examples (split, flatten, trim, blanks dropped), then each line is parsed into a Json value and fed to process, which calls the JS grade(value) function defined in the model’s code:


treatment main(
    const input_file: string = "students.jsonl",
    const output:     string = "grades.txt"
)
  model engine: Grader()
{
    startup()
 
    read: readTextLocal(path=input_file)
    startup.trigger -> read.trigger
 
    splitLines: split(delimiter="\n", inclusive=false)
    lines:      flatten<string>()
    trimmed:    trim()
    read.text -> splitLines.text,splitted -> lines.vector,value -> trimmed.text
 
    isBlank:  exact(pattern="")
    notBlank: not<bool>()
    nonBlank: filter<string>()
    trimmed.trimmed -> isBlank.text
    isBlank.matches -> notBlank.value
    trimmed.trimmed -> nonBlank.value
    notBlank.not    -> nonBlank.select
 
    parsed: toJson()
    body:   unwrapOr<Json>(default=|null())
    graded: process[engine=engine](code="grade(value)")
    result: unwrapOr<Json>(default=|null())
    asText: toString<Json>()
 
    nonBlank.accepted -> parsed.text,json -> body.option,value -> graded.value,result -> result.option,value -> asText.value,into -> logGrades.messages
 
    logGrades: logInfos(label="grade")
 
    withNewline: entry(key="line")
    asLine:      format(format="{line}\n")
    write:       writeTextLocal(path=output)
    logDone:     logInfoMessage(label="js", message="grades written")
 
    asText.into -> withNewline.value,map -> asLine.entries,formatted -> write.text
    write.finished -> logDone.trigger
}

Inside the JavaScript function, value is the parsed object (input.score, input.name), exactly the field access that plain json treatments cannot do, as seen in the previous two tutorial steps. process returns Option<Json> (none if the code throws or returns something that cannot convert to JSON), and unwrapOr supplies a fallback so the pipeline never stalls on one bad record.

Two different unwrap points are worth noting:

unwrapOr<Json>(default=|null()) before process: malformed input JSON quietly becomes null rather than crashing the pipeline.
unwrapOr<Json>(default=|null()) after process: a JS runtime error also produces a graceful null result rather than stopping the run.

Each result is logged and written to grades.txt, one JSON object per line, using the same “entry + format + newline” idiom seen for building text reports throughout the tutorial. Because the model’s code is compiled once at startup, and process’s own code parameter ("grade(value)") is just the expression evaluated per item, the transform logic is written once and reused across every line without recompiling. This makes javascript::process the escape hatch for structured JSON access: whenever a task needs to read or build specific fields of a JSON object or array, it is the tool to reach for rather than trying to assemble it from json package primitives alone.

Dependencies


[dependencies]
std        = "0.10.4"  # core flows, logging, data structures
fs         = "0.10.4"  # local file I/O
json       = "0.10.4"  # JSON parsing and serialisation
javascript = "0.10.4"  # embedded JavaScript engine