Distributed Text Processing

Source: 13_distributed_text_processing

An HTTP server that receives plain-text input and returns it uppercased. The transformation runs on a Mélodium cloud runner provisioned on demand — the javascript package only needs to be available on the runner, not the front-end machine.

Running


melodium run 13_distributed_text_processing/Compo.toml \
  --api_token "my-api-token"


$ curl -X POST http://127.0.0.1:8080/process -d "hello world from melodium"
HELLO WORLD FROM MELODIUM

How it works

The Uppercaser model and the processText treatment are both defined in the same main.mel file, but processText runs remotely:


model distributor: DistributionEngine(
  treatment = "distributed_text_processing/main::processText",
  version   = "0.1.0"
)

The DistributionEngine references the treatment by its fully-qualified path. When the runner executes it, processText instantiates its own Uppercaser model locally on the remote side.

See in Compositeur Studio

The remote treatment

processText converts the raw byte stream to a string via decode, wraps it as JSON using fromString<string>(), processes it through the JavaScript engine, then unwraps the JSON result back to a plain string:


Self.data -> decode.data,text -> wrapStr.value,json -> jsUpper.value,result
          -> unwrapResult.option,value -> resultStr.value,into
          -> unwrapStr.option,value -> encode.text,data -> Self.data

The JSON wrapping/unwrapping is required because JavaScriptEngine.process exchanges Json values, not raw strings.

See in Compositeur Studio

Dependencies


[dependencies]
std        = "0.10.1"  # core flows, logging, data structures
http       = "0.10.1"  # HTTP server and client
net        = "0.10.1"  # IP address helpers
encoding   = "0.10.1"  # UTF-8 encode / decode
javascript = "0.10.1"  # embedded JavaScript engine
json       = "0.10.1"  # JSON parsing and serialisation
work       = "0.10.1"  # cloud runner provisioning
distrib    = "0.10.1"  # stream distribution across runners