Text & Files

Source: tutorial/03_text_and_files See in Playground

Reads a text file line by line, keeps the lines that match a regex pattern, and writes a small report summarising how many (non-blank) lines were read and how many matched.

Running


cd tutorial/03_text_and_files
melodium run Compo.toml --input_file sample.txt --pattern "Mélodium"

This reads sample.txt (shipped alongside the example), logs every line matching the pattern regex, and writes report.txt with the total line and match counts.

Optional: add --api-report and an API token (MELODIUM_API_TOKEN) to see this run’s full trace on Cadence.CI .

How it works

This example uses no models: file reading, regex matching, and text composition are all stateless treatments.

Data flow

From a file to a stream of lines

readTextLocal streams the file’s raw content in chunks. split with delimiter="\n" and flatten turn that into a stream of lines: the same “split then flatten” idiom works for decoding any delimited stream, not just files.


read:       readTextLocal(path=input_file)
readFailed: logInfoMessage(label="read", message="could not read input file")
startup.trigger -> read.trigger
read.failed -> readFailed.trigger
 
splitLines: split(delimiter="\n", inclusive=false)
lines:      flatten<string>()
trimmed:    trim()
read.text -> splitLines.text,splitted -> lines.vector,value -> trimmed.text

Splitting on "\n" leaves one trailing empty piece after the file’s last newline, so a small exact + not + filter chain drops blank lines before anything else runs:


isBlank:  exact(pattern="")
notBlank: not<bool>()
nonBlank: filter<string>()
trimmed.trimmed -> isBlank.text
isBlank.matches -> notBlank.value
trimmed.trimmed -> nonBlank.value
notBlank.not    -> nonBlank.select

This is the general pattern for “keep everything except X”: the same shape used a few lines further to keep only lines matching a pattern, just inverted.

Matching against the pattern

Each line is tested against pattern with matches; the resulting boolean stream drives filter, whose accepted branch is both logged and counted:


isMatch:  matches(regex=pattern)
matching: filter<string>()
nonBlank.accepted -> isMatch.text
nonBlank.accepted -> matching.value
isMatch.matches   -> matching.select
 
logMatches: logInfos(label="match")
matching.accepted -> logMatches.messages

Aggregating a stream to one value

Totals are computed by a small local treatment, finalCount<T>, reused for both the line count and the match count: count numbers every element as it streams by, and trigger .last collapses that running count to its final value once the stream ends:


treatment finalCount<T>()
  input  items: Stream<T>
  output total: Block<string>
{
    index:   count<T>()
    asStr:   toString<u128>()
    lastStr: trigger<string>()
 
    Self.items -> index.stream,count -> asStr.value,into -> lastStr.stream
    lastStr.last -> Self.total
}

Building the report

The two totals and the pattern are combined into a single StringMap with blockEntry/blockInsert (aliases for entry and insert), then formatted into the final report text with format:


withLines:   blockEntry(key="lines")
withMatches: blockInsert(key="matches")
withPattern: blockInsert(key="pattern")
lineTotal.total  -> withLines.value
withLines.map    -> withMatches.base
matchTotal.total -> withMatches.value
withMatches.map  -> withPattern.base
patternBlock.emit -> withPattern.value
 
reportEntries: stream<StringMap>()
reportLine:    format(format="Report for pattern \"{pattern}\": {lines} line(s) read, {matches} matching.")
write:         writeTextLocal(path=output)
logDone:       logInfoMessage(label="report", message="report written")
 
withPattern.map -> reportEntries.block,stream -> reportLine.entries,formatted -> write.text
write.finished  -> logDone.trigger

Dependencies


[dependencies]
std   = "0.10.3"  # core flows, logging, data structures
fs    = "0.10.3"  # local file I/O
regex = "0.10.3"  # regular expressions