Frontends#
Text in, validated IR out. Two syntaxes, one IR – which is what lets a comic script and a YAML document compile identically without anything downstream knowing there is more than one.
scenet.frontends.yaml_front#
The YAML surface syntax: text in, validated IR out.
This is one frontend among several planned – a comic-script frontend will target the same IR. Keeping parsing separate from the IR is what makes that possible without touching anything downstream.
scenet.frontends.script_front#
A frontend for comic script, the format writers already use.
Checked before building this: Fountain has no native panel, caption or SFX support,
and there is no standardised comic script format at all. But the informal industry
convention is stable across publishers – PANEL 1, a prose description, a character
cue in capitals, the dialogue beneath it – so that is what this parses. Dark Horse
publishes its own house format, and it is read as written: PAGE ONE (five panels),
Panel 1. with its description, CAP: and CAP/NAME, numbered lettering, Markdown
around any of it. So is a French script: PLANCHE 1, CASE 2, RÉCITATIF :, the
échelle des plans. Each PAGE heading is a page.
The honest limitation. A prose description like “Alice and Bob face each other on
a rainy street corner” cannot be compiled. Turning that into staging needs natural
language understanding, and guessing would produce panels that are confidently wrong
– far worse than refusing. So descriptions are preserved but not interpreted – each one
is its panel’s description, read out as written by accessible output – and anything the
compiler must know is declared explicitly: cast and staging in a front-matter block,
per-panel settings as @ directives.
---
cast:
ALICE: {reference: alice, at: left_third}
BOB: {reference: bob, at: right_third}
staging:
- ALICE left_of BOB
---
PANEL 1
@shot: full_shot
CAPTION: Midnight. The docks.
Alice and Bob face each other on a rainy street corner.
ALICE
You forgot your umbrella!
BOB (whisper)
I know.
- class scenet.frontends.script_front.ScriptNote[source]#
Bases:
objectSomething the script frontend noticed that is worth saying but is not a fault.
- path#
Where it is, structurally, so a finding’s fingerprint survives an edit above it: the panel’s name and what in it, or the page’s.
- __init__(rule, message, line, path=())#
- scenet.frontends.script_front.default_tiers(count)[source]#
How many panels each tier of a page holds, when the script does not say.
Dark Horse leaves “exact panel layout” to the artist, and the compiler is the artist, so it follows a fixed rule: three tiers at most up to nine panels – the lettering height of a page assumes three – with an odd panel in a tier of its own in the middle, and beyond nine, rows of three with the extras in the earlier tiers.
Example
>>> from scenet.frontends.script_front import default_tiers >>> [default_tiers(count) for count in (1, 4, 5, 7, 10)] [[1], [2, 2], [2, 1, 2], [2, 3, 2], [3, 3, 2, 2]]
- scenet.frontends.script_front.normalise_text(text)[source]#
A script as it is read, whatever the editor that saved it did.
Line endings become line feeds. Python’s text mode does this silently when reading a file, which is why it took a browser to notice: the playground hands over the bytes it was given, and a script saved by a Windows editor – or pasted from one – arrives with CRLF. The front-matter pattern then does not match, the
---fence is read as prose, and the whole document is rejected for having content before the first PANEL heading.A byte-order mark is dropped. Some Windows editors still write one, and it hid the front matter in exactly the same way.
Text is composed (NFC). A macOS editor may store
ÉasEand a combining accent, and a name typed that way in a cue is the same name the front matter composed; compared as stored, the cue named nobody.
Example
>>> from scenet.frontends.script_front import normalise_text >>> normalise_text("\ufeffZOE\u0308\r\n") == "ZO\u00cb\n" True
- Return type:
- class scenet.frontends.script_front.PanelLines[source]#
Bases:
objectWhere a script panel was written, for a finding about it to point at.
- events#
The line each entry of its
scriptwas written on, in order: a speech’s cue, a caption’s or a sound effect’s own line.
- directives#
The line each of its directives was written on, by the path in the panel it sets:
("setting",)for@setting:.
- __init__(heading, events, directives=<factory>)#
- class scenet.frontends.script_front.ScriptDocument[source]#
Bases:
objectA comic script, read and validated, and where each of its panels was written.
- lines#
Panel name to the lines it was written on.
- Type:
- notes#
What reading it noticed that is worth saying but is not a fault – a cue with nothing said under it, a page whose count its panels do not match.
- Type:
- descriptions#
Panel name to its prose, which is kept and never interpreted.
- title_page#
The prose before the first heading: Dark Horse’s format opens with the writer’s name and address.
- page_notes#
Page to the prose between its heading and its first panel.
- layout#
The pages its PAGE headings lay its panels out on; none without them.
- Type:
- property direction: Direction#
Which way the script is read: its front matter’s, which every panel takes.
- __init__(panels, lines, notes=(), descriptions=<factory>, title_page=(), page_notes=<factory>, layout=<factory>)#
- scenet.frontends.script_front.parse_script_document(text, *, source=None)[source]#
Parse comic script, keeping the line each panel and each of its events is on.
parse_scriptis this, less the lines. A checker reporting a fault in a panel that did validate – a balloon over the word limit, say – needs to know where it is.Example
>>> from scenet.frontends.script_front import parse_script_document >>> document = parse_script_document( ... "---\ncast: {A: {reference: alice}}\n---\nPANEL 1\nA\nHi.\n" ... ) >>> document.lines["1"] PanelLines(heading=4, events=(5,), directives={})
- Return type:
scenet.frontends.common#
Shared between the frontends.
Both syntaxes – YAML documents and comic scripts – produce the same
[PanelIR][scenet.ir.PanelIR], and both need the same two things on the way: a rewrite
of the surface conveniences into the IR’s canonical shape, and a way to turn pydantic’s
error list into a diagnostic a person can act on.
These lived in yaml_front and were imported into script_front through their private
names, which put a module’s internals on another module’s import line. Whichever
frontend came first would have owned them by accident. They belong here, where neither
does.
- scenet.frontends.common.LANGUAGE_REFERENCE = 'https://creatoan.github.io/scenet/reference/language/'#
Where the language reference is published, which a finding’s help names.
- scenet.frontends.common.LAYOUT_KEYS = ('page', 'pages')#
Keys a scene may carry beside
panels:that lay its panels out rather than being a default every panel inherits.
- scenet.frontends.common.SITE = 'https://creatoan.github.io/scenet/'#
Where the documentation is published: the root, which serves the latest release. A message read in a terminal or an editor has no folder of documentation beside it, so a path in the repository would lead nowhere; a finding’s help names a page of the site instead.
- scenet.frontends.common.errors_of(exc)[source]#
Pydantic’s error list, without the echoes of a fault already reported.
Two echoes are dropped:
When an item of a
tuple[X, ...]field with a minimum length fails, pydantic also reports the tuple as too short, counting only the items that passed: a tier with a bad height says “Tuple should have at least 1 item after validation, not 0” beside the real fault. Alistdoes not do this. The echo names no fault of its own, so it is dropped whenever a deeper error explains it.A misspelled required key is one mistake that pydantic reports twice: the unknown key, and the required key it should have been. When an unknown key closely matches a required one missing from the same mapping, the two become one finding about the unknown key, which says what it was meant to be.
- Parameters:
exc (
ValidationError) – The validation error pydantic raised.- Return type:
- Returns:
Its errors, in pydantic’s order, less the echoes.
- scenet.frontends.common.expand_place(setting)[source]#
Rewrite
place:into the mass list it stands for.A preset is a library for convenience, never a second format – so it is expanded here, in the frontend, exactly as a staging sentence is. Everything downstream sees one representation of a backdrop.
- Parameters:
setting (
object) – The value of thesettingkey, exactly as the YAML parser produced it – which is why this isobjectand not a mapping: rejecting a setting written as prose is half of what this function is for.- Return type:
- Returns:
The same mapping with
placereplaced bymasses, or unchanged when no place was named. The input is not modified.- Raises:
PanelSyntaxError –
settingis not a mapping (prose is not accepted), the place is not one the library has,placewas given alongsidemasses, or animagebeside a place, masses or a horizon.
Example
>>> from scenet.frontends.common import expand_place >>> [mass.kind.value for mass in expand_place({"place": "shore"})["masses"]] ['sky', 'water', 'ground']
- scenet.frontends.common.normalise(data)[source]#
Rewrite surface conveniences into the IR’s canonical shape.
Three shapes differ between the surface syntax and the IR. Staging is written as sentences. Script entries are written as single-key mappings tagged by verb (
- say: {...},- caption: {...}) so that a new verb can be added without a discriminator field the author has to type; the tag is moved into the payload asverb, which is where the IR’s event models expect to find it. And a setting may name a place, which stands for a list of masses.- Parameters:
data (
dict[str,Any]) – A parsed top-level panel mapping, in surface form.- Return type:
- Returns:
A new mapping in the shape
PanelIRvalidates. The input is not modified.- Raises:
PanelSyntaxError –
stagingorscriptis not a list, a script entry is not a single-key mapping, it names a verb that does not exist, orsettingnames a place that does not exist or lists masses alongside one.
- scenet.frontends.common.normalise_layout(data)[source]#
Rewrite the surface form of
page:andpages:into whatPageLayoutvalidates.One convenience: a panel in a tier or a column may be written as its name alone, which stands for
{use: name}. Anything not shaped as expected is passed through untouched, so the model’s validation reports it with its location rather than this guessing.
- scenet.frontends.common.parse_relation(text)[source]#
Parse
subject predicate objectinto a relation.Written as a sentence rather than a mapping because staging is read far more often than it is written, and
alice left_of bobis legible at a glance in a way that a three-key mapping is not.- Parameters:
text (
str) – One staging entry.- Return type:
- Returns:
The validated relation.
- Raises:
PanelSyntaxError – The entry is not three whitespace-separated words, names an unknown predicate, or relates an actor to itself – the last under rule
reflexive-relation.
Example
>>> from scenet.frontends.common import parse_relation >>> relation = parse_relation("alice left_of bob") >>> relation.subject, relation.predicate.value, relation.object ('alice', 'left_of', 'bob')
- scenet.frontends.common.summarise(exc, *, what='panel')[source]#
Flatten pydantic’s error list into something a human can act on.
pydantic’s default rendering is accurate but verbose; for a language error the useful part is the location and the reason, one line each.
- Parameters:
exc (
ValidationError) – The validation error pydantic raised.what (
str) – What failed to validate, for the first line:panel, orPanel Core.
- Return type:
- Returns:
A multi-line diagnostic, one line per problem, each naming where it is.
scenet.frontends.positions#
Where in the source a value came from.
yaml.safe_load returns values and throws away everything about where they were
written, which is fine for compiling and useless for reporting. yaml.compose parses
the same text into a node tree instead, and every node carries start_mark and
end_mark. So the document is composed a second time, purely to answer “where is
script.0.say.by?”.
`ruamel.yaml` is the library usually reached for here, and its .lc line/column
marks would do the same job. It was not used: PyYAML is already a dependency and this
needs one function, so adding a second YAML implementation to the runtime – and to the
licence gate – would buy nothing.
Marks are 0-based; SARIF is 1-based. The conversion happens once, here, at the boundary.
- class scenet.frontends.positions.Position[source]#
Bases:
objectA one-based line and column, as SARIF and every editor count them.
- __init__(line, column)#
- class scenet.frontends.positions.Region[source]#
Bases:
objectA span of source, from
startup toend.- end#
Where it ends. SARIF treats the end column as exclusive, which is also how PyYAML’s end marks behave, so the two agree without adjustment.
- Type:
- __init__(start, end)#
- scenet.frontends.positions.locate(text, path)[source]#
Find where
pathwas written intext.Walks as far down the path as the document allows and reports the deepest node it reached, so a path naming a field of something malformed still points at the malformed thing rather than giving up and pointing at the document.
- Parameters:
- Return type:
- Returns:
The region, or
Noneif the text does not compose at all – in which case the caller has a syntax error to report and no use for a path. So when one path is not found, no shorter path in the same text is either.
Example
>>> from scenet.frontends.positions import locate >>> region = locate("cast:\n alice: {reference: alice}\n", ("cast", "alice")) >>> region.start.line 2
scenet.safe_yaml#
Scenet YAML: reading a document so that ordinary dialogue is never mistyped.
PyYAML reads YAML 1.1, in which No is false, 1984 a number, 12:30 the sexagesimal
750, 2024-05-01 a date and 01750 the octal 1000. For a language whose values are
mostly lines of dialogue, that refused text: No and silently drew a panel 1000 units
tall. And it reads features no panel needs – tags that build any object they name,
anchors and aliases that expand without bound, merge keys – which are a confusion and an
attack surface.
So every panel, scene, script front matter and puppet is read through load(), by a
loader built from PyYAML’s own parts with three of them replaced. The profile they
implement is stated, normatively, in docs/reference/language/yaml_profile.md:
The resolver (
ProfileResolver) types a plain scalar as the YAML 1.2 JSON schema does:true,false,null(or nothing at all), and integers and floats as JSON spells them. Every other plain scalar is text, exactly as written.The scanner refuses a tag, an anchor or an alias where it is written, and a tab where a token would start, in words an author recognises.
The composer and the constructor refuse a merge key, a key that is not a scalar and a key written twice; every key is text, exactly as written, so
cast: {on: ..., no: ...}names two actors. A number keeps its source (WrittenInt,WrittenFloat), so where the language expects text,as_written()gives back what was typed:text: 1984letters 1984 andtext: 1.50letters 1.50.
Each refusal is a ProfileError, which is a yaml.YAMLError – so anything that
already handles malformed YAML handles it – and carries the rule it breaks and the span
it covers, for a finding at exactly the text at fault.
ruamel.yaml reads YAML 1.2 and refuses a repeated key by default, but a second YAML
implementation in the runtime, and in the licence gate, would buy little: every rule here
is a few lines on top of PyYAML’s machinery, and the same trade was declined for source
positions (frontends/positions.py).
- scenet.safe_yaml.ALIAS_HINT = "a value that starts with '*' is read by YAML as an alias to an anchor, so quote the whole value. To stress a word, write it between underscores instead: _told_"#
What an author who wrote
*told*meant, said in their terms rather than YAML’s.
- exception scenet.safe_yaml.DuplicateKeyError[source]#
Bases:
ProfileErrorA key written twice in one mapping.
The YAML specification requires every key in a mapping to be unique, so a document that repeats one is not valid YAML. PyYAML builds it anyway: the dictionary keeps the last value, and the first disappears without a word (yaml/pyyaml#41, still open). A strict language that rejects every unknown key would otherwise lose a whole known one – a panel, a character, a pose – and compile whatever was left.
- key#
The repeated key, as written.
- first_line#
One-based line of the first occurrence.
lineandcolumnare the second’s, which is where the fault is.
- exception scenet.safe_yaml.NestingError[source]#
Bases:
YAMLErrorThe document is nested more deeply than the parser can follow.
PyYAML parses and builds a document recursively, so nesting a few hundred levels deep reaches Python’s recursion limit.
RecursionErroris not a YAML error, and it escaped every handler that catches one –scenet checkandbuildboth ended in a traceback. No panel is nested more than a handful of levels, so a document this deep is a mistake or an attack, and is reported as one.
- exception scenet.safe_yaml.ProfileError[source]#
Bases:
MarkedYAMLErrorSomething the profile refuses, at the span of source it covers.
- summary#
One line saying what is wrong and what to write instead, without PyYAML’s position block, for a diagnostic that carries the position separately.
- line#
One-based line where the span starts.
- column#
One-based column where it starts.
- end_line#
One-based line where it ends.
- end_column#
One-based column just past its last character.
- class scenet.safe_yaml.ProfileResolver[source]#
Bases:
BaseResolverTypes a plain scalar as the YAML 1.2 JSON schema does, and every other one as text.
PyYAML’s own resolver implements YAML 1.1. This one starts from
BaseResolver, which resolves nothing, and registers four patterns: soyes,True,~,0x400,01750,1_0,.inf,12:30and2024-05-01are all text, and a quoted scalar always is.Example
>>> from scenet.safe_yaml import load >>> load("[true, True, null, ~, 1984, 01750, 1.5, .5]") [True, 'True', None, '~', 1984, '01750', 1.5, '.5']
- class scenet.safe_yaml.WrittenFloat[source]#
Bases:
floatA float, and the text it was written as:
1.50is 1.5, and letters 1.50.Example
>>> from scenet.safe_yaml import WrittenFloat >>> number = WrittenFloat(1.5, "1.50") >>> number * 2, number.source (3.0, '1.50')
- class scenet.safe_yaml.WrittenInt[source]#
Bases:
intAn integer, and the text it was written as.
It is the integer everywhere a number is expected, and its
sourcewhere text is: seeas_written(). JSON writes it as the integer it is.Example
>>> from scenet.safe_yaml import WrittenInt >>> number = WrittenInt(1984, "1984") >>> number + 1, number.source (1985, '1984')
- scenet.safe_yaml.as_written(value)[source]#
A scalar as the text it was written as, for a field that expects text.
trueandfalseare the only spellings of a boolean, so they are their own text. A number read from YAML remembers its source; one built in Python is written as Python writes it.nullis never text – it means no value – and anything else is passed through, for validation to accept or refuse.Example
>>> from scenet.safe_yaml import as_written, load >>> line = load("text: 1.50")["text"] >>> line, as_written(line), as_written(True), as_written(None) (1.5, '1.50', 'true', None)
- Return type:
- scenet.safe_yaml.describe(error)[source]#
The parser’s message, with a hint when the fault is a
*that opened an alias.PyYAML’s own wording is about anchors and aliases, which nobody writing a line of dialogue would recognise.
text: *told* youis the likeliest way to meet it: Markdown stresses a word that way, and in YAML an unquoted value starting with*is an alias.- Parameters:
error (
YAMLError) – What the parser raised.- Return type:
- Returns:
A message for the author.
Example
>>> import yaml >>> from scenet.safe_yaml import describe, load >>> try: ... load("text: *told* you") ... except yaml.YAMLError as exc: ... "quote the whole value" in describe(exc) True
- scenet.safe_yaml.load(text)[source]#
Parse a Scenet YAML document.
- Parameters:
text (
str) – The YAML source.- Return type:
- Returns:
The parsed document, or
Nonefor an empty one. Typed asobject, which is what an untrusted document is: every caller checks its shape before using it.- Raises:
ProfileError – A tag, an anchor, an alias, a merge key, a key that is not a scalar, or a tab where a token would start; or, as a
DuplicateKeyError, a key written twice in one mapping.NestingError – The document is nested too deeply to parse.
yaml.YAMLError – The text is not valid YAML for any other reason.
Example
>>> from scenet.safe_yaml import DuplicateKeyError, load >>> load("cast: {no: {reference: alice}}\nscript: [{say: {by: no, text: No}}]") {'cast': {'no': {'reference': 'alice'}}, 'script': [{'say': {'by': 'no', 'text': 'No'}}]} >>> try: ... load("cast:\n alice: {}\n alice: {}\n") ... except DuplicateKeyError as exc: ... print(exc.key, exc.first_line, exc.line) alice 2 3
- scenet.safe_yaml.mistyped(written, expected)[source]#
What to write instead of text that a yes-or-no or a number field was given.
Scenet YAML reads
yes,OnandTrueas text, and0x400,01750and1_0too – YAML 1.1 read them as a boolean or a number, so they are what an author used to write, and what a model trained on YAML 1.1 still writes. Each says what it is now, and what it would be written as.- Parameters:
- Return type:
- Returns:
The message, or
Nonefor text that spells neither.
Example
>>> from scenet.safe_yaml import mistyped >>> mistyped("yes", "bool").endswith("only true and false as a yes or a no: write true") True >>> mistyped("0x400", "number").endswith("only as JSON writes it, in decimal: write 1024") True >>> mistyped("hello", "number") is None True