Frontends#

Text in, validated IR out. Two syntaxes, one IR – which is what lets a comic script and a YAML document compile identically without anything downstream knowing there is more than one.

scenet.frontends.yaml_front#

The YAML surface syntax: text in, validated IR out.

This is one frontend among several planned – a comic-script frontend will target the same IR. Keeping parsing separate from the IR is what makes that possible without touching anything downstream.

scenet.frontends.script_front#

A frontend for comic script, the format writers already use.

Checked before building this: Fountain has no native panel, caption or SFX support, and there is no standardised comic script format at all. But the informal industry convention is stable across publishers – PANEL 1, a prose description, a character cue in capitals, the dialogue beneath it – so that is what this parses. Dark Horse publishes its own house format, and it is read as written: PAGE ONE (five panels), Panel 1. with its description, CAP: and CAP/NAME, numbered lettering, Markdown around any of it. So is a French script: PLANCHE 1, CASE 2, RÉCITATIF :, the échelle des plans. Each PAGE heading is a page.

The honest limitation. A prose description like “Alice and Bob face each other on a rainy street corner” cannot be compiled. Turning that into staging needs natural language understanding, and guessing would produce panels that are confidently wrong – far worse than refusing. So descriptions are preserved but not interpreted – each one is its panel’s description, read out as written by accessible output – and anything the compiler must know is declared explicitly: cast and staging in a front-matter block, per-panel settings as @ directives.

---
cast:
  ALICE: {reference: alice, at: left_third}
  BOB:   {reference: bob,   at: right_third}
staging:
  - ALICE left_of BOB
---

PANEL 1
@shot: full_shot
CAPTION: Midnight. The docks.
Alice and Bob face each other on a rainy street corner.

ALICE
You forgot your umbrella!

BOB (whisper)
I know.
class scenet.frontends.script_front.ScriptNote[source]#

Bases: object

Something the script frontend noticed that is worth saying but is not a fault.

rule#

Catalogue identifier, a rule whose level is note.

Type:

str

message#

What was noticed, phrased for whoever wrote the script.

Type:

str

line#

One-based line it is on.

Type:

int

path#

Where it is, structurally, so a finding’s fingerprint survives an edit above it: the panel’s name and what in it, or the page’s.

Type:

tuple[str | int, ...]

__init__(rule, message, line, path=())#
scenet.frontends.script_front.default_tiers(count)[source]#

How many panels each tier of a page holds, when the script does not say.

Dark Horse leaves “exact panel layout” to the artist, and the compiler is the artist, so it follows a fixed rule: three tiers at most up to nine panels – the lettering height of a page assumes three – with an odd panel in a tier of its own in the middle, and beyond nine, rows of three with the extras in the earlier tiers.

Example

>>> from scenet.frontends.script_front import default_tiers
>>> [default_tiers(count) for count in (1, 4, 5, 7, 10)]
[[1], [2, 2], [2, 1, 2], [2, 3, 2], [3, 3, 2, 2]]
Return type:

list[int]

scenet.frontends.script_front.normalise_text(text)[source]#

A script as it is read, whatever the editor that saved it did.

  • Line endings become line feeds. Python’s text mode does this silently when reading a file, which is why it took a browser to notice: the playground hands over the bytes it was given, and a script saved by a Windows editor – or pasted from one – arrives with CRLF. The front-matter pattern then does not match, the --- fence is read as prose, and the whole document is rejected for having content before the first PANEL heading.

  • A byte-order mark is dropped. Some Windows editors still write one, and it hid the front matter in exactly the same way.

  • Text is composed (NFC). A macOS editor may store É as E and a combining accent, and a name typed that way in a cue is the same name the front matter composed; compared as stored, the cue named nobody.

Example

>>> from scenet.frontends.script_front import normalise_text
>>> normalise_text("\ufeffZOE\u0308\r\n") == "ZO\u00cb\n"
True
Return type:

str

class scenet.frontends.script_front.PanelLines[source]#

Bases: object

Where a script panel was written, for a finding about it to point at.

heading#

The line of its PANEL heading.

Type:

int

events#

The line each entry of its script was written on, in order: a speech’s cue, a caption’s or a sound effect’s own line.

Type:

tuple[int, ...]

directives#

The line each of its directives was written on, by the path in the panel it sets: ("setting",) for @setting:.

Type:

Mapping[tuple[str, ...], int]

__init__(heading, events, directives=<factory>)#
class scenet.frontends.script_front.ScriptDocument[source]#

Bases: object

A comic script, read and validated, and where each of its panels was written.

panels#

Panel name to scene graph, in the order the panels appear.

Type:

dict[str, PanelIR]

lines#

Panel name to the lines it was written on.

Type:

dict[str, PanelLines]

notes#

What reading it noticed that is worth saying but is not a fault – a cue with nothing said under it, a page whose count its panels do not match.

Type:

tuple[ScriptNote, ...]

descriptions#

Panel name to its prose, which is kept and never interpreted.

Type:

dict[str, tuple[str, ...]]

title_page#

The prose before the first heading: Dark Horse’s format opens with the writer’s name and address.

Type:

tuple[str, ...]

page_notes#

Page to the prose between its heading and its first panel.

Type:

dict[str, tuple[str, ...]]

layout#

The pages its PAGE headings lay its panels out on; none without them.

Type:

PageLayout

property direction: Direction#

Which way the script is read: its front matter’s, which every panel takes.

__init__(panels, lines, notes=(), descriptions=<factory>, title_page=(), page_notes=<factory>, layout=<factory>)#
scenet.frontends.script_front.parse_script_document(text, *, source=None)[source]#

Parse comic script, keeping the line each panel and each of its events is on.

parse_script is this, less the lines. A checker reporting a fault in a panel that did validate – a balloon over the word limit, say – needs to know where it is.

Example

>>> from scenet.frontends.script_front import parse_script_document
>>> document = parse_script_document(
...     "---\ncast: {A: {reference: alice}}\n---\nPANEL 1\nA\nHi.\n"
... )
>>> document.lines["1"]
PanelLines(heading=4, events=(5,), directives={})
Return type:

ScriptDocument

scenet.frontends.common#

Shared between the frontends.

Both syntaxes – YAML documents and comic scripts – produce the same [PanelIR][scenet.ir.PanelIR], and both need the same two things on the way: a rewrite of the surface conveniences into the IR’s canonical shape, and a way to turn pydantic’s error list into a diagnostic a person can act on.

These lived in yaml_front and were imported into script_front through their private names, which put a module’s internals on another module’s import line. Whichever frontend came first would have owned them by accident. They belong here, where neither does.

scenet.frontends.common.LANGUAGE_REFERENCE = 'https://creatoan.github.io/scenet/reference/language/'#

Where the language reference is published, which a finding’s help names.

scenet.frontends.common.LAYOUT_KEYS = ('page', 'pages')#

Keys a scene may carry beside panels: that lay its panels out rather than being a default every panel inherits.

scenet.frontends.common.SITE = 'https://creatoan.github.io/scenet/'#

Where the documentation is published: the root, which serves the latest release. A message read in a terminal or an editor has no folder of documentation beside it, so a path in the repository would lead nowhere; a finding’s help names a page of the site instead.

scenet.frontends.common.errors_of(exc)[source]#

Pydantic’s error list, without the echoes of a fault already reported.

Two echoes are dropped:

  • When an item of a tuple[X, ...] field with a minimum length fails, pydantic also reports the tuple as too short, counting only the items that passed: a tier with a bad height says “Tuple should have at least 1 item after validation, not 0” beside the real fault. A list does not do this. The echo names no fault of its own, so it is dropped whenever a deeper error explains it.

  • A misspelled required key is one mistake that pydantic reports twice: the unknown key, and the required key it should have been. When an unknown key closely matches a required one missing from the same mapping, the two become one finding about the unknown key, which says what it was meant to be.

Parameters:

exc (ValidationError) – The validation error pydantic raised.

Return type:

list[ErrorDetails]

Returns:

Its errors, in pydantic’s order, less the echoes.

scenet.frontends.common.expand_place(setting)[source]#

Rewrite place: into the mass list it stands for.

A preset is a library for convenience, never a second format – so it is expanded here, in the frontend, exactly as a staging sentence is. Everything downstream sees one representation of a backdrop.

Parameters:

setting (object) – The value of the setting key, exactly as the YAML parser produced it – which is why this is object and not a mapping: rejecting a setting written as prose is half of what this function is for.

Return type:

object

Returns:

The same mapping with place replaced by masses, or unchanged when no place was named. The input is not modified.

Raises:

PanelSyntaxError – setting is not a mapping (prose is not accepted), the place is not one the library has, place was given alongside masses, or an image beside a place, masses or a horizon.

Example

>>> from scenet.frontends.common import expand_place
>>> [mass.kind.value for mass in expand_place({"place": "shore"})["masses"]]
['sky', 'water', 'ground']
scenet.frontends.common.normalise(data)[source]#

Rewrite surface conveniences into the IR’s canonical shape.

Three shapes differ between the surface syntax and the IR. Staging is written as sentences. Script entries are written as single-key mappings tagged by verb (- say: {...}, - caption: {...}) so that a new verb can be added without a discriminator field the author has to type; the tag is moved into the payload as verb, which is where the IR’s event models expect to find it. And a setting may name a place, which stands for a list of masses.

Parameters:

data (dict[str, Any]) – A parsed top-level panel mapping, in surface form.

Return type:

dict[str, Any]

Returns:

A new mapping in the shape PanelIR validates. The input is not modified.

Raises:

PanelSyntaxError – staging or script is not a list, a script entry is not a single-key mapping, it names a verb that does not exist, or setting names a place that does not exist or lists masses alongside one.

scenet.frontends.common.normalise_layout(data)[source]#

Rewrite the surface form of page: and pages: into what PageLayout validates.

One convenience: a panel in a tier or a column may be written as its name alone, which stands for {use: name}. Anything not shaped as expected is passed through untouched, so the model’s validation reports it with its location rather than this guessing.

Parameters:

data (dict[str, Any]) – The whole scene document.

Return type:

dict[str, Any]

Returns:

A mapping holding only the layout keys, in canonical form.

scenet.frontends.common.parse_relation(text)[source]#

Parse subject predicate object into a relation.

Written as a sentence rather than a mapping because staging is read far more often than it is written, and alice left_of bob is legible at a glance in a way that a three-key mapping is not.

Parameters:

text (str) – One staging entry.

Return type:

Relation

Returns:

The validated relation.

Raises:

PanelSyntaxError – The entry is not three whitespace-separated words, names an unknown predicate, or relates an actor to itself – the last under rule reflexive-relation.

Example

>>> from scenet.frontends.common import parse_relation
>>> relation = parse_relation("alice left_of bob")
>>> relation.subject, relation.predicate.value, relation.object
('alice', 'left_of', 'bob')
scenet.frontends.common.summarise(exc, *, what='panel')[source]#

Flatten pydantic’s error list into something a human can act on.

pydantic’s default rendering is accurate but verbose; for a language error the useful part is the location and the reason, one line each.

Parameters:
  • exc (ValidationError) – The validation error pydantic raised.

  • what (str) – What failed to validate, for the first line: panel, or Panel Core.

Return type:

str

Returns:

A multi-line diagnostic, one line per problem, each naming where it is.

scenet.frontends.positions#

Where in the source a value came from.

yaml.safe_load returns values and throws away everything about where they were written, which is fine for compiling and useless for reporting. yaml.compose parses the same text into a node tree instead, and every node carries start_mark and end_mark. So the document is composed a second time, purely to answer “where is script.0.say.by?”.

`ruamel.yaml` is the library usually reached for here, and its .lc line/column marks would do the same job. It was not used: PyYAML is already a dependency and this needs one function, so adding a second YAML implementation to the runtime – and to the licence gate – would buy nothing.

Marks are 0-based; SARIF is 1-based. The conversion happens once, here, at the boundary.

class scenet.frontends.positions.Position[source]#

Bases: object

A one-based line and column, as SARIF and every editor count them.

line#

One-based line number.

Type:

int

column#

One-based column number.

Type:

int

__init__(line, column)#
class scenet.frontends.positions.Region[source]#

Bases: object

A span of source, from start up to end.

start#

Where the value begins.

Type:

Position

end#

Where it ends. SARIF treats the end column as exclusive, which is also how PyYAML’s end marks behave, so the two agree without adjustment.

Type:

Position

__init__(start, end)#
scenet.frontends.positions.locate(text, path)[source]#

Find where path was written in text.

Walks as far down the path as the document allows and reports the deepest node it reached, so a path naming a field of something malformed still points at the malformed thing rather than giving up and pointing at the document.

Parameters:
  • text (str) – The YAML source.

  • path (tuple[str | int, ...]) – A pydantic-style location, as in ("script", 0, "say", "by").

Return type:

Region | None

Returns:

The region, or None if the text does not compose at all – in which case the caller has a syntax error to report and no use for a path.

Example

>>> from scenet.frontends.positions import locate
>>> region = locate("cast:\n  alice: {reference: alice}\n", ("cast", "alice"))
>>> region.start.line
2

scenet.safe_yaml#

Reading YAML without losing a key written twice.

The YAML specification requires every key in a mapping to be unique, so a document that repeats one is not valid YAML. PyYAML builds it anyway: the dictionary keeps the last value, and the first disappears without a word. Upstream has long been asked to refuse (yaml/pyyaml#41, still open).

For Scenet that is exactly the failure the rest of the validation exists to prevent. A strict language that rejects every unknown key would otherwise lose a whole known one – a panel, a character, a pose – and compile whatever was left. So every panel, scene, script front matter and puppet is read through load(), a SafeLoader that refuses a repeated key. ruamel.yaml refuses by default, but a second YAML implementation in the runtime, and in the licence gate, would buy this one check; the same trade was declined for source positions (frontends/positions.py).

Two subtleties:

  • A merge key is not a repeat. <<: *base followed by the key again is how YAML spells an override, and PyYAML copies the merged keys into the mapping before building it. The check looks at the keys as written, before that copy.

  • Keys are compared as loaded. 1, 1.0 and true are three keys in the text and one in the result, so they count as a repeat: one of them would be lost all the same.

scenet.safe_yaml.ALIAS_HINT = "a value that starts with '*' is read by YAML as an alias to an anchor, so quote the whole value. To stress a word, write it between underscores instead: _told_"#

What an author who wrote *told* meant, said in their terms rather than YAML’s.

exception scenet.safe_yaml.DuplicateKeyError[source]#

Bases: ConstructorError

A key written twice in one mapping.

A yaml.YAMLError, so anything that already handles malformed YAML handles this.

key#

The repeated key, as loaded.

line#

One-based line of the second occurrence, which is where the fault is.

column#

One-based column of the second occurrence.

first_line#

One-based line of the first occurrence.

summary#

One line saying what is wrong, without PyYAML’s position block, for a diagnostic that carries the position separately.

__init__(key, first, second)[source]#

Build the error from the two key nodes.

Parameters:
  • key (object) – The repeated key, as loaded.

  • first (Node) – The node of its first occurrence.

  • second (Node) – The node of the one that repeats it.

exception scenet.safe_yaml.NestingError[source]#

Bases: YAMLError

The document is nested more deeply than the parser can follow.

PyYAML parses and builds a document recursively, so nesting a few hundred levels deep reaches Python’s recursion limit. RecursionError is not a YAML error, and it escaped every handler that catches one – scenet check and build both ended in a traceback. No panel is nested more than a handful of levels, so a document this deep is a mistake or an attack, and is reported as one.

__init__()[source]#

Say what went wrong in the words the diagnostic will show.

scenet.safe_yaml.describe(error)[source]#

The parser’s message, with a hint when the fault is a * that opened an alias.

PyYAML’s own wording is about anchors and aliases, which nobody writing a line of dialogue would recognise. text: *told* you is the likeliest way to meet it: Markdown stresses a word that way, and in YAML an unquoted value starting with * is an alias.

Parameters:

error (YAMLError) – What the parser raised.

Return type:

str

Returns:

A message for the author.

Example

>>> import yaml
>>> from scenet.safe_yaml import describe, load
>>> try:
...     load("text: *told* you")
... except yaml.YAMLError as exc:
...     "quote the whole value" in describe(exc)
True
scenet.safe_yaml.load(text)[source]#

Parse YAML exactly as yaml.safe_load does, except that a repeated key is an error.

Parameters:

text (str) – The YAML source.

Return type:

object

Returns:

The parsed document, or None for an empty one. Typed as object, which is what an untrusted document is: every caller checks its shape before using it.

Raises:
  • DuplicateKeyError – A mapping has the same key twice.

  • NestingError – The document is nested too deeply to parse.

  • yaml.YAMLError – The text is not valid YAML for any other reason.

Example

>>> from scenet.safe_yaml import DuplicateKeyError, load
>>> load("cast: {alice: {reference: alice}}")
{'cast': {'alice': {'reference': 'alice'}}}
>>> try:
...     load("cast:\n  alice: {}\n  alice: {}\n")
... except DuplicateKeyError as exc:
...     print(exc.key, exc.first_line, exc.line)
alice 2 3