Frontends#

Text in, validated IR out. Two syntaxes, one IR – which is what lets a comic script and a YAML document compile identically without anything downstream knowing there is more than one.

scenet.frontends.yaml_front#

The YAML surface syntax: text in, validated IR out.

This is one frontend among several planned – a comic-script frontend will target the same IR. Keeping parsing separate from the IR is what makes that possible without touching anything downstream.

scenet.frontends.script_front#

A frontend for comic script, the format writers already use.

Checked before building this: Fountain has no native panel, caption or SFX support, and there is no standardised comic script format at all. But the informal industry convention is stable across publishers – PANEL 1, a prose description, a character cue in capitals, the dialogue beneath it – so that is what this parses. Dark Horse publishes its own house format, and it is read as written: PAGE ONE (five panels), Panel 1. with its description, CAP: and CAP/NAME, numbered lettering, Markdown around any of it. So is a French script: PLANCHE 1, CASE 2, RÉCITATIF :, the échelle des plans. Each PAGE heading is a page.

The honest limitation. A prose description like “Alice and Bob face each other on a rainy street corner” cannot be compiled. Turning that into staging needs natural language understanding, and guessing would produce panels that are confidently wrong – far worse than refusing. So descriptions are preserved but not interpreted – each one is its panel’s description, read out as written by accessible output – and anything the compiler must know is declared explicitly: cast and staging in a front-matter block, per-panel settings as @ directives.

---
cast:
  ALICE: {reference: alice, at: left_third}
  BOB:   {reference: bob,   at: right_third}
staging:
  - ALICE left_of BOB
---

PANEL 1
@shot: full_shot
CAPTION: Midnight. The docks.
Alice and Bob face each other on a rainy street corner.

ALICE
You forgot your umbrella!

BOB (whisper)
I know.
class scenet.frontends.script_front.ScriptNote[source]#

Bases: object

Something the script frontend noticed that is worth saying but is not a fault.

rule#

Catalogue identifier, a rule whose level is note.

Type:

str

message#

What was noticed, phrased for whoever wrote the script.

Type:

str

line#

One-based line it is on.

Type:

int

path#

Where it is, structurally, so a finding’s fingerprint survives an edit above it: the panel’s name and what in it, or the page’s.

Type:

tuple[str | int, ...]

__init__(rule, message, line, path=())#
scenet.frontends.script_front.default_tiers(count)[source]#

How many panels each tier of a page holds, when the script does not say.

Dark Horse leaves “exact panel layout” to the artist, and the compiler is the artist, so it follows a fixed rule: three tiers at most up to nine panels – the lettering height of a page assumes three – with an odd panel in a tier of its own in the middle, and beyond nine, rows of three with the extras in the earlier tiers.

Example

>>> from scenet.frontends.script_front import default_tiers
>>> [default_tiers(count) for count in (1, 4, 5, 7, 10)]
[[1], [2, 2], [2, 1, 2], [2, 3, 2], [3, 3, 2, 2]]
Return type:

list[int]

scenet.frontends.script_front.normalise_text(text)[source]#

A script as it is read, whatever the editor that saved it did.

  • Line endings become line feeds. Python’s text mode does this silently when reading a file, which is why it took a browser to notice: the playground hands over the bytes it was given, and a script saved by a Windows editor – or pasted from one – arrives with CRLF. The front-matter pattern then does not match, the --- fence is read as prose, and the whole document is rejected for having content before the first PANEL heading.

  • A byte-order mark is dropped. Some Windows editors still write one, and it hid the front matter in exactly the same way.

  • Text is composed (NFC). A macOS editor may store É as E and a combining accent, and a name typed that way in a cue is the same name the front matter composed; compared as stored, the cue named nobody.

Example

>>> from scenet.frontends.script_front import normalise_text
>>> normalise_text("\ufeffZOE\u0308\r\n") == "ZO\u00cb\n"
True
Return type:

str

class scenet.frontends.script_front.PanelLines[source]#

Bases: object

Where a script panel was written, for a finding about it to point at.

heading#

The line of its PANEL heading.

Type:

int

events#

The line each entry of its script was written on, in order: a speech’s cue, a caption’s or a sound effect’s own line.

Type:

tuple[int, ...]

directives#

The line each of its directives was written on, by the path in the panel it sets: ("setting",) for @setting:.

Type:

Mapping[tuple[str, ...], int]

__init__(heading, events, directives=<factory>)#
class scenet.frontends.script_front.ScriptDocument[source]#

Bases: object

A comic script, read and validated, and where each of its panels was written.

panels#

Panel name to scene graph, in the order the panels appear.

Type:

dict[str, PanelIR]

lines#

Panel name to the lines it was written on.

Type:

dict[str, PanelLines]

notes#

What reading it noticed that is worth saying but is not a fault – a cue with nothing said under it, a page whose count its panels do not match.

Type:

tuple[ScriptNote, ...]

descriptions#

Panel name to its prose, which is kept and never interpreted.

Type:

dict[str, tuple[str, ...]]

title_page#

The prose before the first heading: Dark Horse’s format opens with the writer’s name and address.

Type:

tuple[str, ...]

page_notes#

Page to the prose between its heading and its first panel.

Type:

dict[str, tuple[str, ...]]

layout#

The pages its PAGE headings lay its panels out on; none without them.

Type:

PageLayout

property direction: Direction#

Which way the script is read: its front matter’s, which every panel takes.

__init__(panels, lines, notes=(), descriptions=<factory>, title_page=(), page_notes=<factory>, layout=<factory>)#
scenet.frontends.script_front.parse_script_document(text, *, source=None)[source]#

Parse comic script, keeping the line each panel and each of its events is on.

parse_script is this, less the lines. A checker reporting a fault in a panel that did validate – a balloon over the word limit, say – needs to know where it is.

Example

>>> from scenet.frontends.script_front import parse_script_document
>>> document = parse_script_document(
...     "---\ncast: {A: {reference: alice}}\n---\nPANEL 1\nA\nHi.\n"
... )
>>> document.lines["1"]
PanelLines(heading=4, events=(5,), directives={})
Return type:

ScriptDocument

scenet.frontends.common#

Shared between the frontends.

Both syntaxes – YAML documents and comic scripts – produce the same [PanelIR][scenet.ir.PanelIR], and both need the same two things on the way: a rewrite of the surface conveniences into the IR’s canonical shape, and a way to turn pydantic’s error list into a diagnostic a person can act on.

These lived in yaml_front and were imported into script_front through their private names, which put a module’s internals on another module’s import line. Whichever frontend came first would have owned them by accident. They belong here, where neither does.

scenet.frontends.common.LANGUAGE_REFERENCE = 'https://creatoan.github.io/scenet/reference/language/'#

Where the language reference is published, which a finding’s help names.

scenet.frontends.common.LAYOUT_KEYS = ('page', 'pages')#

Keys a scene may carry beside panels: that lay its panels out rather than being a default every panel inherits.

scenet.frontends.common.SITE = 'https://creatoan.github.io/scenet/'#

Where the documentation is published: the root, which serves the latest release. A message read in a terminal or an editor has no folder of documentation beside it, so a path in the repository would lead nowhere; a finding’s help names a page of the site instead.

scenet.frontends.common.errors_of(exc)[source]#

Pydantic’s error list, without the echoes of a fault already reported.

Two echoes are dropped:

  • When an item of a tuple[X, ...] field with a minimum length fails, pydantic also reports the tuple as too short, counting only the items that passed: a tier with a bad height says “Tuple should have at least 1 item after validation, not 0” beside the real fault. A list does not do this. The echo names no fault of its own, so it is dropped whenever a deeper error explains it.

  • A misspelled required key is one mistake that pydantic reports twice: the unknown key, and the required key it should have been. When an unknown key closely matches a required one missing from the same mapping, the two become one finding about the unknown key, which says what it was meant to be.

Parameters:

exc (ValidationError) – The validation error pydantic raised.

Return type:

list[ErrorDetails]

Returns:

Its errors, in pydantic’s order, less the echoes.

scenet.frontends.common.expand_place(setting)[source]#

Rewrite place: into the mass list it stands for.

A preset is a library for convenience, never a second format – so it is expanded here, in the frontend, exactly as a staging sentence is. Everything downstream sees one representation of a backdrop.

Parameters:

setting (object) – The value of the setting key, exactly as the YAML parser produced it – which is why this is object and not a mapping: rejecting a setting written as prose is half of what this function is for.

Return type:

object

Returns:

The same mapping with place replaced by masses, or unchanged when no place was named. The input is not modified.

Raises:

PanelSyntaxError – setting is not a mapping (prose is not accepted), the place is not one the library has, place was given alongside masses, or an image beside a place, masses or a horizon.

Example

>>> from scenet.frontends.common import expand_place
>>> [mass.kind.value for mass in expand_place({"place": "shore"})["masses"]]
['sky', 'water', 'ground']
scenet.frontends.common.normalise(data)[source]#

Rewrite surface conveniences into the IR’s canonical shape.

Three shapes differ between the surface syntax and the IR. Staging is written as sentences. Script entries are written as single-key mappings tagged by verb (- say: {...}, - caption: {...}) so that a new verb can be added without a discriminator field the author has to type; the tag is moved into the payload as verb, which is where the IR’s event models expect to find it. And a setting may name a place, which stands for a list of masses.

Parameters:

data (dict[str, Any]) – A parsed top-level panel mapping, in surface form.

Return type:

dict[str, Any]

Returns:

A new mapping in the shape PanelIR validates. The input is not modified.

Raises:

PanelSyntaxError – staging or script is not a list, a script entry is not a single-key mapping, it names a verb that does not exist, or setting names a place that does not exist or lists masses alongside one.

scenet.frontends.common.normalise_layout(data)[source]#

Rewrite the surface form of page: and pages: into what PageLayout validates.

One convenience: a panel in a tier or a column may be written as its name alone, which stands for {use: name}. Anything not shaped as expected is passed through untouched, so the model’s validation reports it with its location rather than this guessing.

Parameters:

data (dict[str, Any]) – The whole scene document.

Return type:

dict[str, Any]

Returns:

A mapping holding only the layout keys, in canonical form.

scenet.frontends.common.parse_relation(text)[source]#

Parse subject predicate object into a relation.

Written as a sentence rather than a mapping because staging is read far more often than it is written, and alice left_of bob is legible at a glance in a way that a three-key mapping is not.

Parameters:

text (str) – One staging entry.

Return type:

Relation

Returns:

The validated relation.

Raises:

PanelSyntaxError – The entry is not three whitespace-separated words, names an unknown predicate, or relates an actor to itself – the last under rule reflexive-relation.

Example

>>> from scenet.frontends.common import parse_relation
>>> relation = parse_relation("alice left_of bob")
>>> relation.subject, relation.predicate.value, relation.object
('alice', 'left_of', 'bob')
scenet.frontends.common.summarise(exc, *, what='panel')[source]#

Flatten pydantic’s error list into something a human can act on.

pydantic’s default rendering is accurate but verbose; for a language error the useful part is the location and the reason, one line each.

Parameters:
  • exc (ValidationError) – The validation error pydantic raised.

  • what (str) – What failed to validate, for the first line: panel, or Panel Core.

Return type:

str

Returns:

A multi-line diagnostic, one line per problem, each naming where it is.

scenet.frontends.positions#

Where in the source a value came from.

yaml.safe_load returns values and throws away everything about where they were written, which is fine for compiling and useless for reporting. yaml.compose parses the same text into a node tree instead, and every node carries start_mark and end_mark. So the document is composed a second time, purely to answer “where is script.0.say.by?”.

`ruamel.yaml` is the library usually reached for here, and its .lc line/column marks would do the same job. It was not used: PyYAML is already a dependency and this needs one function, so adding a second YAML implementation to the runtime – and to the licence gate – would buy nothing.

Marks are 0-based; SARIF is 1-based. The conversion happens once, here, at the boundary.

class scenet.frontends.positions.Position[source]#

Bases: object

A one-based line and column, as SARIF and every editor count them.

line#

One-based line number.

Type:

int

column#

One-based column number.

Type:

int

__init__(line, column)#
class scenet.frontends.positions.Region[source]#

Bases: object

A span of source, from start up to end.

start#

Where the value begins.

Type:

Position

end#

Where it ends. SARIF treats the end column as exclusive, which is also how PyYAML’s end marks behave, so the two agree without adjustment.

Type:

Position

__init__(start, end)#
scenet.frontends.positions.locate(text, path)[source]#

Find where path was written in text.

Walks as far down the path as the document allows and reports the deepest node it reached, so a path naming a field of something malformed still points at the malformed thing rather than giving up and pointing at the document.

Parameters:
  • text (str) – The YAML source.

  • path (tuple[str | int, ...]) – A pydantic-style location, as in ("script", 0, "say", "by").

Return type:

Region | None

Returns:

The region, or None if the text does not compose at all – in which case the caller has a syntax error to report and no use for a path. So when one path is not found, no shorter path in the same text is either.

Example

>>> from scenet.frontends.positions import locate
>>> region = locate("cast:\n  alice: {reference: alice}\n", ("cast", "alice"))
>>> region.start.line
2

scenet.safe_yaml#

Scenet YAML: reading a document so that ordinary dialogue is never mistyped.

PyYAML reads YAML 1.1, in which No is false, 1984 a number, 12:30 the sexagesimal 750, 2024-05-01 a date and 01750 the octal 1000. For a language whose values are mostly lines of dialogue, that refused text: No and silently drew a panel 1000 units tall. And it reads features no panel needs – tags that build any object they name, anchors and aliases that expand without bound, merge keys – which are a confusion and an attack surface.

So every panel, scene, script front matter and puppet is read through load(), by a loader built from PyYAML’s own parts with three of them replaced. The profile they implement is stated, normatively, in docs/reference/language/yaml_profile.md:

  • The resolver (ProfileResolver) types a plain scalar as the YAML 1.2 JSON schema does: true, false, null (or nothing at all), and integers and floats as JSON spells them. Every other plain scalar is text, exactly as written.

  • The scanner refuses a tag, an anchor or an alias where it is written, and a tab where a token would start, in words an author recognises.

  • The composer and the constructor refuse a merge key, a key that is not a scalar and a key written twice; every key is text, exactly as written, so cast: {on: ..., no: ...} names two actors. A number keeps its source (WrittenInt, WrittenFloat), so where the language expects text, as_written() gives back what was typed: text: 1984 letters 1984 and text: 1.50 letters 1.50.

Each refusal is a ProfileError, which is a yaml.YAMLError – so anything that already handles malformed YAML handles it – and carries the rule it breaks and the span it covers, for a finding at exactly the text at fault.

ruamel.yaml reads YAML 1.2 and refuses a repeated key by default, but a second YAML implementation in the runtime, and in the licence gate, would buy little: every rule here is a few lines on top of PyYAML’s machinery, and the same trade was declined for source positions (frontends/positions.py).

scenet.safe_yaml.ALIAS_HINT = "a value that starts with '*' is read by YAML as an alias to an anchor, so quote the whole value. To stress a word, write it between underscores instead: _told_"#

What an author who wrote *told* meant, said in their terms rather than YAML’s.

exception scenet.safe_yaml.DuplicateKeyError[source]#

Bases: ProfileError

A key written twice in one mapping.

The YAML specification requires every key in a mapping to be unique, so a document that repeats one is not valid YAML. PyYAML builds it anyway: the dictionary keeps the last value, and the first disappears without a word (yaml/pyyaml#41, still open). A strict language that rejects every unknown key would otherwise lose a whole known one – a panel, a character, a pose – and compile whatever was left.

key#

The repeated key, as written.

first_line#

One-based line of the first occurrence. line and column are the second’s, which is where the fault is.

__init__(key, first, second)[source]#

Build the error from the two key nodes.

Parameters:
  • key (str) – The repeated key, as written.

  • first (Node) – The node of its first occurrence.

  • second (Node) – The node of the one that repeats it.

exception scenet.safe_yaml.NestingError[source]#

Bases: YAMLError

The document is nested more deeply than the parser can follow.

PyYAML parses and builds a document recursively, so nesting a few hundred levels deep reaches Python’s recursion limit. RecursionError is not a YAML error, and it escaped every handler that catches one – scenet check and build both ended in a traceback. No panel is nested more than a handful of levels, so a document this deep is a mistake or an attack, and is reported as one.

__init__()[source]#

Say what went wrong in the words the diagnostic will show.

exception scenet.safe_yaml.ProfileError[source]#

Bases: MarkedYAMLError

Something the profile refuses, at the span of source it covers.

rule#

The rule it breaks, a key into RULES.

summary#

One line saying what is wrong and what to write instead, without PyYAML’s position block, for a diagnostic that carries the position separately.

line#

One-based line where the span starts.

column#

One-based column where it starts.

end_line#

One-based line where it ends.

end_column#

One-based column just past its last character.

__init__(rule, summary, start, end)[source]#

Build the error from what it says and the marks around the text at fault.

Parameters:
  • rule (str) – The rule it breaks.

  • summary (str) – What is wrong, in an author’s words.

  • start (Mark) – PyYAML’s mark at the first character of the span.

  • end (Mark) – Its mark just past the last.

class scenet.safe_yaml.ProfileResolver[source]#

Bases: BaseResolver

Types a plain scalar as the YAML 1.2 JSON schema does, and every other one as text.

PyYAML’s own resolver implements YAML 1.1. This one starts from BaseResolver, which resolves nothing, and registers four patterns: so yes, True, ~, 0x400, 01750, 1_0, .inf, 12:30 and 2024-05-01 are all text, and a quoted scalar always is.

Example

>>> from scenet.safe_yaml import load
>>> load("[true, True, null, ~, 1984, 01750, 1.5, .5]")
[True, 'True', None, '~', 1984, '01750', 1.5, '.5']
class scenet.safe_yaml.WrittenFloat[source]#

Bases: float

A float, and the text it was written as: 1.50 is 1.5, and letters 1.50.

Example

>>> from scenet.safe_yaml import WrittenFloat
>>> number = WrittenFloat(1.5, "1.50")
>>> number * 2, number.source
(3.0, '1.50')
static __new__(cls, value, source)[source]#

Make the float, remembering how it was written.

Return type:

WrittenFloat

class scenet.safe_yaml.WrittenInt[source]#

Bases: int

An integer, and the text it was written as.

It is the integer everywhere a number is expected, and its source where text is: see as_written(). JSON writes it as the integer it is.

Example

>>> from scenet.safe_yaml import WrittenInt
>>> number = WrittenInt(1984, "1984")
>>> number + 1, number.source
(1985, '1984')
static __new__(cls, value, source)[source]#

Make the integer, remembering how it was written.

Return type:

WrittenInt

scenet.safe_yaml.as_written(value)[source]#

A scalar as the text it was written as, for a field that expects text.

true and false are the only spellings of a boolean, so they are their own text. A number read from YAML remembers its source; one built in Python is written as Python writes it. null is never text – it means no value – and anything else is passed through, for validation to accept or refuse.

Example

>>> from scenet.safe_yaml import as_written, load
>>> line = load("text: 1.50")["text"]
>>> line, as_written(line), as_written(True), as_written(None)
(1.5, '1.50', 'true', None)
Return type:

object

scenet.safe_yaml.describe(error)[source]#

The parser’s message, with a hint when the fault is a * that opened an alias.

PyYAML’s own wording is about anchors and aliases, which nobody writing a line of dialogue would recognise. text: *told* you is the likeliest way to meet it: Markdown stresses a word that way, and in YAML an unquoted value starting with * is an alias.

Parameters:

error (YAMLError) – What the parser raised.

Return type:

str

Returns:

A message for the author.

Example

>>> import yaml
>>> from scenet.safe_yaml import describe, load
>>> try:
...     load("text: *told* you")
... except yaml.YAMLError as exc:
...     "quote the whole value" in describe(exc)
True
scenet.safe_yaml.load(text)[source]#

Parse a Scenet YAML document.

Parameters:

text (str) – The YAML source.

Return type:

object

Returns:

The parsed document, or None for an empty one. Typed as object, which is what an untrusted document is: every caller checks its shape before using it.

Raises:
  • ProfileError – A tag, an anchor, an alias, a merge key, a key that is not a scalar, or a tab where a token would start; or, as a DuplicateKeyError, a key written twice in one mapping.

  • NestingError – The document is nested too deeply to parse.

  • yaml.YAMLError – The text is not valid YAML for any other reason.

Example

>>> from scenet.safe_yaml import DuplicateKeyError, load
>>> load("cast: {no: {reference: alice}}\nscript: [{say: {by: no, text: No}}]")
{'cast': {'no': {'reference': 'alice'}}, 'script': [{'say': {'by': 'no', 'text': 'No'}}]}
>>> try:
...     load("cast:\n  alice: {}\n  alice: {}\n")
... except DuplicateKeyError as exc:
...     print(exc.key, exc.first_line, exc.line)
alice 2 3
scenet.safe_yaml.mistyped(written, expected)[source]#

What to write instead of text that a yes-or-no or a number field was given.

Scenet YAML reads yes, On and True as text, and 0x400, 01750 and 1_0 too – YAML 1.1 read them as a boolean or a number, so they are what an author used to write, and what a model trained on YAML 1.1 still writes. Each says what it is now, and what it would be written as.

Parameters:
  • written (str) – The text the field was given.

  • expected (str) – "bool" for a yes-or-no field, "number" for a number.

Return type:

str | None

Returns:

The message, or None for text that spells neither.

Example

>>> from scenet.safe_yaml import mistyped
>>> mistyped("yes", "bool").endswith("only true and false as a yes or a no: write true")
True
>>> mistyped("0x400", "number").endswith("only as JSON writes it, in decimal: write 1024")
True
>>> mistyped("hello", "number") is None
True