NestedDSL

NestedDSL

A composable specification language for the nested layered-control systems of NestedSystems — teams, refined subsystems, targets, per-edge restriction maps, and the dynamics cascade — written as ordinary Julia that runs.

Why a builder block rather than a quoted program

TrackingDSL, the other DSL in this package, parses a block into a symbolic AST and resolves numeric values later from a context dict. This one deliberately does the opposite: @nested_system executes its block, and each declaration appends a term to the fragment under construction.

The reason is the shape of the problems being specified. A nested system's interesting structure is combinatorial — n escort rings, n support pods, a pin on every other agent around each ring — and all of it is a function of parameters known at build time. A declarative language would have to grow loops, conditionals, and functions to express that. Executing the block instead means Julia's own for, if, and functions do that work, and the DSL is left to express only what it is actually good at: the structure of one node.

spec = @nested_system begin
    @dim 4
    for i in 1:n                                     # Julia's loop, not the DSL's
        @team $(Symbol(:ring, i)) = ring(m[i]; radius=r(i))
        @target $(Symbol(:t, i))
        for k in 1:2:m[i]
            @observe via($(Symbol(:ring, i)), redundant_pin(m[i], 4, k)) => $(Symbol(:t, i))
        end
    end
    @bind dynamics=QuadrotorDynamics() K_lqr=K
end
c = compile_nested_system(spec)

Composition

@nested_system returns a SystemFragment — a first-class value describing the contents of one node, with no commitment to where in a tree that node sits. Fragments compose three ways:

Paths inside a fragment are relative, and lowering rewrites them against wherever the fragment lands — so a helper function returning "an escort ring that tracks target t" can be dropped in at the root, or three levels down, unchanged:

escort(name, m, r, tgt) = @nested_system begin
    @team $(name) = ring(m; radius=r)
    @target $(tgt)
    @observe $(name) => $(tgt)
end

fleet = @nested_system begin
    @system wingA begin
        @system flight1 begin
            @include escort(:alpha, 5, 1.0, :t1)
            @include escort(:beta, 5, 1.0, :t2)
            @link centroid(alpha) => centroid(beta)
        end
        @include escort(:gamma, 4, 0.8, :t3)
        @link centroid(flight1) => centroid(gamma)
    end
end

Pipeline

@nested_system  →  SystemFragment  →  validate  →  lower  →  NestedSystemSpec + SheafTower

Coverage of the underlying API

Every feature of the NestedSystems type API has a surface form here: @team for LeafTeam (all four formation kinds, radius, observers), @system for RefinedSystem at unbounded depth, @link for SystemEdge with independent per-endpoint restriction maps, @target/@observe for TargetSpec/Observation with arbitrary many-to-many incidence, @dim/@affine for the remaining NestedSystemSpec fields, and @bind for the whole SystemBinding/AgentBinding cascade. The endpoint forms project, centroid, raw, and via cover every RestrictionSpec, with via accepting an arbitrary one — including helpers like redundant_pin — so the DSL is never a bottleneck on what can be expressed.

NestedDSLTerm

AST node types for the nested-system DSL, plus the SystemFragment value that collects them and the error type the later stages throw.

A fragment is an ordered list of terms, each describing one declaration made at one node of the system tree. Fragments are immutable values: merge concatenates them, and a SubsystemTerm nests one inside another. Nothing here is a NestedSystemSpec yet — names are still names and no index has been assigned — which is what lets a fragment be built in pieces, by ordinary Julia code, and assembled later.

Every term stores already-evaluated Julia values (a Float64 radius, a RestrictionSpec, an AbstractAgentDynamics), not symbols to be resolved against a context dict later. The DSL's macros expand to calls to the constructors in this module, so by the time a term exists its numeric content has been computed by ordinary Julia evaluation in the caller's own scope. Only structural names — child names, target names, member designators — remain symbolic, and those are resolved by NestedDSLLowering.

AffineTerm(affine)

Declares whether restriction maps are affine — the DSL's @affine. Bubbles and conflicts exactly like DimTerm.

BindTerm(path, agent, dynamics, K_lqr, initial_position)

Declares dynamics/gain/initial-position bindings — the DSL's @bind — lowering into the SystemBinding cascade.

An empty path binds the declaring node itself (and hence every agent beneath it, unless overridden); a non-empty path binds a descendant. agent is a local agent index within a leaf team, or nothing to bind the node as a whole. Any field left nothing is simply not declared here and is inherited, exactly as AgentBinding specifies.

DimTerm(D)

Declares the stalk dimension — the DSL's @dim. Bubbles to the root during lowering no matter where it is declared; declaring two different values anywhere in one specification is an error.

LinkTerm(src, dst)

A consensus edge between two direct children of the declaring node — the DSL's @link — lowering to a SystemEdge. Both MemberRefs must have single-component paths, which the validator enforces: SystemEdge connects siblings only.

MemberRef(path, map=project(1))

A reference to a system in the tree, together with the RestrictionSpec it presents on the edge being declared — the DSL's spelling of "this subsystem, seen through this map".

path is a dotted chain of child names relative to the node where the referring term was declared: [:mid, :ringA] for mid.ringA. An empty path is not meaningful and is rejected by the validator.

Endpoint syntax and the MemberRef it builds:

NestedDSLError(msg)

Raised by every stage of the nested DSL — parsing, validation, and lowering — so that a caller can distinguish a malformed specification from an unrelated ArgumentError thrown deep inside the tower compiler.

NestedTerm

Supertype of every declaration a SystemFragment can hold.

ObserveTerm(system, target)

Declares that system observes the target named target — the DSL's @observe — lowering to an Observation. system.path is relative to the declaring node and may be of any depth; system.map is what the observing system presents on the edge.

SubsystemTerm(name, body)

Declares a refined child system whose contents are the fragment body — the DSL's @system — lowering to a RefinedSystem.

Nesting is what makes the DSL's depth unbounded: body may itself contain SubsystemTerms, to any depth. Targets and bindings declared inside body are not rewritten here; lowering walks the tree carrying the current path and prefixes them then, which is what lets the same fragment be spliced in at different depths without change.

SystemFragment(terms=NestedTerm[])

An ordered, immutable bag of declarations describing the contents of one node of a system tree — the value @nested_system returns and the unit of composition in this DSL.

A fragment is not tied to a position in any tree: the paths inside its terms are relative to whatever node it ends up at. That is precisely what makes fragments composable —

• merge(f, g) concatenates two fragments describing the same node;
• @include g splices g's declarations into the fragment being built;
• @system name = g makes g the body of a new child.

so a Julia function may build a fragment from ordinary arguments, a for loop may build one per iteration, and the results merged — with no looping or conditional constructs in the DSL itself.

TargetTerm(name)

Declares a target — the DSL's @target — lowering to a TargetSpec. Targets are global to the whole specification no matter how deeply the declaring fragment is nested, so that a fragment can carry the target it observes along with it.

TeamTerm(name, kind, n_agents, radius, observers)

Declares a leaf team of raw agents — the DSL's @team — lowering to a LeafTeam. See team_term.

merge(fragments::SystemFragment...) -> SystemFragment

Concatenate fragments describing the same node, preserving declaration order. This is the DSL's composition operator: build pieces independently — in a loop, in a helper function, behind an if — and merge them into the fragment for one node.

Merging does not check for duplicate names; that is the validator's job, and deferring it means a partial fragment mid-composition is never spuriously invalid.

affine_term(affine::Bool) -> AffineTerm

Functional constructor for AffineTerm.

bind_term(path, agent; dynamics=nothing, K_lqr=nothing, initial_position=nothing) -> BindTerm

Functional constructor for BindTerm. Rejects a binding that declares nothing at all, since that is always a mistake rather than a no-op the user meant to write.

dim_term(D::Integer) -> DimTerm

Functional constructor for DimTerm.

link_term(src::MemberRef, dst::MemberRef) -> LinkTerm

Functional constructor for LinkTerm.

member_ref(path, map=project(1)) -> MemberRef

Functional constructor for MemberRef; the DSL's endpoint syntax expands to a call to this. path may be a single Symbol or a vector of them.

observe_term(system::MemberRef, target::Symbol) -> ObserveTerm

Functional constructor for ObserveTerm.

subsystem_term(name, body::SystemFragment) -> SubsystemTerm

Functional constructor for SubsystemTerm.

target_term(name::Symbol) -> TargetTerm

Functional constructor for TargetTerm.

team_term(name, kind, n_agents, radius=1.0; observers=[1]) -> TeamTerm

Build a TeamTerm, validating kind and n_agents up front so that a typo surfaces at the declaration rather than at tower assembly.

@team ringA = ring(4; radius=1.0) expands to a call to this function.

terms(f::SystemFragment) -> Vector{NestedTerm}

The fragment's declarations, in declaration order. Returns the internal vector; treat it as read-only.

NestedDSLParser

MLStyle-driven surface syntax for the nested-system DSL: the @nested_system block and the declaration macros (@team, @system, @link, @target, @observe, @bind, @dim, @affine, @include) that go inside it.

The design in one sentence

@nested_system begin … end is a builder block, not a quoted program: it binds a hidden term list and then runs its body as ordinary Julia code, with each declaration macro appending to that list. Everything that is not a declaration macro is left completely untouched.

That single choice is what buys composability. Because the block executes, a for loop is a for loop, an if is an if, and a value computed three lines up is just a variable — the DSL needs no looping or conditional constructs of its own, and none were added. And because the block returns a value — a SystemFragment — a helper function can build one from plain Julia arguments and hand it back to be @included or made into a @system.

escort(name, m, r, tgt) = @nested_system begin
    @team $(name) = ring(m; radius=r)
    @observe $(name) => $(tgt)
end

wing = @nested_system begin
    for i in 1:3                                  # a real Julia loop
        @include escort(Symbol(:ring, i), 5, 1.0, Symbol(:t, i))
    end
    @link centroid(ring1) => centroid(ring2)
end

Names, and when a name is evaluated

Wherever the grammar expects a name — a team or system name, a path component, a target — a bare identifier is taken literally, and $(expr) means "evaluate expr now and use the Symbol it produces". Nothing else is accepted in a name position, so a computed name is always visibly marked rather than silently guessed at.

Every other position is an ordinary Julia expression, evaluated in the caller's scope at the point the declaration runs: radii, agent counts, matrices, dynamics objects, redundant_pin(…) calls, and so on.

Statement reference

Endpoint syntax for @link/@observe is documented on MemberRef.

_endpoint_expr(ex) -> Expr

Lower a @link/@observe endpoint into a member_ref call. See MemberRef for the surface forms and the restriction map each one produces.

_name_expr(ex, what) -> Expr

Lower a name-position expression: a bare identifier becomes a literal Symbol, $(e) becomes a runtime Symbol(e), and anything else is a syntax error.

Refusing the general case is deliberate. If ring1 in a name position could mean "the variable ring1", then every mistyped name would silently become a runtime UndefVarError far from the declaration, and every DSL name would shadow-collide with the caller's locals.

_path_parts(ex) -> Vector

Flatten a dotted path expression (mid.ringA, $(nm).inner) into its component name expressions, outermost first.

_restriction_expr(ex) -> Expr

Lower a member designator inside project(path, i): :name selects a child by name (kept as a literal), anything else is an ordinary expression evaluated for an integer index.

_split_call_args(args) -> (positional, kwargs)

Split a call's argument expressions into positional expressions and name => value keyword pairs, accepting both f(x; k=v) (a :parameters block) and f(x, k=v) (inline :kw).

@affine flag

Declare whether restriction maps are affine (see build_escort_topology). Defaults to true; conflicts are caught exactly as for @dim.

@bind dynamics=dyn K_lqr=K
@bind path.to.system K_lqr=K_soft
@bind team[2] initial_position=p

Bind dynamics, LQR gain, and/or initial position, feeding the most-specific-wins cascade of SystemBinding.

With no path, the binding applies to the node being built and everything beneath it — the usual way to give a whole tree its default dynamics. With a path, it applies to that descendant and its subtree. With a trailing [i], it applies to agent i of a leaf team alone.

Fields are independent: binding only initial_position on one agent leaves it inheriting its dynamics and gain from above.

@dim D

Declare the stalk dimension of the whole specification. Defaults to 4 (SE(3) homogeneous coordinates) if never declared; declaring two different values anywhere is an error, caught by the validator.

@include fragment…

Splice each fragment's declarations into the node currently being built, as though they had been written out at this point.

This is the DSL's composition primitive. Combined with a Julia for loop it replaces any need for iteration in the language itself, and combined with a helper function that returns a fragment it replaces any need for DSL-level abstraction:

@nested_system begin
    for i in 1:n
        @include escort_ring(Symbol(:ring, i), m[i], radius(i))
    end
end

Paths inside a spliced fragment are interpreted relative to this node, since that is where the declarations now live.

@link src => dst

Declare a consensus edge between two direct children of the node being built, with each endpoint presenting whatever its designator says (see MemberRef for the endpoint forms).

@link ringA => ringB                        # each presents its own first member
@link centroid(ringA) => centroid(ringB)    # each presents its members' average
@link pod => project(hub, 3)                # hub presents its third member

Edges connect siblings only — a deeper path is a validation error — because that is exactly what SystemEdge expresses. Coupling two systems in different branches is done by giving them a common parent and linking there.

@nested_system begin … end -> SystemFragment

Run a block of declarations, returning the SystemFragment they build.

The block is executed as ordinary Julia; only the declaration macros listed in the module docstring are interpreted, and everything else — loops, conditionals, local variables, function calls — behaves exactly as it would outside the block, with one exception: the whole block is wrapped in a let, so it opens its own local scope. Variables from the enclosing scope are still visible and mutable, but a bare assignment inside the block creates a new local rather than leaking out (the same as writing a let block yourself at that spot). A nested @nested_system (which @system … begin … end uses internally) shadows the outer builder, so declarations always attach to the innermost enclosing node.

The returned fragment describes one node. It carries no notion of where in a tree it sits, so it may be merged with another fragment for the same node (merge), spliced into one (@include), or made the body of a child (@system name = frag).

To turn a fragment into a solvable problem, see compile_nested_system.

@observe system => target

Declare that system observes the target named target, presenting whatever its designator says (see MemberRef).

Unlike @link, the system may be at any depth below the declaring node, and the incidence is unrestricted: one system may observe several targets and one target may be observed by several systems.

@observe ringA => t1                                   # ringA's first agent tracks t1
@observe centroid(mid.ringA) => t1                     # its centroid tracks t1
@observe via(ringA, redundant_pin(5, 4, k)) => t1      # any RestrictionSpec you like

@system name begin … end
@system name = fragment

Declare a refined child system: either inline, with its own nested block of declarations, or from a SystemFragment computed elsewhere.

Both forms nest to any depth — the inline block is itself a @nested_system block, and a fragment may contain further @system declarations of its own.

@target name…

Declare one or more targets — uncontrolled vertices whose values are supplied as boundary conditions at solve time.

Targets are global: a target declared inside a deeply nested fragment is still a top-level target of the finished specification, which is what lets a reusable fragment carry the target it tracks along with it. Declaring the same target name twice is an error.

@team name = ring(n; radius=r, observers=[…])
@team name = path(n, r)
@team name = team(kind_expr, n; radius=r)

Declare a leaf team of n raw agents in a ring, path, star, or clique formation.

n and radius are ordinary Julia expressions. Use the team(kind_expr, …) form when the formation kind is itself computed. radius may be given positionally or by keyword and defaults to 1.0; observers defaults to [1].

NestedDSLValidator

Purely symbolic checks over a SystemFragment: duplicate names, unresolvable paths, member designators out of range, conflicting @dim/@affine, malformed bindings.

Nothing here computes a rank, a nullspace, or a restriction matrix. That is the same discipline RestrictionSpec follows and it exists for the same reason: a specification error should be reported against the declaration the user wrote, with a path and a name, long before the tower compiler turns any of it into a matrix. By the time NestedDSLLowering runs, every name in the fragment is known to resolve.

This module also owns the structural view of a fragment — fragment_children, node_arity, resolve_fragment_path — which lowering reuses, so the two stages can never disagree about what a path means or in what order children are numbered.

_validate_spec(spec, node, where_)

Check a RestrictionSpec's member designator against node's declared arity. Only project carries a designator; centroid and RawRestriction are checked numerically later, when they meet a concrete D.

collect_target_names(f::SystemFragment) -> Vector{Symbol}

Every target declared anywhere in f's tree, in declaration order (depth-first, following fragment_children). Targets are global regardless of nesting depth; see TargetTerm.

fragment_children(f::SystemFragment) -> Vector{Pair{Symbol,NestedTerm}}

The fragment's direct children — its TeamTerms and SubsystemTerms — paired with their names, in declaration order.

Declaration order is load-bearing, not incidental: it fixes each child's index in the RefinedSystem that lowering builds, which in turn fixes the depth-first agent numbering that SheafTower.agent_vertices uses. Validation and lowering both go through this function so the two can never disagree.

fragment_dim(f::SystemFragment; default=4) -> Int
fragment_affine(f::SystemFragment; default=true) -> Bool

The stalk dimension / affine flag declared anywhere in f's tree, or default if never declared. Two declarations of the same value are fine (a fragment may reasonably restate the dimension it was written for); two different values are an error.

node_arity(node::NestedTerm) -> Int

Number of a node's direct members: raw agents for a TeamTerm, children for a SubsystemTerm. This is the arity a RestrictionSpec is declared against — a refined child counts as one member however many agents it expands to.

resolve_fragment_path(f, path; context) -> (node, indices)

Walk path from fragment f, returning the node it names and the child indices taken to get there. Throws a NestedDSLError naming context and listing the available children when a component does not resolve, or when the path tries to descend into a leaf team.

validate_fragment(f::SystemFragment) -> SystemFragment

Check f as a complete, root-level specification and return it unchanged.

Checks performed, all symbolic:

• every node has at least one child, and no two children of one node share a name;
• @link endpoints name direct children of the linking node, and differ from each other;
• @observe paths resolve, and name a target that is actually declared;
• every project designator is in range for the node it is declared against (an integer within the node's arity, or the name of an actual child);
• @bind paths resolve, agent indices lie within their team, and [i] is used only on a team;
• target names are unique and at least one target exists;
• @dim/@affine do not conflict.

Called automatically by compile_nested_system; call it directly to check a fragment without building anything.

NestedDSLLowering

Turns a validated SystemFragment into the concrete NestedSystems API: a NestedSystemSpec, its compiled SheafTower, the SystemBinding cascade, and the name → index tables that let downstream code keep talking in names.

Lowering is where names become indices. It walks the fragment tree once, carrying the path taken so far, and in that single pass:

• orders each node's children (via fragment_children) to fix their RefinedSystem indices;
• rewrites @link endpoints into SystemEdge index pairs;
• rewrites @observe paths — declared relative to whatever node they appear in — into absolute Observation.system_path index vectors, which is what lets one fragment be spliced in at different depths without editing;
• hoists every @target to the top level and numbers it;
• folds @bind terms into the nested SystemBinding structure;
• records each node's contiguous block of agent indices, mirroring NestedSystems's own depth-first assignment.

That last table is what CompiledNestedSystem exposes as agent_range: the DSL's answer to hand-maintaining an agent_index_ranges([4, 5, 2, …]) call whose argument has to be kept in sync with the tree by hand.

CompiledNestedSystem

Everything a SystemFragment compiles to, with the name tables kept alongside so that callers never have to fall back to raw indices.

Use agent_range, agent_vertices, target_index, and target_vector rather than reaching into ranges/targets directly.

_assign_ranges!(ranges, f, path, next)

Record every node's contiguous block of agent indices, walking children depth-first in declaration order — the same order NestedSystems itself assigns agents in, so these ranges index directly into SheafTower.agent_vertices.

_collect_observations!(out, f, prefix, tindex)

Walk the tree accumulating Observations. prefix is the index path of the node being visited, so an @observe written relative to a deeply nested fragment becomes an absolute path here — the mechanism that makes a fragment position-independent.

_is_trivial(sb::SystemBinding) -> Bool

Whether a binding declares nothing anywhere in its subtree. Trivial child bindings are dropped rather than stored, so that resolve_dynamics never walks a tree of empty placeholders — and so that a SystemBinding built by the DSL is as small and as readable as a hand-written one.

_lower_bindings(f) -> SystemBinding

Flatten every @bind into an absolute-path table, then rebuild the tree-shaped SystemBinding from it.

Going through a flat intermediate is what lets a binding be declared anywhere and still land in the right place: @bind mid.ringA K_lqr=K written at the root and @bind ringA K_lqr=K written inside mid produce the same absolute path and therefore the same binding.

Where two @bind terms set the same field of the same node, the later declaration wins, matching how a plain Julia assignment would read.

_lower_node(f, name) -> RefinedSystem

Build the RefinedSystem for one fragment: its children in declaration order, and its @link terms as SystemEdges between those children's indices.

agent_range(c::CompiledNestedSystem, path...) -> UnitRange{Int}

The contiguous block of agent indices belonging to the system at path, given as symbols (agent_range(c, :mid, :ringA)), a dotted string (agent_range(c, "mid.ringA")), or nothing at all for the whole tree.

These are agent indices — the convention NestedEscortResult.sim_data[step][a] uses. To index solve_hierarchical's output instead, which is keyed by sheaf vertex, go through agent_vertices.

agent_vertices(c::CompiledNestedSystem, path...) -> Vector{Int}

The finest-level sheaf vertices of the agents belonging to the system at path — that is, c.tower.agent_vertices[agent_range(c, path...)]. These index directly into a solve_hierarchical or solve_direct result.

compile_nested_system(f::SystemFragment) -> CompiledNestedSystem

Validate f, lower it to a NestedSystemSpec, compile the SheafTower, and resolve its @bind declarations — the one call that takes a fragment all the way to something solvable.

c = compile_nested_system(@nested_system begin
    @team ring = ring(5; radius=1.0)
    @target t1
    @observe ring => t1
    @bind dynamics=QuadrotorDynamics()
end)

q = solve_hierarchical(c.tower, target_vector(c, Dict(:t1 => [0.0, 0.0, 1.5, 1.0])))

Use nested_spec instead when the tower is not wanted (it is the expensive part).

escort_problem(c, trajectories; velocities=nothing, accelerations=nothing, dt=0.05, steps=200)

Build a NestedEscortProblem from a compiled system, with target trajectories given as a name => (t -> position) dictionary rather than a positionally-ordered vector.

velocities and accelerations follow the same keying and enable feedforward tracking; see run_nested_escort_simulation.

nested_bindings(f::SystemFragment) -> SystemBinding

Resolve f's @bind declarations into the nested SystemBinding cascade, without building a spec or a tower.

nested_spec(f::SystemFragment) -> NestedSystemSpec

Validate f and lower it to a NestedSystemSpec without compiling a tower.

nested_tower(f::SystemFragment) -> SheafTower

Shorthand for build_sheaf_tower(nested_spec(f)).

target_index(c::CompiledNestedSystem, name::Symbol) -> Int

The index of the target named name, i.e. its position in c.spec.targets and its vertex in c.tower.target_vertices.

target_vector(c::CompiledNestedSystem, values::AbstractDict) -> Vector

Order a name => value dictionary into the positional vector that solve_hierarchical/solve_direct expect, so that callers can keep referring to targets by the names they declared.

Every declared target must be present; an unknown name is an error rather than being ignored.