Theoretical foundations and architecture

Operator-grade configuration generation is built on Python, the Jinja2 template engine and delivery libraries (Scrapli, Netmiko, NAPALM, Nornir). The Infrastructure as Code (IaC) principle separates data, templates and transport: 1. Service data is stored in YAML or JSON (customer name, RD, RT, CE prefixes, QoS parameters) and is checked before rendering by a schema (Pydantic, JSON Schema) and business rules. 2. A Jinja2 template (.j2) holds the syntax of one vendor (Cisco, Huawei, Juniper, MikroTik) with variable substitution ({{ vrf_name }}, {{ rd }}) and control structures ({% for peer in peers %}, {% if vlan %}); StrictUndefined refuses to render an empty value. 3. A Python script renders the configuration and delivers it: Netmiko and Scrapli send CLI lines without transactions (the result must be checked), NAPALM offers load_merge_candidate, compare_config, commit_config and rollback only for supported drivers (for example ios and junos; the core distribution has no drivers for Huawei VRP and RouterOS), and Nornir runs tasks in parallel. This removes typos in hundreds of similar blocks, but it does not replace data validation, a diff before applying and post-checks.

The source of truth is separate from the template

The Infrastructure as Code approach for a PE rests on separating three things: data (what must be configured), the template (how it is written on a specific platform) and transport (how it is delivered). The data describes intent: customer A, RT 65000:100, a PE-CE interface. It is stored as YAML/JSON in Git or in NetBox. The template is a Jinja2 file per vendor, with no business logic. Transport is Netmiko, NAPALM, Scrapli or Nornir. One intent renders into four syntaxes, and there are no manual edits on devices.

yaml
# data/customers.yml (source of truth, no secrets)
asn: 65000
vrfs:
  - {name: A, rt: "65000:100", rd_offset: 0, iface: ether2}
  - {name: B, rt: "65000:200", rd_offset: 10, iface: ether3}
  - {name: SS, rt: "65000:900", rt_import: ["65000:900", "65000:100"],
     rd_offset: 20, iface: ether4}

The RD is built by the formula asn:(pe_id + rd_offset) and is unique on each PE, which is needed to announce prefixes of one VRF separately from two PEs (multipath, fast convergence).

Data validation before rendering

The data is checked with a schema: Pydantic (types, the regular expression ^\d+:\d+$ for RT and RD), JSON Schema or yamllint. Business rules are written in code: the customer RT is in the allowed list, VRF names are unique, the RD is unique within a PE, the interface belongs to the PE. If the data is wrong, the template must not be called: an error at this stage costs nothing, on the device it costs an incident.

Jinja2: what you really need for configurations

  • StrictUndefined: by default an unknown variable renders as an empty string, and you get route-target export without a value. With StrictUndefined the render fails with UndefinedError.
  • trim_blocks=True, lstrip_blocks=True, keep_trailing_newline=True: {% %} tags leave no empty lines or indentation. Indentation matters for configurations (in Cisco and Huawei nesting is determined by spaces).
  • Loops with loop.index, the filters sort(attribute="name") (stable output order, therefore a clean diff), default([...]) for optional fields, join(","), selectattr, map, unique.
  • Arithmetic in expressions (pe_id + v.rd_offset), macro and include for repeated blocks, namespace for counters inside loops.
  • autoescape is off for configurations: this is not HTML. But templates from an untrusted source must not be rendered either; SandboxedEnvironment exists for them.
  • Custom filters (for example an RD check) are registered in env.filters.

The render must be deterministic: the same data gives a byte-identical result, otherwise the diff in CI is noisy.

python
import yaml
from jinja2 import Environment, FileSystemLoader, StrictUndefined

env = Environment(
    loader=FileSystemLoader("templates"),
    undefined=StrictUndefined,
    trim_blocks=True, lstrip_blocks=True, keep_trailing_newline=True,
)
data = yaml.safe_load(open("data/customers.yml"))
cfg = env.get_template("pe_ios.j2").render(**data, pe_id=1)

Transport: Netmiko, NAPALM, Scrapli, Nornir

  • Netmiko (ConnectHandler, types cisco_xe, juniper_junos, huawei_vrpv8, mikrotik_routeros) sends CLI lines. It knows nothing about transactions: send_config_set does not stop on % Invalid input unless error_pattern is set. Check the output.
  • NAPALM provides a unified API load_merge_candidate, compare_config, commit_config, discard_config, rollback for the drivers ios, junos and others; the core distribution has no drivers for Huawei VRP and RouterOS. On Junos commit_config(revert_in=...) implements commit confirmed, while on IOS-XE rollback is based on configure replace.
  • Scrapli is faster than Netmiko on large fleets and has typed drivers; vendor platforms must be chosen correctly (the IOS-XE driver does not suit Junos).
  • Nornir runs tasks in parallel (num_workers), keeps the inventory in Python and is used for complex sequences: canary first, then the rest.

Idempotency and diff

Pushing the whole config on every run is bad: it adds noise to the logs and raises risk. You need to compute the difference. On Junos compare_config shows the candidate diff. On IOS-XE the rendered lines are compared with show running-config | section vrf definition. For RouterOS the comparison is with /export. If the diff is empty, nothing is sent to the device: this is idempotency at the script level.