Svalefelt

Svalefelt Scheduler / What it is, and what it does

What it is, and what it does

Who Svalefelt Scheduler is for, what it refuses to be, and everything it does today — jobs, triggers, steps, handlers, the Nova batch server and the web UI.

8 min read

A job scheduler built to sit in front of an unstable Nova batch server: it owns the scheduling, the preparation, the checking and the recovery, and hands the batch server only the parts nothing else can do. It is also a perfectly ordinary scheduler for scripts, SQL and file transfers, and runs without any Nova at all.

Who it is for

The person who runs the night. Files arrive, imports have to happen in order, exports have to go out, and in the morning someone has to know what did not happen and why. You want the reasons on one page, the jobs in files you can read and diff, and a tool that tells you the batch server is dead before the first import is missed.

The developer who inherits a Nova schedule. Two hundred jobs in ncScheme, chained by trigger names, with their settings in a binary blob. You want them as readable files, with the chain visible as ordered steps, and a way to run one by hand and see what it did.

The integrator. You move a system between environments, and every job carries a path to a share that has a different name on the other side. You want the paths listed and checked before the night, not discovered by a failed run.

What it is not

  • Not a workflow engine. No DAGs across jobs beyond "run after that job", no fan-out, no parallel branches inside a job. A job is an ordered list of steps; the scheduler runs jobs in parallel, not steps.
  • Not a Nova client. It books tasks on the batch server and reads the results and the import log; it does not open cases, print letters by itself or talk to Nova's business logic. What the batch server cannot do, this cannot make it do.
  • Not a monitoring platform. It watches its own jobs and the batch servers it is told about. There is no alerting channel of its own - a handler runs your script, and the log is a file your monitoring can read - no metrics endpoint yet, and no history of the server's health beyond the last sixty heartbeats.
  • Not multi-user. Sign-in is a placeholder behind a real token flow; there are no roles, and every operator can do everything.
  • Not a database migration or deployment tool. SQL steps run scripts; nothing versions a schema.

Features

Jobs as files

  • One YAML (or JSON) file per job: name, description, group, enabled, triggers, steps, handlers, environment.
  • The folder is watched; a changed file is reloaded without a restart.
  • A file with an unknown property, a missing setting or a misspelled step setting is reported as a problem and is never scheduled.
  • scheduler validate checks a file or the folder with an exit code, for CI.
  • Steps and triggers may carry a free-text description; the editor shows it, or makes one from the settings.

Triggers

  • Schedule: five-field cron, local time, with a preview of the next occurrences in the editor.
  • File: a folder and a pattern, checked at an interval; fires once per file that appears and hands the file to the steps as TRIGGER_FILE / TRIGGER_FILE_NAME.
  • SQL: a query on a named connection, checked at an interval; fires when the scalar is truthy.
  • Job: fires when another job ends with success, failure or either; the steps get TRIGGER_JOB and TRIGGER_JOB_STATUS.
  • A job never overlaps itself; several triggers on one job all start the same job.

Steps

  • script: a shell command or a .sh/.bat/.cmd/.ps1 file (cmd on Windows, sh elsewhere; the shell can be overridden).
  • python: inline code or a .py file, with arguments.
  • sql: inline SQL or a .sql file against a named connection; GO splits batches.
  • file-import / file-export: a folder bulk-loaded into a main plus Rows/Csv table pair inside a transaction with a verify script, files archived on success; rows with ExportedTime is null written out as files and stamped.
  • Every step: timeoutSeconds, retries with retryDelaySeconds, continueOnError, a working directory, its own environment variables.
  • Output file check: producesFile (wildcards, ${VAR}) and producesMinBytes - the step fails unless a new or changed file of at least that size is there afterwards; an old file does not count, no clocks are compared, and the failure is not retried because the work already ran.
  • Environment merges job < trigger < step < the built-ins SCHEDULER_JOB, SCHEDULER_RUN_ID, SCHEDULER_TRIGGER.

Nova batch server steps

  • Imports nova.impinit, nova.imprex, nova.impsup, nova.impsupab: book the import, wait for the batch server's result, then check Nova's import log (niIELogg) for a row written after the booking: OK, EMPTY (zero records; fails unless allowEmpty), FAILED, NOT_FOUND.
  • nova.execob: an OB-code script. nova.exec: a program or command line, wait deciding whether the exit code is the result.
  • Fifteen batch tasks with forms generated from live schedules: nova.batchsql (export, report and update definitions), nova.batchied, nova.batchprint, nova.filemove, nova.batchcreaccount, nova.batchcrecostinv, nova.batchpaycommit, nova.batchinterestupd, nova.batchaap, nova.exprex, nova.expsup, nova.impag, nova.expag, nova.batchgetemail, nova.cleanbatchhistory. The batch server's own result is checked; nothing more is verified.
  • nova.raw: any task, TaskData written as INI text.
  • Results a job can react to: OK, FAILED, MUTEX (another task of the same type holds Nova's lock), FILE_MISSING, TIMEOUT, DUPLICATE_RUN; retryOn says which are worth another attempt.
  • A booking that never started is withdrawn on timeout or cancellation; finished bookings are removed; leftovers of a crashed scheduler are found by their BookedBy and cleaned up.
  • Nothing is installed in the Nova database: the scheduler only inserts into and deletes from ncScheme, and reads ncSrvHst and niIELogg.
  • nova-server: start, stop or restart a configured batch server from a job, waiting for a heartbeat to come through after a start.
  • scheduler nova convert: an ncScheme extract becomes job files - chains as steps, triggers as job triggers, repeats as cron, verified to rebuild the original TaskData, disabled until enabled on purpose.

Watching the batch server

  • Configured per Nova system: a named connection, a heartbeat interval, and optionally the Windows service (local or remote) or the Docker container the server runs as.
  • The heartbeat is a real job through the batch server; ok / late / failing / off.
  • Service state, start, stop, restart from the State page and from scheduler nova; stop and restart are confirmed and logged with the user's name.
  • Optional automatic restart after N failed heartbeats in a row.

Event handlers

  • onStart (may skip the run), onError (answers continue, restart-step, restart-job or stop, with a cap on restarts), onFinally (always runs).
  • Scoped to a step: only it, it and after, it and before, all but it; and to stopping failures or continue-on-error failures.
  • Script and Python handlers answer with a SCHEDULER_RESULT= line and get the context as SCHEDULER_* variables; SQL handlers answer with a scalar and get it as parameters.

Web UI

  • Dashboard: runs last hour / today / yesterday with failures, running now, jobs, one tile per batch server, latest runs, latest failures with the failing step and the reason.
  • Jobs: grouped tables; last run, elapsed (live while running), status, next run; row click opens the editor; Run on manual jobs.
  • Editor: job details, triggers, steps, handlers and variables as collapsible lines; a form per step kind, generated from one catalogue; grouped Add menus; Run now, Validate, Delete; a YAML tab that converts through the scheduler's own parser.
  • Log: grouped by last hour / earlier today / earlier this week / older, or flat; job and failed-only filters; auto-refresh; every run opens its full log.
  • State: heartbeats and server controls per Nova system; every path the jobs use, checked read-only from this machine.
  • Vue 3 and Element Plus vendored: no node, no build step, works offline. Dates follow the browser's language. One (light) theme.
  • Sign-in is a mock name prompt behind a real JWT + refresh-token flow.

Running it

  • scheduler serve on Linux or Windows; install registers a Windows service (with crash restart) or a systemd unit; warnings reach the Windows event log.
  • scheduler run <job> from a shell, output streamed, exit code 0 on success; list; validate; paths; nova status|beat|start|stop|restart|convert.
  • Configuration: appsettings.json next to the binary, appsettings.local.json, SCHEDULER_* environment variables (SCHEDULER_Connections__name, SCHEDULER_Nova__Systems__name__...), command-line options. Jobs reference connections by name only.
  • /healthz for a load balancer; run logs as JSONL files per job.
  • Releases: a self-contained Windows zip and a Docker image, built from version.txt, with this version's release notes inside.

A Nova test environment on Linux

  • docker/nova-batch/nova-env.sh creates complete environments: SQL Server with a restored Nova database and the real batch server plus CCS under Wine, several side by side. The scheduler's Nova steps and the heartbeat were verified against it, with the same results as on Windows.

Where it is still thin

  • Exports are not verified against Nova's import log; only imports are. The output file check covers exports for now.
  • The schema-derived Nova tasks have exact layouts but only BATCHSQL exports and FILEMOVE have been run against a batch server so far; BATCHSQL reports and updates return OK without running their query in the Wine environment, and have not yet been tried on Windows.
  • Jobs stuck at Running=1 in Nova's own schedule are not yet unstuck, and existing ncScheme jobs are not triggered in place; the scheduler books its own.
  • No roles, no alerting channel, no metrics.