Intelligent Automation

The workflow is a versioned, observable artefact. Events are integration primitives. Infrastructure is code, and code is the source of truth. The operating model is what keeps automation current with the systems it orchestrates.

On this page
  1. Top of page
  2. What intelligent automation is
  3. The architecture
  4. State of the practice
  5. Questions we get asked
  6. Patterns we design our practice to avoid
  7. Evidence & references
  8. What to read next

§ 01 ·

What intelligent automation is

Intelligent automation is the discipline of treating workflows, integrations, and infrastructure as first-class artefacts: versioned, observable, tested, deployed through a pipeline, owned end-to-end. Automation is the same kind of artefact as application code. Automation is treated the same way.

Three layers drive the discipline.

  • The workflow is the sequence of steps that describes what runs when.
  • The integration is the events, API calls, messages that connect systems.
  • The infrastructure is the compute, storage, networks that workflows run on.

Each layer has its own discipline. Each layer is required. None of the three is sufficient on its own.

Design observability in from the start, not after the workflow goes into production. Pattern: automation as a first-class artefact: versioned, observable, tested, owned.

Two failure modes recur.

  • The laptop script. Automation lives on someone's laptop, undocumented; the person leaves and the automation breaks.
  • The unobserved automation. The workflow is observed by running it and watching output; the automation produces wrong results for six months before anyone notices.

Both are first-class-artefact gaps. Both close with the same tooling application code uses.

§ 02 ·

The architecture

Throughout this page, a first-class artefact is one that is versioned, reviewed, tested, deployed through a pipeline, owned end-to-end. NOETRONIQ treats workflows, infrastructure, and automation itself as first-class artefacts by the same discipline application code uses.

Four primitives drive the discipline.

The workflow

Workflows persist. Workflows running for days, with thousands of steps, survive any single step's failure because the durable execution pattern persists workflow state and resumes from persisted state.

Workflows are first-class artefacts:

  • Versioned in source control.
  • Reviewed before merge.
  • Tested before deploy.
  • Deployed through the same pipeline as application code.
  • Owned end-to-end.

The integration

Events are integration primitives. Event schemas are contracts. Schema registries are the source of truth. Breaking changes trigger new versions. Deprecations notify consumers.

Three roles divide the ownership.

  • Producers own events.
  • Consumers own reactions.
  • Registries own contracts.

The infrastructure

Infrastructure is code. Code is the source of truth. Plans, applies, drift detection are the three stages of the same artefact. Infrastructure tests run in the same pipeline as application code.

The operating model

Platforms are paved roads. Workflow engines, event brokers, infrastructure-as-code are available by default. Automation practice is a property of the platform; the practice keeps automation current with systems it orchestrates.

§ 03 ·

State of the practice

3
Layers of intelligent automation (workflow, integration, infrastructure), each with its own discipline, each required.
Industry observation; McKinsey, others
1
Right answer to which workflow engine: the one the team will actually use and operate.
Industry observation
~5x
Typical productivity gain of well-designed intelligent automation over manual process (varies by use case).
Industry observation; McKinsey, others

§ 04 ·

Questions we get asked

Fit the engine to the team's skill set, the team's operating model, the organisation's platform. Three common profiles:

  • Teams already running Kubernetes land on Argo Workflows or Temporal.
  • Teams on AWS Step Functions or Azure Durable Functions land on cloud-native durable execution.
  • Teams needing open-source forks without vendor lock-in land on OpenTofu or Terraform-compatible alternatives.

Choice is a fit decision, not a correctness decision.

The durable execution pattern persists workflow state, resumes workflows from persisted state when steps fail, and prevents duplicate step runs unless workflows explicitly retry. Implementations include Temporal, AWS Step Functions, Azure Durable Functions, and academic and open-source projects. Reference implementation: Temporal.

Automation is a first-class artefact. Tests are a property of the artefact. Tests run in the pipeline. Coverage includes:

  • Workflow logic: unit tests for workflow steps.
  • Integration: contract tests for event schemas.
  • Infrastructure: plan tests for infrastructure-as-code.

Pipelines gate deploys. Artefacts gate pipelines.

The human-in-the-loop pattern fits high-stakes, judgement-dependent, externally-consequential decisions. The rest of the workflow runs automated. The human-in-the-loop becomes a bottleneck only when the human sits on decisions automation could handle. Bottlenecks signal human-in-the-loop placement has gone wrong.

Right next steps ground in the team's existing operating model.

  • Teams on Kubernetes start with one workflow engine, run one workflow in production, learn the durable execution pattern.
  • Teams starting from scratch scope one automation, run one workflow in production, learn the practice from one workflow.

Next steps are fit decisions, not correctness decisions.

§ 05 ·

Patterns we design our practice to avoid

An automation that lives on a single laptop carries a single point of failure. Discipline: workflow as code. Workflows:

  • Version and review through the same pipeline as application code.
  • Deploy through that pipeline.
  • Survive the departure of the people who wrote them.

An automation that is observed by running it and watching output produces wrong results for six months before anyone notices. Discipline: observability as a property of the workflow.

  • Logs live with the workflow.
  • Metrics live with the workflow.
  • Traces live with the workflow.
  • UI lives with the workflow.

Observability is a property of the workflow, not an add-on.

An event schema owned by the producer is a schema the consumer cannot evolve against. Schemas are contracts. Contracts are shared artefacts. Schema registries are the source of truth. Breaking changes trigger new versions. Deprecations notify consumers.

Infrastructure as code that has not been tested is hand-written, unverified infrastructure. Discipline: plans, applies, drift detection. Infrastructure tests run in the same pipeline as application code.

A workflow that is not durable loses state when steps fail. Discipline: durable execution.

  • Workflow state persists.
  • Workflows resume from persisted state.

Non-durable workflows cannot be trusted for workflows that matter most.

A human-in-the-loop that has become the bottleneck slows workflows without adding value. The human-in-the-loop fits high-stakes, judgement-dependent, externally- consequential decisions. The rest of the workflow runs automated.

An automation that is not a first-class artefact is not versioned, not reviewed, not tested, not deployed through a pipeline. Automation is the same kind of artefact as application code. Automation is treated the same way.

§ 06 ·

Evidence & references

Designing Data-Intensive Applications
Kleppmann, O'Reilly, 2017

Reference for distributed systems discipline. Coverage of event logs, change-data-capture, stream processing is the most rigorous available. The book addresses every automation problem in a distributed system.

Workflow Data Patterns
Russell, van der Aalst, ter Hofstede, 2016

Academic reference for workflow patterns. The catalogue of 40+ patterns (sequence, parallel split, exclusive choice, deferred choice, milestone, cancel case, and others) is the most rigorous treatment of workflow semantics. The reference is dense. Patterns are universal.

§ 07 ·

What to read next

Related elaborations across the site, plus the conversation to start.

Building an intelligent automation practice?

Weighing an intelligent automation initiative, evaluating a workflow engine, or making an existing automation more than a script on someone's laptop? Useful at the boundary between workflow and implementation. A short conversation is the right next step.

Learn more