Degrees of Autonomy: Agentic AI as a Spectrum

Lecture 03 notes: from fixed workflows to highly autonomous agents, who decides each step, and how to choose the right degree of autonomy for a problem.

Notes on the third lecture of Andrew Ng’s Agentic AI course (DeepLearning.AI, Module 1). This is my personal study notebook; it is not affiliated with or endorsed by the course or its authors. Paraphrases of the lecture are marked From the lecture. Everything else is my own working.

Agentic AI — Lecture 03: Degrees of autonomy

Are all AI agents equally autonomous?

Does an AI system need to make every decision on its own to count as an agent? It is a tempting question, and an easy one to argue about without end. Lecture 02 ended with a related one: when should the model, rather than the developer, decide what happens next?

From the lecture

Andrew Ng suggests that, rather than debating whether a system is a “true” agent, it is more productive to recognise that systems can be agentic to different degrees.

He also points out that a workflow can be valuable even when most of its steps are decided in advance by a software engineer.

That reframing moves the question from definitions to design. The core idea of this lecture is that agentic AI sits on a spectrum, not in a yes-or-no category. The interesting engineering question becomes: how much should this particular system decide for itself?

The spectrum of autonomy

The lecture describes three broad regions along that spectrum. Read them from left to right, or top to bottom on a phone.

Less autonomous → More autonomous

  1. Less autonomous
    • All workflow steps are predetermined.
    • Tool calls are hard-coded by the developer.
    • Most of the autonomy is in the content the LLM generates.
  2. Semi-autonomous
    • The agent makes some decisions.
    • It can choose tools based on the task.
    • The available tools are predefined by the developer.
  3. Highly autonomous
    • The agent makes many decisions independently.
    • It may decide the sequence of actions.
    • Some systems can create new functions or tools as they go.
Figure 1. Degrees of autonomy, after the spectrum shown in the lecture. My own drawing. The regions shade into each other: they are useful landmarks, not fixed or standardised categories, and further right is not automatically better.

Notice what changes from left to right. It is not how clever the model is. It is who decides the steps: the developer, writing code in advance, or the model, at run time.

One task, two degrees of autonomy

The lecture makes this concrete with a single task: write an essay about black holes. Switch between the two workflows below, and use “Who decides?” to highlight the decisions made by the developer or by the AI.

From the lecture

Andrew Ng uses a colour convention in his workflow diagrams: red for user inputs, grey for LLM calls, and green for software actions and tools. The diagrams below follow it.

A · Less autonomous

The developer fixed the sequence and every tool call in advance. The LLM’s freedom is in what it writes: the search terms and the essay.

  1. User request (User input; person decides) — “Write an essay about black holes.”
  2. LLM generates search terms (LLM call; developer decided) — The model chooses the words; the developer decided that this step always happens.
  3. Predefined web search (Software / tool; developer decided) — Always the same search tool.
  4. Fetch web pages (Software / tool; developer decided) — A fixed number of top results.
  5. LLM writes the essay (LLM call; developer decided) — From whatever was fetched.
  6. Output (Output; developer decided) — Returned after one pass.

B · More autonomous

The model decides what to do next at several points. The developer still decides which tools exist and what limits apply.

  1. User request (User input; person decides) — “Write an essay about black holes.”
  2. LLM selects a research strategy (LLM call; AI decides) — Decides what it needs to find out first.
  3. Chooses sources and tools (LLM call; AI decides) — Picks from the tools the developer provided. Options: Web search; News search; Academic papers.
  4. Decides how many pages to fetch (Software / tool; AI decides) — More for a broad question, fewer for a narrow one.
  5. Processes documents when needed (Software / tool; AI decides) — For example, converts a PDF to text, only if a source is a PDF.
  6. Drafts the essay (LLM call; AI decides)
  7. Reflects, and may research further (LLM call; AI decides) — Reads its own draft and looks for gaps. May return to choosing sources
  8. Final output (Output; AI decides) — The model decides when the essay is finished.
Figure 2. The lecture’s black-hole essay example at two degrees of autonomy. Step wording and details are my own.

Both workflows call an LLM several times, and both are agentic in the lecture’s sense. The difference is where the decisions live. In A they are in the developer’s code, where you can read them. In B many of them happen at run time, so you can only see them by watching what the system actually did.

What changes as autonomy grows

Moving along the spectrum is a trade, not an upgrade. This is how I summarise the tendencies at the two ends:

Dimension

Less autonomous

Highly autonomous

Workflow control

Predetermined

More dynamic

Tool selection

Often hard-coded

May be selected by the agent

Predictability

Generally higher

Generally lower

Flexibility

More limited

Potentially greater

Debugging

Often easier

May be more complex

Risk management

More constrained

Requires stronger safeguards

These are broad tendencies, not guaranteed properties. A carefully engineered autonomous system can be quite predictable, and a sloppy fixed workflow can be hard to debug. But they point in a consistent direction: each decision handed to the model is one fewer decision you can read in the code.

Engineering principle

Choose the degree of autonomy the problem requires, rather than maximising autonomy because it is possible.

Engineering application: food-safety temperature records

To test my understanding, I took the temperature-records example from Lecture 02 and asked how it would look at each degree of autonomy. This is my own application of the lecture, not an example from it.

Less autonomous

Every step is fixed. The model only writes the summary. This is the pipeline from Lecture 02, and for a daily report it may be all that is needed.

  1. Retrieve today’s records (Software / tool; developer decided) — A fixed database query.
  2. Validate readings with deterministic rules (Software / tool; developer decided) — Missing values, impossible readings, unit conversion, threshold check.
  3. LLM summarises the exceptions (LLM call; developer decided) — Prose only; every number comes from the code.
  4. Human reviews the report (Human review; person decides) — Nothing is shared until a person approves it.

Semi-autonomous

When validation finds a gap, the model chooses how to resolve it, but only from tools the developer has provided.

  1. Retrieve today’s records (Software / tool; developer decided)
  2. Validate readings (Software / tool; developer decided) — Finds, for example, a missing reading for one delivery.
  3. AI decides how to handle the gap (LLM call; AI decides) — Chooses one of the predefined options. Options: Retrieve the missing records; Consult approved SOP documents; Ask a person to clarify.
  4. Run the chosen tool (Software / tool; AI decides) — Read-only tools; anything else is refused.
  5. LLM summarises the exceptions (LLM call; developer decided)
  6. Human reviews the report (Human review; person decides)

Highly autonomous

Given an open question, the system plans its own investigation and chooses which permitted tools to run, and in what order.

  1. Request (User input; person decides) — “Find out why deliveries from one depot keep breaching the limit.”
  2. AI plans an investigation (LLM call; AI decides) — Breaks the question into steps it decides on.
  3. Executes permitted tools (Software / tool; AI decides) — Every call passes a permission check first. Options: Query records; Read SOPs; Compare depots; Read vehicle sensor logs.
  4. Reflects on what it found (LLM call; AI decides) May plan more steps, up to a fixed step limit
  5. Proposes actions (LLM call; AI decides) — For example, recalibrating a sensor. It cannot carry them out itself.
  6. Person approves any action (Human review; person decides) — High-risk actions, such as discarding stock, always need a person.
  7. Investigation report (Output; AI decides)
Figure 3. My own engineering application of the lecture: one domain at three degrees of autonomy.

The further right the system sits, the more its safety depends on the boundaries around it rather than on the steps inside it. For the highly autonomous version, I would want at least:

  • Permissions. An allow-list of tools, read-only by default.

  • Limits. A maximum number of steps, tool calls and cost per run.

  • Human approval for anything costly, irreversible or safety-related.

  • A full log of every decision and tool call, so a run can be reconstructed afterwards.

For the daily report, the less autonomous version is the better design: it is easier to test, and nothing in the task needs the flexibility. The investigation is different. Its steps cannot be known in advance, and that is where more autonomy earns its cost.

Takeaways

  1. Agentic AI is a spectrum of autonomy, not a yes-or-no label.

  2. Less autonomous systems can deliver substantial value; most of their steps being fixed is not a weakness.

  3. Greater autonomy can increase flexibility, but may reduce predictability and control.

  4. An engineer’s responsibility is to choose the level of autonomy that suits the task.

My engineering takeaway

The goal is not to build the most autonomous agent. The goal is to build the most reliable system for the problem.

Think before you continue

Question

If an AI workflow uses an LLM to generate search queries, but the programmer controls every other step, is it still agentic?

Reveal an explanation

Yes, to a small degree. Seen as a spectrum, it sits at the less autonomous end: the sequence and the tools are fixed, and the model’s autonomy is in the queries it writes. That small decision still shapes everything downstream, since the queries decide what gets found.

The more useful question is the one this lecture keeps returning to: does that level of autonomy suit the task, and does the system reliably do its job?

I am left with a question I expect to keep coming back to: if the path a system takes can change from one run to the next, how do you test it and know it is working?