Benefits of Agentic AI Workflows: Performance, Parallelism and Modularity

Lecture 04 notes: why a better workflow can beat a better model, when parallel steps save time, and why interchangeable components matter.

Notes on the Benefits of agentic AI lecture in Module 1 of Andrew Ng’s Agentic AI course (DeepLearning.AI). This is my personal study notebook; it is not affiliated with or endorsed by the course or its authors. Paraphrases of the lecture are marked From the lecture. Everything else is my own working.

Agentic AI — Module 1: Benefits of agentic AI

Why does workflow design matter?

What if improving an AI application isn’t only about choosing a more powerful model, but also about designing a better workflow around it?

The previous lectures described what an agentic workflow is. This one asks what it buys you. The lecture names three benefits: better task performance, the ability to run independent steps in parallel, and modularity, meaning parts that can be swapped independently.

Better performance comes from the things Lecture 02 introduced: several steps instead of one, tools, review and revision. But none of these benefits is automatic. A workflow is a design, and a poor design can be slower, costlier and no better than a single call. Whether it helps is something you measure.

Benefit one: better task performance

From the lecture

Andrew Ng shows results on HumanEval, a benchmark of programming problems scored by whether the generated code passes unit tests. Used in a single pass, GPT-4 scores clearly higher than GPT-3.5. But wrapping a model in an agentic workflow, using techniques such as reflection, tool use, planning and multiple agents, raises its score further.

ModelApproachHumanEval pass rate (%)Source
GPT-3.5Zero-shot (non-agentic)48.1%Lecture slide (about 48%); The Batch, issue 241 (48.1%)
GPT-3.5In an agentic workflow (best reported)95.1%The Batch, issue 241: “up to 95.1%”
GPT-4Zero-shot (non-agentic)67.0%Lecture slide (about 67%); The Batch, issue 241 (67.0%)

The lecture slide also plots individual results for reflection, tool use, planning and multi-agent techniques. I have not placed those points here: I could not verify their exact values, and reading them off a slide would add false precision.

Figure 1. HumanEval results from a particular evaluation published in 2024, redrawn from the lecture and Andrew Ng’s article in The Batch. They are a historical snapshot, not a ranking of current models.
Source note

In the spoken lecture, the GPT-3.5 baseline sounds like “40%”, while the slide labels it at about 48%. Andrew Ng’s written account in The Batch (Agentic Design Patterns, Part 1) gives 48.1% for GPT-3.5 and 67.0% for GPT-4, both zero-shot, and says GPT-3.5 in an agent loop reaches “up to 95.1%”. I use the written figures.

Press “Compare” on the chart to see the point of the slide. Upgrading the model added about 19 points. Wrapping the older model in a workflow added up to 47. A well-designed agentic workflow can sometimes produce a larger improvement than switching to a more capable model.

“Sometimes” is doing real work in that sentence. This is one benchmark, with tests that give a clear pass or fail signal, which is exactly the kind of feedback that makes reflection and revision effective. The 95.1% is the best reported result, not a typical one, and none of it says the same gain will appear on your task. That is why the next step after building a workflow is always evaluation.

Benefit two: parallelism

From the lecture

Returning to the black-hole essay, Andrew Ng shows a research agent that asks an LLM for three search strategies, runs the three web searches in parallel, takes the top three results from each, fetches all nine pages in parallel, and passes everything to an LLM to write the essay.

Compare the two shapes below. Then drag the slider to limit how many operations may run at once, as a rate limit or a small worker pool would, and watch the timeline.

Write an essay about black holes → LLM: three search strategies → three web searches → top three results each: nine page fetches → combine → LLM: write the essay.

With hypothetical durations, running every step one after another takes 27 s; running the three searches together and then the nine fetches together takes 9 s. Overhead and rate limits are ignored.

The simplest version of the arithmetic: nine independent page downloads that take 2 seconds each take about 18 seconds one after another, and about 2 seconds all at once, if you ignore overhead and concurrency limits. This is a simplified hypothetical, not a measurement from the lecture.

Parallelism helps only when operations are independent. In practice, the saving shrinks because of:

  • Dependencies. The fetches cannot start until the searches have returned URLs, and the essay cannot be written until the pages arrive.

  • Rate limits and resources. An API may allow only a few requests at a time; the slider shows what that does.

  • The slowest operation. A parallel stage takes as long as its slowest member. One slow page delays the whole stage.

Benefit three: modularity

From the lecture

Andrew Ng points out that agentic workflows are built from components that can be changed independently: you can try a different web search engine, add a news search tool, or use different language models for different steps.

Try it below. Change any component and see what is affected. Notice that one of the options changes the shape of its output, so it would need an adapter before the next stage could use it.

  1. User request (text)
  2. Planning model (question → list of queries) — interchangeable: Large model; Small, fast model
  3. Search provider (query → ranked URLs) — interchangeable: Web search engine A; Web search engine B; News search
  4. Document retrieval (URLs → clean text) — interchangeable: HTML-to-text fetcher; Fetcher with PDF conversion
  5. Synthesis model (texts + question → report) — interchangeable: Large model; Mid-sized model
  6. Report (Markdown)
Figure 2. A research workflow as interchangeable parts. Component names are generic placeholders, not product recommendations. My own diagram.

Each stage has a contract: what it receives and what it returns. As long as a replacement honours the contract, the rest of the workflow does not need to change. That is what makes it practical to experiment with models and tools, maintain a system over time, and optimise one slow or expensive step without touching the others.

It also means the best model for one step need not be the best for another. Planning, ranking and long-form writing ask different things of a model, and a smaller model may be good enough for some of them. Swapping is not free, though: a replacement needs a compatible interface, its own tests and an end-to-end evaluation before it ships.

The three benefits side by side

Benefit

What it enables

Engineering consideration

Performance

Multi-step refinement and better task execution

Requires evaluation

Parallelism

Concurrent independent operations

Dependencies and resource limits

Modularity

Replaceable models and tools

Interface compatibility and testing

Engineering reflection: school meal-delivery temperature records

To check that I can use these ideas, I applied them to the temperature-records example from earlier lectures, now across several school meal-delivery locations. This is my own engineering reflection, not an example from the lecture. Switch between the three benefits to see where each one fits.

Performance

Multiple steps with checks between them, rather than one prompt asking for a compliance report.

  1. Validate every record with rules (Software / tool; developer decided) — Missing readings, impossible values, units, thresholds. Ordinary, tested code.
  2. Examine each exception (Software / tool; developer decided) — Collect the evidence: sensor log, delivery time, location.
  3. LLM drafts an explanation of the exceptions (LLM call; developer decided) — Prose only; every figure comes from the rules.
  4. Review the draft against the evidence (LLM call; developer decided) — A second pass checks each claim against the records. Revise if a claim is unsupported
  5. Person reviews and signs off (Human review; person decides) — Compliance is a human decision, made on validated evidence.

Parallelism

Each location’s records are independent, so they can be fetched and validated at the same time.

  1. Fetch and validate records (Software / tool; developer decided) — One task per location, all at once. Options: Location A; Location B; Location C; Location D.
  2. Wait for every location (Software / tool; developer decided) — The summary depends on all of them; a slow location delays the report.
  3. LLM summarises the exceptions (LLM call; developer decided)
  4. Person reviews (Human review; person decides)

Modularity

Each part can change without redesigning the rest.

  1. Data source (Software / tool; developer decided) — One location uploads spreadsheets, another has a sensor API. Options: Spreadsheet import; Sensor platform API.
  2. Rule validation (Software / tool; developer decided) — Thresholds and checks in a tested, versioned module, updated when guidance changes.
  3. Summarisation model (LLM call; developer decided) — Swappable, and re-evaluated on the same past reports before any change.
  4. Person reviews (Human review; person decides)
Figure 3. My own application of the lecture’s three benefits. Compliance is decided by validated rules, evidence and a person, never by a language model alone.

The thread through all three: the LLM explains and drafts, but it does not decide whether a delivery was compliant. Validated rules decide that, using the evidence, and a person signs it off.

Think before you continue

Try to answer each question in a sentence before opening the explanation.

Think before you continue

1. Can a smaller model with an effective workflow outperform a larger model used in a single pass?

Reveal an explanation

It can. In the HumanEval results above, GPT-3.5 in an agentic workflow (up to 95.1%) scored well above GPT-4 used in one pass (67.0%). But this is evidence for one benchmark with automatic tests, not a law. Whether it holds for your task is something to evaluate.

Think before you continue

2. Why can’t every step in an agentic workflow run in parallel?

Reveal an explanation

Because many steps depend on the output of earlier ones. You cannot fetch pages before a search returns their URLs, or write the essay before the pages arrive. Only independent operations, such as the three searches or the nine fetches, can overlap, and even then rate limits and the slowest operation set the pace.

Think before you continue

3. What are the advantages of separating a research agent into independent components?

Reveal an explanation

Each component can be tested, improved, replaced or scaled on its own. You can try a different search provider or a cheaper model for one step, and a failure is easier to trace to the stage that caused it. The price is designing clear interfaces and re-evaluating the whole workflow after each change.

Key takeaways

  • Agentic workflows can significantly improve task performance, sometimes by more than a model upgrade.

  • Parallelism can reduce execution time, but only for operations that do not depend on each other.

  • Modularity lets developers experiment with, and replace, individual models and tools.

  • Strong engineering means measuring quality, latency, cost and reliability together.

My engineering takeaway

A powerful AI application is not defined only by its model. Its architecture and workflow design matter just as much.

This is my synthesis of the lecture, not a quotation.