The AI orchestration play

Why this is different (and a step change) from workflow automation

I’ve already written about how I believe a key piece of value accrual in AI will start appearing at the app layer. Another piece of that value puzzle is AI orchestration, and this is about to be a big deal.

You’ll see AI orchestration defined in a few different ways, but to me, it means:

I’m a consumer or a business that already uses multiple software services to run my day-to-day operations, now how do I stitch together outcomes from different AI models, apps and agents to take concrete steps towards achieving my objective?

The Context

Let’s consider a small business - the software stack they use today might look something like this:

And here’s what it might look like for an enterprise:

So the idea is the same, regardless of what type of business it is. Different pieces of the software stack perform different functions, like Google Workspace for email, Slack for messaging, Salesforce for CRM, Workday for HR, Asana for project management.

In the last ~2 years, every piece of this software stack has now become AI-powered - all running their own trained models, curating their own data, building their own context, generating their own output:

But these different tools are all still operating in silos, and don’t automatically talk to each other.

Enter automation

The concept of workflows, and workflow automation, is not new. IFTTT, Workato and, in particular, Zapier, have all been in the space for several years.

Let’s say you’re a small business that wants to communicate a new sale, recorded through Shopify, to your team Slack channel. This is what a simple Zapier workflow might look like:

Workflow vs AI Orchestration

I’ve seen many people use workflows and agents interchangeably, but they’re not the same.

Workflow orchestration:

• Uses rule-based routing that is predetermined internet glue, connecting different pieces of your software stack

• Executes fixed sequences through data pipes

• Doesn’t handle ambiguity or exceptions well, without a lot of effort

• Is great for repetitive, well-defined tasks that don’t break easilly

• Is an efficiency play

AI orchestration, on the other hand:

• Uses intelligence and reasoning about what to do

• Breaks down sophisticated tasks into subtasks dynamically

• Can synthesize information from different sources within your company to make decisions, all while understanding the business context

• Adapts and can handle ambiguity in the event of service failures or errors

• Expands what can be automated

• Could potentially replace entire job functions

The last point is key here - I believe successful AI orchestrators can dramatically expand the TAM from “make things faster and more efficient” to “do the knowledge work itself”:

The major players

Needless to say this is a tremendously valuable platform layer to control, because it dictates the interpretation and usage of all underlying systems. It might be easy enough to switch out a single underlying SaaS tool (+agent), but it is certainly not easy to replace a layer that understands the full scope of your business context and continues to learn and get better. It’s a huge moat.

There’s so much jostling in the space already, but I don’t believe there will be a single winner. There’s plenty of room in this market to address different niches:

• Enterprises will center around their cloud provider’s solution, like Microsoft Azure AI Foundry, or Amazon SageMaker AI

• Technical teams will choose to deploy their own orchestration using LangChain or similar

• Smaller businesses will choose no-code tools like Zapier that have a large library of integrations to cover edge cases

• AI platforms are also pushing their own plays, like OpenAI and ChatGPT Actions

Of these, I’m fascinated by the position Zapier find themselves in. They’ve spent ~15 years building the perfect company for this agentic moment. Zapier now boasts 7,000+ integrations with a likely very long tail of SaaS tools and services, so if you’re trying to create workflows of any kind, the odds are pretty good that they’ve got you covered.