AI Is Becoming a Systems Problem

Something about the progress in AI has been on my mind.

The capabilities are remarkable, and they keep getting better. Much of that progress has come with an appetite for more compute, more memory, more infrastructure, more energy.

That isn’t particularly surprising. When you’re pushing the boundaries of what’s possible, you use the resources available to you.

But it raises an interesting question.

What happens when we want this intelligence everywhere?

In phones, cars, glasses, devices we use all day, and probably in things we haven’t imagined yet.

Those systems live in a very different world. Compute is finite. Memory is finite. Batteries run down. Devices get hot. Networks disappear.

You can’t take the data center with you.

I’ve spent much of my career building systems where constraints aren’t an inconvenience. They’re part of the architecture. Perhaps that’s why I keep coming back to the same thought:

AI is becoming a systems problem.

This feels familiar

There is a pattern in computing that I’ve seen more than once.

A new technology arrives and we naturally focus on the new capability. That’s where the excitement is. Then, at some point, the capability becomes good enough that the harder questions begin to move elsewhere.

Mobile computing went through this.

The smartphone wasn’t the result of one breakthrough. It came from getting compute, connectivity, sensors, software, applications and power management to work together inside a device with very real constraints.

You couldn’t optimize every piece independently and expect the product to work. The tradeoffs were part of the product.

Eventually, the system became the product.

I wonder if we’re approaching a similar point with AI.

The constraint may be the opportunity

Model innovation isn’t going to stop. Nor should it.

But as AI moves beyond the data center and into the things around us, another direction of innovation becomes increasingly interesting:

Doing more with less.

Less compute. Less memory. Less movement of data. Less energy. Less dependence on connectivity.

That sounds like an efficiency problem. I think it’s bigger than that.

Constraints force choices. Where should work happen? What needs to happen immediately? What can wait? What belongs close to the user, and what belongs elsewhere?

Those choices eventually shape the architecture, and ultimately the experience.

There is rarely a perfect answer.

There are choices. And consequences.

What I’m curious about

We’ve spent the last few years discovering what happens when we scale the model.

I think we’re only beginning to explore what happens when we optimize the system around it.

Where should intelligence live? How will computing platforms evolve around it? What happens when hardware and software are designed around these new constraints rather than adapted to them?

I don’t know exactly where this leads.

That’s what makes it interesting.

My hunch is that some of the most important advances ahead won’t come from any one component. They’ll come from making the whole system work better together.

We’ve learned a great deal by asking:

How much more can the model do?

I’m increasingly curious about a different question:

How much more can the system do with what it has?