AI should not be the answer to everything

I wont be talking about the code slop here. Promise :)

There is something interesting happening in the software industry right now. AI has become so accessible that we have started reaching for it before we even understand the problem we are trying to solve.

Everywhere I look, there is a conversation about LLMs, agents, AI-powered workflows, and the next big AI integration. Teams are under pressure to adopt them. Leaders want AI roadmaps. Developers are expected to become proficient with AI tools, build AI-powered features, and find opportunities to integrate models into existing products.

And somewhere in this race, I feel we have started forgetting something fundamental: AI is much bigger than LLMs, and software engineering is much bigger than AI.

I am not against LLMs. Quite the opposite. What we have achieved with models such as GPT, Claude, Gemini, and others is remarkable. But I believe we are entering a phase where the ease of accessing these models is making us less deliberate about when we should actually use them.

Every problem now seems like an LLM problem

I recently had a conversation with a senior director at a multinational SaaS company. We were working for one of the projects in his team, and his suggestion was to use LLMs to predict infrastructure scale-up costs when application traffic increases by some amount.

My first reaction was simple: why?

If we know our current infrastructure configuration, resource utilization, traffic patterns, scaling rules, and the relationship between workload and resource consumption, why would we immediately jump to reach out for an LLM?

We could build a deterministic cost estimator using resource requirements, scaling assumptions, pricing information, and known constraints. If the relationship between traffic and resource consumption is nonlinear or uncertain, we could use historical data and statistical models to improve the estimate.

Sure that would take some time to build, while LLMs can be instantly integrated - but that would be free or at a minimal cost. Whereas, LLMs would charge for every token consumed.

An LLM might help explain the results in natural language or interpret a loosely worded request. But should it be responsible for the underlying calculation?

This is the distinction I think we are losing.

We are increasingly treating the availability of a capable model as evidence that the model is the right tool for the job. And because integrating an API can be faster than designing a proper solution, we sometimes mistake implementation speed for engineering efficiency.

The result? A problem that could have been solved with a few well-defined formulas becomes a model invocation, complete with token costs, latency, failure modes, monitoring requirements, and additional infrastructure.

We have made the solution easier to write, perhaps. But have we made the system better?

We Have Forgotten That AI Existed Before LLMs

Ask a developer today about AI, and the conversation will often jump straight to ChatGPT, agents, RAG pipelines, and model context protocols.

But AI and machine learning did not begin with generative AI.

For decades, we have had regression models, decision trees, support vector machines, clustering algorithms, Bayesian methods, and countless other techniques. Deep learning brought us neural networks, convolutional neural networks, recurrent neural networks, and transformers. Computer vision and natural language processing developed specialized approaches to problems ranging from image classification to text processing.

These techniques have not suddenly become obsolete.

A regression model can predict a numerical value. A classifier can categorize an input. A rules engine can enforce business constraints. A database query can retrieve structured information. A traditional optimization algorithm can find an efficient allocation of resources.

And none of these necessarily requires an LLM.

What changed is not that these approaches stopped working. What changed is that one category of AI became dramatically easier to access, demonstrate, and integrate.

An API call to a powerful model is an attractive shortcut. You can build a convincing prototype without understanding the underlying mathematics, selecting a suitable algorithm, or collecting and preparing a training dataset.

That accessibility is valuable. But it also creates a risk: people may begin to assume that because LLMs can solve a problem, they are the best way to solve it.

Capability is not the same as suitability.

Let's classify our problem solving approaches into 3 categories

I find it useful to think about the available approaches in three broad categories. The categories can overlap most of the times, and the right approach depends on the problem being solved.

1. Deterministic approaches: When the rules are known

These are the traditional software engineering techniques we have relied on for decades.

We define the inputs, specify the rules, implement the logic, and obtain predictable outputs. Given the same inputs and state, the system produces the same result.

Examples include:

• Calculating invoices, taxes, and financial totals.
• Enforcing authorization policies and validating business rules.
• Applying database queries, sorting records, and validating data.
• Executing predefined workflows and checking whether a system meets specified constraints.
• ... and many other use cases, we have been solving using traditional software development

The advantage is not merely that these approaches are cheap. They are generally easier to test, audit, debug, and reason about. If something fails, you know for sure - why it failed.

If we understand the problem well enough to express it in deterministic rules, introducing an LLM into the critical calculation or decision path doesn't make sense.

Why pay for probabilistic behavior when you can already model a predictable behavior?

2. Approximate approaches: When patterns must be learned

Some problems cannot be solved adequately using hand-written rules alone. The relationships are too complicated, the data is noisy, or the underlying patterns are difficult to describe explicitly.

This is where statistical methods, numerical techniques, and machine learning become useful.

Examples include:

• Predicting demand or infrastructure utilization from historical data.
• Detecting fraudulent transactions.
• Forecasting sales or customer churn (which is a very important problem for telecom industry).
• Classifying images using convolutional neural networks.
• Detecting anomalies in system metrics.
• Ranking recommendations based on observed user behavior.

The system learns patterns from data and produces estimates or predictions. Those outputs may be right or wrong (since it is not deterministic), but their quality can be measured using appropriate evaluation methods.

Suppose infrastructure costs depend on traffic, cache hit rates, workload composition, autoscaling behavior, and database utilization. A regression model or another suitable predictive approach might help estimate future costs using historical observations.

The important point is that we are not blindly introducing an LLM. We are selecting a model based on the nature of the problem, the available data, and the accuracy we need.

Sometimes a simple linear regression is enough. Sometimes a more sophisticated model is justified. Sometimes a deterministic baseline performs just as well.

3. LLMs and other generative approaches: When the input is difficult to formalize

Then there are problems where the inputs are unstructured, the instructions are expressed in natural language, or the task requires interpreting context that is difficult to capture with conventional rules.

Examples include:

• Summarizing lengthy customer support conversations.
• Extracting meaning from unstructured documents (for example vendor bills, insurance policies, etc.)
• Translating natural-language questions into database queries (Uber also built an interesting solution around this)
• Helping users navigate complex information (very common these days)
• Generating an initial draft from a collection of source materials (one of the must haves - since people no longer like to fill out forms)
• Interpreting ambiguous requests and coordinating multi-step workflows.

LLMs are particularly useful here because they can work with language and context without requiring us to enumerate every possible phrasing or scenario.

Even here, an LLM is not automatically the best choice. A traditional NLP pipeline, a smaller specialized model, a search system, or a combination of approaches might be cheaper and more reliable.

And for complex workflows, an agent is not necessarily required just because multiple steps are involved. If the steps are known in advance, a conventional workflow engine may be easier to control.

The goal is not to use the most sophisticated technology available. It is to use the simplest approach that satisfies the requirements.

Making LLMs "too easy" to adopt has a hidden cost

The financial cost of AI is the most obvious concern. Model usage can grow rapidly when developers integrate APIs into features without carefully estimating request volumes, context sizes, retries, or inference costs.

But token consumption is only one part of the problem.

Every additional model dependency introduces potential latency, availability concerns, version changes, output variability, security considerations, and evaluation requirements. Systems may need fallbacks, guardrails, tracing, prompt management, and ongoing monitoring.

An agent that autonomously calls tools can multiply these costs if its execution is poorly constrained. A feature that looks inexpensive in a demonstration may become surprisingly expensive at production scale.

There is also an environmental dimension.

AI inference consumes electricity, and the total impact depends on factors such as model size, hardware utilization, request volume, cooling, and the electricity mix powering the infrastructure. Data centres also require physical infrastructure and, in some locations, substantial water resources for cooling. Training large models adds further computational demands.

Not every LLM call has a dramatic environmental footprint. But unnecessary calls multiplied across millions of requests are still unnecessary consumption.

If a simple calculation can be performed locally in milliseconds, sending the request to a remote model adds computation which could have been avoided, network traffic, and energy demand.

Efficiency is not just a cloud cost optimization exercise. It is also a matter of responsible engineering.

The Technical Debt We May Have to Repay

I suspect that many organizations will eventually go through a phase of reassessing their AI integrations.

As usage grows, budgets tighten (with diminishing returns on AI investments), and production systems become more complicated, teams will start asking questions that should have been asked at the beginning.

Why are we using an LLM here? Could a rule-based system achieve the same result? Could we use a regex matching? Would a smaller model suffice? Do we even need a model? What happens if the provider changes its pricing or becomes unavailable?

Answering these questions will not always be straightforward.

The original implementation may have spread across multiple services, workflows, and product features. Business logic may have become embedded in prompts. Developers may have built dependencies around a particular model's behavior. Test suites may not adequately measure output quality, and nobody may have established a deterministic baseline against which to compare the system.

Replacing these integrations then becomes more than removing an API call. It can require redesigning workflows, reconstructing business rules, building evaluation datasets, and migrating production dependencies without disrupting users.

Leave alone the sloppy code, that is the technical debt we should be thinking about. This would be a much bigger debt problem than what we have ever seen.

The irony is that we may spend less time building the initial solution only to spend considerably more time operating, debugging, and eventually simplifying it.

Fast implementation is valuable. But implementation speed without architectural discipline can merely shift costs into the future.

Organizations need governance around adoption of LLMs

I believe organizations need a more deliberate approach to AI adoption.

Before introducing an LLM or an agent, teams should document the problem, establish measurable success criteria, and evaluate alternative approaches. A simple decision framework could look like this:

Organizations should maintain visibility into where AI is used, which models are involved, what those integrations cost, and whether simpler alternatives have been evaluated.

This does not mean every small feature needs a lengthy approval process. Governance should be proportionate to the risk and scale of the application.

More importantly, leaders need to understand these distinctions themselves. If leadership treats AI adoption as a target in its own right, teams will naturally optimize for visible AI integrations rather than meaningful outcomes.

Developers should not feel that using fewer LLMs makes them less innovative. Sometimes, the most thoughtful engineering decision is to remove a model from the architecture.

A Practical Decision Matrix for Choosing the Right Approach

Here is a simple matrix teams can use during design reviews before deciding to introduce an LLM, an agent, or another AI technique. This is what I find particularly very useful.

The matrix is a starting point, not a set of rigid rules. A problem can combine several approaches. For example, an LLM might interpret a user's request, a deterministic service might perform the calculation, and conventional validation might check the result. (Yes, this is what most AI agents built on top of MCPs do right now!)

The objective is to assign each part of the problem to the technique best suited to it.

Before approving an AI integration, ask five questions:

1. Necessity: What specific limitation prevents us from solving this with conventional software?
2. Alternatives: Have we compared a deterministic baseline, a simpler model, and an LLM where appropriate?
3. Quality: How will we measure correctness, reliability, and failure rates?
4. Economics: What are the expected costs at production scale, including inference, retries, monitoring, and maintenance?
5. Reversibility: Can we replace the model or remove the AI dependency without redesigning the entire system?

One final principle matters: compare approaches on total cost of ownership, not API pricing alone. A slightly more expensive model may save significant engineering effort, while a seemingly cheap integration may create substantial operational overhead.

The right decision is the one

We Need Better Engineers, Not Just More AI Integrations

The biggest risk I see is not that companies are using too much AI today. It is that we may gradually lose the habit of asking whether we need it at all.

When a new technology becomes popular, there is always a temptation to apply it everywhere. But engineering has never been about using the newest tool for every problem. It has been about understanding constraints, evaluating trade-offs, and choosing an appropriate solution.

LLMs have expanded what we can build. They have lowered barriers to experimentation and made previously difficult interfaces and workflows accessible to far more people. That is an extraordinary achievement.

But they have not eliminated the need for statistics, machine learning, algorithms, systems design, or fundamental software engineering.

We should teach developers not only how to call a model API, construct an agent, or assemble a RAG pipeline, but also how to recognize when those tools are unnecessary. We should encourage teams to understand the underlying problem before reaching for a familiar solution.

The future of good AI engineering is not about using an LLM wherever possible. It is about knowing when an LLM is the right choice, when another model is better, and when no AI is needed at all.

The next competitive advantage may not belong to the companies that integrate the most AI. It may belong to the ones that know precisely where AI creates value, where it does not, and how to build systems that remain efficient, reliable, and maintainable over time.

Because at the end of the day, our job is not to maximize AI usage.

It is to solve problems well.