Stop Guessing Your App's Resource Requirements

After development comes deployment - whether on-premise or on a cloud based environment. And then we face a simple question: how much resource should I assign to this system? What is the ideal numbers? If we get this wrong, we often need to go back time and again to fine tune - either to ensure our application is capable of handling the targeted load, or to avoid
paying for resources we are not using.
This article explains the approach step by step. So that we spend just enough time
upfront to avoid spending exponentially more time and money at later stages.

Who Is This For?

This article is written primarily for developers. But if you are a manager or a CTO, there are sections written specifically for you. Feel free to jump straight there.

• 👉 If you are a Manager or Project Manager
• 👉 If you are a CTO or Architect

For everyone else - the full article is worth reading top to bottom at least once. But if you are revisiting a specific topic, jump to whatever is relevant.

Table of Contents

1. Local is the Starting Point
2. When Should You Start Thinking About Right Sizing?
3. How Long Will This Actually Take?
4. Start With What You Have - Your Local Setup
5. Setting Up Your Load Generation - The Hammer
6. The Cost of Testing - This Is Not Free
7. Sizing Your Pod
8. More Resources Per Pod or More Pods?
9. Scaling - Easy to Set Up, Hard to Get Right
10. Periodic Right-Sizing - You Are Not Done Yet

Local is the Starting Point

Local system is always where we start. To try things out, to check if things work. But 99% of what we test locally is the sunny day scenario. Does the MVP work? Does the happy path hold? Even if you're diligent enough to test negative scenarios, you're almost certainly not testing production-level load on your laptop. Which means you have no idea what resources your app actually needs when it matters.

This is where the problem starts.

On local, we routinely kill the heavy IDE, close browser tabs, shut down background processes -without ever stopping to ask: how much memory and CPU does this app actually need to run? With modern systems sometime we don't worry about even that.

To come to the correct numbers we need to understand resource requirements across four distinct states your app will be in:

• Startup - some apps run heavy initialization scripts, pre-load caches, or run migrations. Resource consumption here can spike significantly above idle.
• Idle - no load, app is just running. This is your floor. The minimum you'll always be paying for.
• Normal load - the load your system sees 90-95% of the time. This is what you'll be sized for, and what your infrastructure bill is mostly based on.
• Peak load - the theoretical maximum your system can handle successfully. Beyond this you make no guarantees. But you've tested up to this number, and you own it.

For each of these states, we're primarily concerned with three resources: CPU, memory, and disk. We won't go into network throughput, Kafka storage sizing, or DB memory in this article - each of those deserves its own deep dive.

Could you just throw maximum resources at every pod and call it a day? Sure. It'll probably work. But it'll cost you - and we'll get to exactly how much that means in real dollars later in this article.

For now, let's start from the beginning and build towards a number you can actually defend.

When Should You Start Thinking About Right-Sizing?

There is no single right answer here. Most people will say "as early as possible" - but that's easier said than done. You cannot right-size an application during requirement gathering or system design. The numbers simply don't exist yet.

But here's the conundrum: managers and CTOs need to tell stakeholders how much this system will cost to run. And stakeholders want a number long before the app is built.

So let's split this into two realities.

Before the App is Built - Four Inputs Must Be Locked First

From a dev or architect perspective, you cannot give any number with confidence until you have answers to these four questions:

• What is your normal load? The average load you will receive 95% of the time.

• What is your peak load? The maximum load the system must support.

• What is your SLA? Acceptable latency, error rate, response time.

• Is this mission critical? Can it go down, even briefly, or does it need to be up at all times?

Without these four inputs, any number you give is a guess dressed up as an estimate.

🏗️ CTO/Architect lens: If you're being pushed for numbers before these inputs are finalized, lean on your experience. Something like: "Based on similar systems, we're looking at a minimum of 3 pods and a maximum of 25, with 512MB memory and 2 CPU cores each - but treat these as directional until the app is built and we've run actual load tests." Always attach that caveat. Without it you'll be held to a number that was never real.

After the App is Built - There's Still One More Gate

Even with all four inputs locked, even with the app fully developed, you still cannot give precise numbers until you run real workloads through the actual application.

A real message, going through your real processing pipeline, hitting your real database - that is the only thing that tells you the truth about resource consumption. Synthetic estimates and architectural assumptions will get you in the ballpark. Only a running app under real load gives you the number you can actually put in a configuration file and defend.

Everything from here onwards is about how to get to that number systematically, without spending months and a small fortune figuring it out in production.

How Long Will This Actually Take?

The first question your manager will ask is "how long will this take?" Followed closely by "can we get a close enough number faster?" And inevitably: "can we just throw more people at it and get it done quicker?"

This is not a half day task. It is not a two day task. It is an iterative process, and the real value - the savings, the stability, the confidence - comes from doing it thoroughly.

Here is why it takes as long as it does: a 2 node setup with 1 CPU core and 1GB RAM each will almost always perform differently than a single node with 2 cores and 2GB RAM. And figuring out whether a 500 millicores/256MB config outperforms a 1 core/768MB config requires someone to actually run both, under real load, and document the findings. Otherwise you or someone on your team will repeat the same test six months from now and waste the same time all over again.

What you can realistically expect:

• Medium to large systems - 2 to 3 configurations tested, verified, and documented per person per day
• Small to medium systems - 5 to 10 configurations per person per day If you want to parallelize to go faster, you need separate test environments for each person. One shared environment means everyone is queuing, not parallelizing. And each environment has its own cost - we'll get to that later.

The larger and more complex the system, the more configurations you need to explore. Plan accordingly and set that expectation with your manager upfront. 2 weeks of thorough testing now saves months of firefighting in production.

💼 Manager's lens: When a dev says this will take a week, they're not being slow - they're being honest. Cutting this short doesn't save time, it moves the problem to production where it costs significantly more to fix. The configuration report generated here is a long term asset. It prevents the same work from being repeated every time someone questions the cluster setup.

The Example We Will Use Throughout This Article

To keep things concrete, we will use the same pipeline across all examples and screenshots:

JMeter → Kafka → Spring Boot app → PostgreSQL

JMeter drops messages into a Kafka topic. A Spring Boot application consumes those messages, does some processing, and writes to PostgreSQL. Simple enough to follow along, realistic enough to reflect what some production systems actually look like.

All numbers, screenshots, and cost examples in this article are based on this pipeline.

Start With What You Have - Your Local Setup

Your local machine is the cheapest and most configurable test environment you have access to. Use it well. Extract as much information as possible from it before you spend a single dollar on cloud infrastructure.

We will do this in two steps. First, find your idle numbers. Then, start dropping load and watch what happens.

Step 1 - Find Your Idle Numbers

Here is something most people don't realize: idle resource consumption is almost always the same regardless of where your app is running. A Spring Boot app consuming 320MB at idle on your laptop will consume roughly the same on a cloud VM. This makes your local machine the perfect place to find your baseline.

Start your application. Let it fully initialize - wait for all startup scripts, cache loads, and connection pools to settle. Then leave it alone. No requests, no load. Just running.

Now find your process and note down its resource consumption.

Important: always use absolute numbers, not percentages. 40% CPU means nothing without knowing how many cores you have. 320MB memory is a number you can actually use.

Note down:

• Memory - how much RAM is the process consuming at rest

• CPU - how many cores or millicores is it consuming at rest

• Disk - only if your app is read/write heavy. At idle this should be negligible

These are your base numbers. Write them down. Everything else is built on top of this.

How to Find These Numbers

The recommended tool across all platforms is htop - it is clean, filterable, and shows exactly what you need. top, Activity Monitor (Mac), ps aux, and pidstat (Linux) will also give you the same information if htop is not available.

Linux/Mac
Install htop.

htop

Press F4 to filter, type java. You will see memory and CPU per process in a clean, real time view.

What to read: Look at the RES column for memory - resident memory, the actual RAM your process is using. Ignore VIRT. For CPU read the CPU% column.

You can also use Activity Monitor (Spotlight → Activity Monitor) or top in terminal - both show the same numbers, htop is just easier to work with.

Windows

Use Task Manager → Details tab → find java.exe. Right click → Select Columns → add Memory (private working set) and CPU.

Microservices - Same Approach, One Service at a Time

If you are running multiple services locally, don't try to measure everything at once. Start each service one by one and note its idle consumption before starting the next one. This gives you a per-service baseline rather than a combined number you can't break down later.

If your service depends on other processes to run - Kafka, PostgreSQL, a sidecar - identify your specific process clearly in the tool and measure only that. Not the total system consumption.

Step 2 - Start Dropping Load

Now the interesting part.

Run a single message through the pipeline. One JMeter request → Kafka → Spring Boot → PostgreSQL. Watch your process in htop. Note the maximum CPU and memory your process hit during that single request. Not the average - the maximum.

Now calculate your first delta:

Delta = (Peak resource during 1 request) - (Idle resource)

Write that down too.

Now increase sequentially. 2 messages. 5 messages. 10 messages. 25 messages. Do not jump randomly. The whole point is to see a pattern emerge. Gut feeling jumps destroy the pattern.

For each run note down peak CPU and memory. Build a simple table like below:

Now look at the pattern. Is the delta growing at roughly the same rate as the message count? That's linear. Is it growing faster than the message count? That's exponential - and you need to understand why before you go any further.

Use AI to spot the pattern if it isn't obvious. Dump your table into Claude or ChatGPT and ask it to identify whether the growth is linear, exponential, or something else. Takes 30 seconds.

Linear vs Exponential - Why It Matters

Linear growth means your app is well behaved. Resource consumption scales predictably with load. You can extrapolate with reasonable confidence.

Exponential growth means something is wrong. A memory leak, an N+1 query, unbounded caching, connection pool exhaustion - something in your code does not scale. Fix this before you do anything else. No amount of right-sizing will save an app with exponential resource growth. It will just fail more expensively.

🏗️ Architect's lens: Exponential resource growth under load is a design problem, not a configuration problem. Throwing more pods at it is not a solution - it buys time at best. This local testing phase is often where these problems surface for the first time, and the cheapest place to fix them.

This is one of the most valuable things about this local exercise - you will catch code problems here that will never show up in casual single request testing. A happy path test will never tell you your app has an N+1 query. Twenty five sequential requests will.

Extrapolating to Your Target Load

Once your pattern is clear and your app is behaving linearly, you can extrapolate.

Let's say your target is 1000 requests per second as your normal load. You have data up to 25 requests. Here is how to get to a starting configuration number:

Step 1 - Run 100 requests if your local machine can handle it. Note peak CPU and memory.
Step 2 - Calculate delta for 100 requests:

Delta(100) = Peak(100) - Idle

Step 3 - Extrapolate to 1000:

Estimated resource for 1000 req = (Delta(100) × 10) + Idle + 10% buffer

Using our example numbers - let's say 100 requests gave us peak memory of 880MB and peak CPU of 10.3 cores:

Delta(100) = 880MB - 320MB = 560MB memory / 10.25 cores CPU
Estimated for 1000 req:
Memory = (560MB × 10) + 320MB + 10% buffer = 5600 + 320 + 592 = ~6500MB (~6.5GB)
CPU = (10.25 × 10) + 0.05 + 10% buffer = 102.5 + 0.05 + 10.3 = ~113 cores

This is your single machine number. The total resource you would need if you were running everything on one box to handle 1000 RPS.

Don't panic at that number - we are not putting this on one machine. This is the input to the next step: distributing across multiple pods. But you need this number first.

⚠️ One important caveat: This extrapolation assumes linear scaling holds at 1000x. It may not. Use this as your starting point - not your final answer. The further you extrapolate beyond your tested range, the less confident you should be. Always verify with an actual load test at scale, which we will cover in the next sections.

Setting Up Your Load Generation - The Hammer

By this point you know your target peak load. You have extrapolated numbers from your local testing. Now comes the part where you actually verify if those numbers hold under real sustained load.

For that you need a system that can generate that load reliably. And this is where most people underestimate the setup.

Now Your Local Machine Is Not Enough

Your local machine was perfect for finding idle numbers and plotting the load curve. But it has already shown you its limits - you know roughly how many requests it can handle before it starts struggling. If your target peak is 1000 RPS and your local machine tops out at 100 RPS, you cannot use it as your load generator for peak testing.

You need a dedicated setup for load generation. Either two separate machines - one to generate load, one to run your application - or one powerful machine with enough headroom to do both without contaminating your results. Running the load generator on the same machine as your application is one of the most common mistakes in load testing. The load generator competes for the same CPU and memory as your app, and your results become meaningless.

The Pre-loading Hack - Save Money and Infrastructure

Here is a trick that can save you significant infrastructure cost, especially when your target throughput is high.

Instead of running your load generator and your application simultaneously at full throttle, do this:

1. Stop your processing service
2. Pre-load your target message count into Kafka using a small, cheap system
3. Start your processing service and let it consume from Kafka at full speed

Why does this help? Because generating 1000 RPS continuously while simultaneously processing 1000 RPS requires two powerful systems running in parallel for the entire duration of the test. With pre-loading, you only need a powerful system for the consumption and processing phase. The load generation phase can happen slowly, cheaply, on a much smaller machine.

Here is what the difference looks like in practice:

Without pre-loading:

• Need a dedicated load generator capable of sustained 1000 RPS for 20+ minutes
• Both systems running simultaneously for the full test duration
• To reach a clean 15 minute window at peak load, you are looking at 20-25 minutes of full infrastructure running
• At 1000 RPS that is approximately 1.2 to 1.5 million messages generated and processed

With pre-loading:

• Use a small cheap instance to push 1 million messages to Kafka - no time pressure, no throughput requirement
• Spin up your application, start consuming
• Your application runs at full speed against the pre-loaded queue
• Load generator cost is near zero. You only pay for the application infrastructure during the actual test.

We will put exact dollar amounts on this difference in the next section.

Setting Up JMeter

JMeter is the standard tool for this. It has broad support for data sources, Kafka integration, configurable throughput, and warmup support - which covers most scenarios.

Target throughput - configure JMeter's Constant Throughput Timer to keep load close to your target RPS. Without this JMeter will generate load as fast as it can, which is not what you want. You want sustained, controlled throughput that mirrors real traffic.

Warmup - always warm up before your measurement window starts. A practical rule: let the system run for at least 5 minutes before you start recording results. During this initial period the system is not running at peak performance - connection pools are stabilizing, caches are warming, the JVM is settling. Results from this window are not representative. JMeter's Ramp-Up Period configuration handles this - set it to gradually increase load over the first 5 minutes rather than hitting full throttle immediately.

Test duration - configure JMeter to run long enough to give you a clean measurement window. You need at minimum 15-20 minutes of sustained load at target throughput to have confidence in your results. Anything shorter and you are not seeing steady state behavior.

Stopping after N messages - if you are using the pre-loading approach, configure JMeter to generate exactly the number of messages you need and stop. No need to manage throughput rate in this case - just set the count and let it run.

Unique messages - if your system detects and rejects duplicates, use JMeter's ${__UUID()} function to generate a unique ID for each message. One line in your message template, zero duplicates.

Corrupted data - review your sample data before running. If your system has validation rules that flag certain patterns as corrupted, make sure your generated data does not accidentally trigger them. A test run where 20% of messages are rejected as invalid is not a useful data point.

If JMeter Does Not Meet Your Needs

JMeter covers the vast majority of load testing scenarios. If you hit a case it genuinely cannot handle, write a custom script. The goal is simple: generate enough load that your system runs at target throughput for 15-20 minutes continuously. How you get there is secondary.

The Cost of Testing - This Is Not Free

In the era of cloud computing, spinning up a supercomputer is a single Terraform script away. Most developers have never felt the pain of a surprise AWS bill because most organisations do not share cloud cost details with their engineering teams. Understandable - but it creates a blind spot. Developers make infrastructure decisions every day without understanding what those decisions actually cost.

This section tries to improve that. We will walk through the real cost of right-sizing, scenario by scenario, using our standard benchmark:

• Pipeline: JMeter → Kafka → Spring Boot → PostgreSQL
• Target load: 1000 RPS
• Message size: 5KB
• Processing time: ~300ms per message
• Test duration: 30 minutes of sustained peak load

Start Here - Your Laptop

Running a MacBook Pro M4 at full load for 24 hours consumes 0.72 kWh - less than one unit of electricity. At US rates that is roughly $0.09.. India current ~INR10.

Keep that number in your head. Everything below is measured against it.

The Numbers That Seem Reasonable

Let's start with what looks like a modest, careful setup.

Idle infrastructure - just keeping things running:

$7/day. Not alarming. Less than a coffee.

A single 30 minute test run, being diligent:

Still not scary. $2.45 per round, 2-3 rounds a day, $5-7/day in active testing plus $7/day idle. Call it ~$12-15/day total. A week of testing: ~$85-100.

Manageable, right?
Here is where it changes.

The Production Mirror Reality

For your right-sizing numbers to mean anything, your test environment must match production exactly. Same instance types, same Kafka broker count, same RDS instance class. Testing on a smaller setup and deploying to production-grade infrastructure gives you numbers that are essentially fiction.

So let's talk about what production-grade actually means at 1000 RPS with 5KB messages.

Your data volume:

1000 RPS × 5KB × 3600 seconds = 18GB per hour
18GB × 24 hours = 432GB per day
432GB × 365 days = ~158TB per year
158TB × 3 years = ~470TB

We are ignoring theoretical storage limits and sharding costs here - for a single round of testing you can get away with a single deployment. But your infrastructure needs to be sized for production lifetime data volumes to give you realistic performance numbers.

What Kafka actually needs to cost at this scale:

What PostgreSQL actually needs at this scale:

Kafka + PostgreSQL alone: ~$188/day.

Just those two components. Before a single application pod. Before your load generator. Before anything else.

The Real Cost of One Test Round

30 minutes of peak load testing on production-grade infrastructure, being completely diligent - warmup, test, tear down immediately:

2-3 rounds per day: ~$20-36/day in active testing.
Plus infrastructure idle cost: ~$188/day.
One serious day of right-sizing: ~$200-220.
One week: ~$1,400-1,500.

The Contrast

Your laptop ran at full load for 24 hours for $0.09.

One day of properly configured test infrastructure - the kind that actually gives you numbers you can trust - costs more than most people's monthly electricity bill.

This is not an argument against doing right-sizing properly. It is an argument for doing it efficiently. Every wasted test run, every idle hour, every forgotten Kafka topic sitting in storage - it all has a real dollar amount attached to it.

What Silently Bleeds Money

These are the habits that turn a $1,500 week into a $3,000 week:

1. Running tests longer than needed
5 extra minutes at 1000 RPS = 300,000 extra messages = 1.5GB of extra data your service processes and stores. Zero testing value. Pure cost.

2. Leaving infrastructure idle between runs
At ~$188/day for Kafka and PostgreSQL alone, every hour of idle time costs ~$8. A forgotten long weekend costs ~$560.

3. Orphaned storage
One 30 minute test run generates ~9GB in Kafka and ~9GB in PostgreSQL. After a week of testing without cleanup that is ~63GB of stale test data you will never look at again - and are still paying to store.

4. Not scaling down between iterations
Leaving 10 pods running overnight instead of 3 costs an extra ~$8/day in application pods alone.

The Rules - Non Negotiable

1. Spin up fresh for each iteration. Start clean, test, shut down completely.
2. Clean up storage after every run. Purge Kafka topics. Truncate test DB tables.
3. Never leave test infrastructure running unattended. Not overnight. Not over a weekend.
4. Use the pre-loading approach wherever possible - it reduces your load generator cost and gives you cleaner test conditions.
5. Track your runs. Know how many you have done and what each one cost. It keeps the team honest.

💼 Manager's lens: A thorough right-sizing exercise done properly costs roughly $1,500 in cloud infrastructure for a medium sized system. That sounds significant until you compare it to running an over-provisioned production cluster for 12 months. Over-provisioning by just 20% on a $10,000/month production cluster costs $24,000 a year. The right-sizing exercise pays for itself in the first month.

Sizing Your Pod

The question that decides how resource-optimized your application is: what should be the size of a single pod?

Before we get to the systematic approach, let's be honest about how most teams actually answer this question.

How The Industry Actually Does It

Gut feeling

This is the most commonly used method. Individual experience shapes the number - and you would be surprised how many teams start with 1 core and 2GB memory as a default and work from there. Nothing fundamentally wrong with it, but reaching the right configuration from this starting point is the most time consuming path.

Copy production

Find a service in production with similar load or functionality and copy its configuration. Sounds reasonable. Except most people never check whether that existing service is quietly throttling or wastefully over-provisioned. You are copying a number with unknown history. Still time consuming to correct later.

Start big, scale down

Go with an overkill configuration - say 4 cores and 8GB RAM. Monitor production. If memory never crosses 60%, reduce it. Repeat. Over time you land somewhere close to right. This works, but you are paying for over-provisioned resources for months while you get there.

Systematic approach

This is what we will cover here. It takes more time upfront. But it gives you the best possible configuration in the fewest iterations. Your local testing data is the foundation - which is why we told you to save those numbers.

Finding Your Single Pod Ceiling

Go back to your local testing data. You were systematically increasing load - 1 message, 2, 5, 10, 25, 100. At some point something interesting happened: you asked the system for more throughput and it stopped delivering it.

You pushed for 50 RPS. The system only processed 40 RPS. It could not keep up anymore.

That is your ceiling. That is the point where throwing more requests at it stops producing more output.

This is different from CPU hitting 90% or memory spiking - those are symptoms. Throughput degradation is the actual signal. When your system cannot deliver the RPS you are asking for, you have found your limit.

Record two things at this point:

• The RPS at which throughput started degrading
• The CPU and memory the process was consuming at that point

Setting Your Per Pod Target

Now take 60-70% of that ceiling number. That is your per pod target load.

Why 60-70% and not 100%? Three reasons:

• GC pressure - JVM garbage collection causes periodic CPU and memory spikes. You need headroom for that.
• Traffic spikes - real traffic is never perfectly flat. You need room to absorb sudden bursts without immediately saturating.
• Pod failure headroom - if one pod goes down, its load redistributes to surviving pods. Those pods need enough spare capacity to absorb that without degrading.

Example:

Throughput started degrading at 50 RPS on your local process. Your per pod target is:

50 × 0.65 = ~35 RPS per pod

Now go back to your extrapolation data. You already calculated resource consumption at various load points. Find the configuration that comfortably handles 35 RPS - that is your pod size.

Let's say that works out to 1 core CPU and 2GB RAM per pod.

Calculating Pod Count

You have your per pod target. You have your peak load. Pod count follows directly:

Max pods = Peak load / Per pod target

= 1000 RPS / 35 RPS

= ~29 pods

Add a 10% buffer for future development - new features, code changes, anything that might slightly reduce throughput:

29 × 1.1 = ~32 pods maximum

Minimum pod count is always 3 regardless of load calculation. This is the industry standard for basic disaster recovery - if one pod goes down during a rolling deployment or a node failure, you still have two pods serving traffic while the third recovers.

So your range is:

Minimum: 3 pods

Maximum: 32 pods

Your autoscaler works within this range.

A Note on JVM Overhead

Since our example uses Spring Boot - always account for JVM baseline memory on top of your application's actual usage. A Spring Boot app with 512MB of actual heap usage will typically consume 700-800MB of real memory once you add JVM overhead, metaspace, thread stacks, and off-heap buffers.

Never set your pod memory limit equal to your heap size. Set it at least 20-25% higher. Otherwise your pod will get OOMKilled regularly and you will spend hours debugging what looks like a memory leak but is actually just insufficient allocation.

🏗️ Architect's lens: The systematic approach described here takes a few days of upfront work. The gut feel approach takes 5 minutes and then 6 months of production tuning. The systematic approach is cheaper. It just does not feel that way until you have done both.

More Resources Per Pod or More Pods?

You have your initial configuration running. Things are working. Then your monitoring starts showing something uncomfortable - one or more resources consistently hitting 90-95% utilization. Not occasional spikes. Hours of sustained pressure.

That is your cue to revisit your pod configuration.

Two Scenarios You Will Face

Scenario A - One resource is at its limit, the other is not

Memory is maxed out but CPU is sitting at 40%. Or CPU is saturated but memory has plenty of headroom. This tells you something specific about your workload - it has become more memory intensive or more CPU intensive than your original testing suggested.

Scenario B - Both resources are hitting their limits

Either together or at different times. This means your pod is genuinely overloaded - it is being asked to do more than it was sized for.

Both scenarios need the same disciplined approach.

Local First or Direct Tuning?

You have two options:

Local testing first - run through the same process as before. Useful if you have time and the problem is complex enough to warrant it.

Tune directly on the deployed system - faster, but riskier. Only do this if your local testing has already given you all the data it can, or if time pressure is real.

Whichever path you choose, the tuning process is the same. Small increments. One change at a time. Observe before changing again.

The Tuning Process - Step by Step

Step 1 - Increase only the resource that is at its limit

Do not change both at once. Increase in small increments:

• Memory: +/-128MB at a time
• CPU: +/-100 to 250 millicores at a time

Step 2 - Observe the response

After each increment watch two things: throughput and resource utilization.
The key question is: did throughput improve meaningfully?

Doubling a resource will never give you double the throughput - too many other factors are at play: I/O wait, GC cycles, database bottlenecks, network latency. But you should see a meaningful improvement. A rough rule:

If you increase a resource by 2x and throughput improves by less than 1.5x
→ that resource is no longer your bottleneck
→ stop increasing it and look elsewhere

Step 3 - Fine tune

Once throughput is improving proportionally and utilization is back in range, stop making large changes. Adjust by a maximum of 10-15% in either direction to find the precise sweet spot.

Step 4 - Watch for underutilization of both resources

Sometimes increasing one resource suddenly results in both CPU and memory being underutilized. This is a signal you have overshot. Go in the other direction - reduce both resources slightly and observe.

Step 5 - When increasing resources stops helping

If you have increased a resource meaningfully and throughput is simply not responding - your bottleneck is elsewhere. It could be your database, your Kafka consumer configuration, your network, or something in your code.

At this point:

• Record the new throughput ceiling for a single pod
• Update your maximum pod count accordingly

New max pods = Peak load / New per pod throughput ceiling

When to Add More Pods Instead

If tuning a single pod keeps hitting a ceiling regardless of resources, the answer is horizontal scaling - more pods, not bigger pods.

More pods also makes sense when:

• Your per pod resource config is already at a reasonable size and increasing further feels disproportionate
• You want better fault tolerance
• Cost favors it - smaller pods you can scale down overnight are cheaper than large pods running 24/7

The general rule: if increasing resources gives diminishing returns below the 1.5x threshold, stop scaling up and start scaling out.

🏗️ Architect's lens: The single most common mistake here is treating a resource problem as always a resource solution. If your app has an N+1 query, a memory leak, or an inefficient algorithm, no amount of CPU or memory will fix it permanently - it will just delay the next crisis. Before tuning resources, always rule out a code problem first.

Scaling - Easy to Set Up, Hard to Get Right

Autoscaling is one of those features that looks straightforward until you have a system scaling up and down every 2 minutes at 2am for no apparent reason, or refusing to scale when you desperately need it to. The rules below will not cover every edge case but they will give you a solid, defensible starting point.

Rule 1 - Your Primary Metric Must Reflect Your Actual Workload

If your application reads from Kafka, Kafka consumer lag must be part of your scaling criteria. CPU and memory alone will not tell you your app is falling behind - a backlog of 2 million unprocessed messages will not show up in CPU utilization until it is far too late.

If your app is a set of API endpoints, incoming request rate or request queue depth should be your primary metric.

If your app is memory intensive or does heavy disk reads and writes, memory utilization or disk I/O should be in your criteria.

Match your scaling trigger to what your application actually does. Generic CPU-only scaling is better than nothing - but only just.

Rule 2 - Combine Multiple Criteria With OR Conditions

Use OR not AND.

AND means all conditions must be true simultaneously before scaling - too conservative, will miss real load scenarios.

OR means any one condition being true triggers scaling - more responsive, catches different types of load pressure.

A practical example for our pipeline:

Scale up if:
Kafka consumer lag > 100,000 messages
OR Memory utilization > 75%
OR CPU utilization > 75%

Rule 3 - Be Measured About How Many Pods You Add or Remove

Never increase by more than 10% of your maximum pod count in one scaling event.

If your maximum is 30 pods, scale up by 2-3 pods at a time. Not 10. Not 15.

You can be slightly more aggressive scaling down - but still gradual. Going from 30 pods to 15 in one step is unnecessary and creates instability. Scale down by 3-5 pods at a time.

Rule 4 - Two Timers, Not One

Stabilization window - 30 seconds
The condition must persist continuously for 30 seconds before any scaling action is triggered. A single memory spike that lasts 3 seconds does not warrant spinning up new pods.

Cooldown period - 1 minute
After a scaling event, wait 1 full minute before evaluating conditions again. New pods need time to start, register, and begin absorbing load before you trigger another scale event.

Getting these two timers wrong is the most common reason autoscalers behave erratically.

Rule 5 - How Aggressive You Are Depends on Criticality

• Mission critical system - scale up when you hit 70% of your threshold.
• Standard production system - 75-80% is a reasonable trigger point.
• Low traffic or non-critical system - you can afford to wait until 85-90%.

A Sensible Starting Point

Start here. Observe for a week. Then fine tune based on what you actually see.

💼 Manager's lens: Autoscaling is not a "set it and forget it" feature. The initial configuration is a starting point. Budget time for at least one tuning iteration after the first week of production traffic. An autoscaler configured on assumptions and never revisited is almost as risky as no autoscaler at all.

Periodic Right-Sizing - You Are Not Done Yet

By this point you have run multiple iterations of testing. You have tables, notes, observations. Some findings will be clean enough to summarise in a table. Others will be complex enough that a detailed write-up makes more sense. Do not force everything into a table - use whichever format captures the nuance.

If you plot your findings on a graph - resource consumption or throughput against configuration - the tipping point where the curve flattens is your optimal configuration. That is where adding more resources stops producing meaningful returns.

Take help of AI to generate summaries or spot patterns across your test runs. You have the data - use the tools available to make sense of it faster.

What Comes Next - Monitoring

Monitoring is a massive topic that deserves its own article. But these four rules will keep you covered:

1. Periodic checks

• Recently right-sized system: check every month until resource consumption feels predictable
• Stable older system: every 6 months is sufficient
• Create calendar reminders or Jira tickets. If it is not scheduled it will not happen.

2. Set up alerts

Consistent high resource consumption should trigger an alert - not a page at 3am, but a notification that gets looked at within a working day. A spike is noise. Hours of sustained 85-90% utilization is a signal.

3. Benchmark after major code changes

After every significant feature release or logic change, run a benchmark. Can your system still handle the peak load you claimed? If performance is off by a small margin - ignore it. If it is off by 20% or more, your previous right-sizing data is no longer valid and you need to start the exercise again.

Code changes can completely invalidate your previous findings. A new feature that adds a database join to every request, a new library with different memory characteristics, a change in payload size - any of these can shift your numbers significantly.

4. Document everything

Every test run. Every configuration tried. Every finding. The next developer who picks this up should not need to spend $200 per test cycle rediscovering what you already know.

A simple document with your test configurations, results, and conclusions is worth more than you think. Especially 18 months later when nobody remembers why the pod count is set to 28.

💼 Manager's lens: Right-sizing is not a one time project. It is a periodic engineering activity with a clear ROI. The alternative - discovering in production that your system can no longer handle peak load after a major release - costs significantly more in incident response, emergency scaling, and customer impact than a scheduled benchmarking exercise every six months.

If You Are a Manager or Project Manager

You do not need to understand the technical details of pod sizing or JVM overhead. But you do need to understand two things: what this exercise costs and what skipping it costs.

What You Are Approving

When your team says right-sizing will take a week, they are not being slow. They are being honest. Each testing iteration takes time to set up, run, observe, and document. You cannot parallelize this by adding more people unless you also add more test environments - and each environment has a cost.

A realistic week of right-sizing for a medium sized system:

• ~$1,500 in cloud infrastructure
• 1 engineer, approximately 5 working days
• 2-3 testing iterations per day maximum

That is the investment. Here is the return.

What Skipping This Costs

The most common alternative to proper right-sizing is over-provisioning - give the system more resources than it needs and hope for the best. It feels safe. It is not free.

Over-provisioning by just 20% on a $10,000/month production cluster costs $24,000 per year. For doing nothing except running pods that are larger than they need to be.

And that assumes nothing goes wrong. An under-provisioned system that hits peak load without enough resources costs significantly more - in incident response time, emergency scaling, engineering hours, and customer impact.

What You Should Ask Your Team

• Is there a documented report from this right-sizing exercise?
• Have we scheduled periodic benchmarking - at minimum every 6 months?
• Do we have alerts set up for sustained high resource consumption?
• After the last major release, did we verify our system can still handle peak load?

If the answer to any of these is no, you have a risk sitting in your infrastructure that has a dollar amount attached to it.

The Bottom Line

The configuration report generated from a proper right-sizing exercise is a long term asset. It prevents the same work from being repeated every time someone questions the cluster setup. It gives your team a defensible answer when stakeholders ask why infrastructure costs what it costs.

A week of right-sizing now versus months of firefighting later. The math is not complicated.

If You Are a CTO or Architect

You already understand the technical landscape. This section is about how to think about right-sizing at a systems level - and how to make the case for it when the benefits are not immediately visible on a dashboard.

The Risk You Are Actually Managing

An application that has never been properly right-sized is an application whose behavior under peak load is unknown. You may have autoscaling configured. But if the scaling rules are based on guesswork - which resource to watch, at what threshold, with what cooldown - you do not have a scaling strategy. You have a hope.

The systematic approach in this article does one thing above everything else: it replaces unknown behavior with known behavior. You know your saturation point. You know your per pod ceiling. You know your scaling range. When peak load hits, you are not watching dashboards hoping the system holds - you have already seen it hold under controlled conditions.

What Right-Sizing Actually Exposes

One of the most valuable outcomes of this exercise is what it reveals about your code - not your infrastructure.

Exponential resource growth under load is not a configuration problem. It is a design problem. N+1 queries, memory leaks, unbounded caches, connection pool exhaustion - these show up clearly when you systematically increase load from 1 to 100 requests. They will never show up in a happy path test.

Finding these problems during a right-sizing exercise costs an afternoon. Finding them in production during peak load costs significantly more.

The Maintenance Argument

Right-sizing is not a one time exercise. Code changes shift performance characteristics. New features add database calls. Libraries change memory behavior. A system that was perfectly sized 6 months ago may be operating at 80% capacity today after three major releases - and nobody noticed because there was no scheduled benchmarking.

Every major release should be followed by a benchmark verification. If throughput has dropped by more than 20% from your baseline, your right-sizing data is no longer valid and the exercise needs to be repeated.

How to Present This to Stakeholders

When you need budget and time approval for a right-sizing exercise, the conversation is straightforward:

• Cost of the exercise: ~$1,500 in infrastructure, ~1 week of engineering time
• Cost of over-provisioning without it: 20% waste on a $10,000/month cluster = $24,000/year
• Cost of an under-provisioning incident: engineering hours, emergency scaling, potential customer SLA breaches
• Payback period: the exercise pays for itself within the first month of optimized production costs

The benefits are not always visible as a moving needle on a dashboard. But they show up in your monthly cloud bill, in your incident count, and in the confidence your team has when peak load season arrives.

A right-sized system is not just cheaper to run. It is more predictable, more maintainable, and easier to defend when something goes wrong.

All AWS prices referenced in this article are approximate us-east-1 on-demand rates. Actual costs vary by region and reserved instance pricing. Numbers are estimates for order-of-magnitude awareness.