Between slide decks promising "infinite scale" and the quiet panic of a 3 AM PagerDuty alert, there is a question worth answering: what does a Java system handling one million transactions per second actually look like in production?

No single Java framework gets you there on its own. Spring Boot did not build Twitter. Quarkus did not keep Grab alive during peak monsoon hours. The layers above and below your JVM process did. And whether Java is even the right language for your hot path is a question worth asking before you have 50 pods and a runbook.

The layered architecture

Every system at this scale is built in rings. Each ring absorbs what it can and passes only what it must to the next layer down.

"The framework is your engine. The architecture around it determines whether you survive the load, and whether your on-call rotation survives the night."

A lesson learned in production, not in tutorials

The thread model question

Traditional Spring Boot with Tomcat gives you one OS thread per request. At 1M TPS with average latency of 60ms, that means roughly 60,000 concurrent threads. Memory exhaustion, context-switching overhead, and a performance cliff you hit hard and without much warning.

Option A: Reactive with WebFlux

Spring WebFlux on Netty uses a small fixed pool of event-loop threads, typically 2 x CPU cores, and handles concurrency through non-blocking I/O. The catch is the mental overhead — reactive code is hard to write, harder to debug, and harder to hire for.

@GetMapping("/product/{id}")
public Mono<Product> getProduct(@PathVariable Long id) {
    return cacheService.get(id)
        .switchIfEmpty(productRepo.findById(id)
            .doOnNext(p -> cacheService.set(id, p)));
    // non-blocking all the way down
}

Option B: Virtual threads on JDK 21

Project Loom lets you write normal synchronous code and get non-blocking behaviour underneath. The JVM multiplexes millions of lightweight virtual threads onto a small pool of OS carrier threads. For most teams today this is the right call. Simpler code, comparable throughput, no reactive pyramid of doom to maintain.

# Spring Boot 3.2+ - one line to unlock it
spring.threads.virtual.enabled=true

# Tune HikariCP separately - virtual threads won't save a saturated pool
spring.datasource.hikari.maximum-pool-size=50
spring.datasource.hikari.minimum-idle=10

What actually breaks first

Connection pool exhaustion. HikariCP defaults are sized for modest load. A misconfigured pool becomes a synchronised chokepoint fast. Tune maximumPoolSize and watch pool metrics before go-live, not after.

GC stop-the-world pauses. A 200ms pause at 16K req/sec means thousands of requests timing out at once. Use G1GC or ZGC. Java 21's generational ZGC is currently the best choice for latency-sensitive workloads.

Slow queries without indexes. An un-indexed column that is fine in dev becomes a full table scan in production. Run EXPLAIN ANALYZE before you ship, not after your first incident.

Debug logs left on in prod. DEBUG logging at 16K req/sec generates gigabytes per minute. Your logging infrastructure falls over before your service does. Async appenders, structured JSON, log sampling. Non-negotiable at this traffic level.

What else can handle this scale, beyond Java

Java is excellent at this scale. It is also not the only answer. A few other languages are worth knowing about, and some of them beat Java in specific situations.

Which one to actually pick

If your team knows Java well, use Java with virtual threads. The ecosystem, tooling, and hiring pool are unmatched. If you are building a proxy, a CLI tool, or anything infrastructure-adjacent, Go is the pragmatic call. If you have a p99 latency SLA and a team with patience, Rust is the right answer. If you need millions of concurrent long-lived connections for chat, gaming, or real-time data, look at Elixir before anything else.

At genuine 1M TPS scale, the language at the hot path matters less than most engineers like to debate. The CDN absorption rate, the cache hit ratio, the Kafka consumer group lag, the HikariCP pool size — those are the numbers that decide whether you page or sleep. Pick a language your team is productive in, tune the layers above and below, and put your architectural energy where the load actually lives.

"Architecture scales. Clever framework configuration does not. And neither does arguing about languages at 2 AM."

Earned opinion

One last thing on observability

At this scale you cannot debug by reading logs. By the time you find the relevant entry, the incident is over and your P0 bridge is still open. You need Prometheus metrics, distributed traces via OpenTelemetry, and dashboards that exist before the problem arrives. Five numbers to watch: p99 latency per endpoint, error rate, HikariCP active connections, Redis hit ratio, GC pause duration. If those are green, everything else is usually fine. If one spikes, you know where to look.