You’re late for a meeting. You jab the elevator button and wait. An elevator going down passes your floor. Then another one. Finally, a car arrives, already half full, stopping at three more floors on the way up. The whole system feels busy, but it’s not getting you where you need to go. This isn’t just a building design problem. It’s a perfect metaphor for a trap we fall into constantly when building software.
As a recent interactive post on Hacker News perfectly illustrated, the most intuitive way to schedule elevators is often the least efficient. The simple algorithm, sending the nearest available car, creates chaos. A smarter, grouped system feels slower for a moment but improves the whole building's flow. This exact tradeoff between local optimization and global throughput is probably happening in your stack right now, and it’s costing you more than just a few seconds of wait time.
The "Nearest Car" Anti-Pattern in Software
The simple elevator algorithm is a greedy algorithm. It makes the locally optimal choice at each step, hoping to find a global optimum. Send the closest car. It feels right. It's easy to implement. But it leads to a fleet of elevators crisscrossing a building inefficiently, wasting energy and time. We build software this way all the time.
Naive Load Balancing
Think about the most basic form of load balancing: round-robin. Request A goes to server 1, B to server 2, C to server 1 again. It’s simple and fair, like sending the 'next' elevator car. But what if server 1 is already struggling under the weight of a heavy, long-running task from a previous request? A pure round-robin balancer doesn’t care. It will happily send another request to the overloaded instance, leading to timeouts and a degraded user experience. This is the 'nearest car' anti-pattern. The balancer makes a simple choice without any context about the state of the system.
A smarter approach, like a least-connections algorithm, is a step in the right direction. It sends the new request to the server with the fewest active connections. This is the beginning of a 'group control' system. It requires the load balancer to have more state, to know what the entire pool of servers is doing, but the payoff is a much more resilient system.
Greedy Job Queues
Background job processors are another prime example. Let’s say you're running an e-commerce platform and use a single Redis queue for all background tasks, processed by a pool of workers. A user signs up, triggering a welcome email. Then another user uploads a batch of 50 high-resolution product photos that each need resizing, watermarking, and CDN distribution. All 51 jobs go into the same queue.
If you're using a simple FIFO (First-In, First-Out) worker, that welcome email is now stuck behind potentially minutes of heavy image processing. The new user thinks your service is broken because they never got their confirmation email. You made a locally optimal choice (add the job to the queue, let the first available worker pick it up), but the global result is a terrible user experience.
This is where the 'group control' logic from the elevator example comes in. A better system would use multiple queues, maybe high_priority_emails and bulk_image_processing. Or it would use a single queue with a priority field that workers respect. The password reset email gets handled in seconds, even if it means one of the 50 image jobs has to wait a little longer. You've optimized for system-wide user experience, not just for keeping workers busy.
The Team Tasking Fallacy
This isn't just a technical problem. It happens with engineering teams. A critical bug comes in. Who takes it? The manager might assign it to the first developer who signals they have bandwidth. That’s the 'nearest car' approach.
But what if another developer, who is currently wrapping up a different task, wrote the original feature and has all the context? They might be able to fix the bug in 30 minutes, while the first available developer might take four hours just to understand the code. Waiting an hour for the expert to become free is the 'group control' solution. It feels slower at first, but it's a massive win for the team's overall throughput.
Implementing "Group Control" in Your Architecture
Recognizing the anti-pattern is the first step. The next is architecting a system that has a wider view. This means building components that are aware of the state of the whole system, not just their own little world. It’s a move from isolated, greedy components to a coordinated, strategic system.
Centralized vs. Decentralized Schedulers
The elevator article's 'group control' system is a centralized scheduler. A single 'brain' sees all the button presses and all the elevator locations and makes a globally optimal decision. In software, Kubernetes' kube-scheduler is a perfect example. It looks at pod requirements, node resources, affinity rules, and taints to make a very smart decision about where to place a new workload. It doesn't just put it on the first node with free RAM.
The tradeoff is that a centralized scheduler can become a single point of failure or a bottleneck itself. If your scheduler goes down, no new work gets assigned. This leads some teams to decentralized approaches, where services use a gossip protocol or a distributed hash table to share state and make local decisions based on a partial, but still useful, view of the world. There's no single right answer, but you must make a conscious choice. Sticking with the default, 'nearest car' behavior is a choice, and it's usually the wrong one at scale.
A Practical Example with a Message Queue
Let's go back to the job queue problem. Instead of a single, dumb queue, you can use the more advanced features of a tool like RabbitMQ to create a 'group control' system. You can use a topic exchange, which allows you to route messages based on patterns in a routing key.
Imagine you publish all jobs to one exchange. A high-priority job might have the routing key jobs.transactional.high. A bulk image processing job might be jobs.batch.low.
You can then have different sets of workers listening for different patterns:
- A dedicated pool of workers binds a queue to listen for
jobs.transactional.#. They only ever see and process high-priority transactional tasks like emails and notifications. - A larger, scalable pool of workers on cheaper spot instances binds a queue to listen for
jobs.batch.#. They chew through the heavy, non-time-sensitive work.
Here’s what the publisher-side pseudocode might look like:
// Don't just dump everything into one queue
// Use metadata to allow for intelligent routing
function dispatchPasswordReset(userId) {
const job = { userId: userId, template: 'reset_email' };
// The routing key provides the context for the 'group controller' (the exchange)
messageBroker.publish('jobs_exchange', 'jobs.transactional.high', job);
}
function dispatchImageProcessing(imageBatch) {
const job = { batchId: imageBatch.id };
messageBroker.publish('jobs_exchange', 'jobs.batch.low', job);
}
This architecture is the direct equivalent of having express elevators for certain floors and local elevators for others. It requires more setup than a single queue, but it prevents a catastrophic failure where a user can't reset their password because someone else is uploading photos.
The Real Cost: Latency vs. Throughput
This brings us to the core tradeoff: optimizing for latency versus optimizing for throughput. The 'nearest car' approach is an attempt to minimize the latency for one specific request.
Building something in this space? AgileStack helps teams ship enterprise-grade software without the consulting-firm overhead. Book a 30-minute call and tell us what you're working on.