Recently, there’s been a growing architectural trend among large tech companies: removing external message brokers or Redis from their tech stacks in favor of using their existing relational databases for simple job queues.

But why drop a blazing-fast in-memory datastore like Redis for database tables? The answer lies in transactional integrity, infrastructure complexity, and lock contention.

To understand the real-world impact of this shift, I built a Spring Boot 4.1 application (powered by Java 21) to benchmark three different queueing strategies. We simulated 10 concurrent workers competing to process 100 pending jobs. The results perfectly illustrate the hidden traps of distributed locking.

The Problem: The Concurrency Race Condition

Imagine you have a table of jobs with a status column (PENDING, COMPLETED). You have multiple worker nodes constantly querying the database for the next available PENDING job.

If 10 workers execute SELECT * FROM jobs WHERE status = 'PENDING' LIMIT 1 at the exact same millisecond, they will all fetch the exact same job (e.g., Job ID 1). If they all process it, you end up with duplicated work, wasted CPU cycles, and data inconsistency.

Here is how different architectures attempt to solve this.

Strategy 1: The Naive Redis Lock (The Trap)

The classic approach is to introduce Redis as a distributed lock manager.

  1. The worker fetches the next PENDING job from the database.
  2. It attempts to acquire a lock in Redis using SETNX (Set if Not eXists) for that specific Job ID.
  3. If acquired, it processes the job, updates the database to COMPLETED, and deletes the Redis lock.

The Benchmark Result

Type: REDIS | TotalJob: 100 | Worker: 10 | Time: ~390ms | Success: 104 ❌ | Skipped: 1722

Wait, we only had 100 jobs, but the success count is 104! What happened?

The Gotcha: This is a classic distributed systems race condition caused by Spring's @Transactional boundary. The worker deletes the Redis lock in a finally block before the Spring proxy has committed the database transaction.

During that tiny millisecond window, the lock is gone, but the database still shows the job as PENDING. Another worker swoops in, grabs the new lock, and processes the job a second time.

Strategy 2: The Double-Checked Redis Lock

To fix the naive approach, we must implement Double-Checked Locking.
After acquiring the Redis lock, the worker must query the database again to verify the job is still PENDING before processing it. If another worker already completed it while we were waiting, we abort.

The Benchmark Result

Type: REDIS_DOUBLE_CHECKED | TotalJob: 100 | Worker: 10 | Time: ~590ms | Success: 100 ✅ | Skipped: ~2600

The Gotcha: We fixed the bug—exactly 100 jobs were processed. But look at the performance penalty.
Because workers fetch a job without a lock initially, all 10 workers grab Job 1. One gets the Redis lock, and the other 9 fail. Those 9 workers immediately fetch Job 1 again.

This creates massive lock contention. To process just 100 jobs, our application generated over 2,500 wasted network trips between the app, Redis, and MySQL.

Strategy 3: MySQL FOR UPDATE SKIP LOCKED (The Elegant Solution)

What if the database could just handle the queueing natively? Enter SKIP LOCKED.

Instead of a standard SELECT, the worker executes: 

SELECT * FROM jobs WHERE status = 'PENDING' ORDER BY id ASC LIMIT 1 FOR UPDATE SKIP LOCKED;

When Worker A runs this query, MySQL instantly places a row-level lock on Job 1.
When Worker B runs the exact same query a millisecond later, MySQL sees that Job 1 is locked, skips it, and immediately returns and locks Job 2.

The Benchmark Result

Type: SKIP_LOCKED | TotalJob: 100 | Worker: 10 | Time: ~214ms | Success: 100 ✅ | Skipped: 10

The Results: Breathtakingly efficient. Processing time was cut by more than half compared to the safest Redis approach. Lock contention practically vanished (only 10 skipped attempts).

The MySQL Gap Lock Deadlock (Pro-Tip)

If you implement this in MySQL (InnoDB), you might initially encounter a Deadlock found exception. This happens because MySQL's default isolation level is REPEATABLE READ, which uses Gap Locks (locking the spaces between index records).

To make SKIP LOCKED work perfectly in MySQL, you must lower the transaction isolation level for the worker method to READ_COMMITTED: 

@Transactional(isolation = Isolation.READ_COMMITTED)
public boolean processNextJob() { ... }

This instructs MySQL to only lock the specific record (Record Lock), completely eliminating the deadlocks.

Conclusion

Redis is an incredible tool, but using it to manage locks for database records often leads to the "Two Database Problem"—syncing state between memory and disk introduces network latency, edge cases, and transaction boundary headaches.

For low-to-medium scale job queues, leveraging MySQL or PostgreSQL's native SKIP LOCKED functionality keeps your architecture simple, your transaction boundaries perfectly intact, and your performance incredibly high.

You can find the full source code, Docker setup, and benchmark suite for this project on my GitHub.
👉 https://github.com/ercansormaz/skip-locked-vs-redis