Skip to main content

🎯 Lesson Learned: RabbitMQ Default Timeouts

 πŸ‘‹ Hello Team!

Today I want to share an important lesson we discovered in production 


πŸ‘‰ Default timeouts matter! Especially when you’re working with RabbitMQ and long‑running consumers.


πŸ•’ What’s the default timeout in RabbitMQ?

✅ By default, RabbitMQ expects a message acknowledgment from the consumer within 30 minutes.


✅ If no ACK is received in that window, RabbitMQ assumes:

“This consumer might have failed!”
…and re‑queues the message to another consumer.


Why did this become a problem for us?

Our consumers trigger shell scripts that sometimes run for 2–3 hours.
πŸ› ️ In development, our test scripts were small (20–25 mins), so we never hit the 30‑minute limit.


πŸš€ But in production:

  • After 30 minutes, RabbitMQ re‑delivered the same message.

  • That led to multiple consumers processing the same job.

  • A single 1‑hour job started running 5 to 10 times in parallel! 😱


πŸ”Ž Root Cause Analysis (RCA)

✔️ RabbitMQ was working as designed—but our process simply exceeded its default timeout.


The Fix

We updated the message acknowledgment timeout from 30 minutes → 24 hours.


✅ This aligns with our actual processing time.
✅ Now, each job runs exactly once—no more duplicate executions. πŸŽ‰


Key Takeaway

πŸ’‘ Always review and configure default timeouts when integrating systems—especially if you have long‑running consumers or processes.


A small setting can make a huge difference in production stability. πŸš€


 

Comments

Popular posts from this blog

🐱 Tomcat vs ⚡ Netty – Which One Should You Use?

🐱 Tomcat vs ⚡ Netty – Which One Should You Use? So recently I got curious about this too πŸ€”. Everywhere in Spring Boot tutorials we see Tomcat . Then suddenly while exploring Spring WebFlux , the name Netty pops up. And I was like – “Wait, who’s this Netty guy trying to replace Tomcat?” πŸ˜… Let’s break it down with real-time examples , icons , and fun comparisons . 🐱 Tomcat – The Traditional Web Server Type: Servlet Container (blocking I/O) World: Used with Spring MVC Style: Thread-per-request model πŸ‘©‍πŸ’» Pros: Stable, widely used, battle-tested Cons: Struggles with huge concurrent connections πŸ‘‰ Example in real life: Tomcat is like a restaurant with fixed waiters 🍴. - Each customer = one thread/waiter - If too many customers come in at once → waiters run out → customers wait outside πŸšͺ ⚡ Netty – The Reactive Rockstar Type: Asynchronous Event-Driven Network Framework World: Default for Spring WebFlux Style: Event-lo...

🎭 Spring’s Secret: Why @Transactional & Friends Betray You Silently

πŸ’‘ Lesson Learned — Not a Prod Bug, But a Real Pain No, this wasn’t a production outage. Nobody screamed at me. But I sat for 3 hours wondering: “Why the heck is my @Transactional not rolling back!?” 😡‍πŸ’« “Why is Redis cache not working?” 🀯 Turned out, the issue was one silent villain: 🧱 Self-invocation 🀷 What Is @Transactional ? If you're new: @Transactional = Tells Spring to start a DB transaction when a method is called. It’ll commit if everything’s okay. It’ll rollback if something fails. 🧠 Think of it like wrapping your code in: try { beginTransaction(); // your logic commit(); } catch(Exception e) { rollback(); } πŸ•΅️ Real-Life Analogy — The Gateway Community 🏘️ Let me tell you about my society — it has a strict watchman at the gate. Here’s how it works: πŸ›‚ Watchman = Spring Proxy 🏠 Your apartment = Your service class πŸšͺ Your room = A method inside that class πŸƒ Scenario 1: Outsider Visits Your friend from outside...

🌟 My Journey – From Zero to Senior Java Tech Lead 🌟

 There’s one thing I truly believe… If I can become a Java developer, then anyone in the world can. πŸ’― Sounds crazy? Let me take you back. πŸ•“ Back in 2015… I had zero coding knowledge . Not just that — I had no interest in coding either. But life has its own plans. In 2016, I got a chance to move to Bangalore and joined a Java course at a training center. That’s where it all started — Every day, every session made me feel like: "Ohhh! Even I can be a developer!" That course didn’t just teach Java — it gave me confidence . πŸ§ͺ Two Life-Changing Incidents 1️⃣ The Interview That Wasn't Planned Halfway through my course, I had to urgently travel to Chennai to donate blood to a family member. After that emotional rollercoaster, I found myself reflecting on my skills and the future. The next day, as I was preparing for my move to Bangalore to complete the remaining four months of my course, I randomly thought — "Let me test my skills... let me just see...