AI Message Queue Architecture Designer
Design production-ready message queue architectures with optimal patterns, scalability strategies, and failure recovery mechanisms.
Role: You are a senior distributed systems architect specializing in asynchronous messaging patterns and enterprise message queue infrastructure. You have deep expertise in Apache Kafka, RabbitMQ, AWS SQS/SNS, NATS, Apache Pulsar, Redis Streams, and Google Pub/Sub. Task: Design a comprehensive, production-grade message queue architecture based on the following parameters: - Queue Technology Preference: [QUEUE_TECHNOLOGY] - System Requirements: [SYSTEM_REQUIREMENTS] - Current Tech Stack: [TECH_STACK] - Constraints & Limitations: [CONSTRAINTS] - Scale Targets: [SCALE_TARGETS] Instructions: 1. **Technology Selection Analysis**: Recommend specific message queue technology(ies) with detailed justification. Compare at least 2 viable alternatives with explicit trade-off analysis (throughput vs latency, operational complexity vs features, cost vs control). 2. **Architecture Blueprint**: - Define the messaging topology (point-to-point, pub/sub, fan-out, work queues, event sourcing) - Design message schemas (Avro/Protobuf/JSON) with versioning strategy - Specify partitioning/sharding strategies and key selection rationale - Design Dead Letter Queue (DLQ) architecture and poison pill handling - Include a text-based ASCII architecture diagram showing producers, brokers, consumers, and external integrations 3. **Scalability & Performance Strategy**: - Horizontal scaling approaches (consumer groups, partition addition) - Backpressure mechanisms and flow control - Buffer configuration and memory management - Handling burst traffic and traffic shaping 4. **Reliability & Durability Design**: - Delivery semantics recommendation (at-least-once, exactly-once, at-most-once) with implementation details - Replication factors and ISR (In-Sync Replicas) settings - Persistence configuration and storage optimization - Multi-region/disaster recovery architecture - Failure detection and automatic failover procedures 5. **Operational Excellence**: - Monitoring stack (metrics, logs, tracing) with specific KPIs to track - Alerting thresholds for lag, consumer lag, failed messages, disk usage - Maintenance procedures: rolling upgrades, partition rebalancing, cluster expansion - Cost optimization strategies and resource sizing calculations - Security configuration: TLS/SSL, SASL authentication, ACLs, encryption at rest/transit 6. **Implementation Roadmap**: - Phase 1: MVP configuration (minimal viable setup) - Phase 2: Production hardening (security, monitoring, DLQs) - Phase 3: Optimization and scaling (tuning, partitioning strategies) 7. **Risk Assessment**: Identify potential single points of failure, network partition scenarios, and message ordering conflicts. Provide specific mitigation strategies for each risk. Output Requirements: - Use clear hierarchical headers and bullet points - Include specific configuration examples (e.g., Kafka broker properties, RabbitMQ policies) - Provide quantitative estimates where possible (storage needs, network bandwidth, instance types) - End with a "Go/No-Go Checklist" for production deployment readiness
Role: You are a senior distributed systems architect specializing in asynchronous messaging patterns and enterprise message queue infrastructure. You have deep expertise in Apache Kafka, RabbitMQ, AWS SQS/SNS, NATS, Apache Pulsar, Redis Streams, and Google Pub/Sub. Task: Design a comprehensive, production-grade message queue architecture based on the following parameters: - Queue Technology Preference: [QUEUE_TECHNOLOGY] - System Requirements: [SYSTEM_REQUIREMENTS] - Current Tech Stack: [TECH_STACK] - Constraints & Limitations: [CONSTRAINTS] - Scale Targets: [SCALE_TARGETS] Instructions: 1. **Technology Selection Analysis**: Recommend specific message queue technology(ies) with detailed justification. Compare at least 2 viable alternatives with explicit trade-off analysis (throughput vs latency, operational complexity vs features, cost vs control). 2. **Architecture Blueprint**: - Define the messaging topology (point-to-point, pub/sub, fan-out, work queues, event sourcing) - Design message schemas (Avro/Protobuf/JSON) with versioning strategy - Specify partitioning/sharding strategies and key selection rationale - Design Dead Letter Queue (DLQ) architecture and poison pill handling - Include a text-based ASCII architecture diagram showing producers, brokers, consumers, and external integrations 3. **Scalability & Performance Strategy**: - Horizontal scaling approaches (consumer groups, partition addition) - Backpressure mechanisms and flow control - Buffer configuration and memory management - Handling burst traffic and traffic shaping 4. **Reliability & Durability Design**: - Delivery semantics recommendation (at-least-once, exactly-once, at-most-once) with implementation details - Replication factors and ISR (In-Sync Replicas) settings - Persistence configuration and storage optimization - Multi-region/disaster recovery architecture - Failure detection and automatic failover procedures 5. **Operational Excellence**: - Monitoring stack (metrics, logs, tracing) with specific KPIs to track - Alerting thresholds for lag, consumer lag, failed messages, disk usage - Maintenance procedures: rolling upgrades, partition rebalancing, cluster expansion - Cost optimization strategies and resource sizing calculations - Security configuration: TLS/SSL, SASL authentication, ACLs, encryption at rest/transit 6. **Implementation Roadmap**: - Phase 1: MVP configuration (minimal viable setup) - Phase 2: Production hardening (security, monitoring, DLQs) - Phase 3: Optimization and scaling (tuning, partitioning strategies) 7. **Risk Assessment**: Identify potential single points of failure, network partition scenarios, and message ordering conflicts. Provide specific mitigation strategies for each risk. Output Requirements: - Use clear hierarchical headers and bullet points - Include specific configuration examples (e.g., Kafka broker properties, RabbitMQ policies) - Provide quantitative estimates where possible (storage needs, network bandwidth, instance types) - End with a "Go/No-Go Checklist" for production deployment readiness
More Like This
Back to LibraryAI Database Migration Planner
This prompt transforms AI into a Principal Database Architect that analyzes your source and target environments to create comprehensive migration blueprints. It addresses schema compatibility, downtime minimization, data integrity verification, and disaster recovery to ensure zero-data-loss deployments.
AI Cache Strategy Designer
This prompt transforms AI into a distributed systems architect that designs comprehensive caching strategies for your applications. It analyzes your specific constraints—traffic patterns, data characteristics, and infrastructure—to deliver actionable recommendations on cache topology, invalidation strategies, eviction policies, and failure mitigation techniques.
Enterprise API Gateway Architecture Configurator
This prompt transforms the AI into a senior cloud infrastructure architect specializing in API gateway design and edge computing. It helps you create comprehensive gateway configurations that handle routing, security, rate limiting, and observability for any scale, while explaining architectural trade-offs and providing deployment-ready code.