-
Notifications
You must be signed in to change notification settings - Fork 291
Processing Groups
Pablo Cantero edited this page Jun 4, 2020
·
20 revisions
Processing Groups is a way to set specific configurations, such as queues, concurrency, long polling per queue(s).
Given this configuration:
concurrency: 25
queues:
- queue1
- queue2
- queue3Shoryuken will fetch messages from queue1, then queue2, then queue3, then repeat. See Polling strategies for more information.
If you don't want to wait on fetching for queue1 to fetch for queue2, you can move them into separate groups:
groups:
group1:
concurrency: 10
queues:
- queue1
group2:
concurrency: 10
delay: 10
queues:
- queue2Is that all? No, it is not! If you want to process only one message at time for queue1, but keeping concurrency: 25 for queue2 and queue3:
concurrency: 25
queues:
- queue2
- queue3
groups:
group1:
concurrency: 1
queues:
- queue1Note: If you want to make sure to process only one message at time for a given queue while running multiple Shoryuken processes, have a look at FIFO Queues.
groups:
group1:
polling_strategy: StrictPriority
queues:
- queue1
group2:
polling_strategy: WeightedRoundRobin # Default, declaration optional
queues:
- queue2See Polling strategies.
# config/initializers/shoryuken.rb
Shoryuken.sqs_client_receive_message_opts[:group1] = { wait_time_seconds: 20 }See Long Polling.