Multi-queue processing allows you to split CIR410 – Cycle-end Processing
across multiple queues, with each queue dedicated to a different range
of customer numbers. For very large databases, this can help speed the
total wall-time processing required for cycle-end. Multi-queue
processing only works if you are also running DBR.
You can increase the throughput for this process by dividing the input
file into separate queues (a maximum of nine queues is allowed), and
running the CIR4MQ – Multi-queue Cycle-end process instead. CIR4MQ is
identical to CIR410, except that it is designed to split processing for
the publication across multiple queues.
Setting up the Queues
This is controlled at the publication level. To enable
multi-queue streaming for a pub, go to the Cycle End Queues tab when
inquiring on a pub at CIRPUB.
Use the Add button to specify a queue and starting customer number. You
can use the Customize button to add other available columns to the view.
The columns available in the view are:
- Queue: queue number for the customer range in question; selected by
the user from the available defined queues)
- Start customer: starting customer number for the queue;
user-assigned. If the lowest defined customer number (across all queues)
is not zero, the system resets it to zero to ensure that no customers
are missed.
- Issue date: date of the issue for which the cycle-end is being
performed. Populated by the system as the queue completes. Used (with
last cycle-end number) to recognize that a queue has completed, and the
order of queue completion.
- Last cycle end number: system-assigned number for each queue as it
finishes processing. Advantage uses this, plus the issue date,
to determine the order of queue completion. The last queue to complete
also performs process-wide wrap-up tasks.
- Request number: system-assigned number that uniquely identifies an
Advantage process request. Used for internal purposes and
troubleshooting.
- Update date: last date a queue definition was added or changed at
CIRPUB.
- Update user: user ID of the person to last perform a queue
definition add or update.
Running CIR4MQ
Queues are defined by publication (not issue date), so that the
multi-queue information present at CIRPUB is the information that CIR4MQ
uses when it initiates the cycle-end(s).
If a queue fails, all of its updates are rolled back to the beginning, as
if that queue had never started. If the queue determines that it is the last
to finish (based on the other CEQ records) then it completes the global
wrap-up. If an error occurred after that point, the CEQ update is also
rolled back, along with any global updates. In an error situation either
before or after the wrap-up section, all that is necessary is to correct the
problem and restart that queue.
When CIR4MQ is initiated, it checks the queue information for the
publication involved. The process then initiates a separate cycle-end
process for each queue (and corresponding starting customer number). These
processes can run concurrently.
As the queues finish, the Issue Date field is updated to reflect
successful completion of that queue. Also, the Cycle End Last Number is
incremented on a queue-by-queue basis. These two fields---Issue Date and
Cycle End Last Number---allow the process to track the order in which the
queues are completing, and also which of the queues is the very last to
complete. The last queue-based process to finish conducts final reporting
for the issue.
Multi-queuing may be used with any of the CIR4MQ processing modes:
report, trial, update, mail, etc.
CIRSMQ (Multi-queue Setup Processing)
An alternative to using the Cycle End Queue tab at CIRPUB is to run
the CIRSMQ process. The process includes a run-time setting for the number
of cycle-end queues. The process then evenly divides the customer base
across the queues---in effect providing an automated shortcut to the ADD
function on the Cycle End Queue tab.
