### ----------------------------------------------------
### -- ParaSeq
### -- Licenses: Artistic-2.0
### -- Authors: Elizabeth Mattijsen
### -- File: README.md
### ----------------------------------------------------
### -- Chunk 2 of 2
ParaQueue`.

The `ParaQueue` that is pushed to the result queue by the batcher ensures that the order in which batches of work are handed out, will also be the order in which the results will be delivered. The fact that the result of processing is pushed to this `ParaQueue` itself, makes the deliverer block until the worker thread has pushed its result (even if other threads may have been ready earlier).

Step 8: set up the waiting queue
--------------------------------

To communicate back performance statistics from a worker thread finishing their work, another `ParaQueue` is created. Its sole purpose is to collect all `ParaStats` objects that each worker thread has completed **as soon as they are done**. It basically also represents the number of result batches that are waiting to be delivered: hence the name "waiting queue".

This queue is inspected by the delivery thread whenever it has completed delivery of a batch. It's possible that this queue is empty. It can also contain the statistics of more than one worker thread.

Step 9: start the batcher thread
--------------------------------

The batcher thread keeps running until the source iterator is exhausted. When it is exhausted, it pushes an `IterationEnd` to the result queue, indicating that no more batches will be coming. It's the only time a non-`ParaQueue` object is pushed to the result queue.

The batcher is basically a loop that blocks on the messages from the deliverer. Whenever a message is received (with the proposed batch size, calculated from any available performance statistics), it adjusts for correct granularity (as the deliverer is unaware of any granularity restrictions). The batcher then fetches that many values from the source iterator and schedules another job for a worker thread with the values obtained (if any).

Step 10: start delivering
-------------------------

At this point a `ParaIterator` object is created and installed as the (result) `.iterator` of the invocant. The paralellizing logic in the hypered variant of the method is started, which then in turn starts feeding the `ParaIterator`.

It is then up to the code that iterators over the (result) iterator to consume the values delivered. If no values are being delivered, production of values will halt after all inital "degree" batches have been produced.

When the deliverer is done with a result batch, it removes all available statistics information and adds these to the `stats` of the `ParaSeq` object. It also uses this information to calculate an optimum batch size for any other batches to be processed (unless this is inhibited by the `:!auto` named argument to `.&hyperize`).

The algorithm for batch size calculation is pretty simple at the moment. To allow for optimum responsiveness of the program, as well as any other processes running on the computer, a time slice of 1_000_000 nanoseconds (aka 1 millisecond) has been chosen.

If a batch took almost exactly that amount of time to produce values, then the size of that batch is assumed to be the optimum size. If the batch took longer, then the batch size should be reduced. If the batch took less than that to complete, then the batch size should be increased.

Decision tree
-------------

      source
       \- hyperize
           |- invocant (degree == 1)
           |- Seq      (source exhausted with initial batch)
           |- ParaSeq
               \- method call
                   |- endpoint
                   |- coercer
                   |- hypering

Hypering control flow
---------------------

        /---------------------------------------------------------\
        |                |                                        |
        |               ⬇                                         |
        |         --------------                 ------------     |
        |         | statistics |         /-----⮕ | producer | ⮕ -|
        |         --------------        /        ------------     |
        ⬇               ⬇             /                          |
    -----------    ------------    -----------   ------------     |
    | results | ⮕ | deliverer | ⮕ | batcher | ⮕ | producer | ⮕ -|
    -----------    ------------    -----------   ------------     |
                        |              \                          |
                        |               \        ------------     |
                        |                \-----⮕ | producer | ⮕ -/
                        ⬇                        ------------
                     values
              consumed by program

AUTHOR
======

Elizabeth Mattijsen <liz@raku.rocks>

Source can be located at: https://github.com/lizmat/ParaSeq . Comments and Pull Requests are welcome.

If you like this module, or what I’m doing more generally, committing to a [small sponsorship](https://github.com/sponsors/lizmat/) would mean a great deal to me!

COPYRIGHT AND LICENSE
=====================

Copyright 2024 Elizabeth Mattijsen

This library is free software; you can redistribute it and/or modify it under the Artistic License 2.0.



