### ----------------------------------------------------
### -- MUGS
### -- Licenses: Artistic-2.0
### -- Authors: Geoffrey Broadwell
### -- File: docs/design/concurrency.md
### ----------------------------------------------------
% Concurrency

*Reviewed 2021-03-03 by japhb*


By its very nature, MUGS must be highly concurrent.  A single server may be
handling dozens or even hundreds of games, each with many participants each
in several games at once, and there may be thousands of connected sessions.
Alternately, there may be fewer connected sessions, but with each of them
playing very intense twitch games, for a similar total level of concurrency
but with a very different "shape".

Worse, there is no guarantee that all participants are benign; on the open
Internet, it is highly likely that at some point malicious players will
sign up (or even that malicious or cheating servers appear).  Concurrency
handling is an obvious target for someone wishing to DoS the other
participants.


# Problems

High concurrency immediately presents three major problems:

* It's hard to reason about and code correctly.
* It quickly exposes any latent threading hazards.
* It is very difficult to scale without biasing towards a single use case.

To manage the first two problems, MUGS eschews low-level concurrency primitives
wherever possible in favor of the higher-level tools available in Raku, such as
Supplies and Channels for event delivery and queuing.  Similarly, rather than
hand-coded event loops, MUGS uses the Raku-native concurrency-managing `react`
construction.  When there is a choice for slightly higher efficiency or
likelihood of correct implementation, the latter is nearly always the right
choice.

The last "problem" is actually an entire family of problems, and the desire to
support many concurrent use cases while serving possibly-malicious players
unfolds into a set of design constraints.


# Constraints

MUGS attempts to use execution resources efficiently, without allowing abusive
users to ruin everyone else's experience.  In fact, the design must balance
several key goals:

* Resource hogs only hurt themselves
* Other types of abusers are easy to isolate and contain
* Games are isolated from each other as much as possible
* The service remains smooth and responsive for everyone (on a human scale)
* Overall concurrency scaling is maximized


# Threads and Tasks

While Raku's scheduler is fully pluggable, Rakudo's default is a green-thread
task scheduler which tries to fully utilize a smaller number of OS threads by
sharing them across the offered workload.  Each execution unit is a "task", and
thus it makes more sense to talk about how MUGS apportions work across tasks.

The client side is relatively simple, with only 3 main tasks:

* A task for the session's network listener/dispatcher
* A task for the UI (for UI toolkits that require a dedicated UI thread)
* A task handling everything else

Each server-bound request also creates a Promise that will be kept or broken
when the server responds to that request.  *(NYI: Removing old Promises left
by an unresponsive or misbehaving server)*

The server is more complex:

* A task watching for shutdown signals
* A task watching for new network connections
* A task handling out-of-game requests *(NYI, currently using session tasks)*
* One task for every active game
* One task for every active client connection (server session)

Thus the server has `(3 + num-games + num-sessions)` tasks, multiplexed across
the OS threads provided by Rakudo's scheduler.


# Supplies and Channels

In order for all these cooperating tasks to communicate, especially since each
session may need to talk to many games and vice-versa, MUGS uses thread-safe
Raku constructs for internal messaging.

Cro, used for HTTP and WebSocket traffic, is heavily Supply-based; on both
ends of the connection, the body of the Cro-native messages are converted to
MUGS::Message objects, while still transiting through Supply chains.  (When
using an internal stub server rather than a network server, this is simulated
by just connecting client and server Supplies together; everything else remains
the same.)

On the client side, received server responses are used to fulfill Promises from
the client's async requests; server push messages are simply dispatched as
appropriate.

On the server side, request messages received from a particular session are
checked for the most basic server-protecting validity (e.g. that the message
cleanly parsed, and that the session is fully logged in if the request is for
anything other than to authorize a login or create a new user).  After those
basic checks, in-game requests are sent to the unique Channel that serves as
the input queue for that game, where they undergo full validation and dispatch.

When a game wishes to send a message to a user (as it will at least for every
incoming request), it sends it to the unique outgoing Channel associated with
each user's Session.

*Out-of-game requests are currently handled differently; they are dispatched
from the user's session directly, rather than via an incoming queue as with
games, though they still send responses via the session's outgoing Channel.
This design is still in flux, and may be replaced by dispatching to one or more
out-of-game request Channels in the same style as for in-game requests.*


# Design Assessment

The current design supports several of the goals/constraints:

* Each server session runs in its own task and self-throttles at the maximum
  load a single session thread can support.
* Each game runs in its own task, taking inputs via Channel and serving
  incoming requests on a FIFO basis.
* Server sessions are isolated from load in each game, at least until total
  server load is too high for busy session and game tasks to each get scheduled
  on their own CPU core.

There are still a few problems though:

* Game input can back up and lag the game; this can occur either because:
  * Input request load is very high and a single thread can't process requests
    fast enough to keep up.
  * There is a high fanout from input requests to output messages (for example,
    if each player action requires updating the visible game state for every
    other player in the game).
* A very noisy game may be able to starve (or at least greatly lag) quieter
  games sharing the same session.
* The client is too trusting that the server will respond to all of its
  requests; outstanding request Promises should time out somehow.
* While structurally resistant to certain types of bottleneck/overload, the
  design hasn't been proved hardened, or tested under very high offered load
  (yet).

Open questions:

* How can MUGS keep a very noisy client from lagging the rest of the game?
  * Can the first lag case be handled by allowing games to spawn subtasks?
  * Can the second lag case be handled by having dedicated fanout tasks whose
    only job is to send game updates to many players at once?
* Would scalability be better or worse if there was a separate output queue
  per game for each session?
* Does Rakudo guarantee that the `whenever` blocks sharing a single `react`
  cannot be starved by heavy load from other `whenever` sources?
  * If not, is there a workaround, or does MUGS need to implement its own fair
    queuing?


