txt: ## containers.md
  Containers
  
  A low-level explanation of Raku containers
  
  This section explains how raw data, variables and containers relate to each other in Raku. The different types of containers used in Raku are explained and the actions applicable to them like assigning, binding and flattening. More advanced topics like self-referential data, type constraints and custom containers are discussed at the end.
  
  For a deeper discussion of the various kinds of *ordered* containers in Raku, see the overview of [lists, sequences, and arrays](/language/list); for *unordered* containers, see [sets, bags, and mixes](/language/setbagmix).
  
  # [What is a variable?](#Containers "go to top of document")[§](#What_is_a_variable? "direct link")
  
  Some people like to say "everything is an object", but in fact a variable is not a user-exposed object in Raku.
  
  When the compiler encounters a variable scope declaration like `my $x`, it registers it in some internal symbol table. This internal symbol table is used to detect undeclared variables and to tie the code generation for the variable to the correct scope.
  
  At runtime, a variable appears as an entry in a *lexical pad*, or *lexpad* for short. This is a per-scope data structure that stores a pointer for each variable.
  
  In the case of `my $x`, the lexpad entry for the variable `$x` is a pointer to an object of type [`Scalar`](/type/Scalar), usually just called *the container*.
  
  # [Scalar containers](#Containers "go to top of document")[§](#Scalar_containers "direct link")
  
  Although objects of type [`Scalar`](/type/Scalar) are everywhere in Raku, you rarely see them directly as objects, because most operations *decontainerize*, which means they act on the [`Scalar`](/type/Scalar) container's contents instead of the container itself.
  
  In code like
  
  ```raku
  my $x = 42;
  say $x;
  ```
  
  the assignment `$x = 42` stores a pointer to the [`Int`](/type/Int) object 42 in the scalar container to which the lexpad entry for `$x` points.
  
  The assignment operator asks the container on the left to store the value on its right. What exactly that means is up to the container type. For [`Scalar`](/type/Scalar) it means "replace the previously stored value with the new one".
  
  Note that subroutine signatures allow passing containers around:
  
  ```raku
  sub f($a is rw) {
      $a = 23;
  }
  my $x = 42;
  f($x);
  say $x;         # OUTPUT: «23␤»
  ```
  
  Inside the subroutine, the lexpad entry for `$a` points to the same container that `$x` points to outside the subroutine. Which is why assignment to `$a` also modifies the contents of `$x`.
  
  Likewise, a routine can return a container if it is marked as `is rw`:
  
  ```raku
  my $x = 23;
  sub f() is rw { $x };
  f() = 42;
  say $x;         # OUTPUT: «42␤»
  ```
  
  For explicit returns, `return-rw` instead of `return` must be used.
  
  Returning a container is how `is rw` attribute accessors work. So
  
  ```raku
  class A {
      has $.attr is rw;
  }
  ```
  
  is equivalent to
  
  ```raku
  class A {
      has $!attr;
      method attr() is rw { $!attr }
  }
  ```
  
  Scalar containers are transparent to type checks and most kinds of read-only accesses. A `.VAR` makes them visible:
  
  ```raku
  my $x = 42;
  say $x.^name;       # OUTPUT: «Int␤»
  say $x.VAR.^name;   # OUTPUT: «Scalar␤»
  ```
  
  And `is rw` on a parameter requires the presence of a writable Scalar container:
  
  ```raku
  sub f($x is rw) { say $x };
  f 42;
  CATCH { default { say .^name, ': ', .Str } };
  # OUTPUT: «X::Parameter::RW: Parameter '$x' expected a writable container, but got Int value␤»
  ```
  
  # [Callable containers](#Containers "go to top of document")[§](#Callable_containers "direct link")
  
  Callable containers provide a bridge between the syntax of a [`Routine`](/type/Routine) call and the actual call of the method [CALL-ME](/type/Callable#method_CALL-ME) of the object that is stored in the container. The sigil `&` is required when declaring the container and has to be omitted when executing the [`Callable`](/type/Callable). The default type constraint is [`Callable`](/type/Callable).
  
  ```raku
  my &callable = -> $ν { say "$ν is ", $ν ~~ Int ?? "whole" !! "not whole" }
  callable(⅓);   # OUTPUT: «0.333333 is not whole␤»
  callable(3);   # OUTPUT: «3 is whole␤»
  ```
  
  The sigil has to be provided when referring to the value stored in the container. This in turn allows [`Routine`](/type/Routine)s to be used as [arguments](/language/signatures#Constraining_signatures_of_Callables) to calls.
  
  ```raku
  sub f() {}
  my &g = sub {}
  sub caller(&c1, &c2){ c1, c2 }
  caller(&f, &g);
  ```
  
  # [Binding](#Containers "go to top of document")[§](#Binding "direct link")
  
  Next to assignment, Raku also supports *binding* with the `:=` operator. When binding a value or a container to a variable, the lexpad entry of the variable is modified (and not just the container it points to). If you write
  
  ```raku
  my $x := 42;
  ```
  
  then the lexpad entry for `$x` directly points to the [`Int`](/type/Int) 42. Which means that you cannot assign to it anymore:
  
  ```raku
  my $x := 42;
  $x = 23;
  CATCH { default { say .^name, ': ', .Str } };
  # OUTPUT: «X::AdHoc: Cannot assign to an immutable value␤»
  ```
  
  You can also bind variables to other variables:
  
  ```raku
  my $a = 0;
  my $b = 0;
  $a := $b;
  $b = 42;
  say $a;         # OUTPUT: «42␤»
  ```
  
  Here, after the initial binding, the lexpad entries for `$a` and `$b` both point to the same scalar container, so assigning to one variable also changes the contents of the other.
  
  You've seen this situation before: it is exactly what happened with the signature parameter marked as `is rw`.
  
  Sigilless variables and parameters with the trait `is raw` always bind (whether `=` or `:=` is used):
  
  ```raku
  my $a = 42;
  my \b = $a;
  b++;
  say $a;         # OUTPUT: «43␤»
  
  sub f($c is raw) { $c++ }
  f($a);
  say $a;         # OUTPUT: «44␤»
  ```
  
  # [Scalar containers and listy things](#Containers "go to top of document")[§](#Scalar_containers_and_listy_things "direct link")
  
  There are a number of positional container types with slightly different semantics in Raku. The most basic one is [`List`](/type/List), which is created by the comma operator.
  
  ```raku
  say (1, 2, 3).^name;    # OUTPUT: «List␤»
  ```
  
  A list is immutable, which means you cannot change the number of elements in a list. But if one of the elements happens to be a scalar container, you can still assign to it:
  
  ```raku
  my $x = 42;
  ($x, 1, 2)[0] = 23;
  say $x;                 # OUTPUT: «23␤»
  ($x, 1, 2)[1] = 23;     # Cannot modify an immutable value
  CATCH { default { say .^name, ': ', .Str } };
  # OUTPUT: «X::Assignment::RO: Cannot modify an immutable Int␤»
  ```
  
  So the list doesn't care about whether its elements are values or containers, they just store and retrieve whatever was given to them.
  
  Lists can also be lazy; in that case, elements at the end are generated on demand from an iterator.
  
  An [`Array`](/type/Array) is just like a list, except that it forces all its elements to be containers, which means that you can always assign to elements:
  
  ```raku
  my @a = 1, 2, 3;
  @a[0] = 42;
  say @a;         # OUTPUT: «[42 2 3]␤»
  ```
  
  `@a` actually stores three scalar containers. `@a[0]` returns one of them, and the assignment operator replaces the integer value stored in that container with the new one, `42`.
  
  # [Assigning and binding to array variables](#Containers "go to top of document")[§](#Assigning_and_binding_to_array_variables "direct link")
  
  Assignment to a scalar variable and to an array variable both do the same thing: discard the old value(s), and enter some new value(s).
  
  Nevertheless, it's easy to observe how different they are:
  
  ```raku
  my $x = 42; say $x.^name;   # OUTPUT: «Int␤»
  my @a = 42; say @a.^name;   # OUTPUT: «Array␤»
  ```
  
  This is because the [`Scalar`](/type/Scalar) container type hides itself well, but [`Array`](/type/Array) makes no such effort. Also assignment to an array variable is coercive, so you can assign a non-array value to an array variable.
  
  To place a non-[`Array`](/type/Array) into an array variable, binding works:
  
  ```raku
  my @a := (1, 2, 3);
  say @a.^name;               # OUTPUT: «List␤»
  ```
  
  # [Binding to array elements](#Containers "go to top of document")[§](#Binding_to_array_elements "direct link")
  
  As a curious side note, Raku supports binding to array elements:
  
  ```raku
  my @a = (1, 2, 3);
  @a[0] := my $x;
  $x = 42;
  say @a;                     # OUTPUT: «[42 2 3]␤»
  ```
  
  If you've read and understood the previous explanations, it is now time to wonder how this can possibly work. After all, binding to a variable requires a lexpad entry for that variable, and while there is one for an array, there aren't lexpad entries for each array element, because you cannot expand the lexpad at runtime.
  
  The answer is that binding to array elements is recognized at the syntax level and instead of emitting code for a normal binding operation, a special method (called [`BIND-POS`](/routine/BIND-POS)) is called on the array. This method handles binding to array elements. There is also an equivalent method for binding to hash elements as well (called [`BIND-KEY`](/routine/BIND-KEY))
  
  Note that, while supported, one should generally avoid directly binding uncontainerized things into array elements. Doing so may produce counter-intuitive results when the array is used later.
  
  ```raku
  my @a = (1, 2, 3);
  @a[0] := 42;         # This is not recommended, use assignment instead.
  my $b := 42;
  @a[1] := $b;         # Nor is this.
  @a[2] = $b;          # ...but this is fine.
  @a[1, 2] := 1, 2;    # runtime error: X::Bind::Slice
  CATCH { default { say .^name, ': ', .Str } };
  # OUTPUT: «X::Bind::Slice: Cannot bind to Array slice␤»
  ```
  
  Operations that mix Lists and Arrays generally protect against such a thing happening accidentally.
  
  # [Flattening, items and containers](#Containers "go to top of document")[§](#Flattening,_items_and_containers "direct link")
  
  The `%` and `@` sigils in Raku generally indicate multiple values to an iteration construct, whereas the `$` sigil indicates only one value.
  
  ```raku
  my @a = 1, 2, 3;
  for @a { };         # 3 iterations
  my $a = (1, 2, 3);
  for $a { };         # 1 iteration
  ```
  
  `@`-sigiled variables do not flatten in list context:
  
  ```raku
  my @a = 1, 2, 3;
  my @b = @a, 4, 5;
  say @b.elems;               # OUTPUT: «3␤»
  ```
  
  There are operations that flatten out sublists that are not inside a scalar container: slurpy parameters (`*@a`) and explicit calls to `flat`:
  
  ```raku
  my @a = 1, 2, 3;
  say (flat @a, 4, 5).elems;  # OUTPUT: «5␤»
  
  sub f(*@x) { @x.elems };
  say f @a, 4, 5;             # OUTPUT: «5␤»
  ```
  
  You can also use `|` to create a [`Slip`](/type/Slip), introducing a list into the other.
  
  ```raku
  my @l := 1, 2, (3, 4, (5, 6)), [7, 8, (9, 10)];
  say (|@l, 11, 12);    # OUTPUT: «(1 2 (3 4 (5 6)) [7 8 (9 10)] 11 12)␤»
  say (flat @l, 11, 12) # OUTPUT: «(1 2 3 4 5 6 7 8 (9 10) 11 12)␤»
  ```
  
  In the first case, every element of `@l` is *slipped* as the corresponding elements of the resulting list. `flat`, in the other hand, *flattens* all elements including the elements of the included array, except for `(9 10)`.
  
  As hinted above, scalar containers prevent that flattening:
  
  ```raku
  sub f(*@x) { @x.elems };
  my @a = 1, 2, 3;
  say f $@a, 4, 5;            # OUTPUT: «3␤»
  ```
  
  The `@` character can also be used as a prefix to coerce the argument to a list, thus removing a scalar container:
  
  ```raku
  my $x = (1, 2, 3);
  .say for @$x;               # 3 iterations
  ```
  
  However, the *decont* operator `<>` is more appropriate to decontainerize items that aren't lists:
  
  ```raku
  my $x = ^Inf .grep: *.is-prime;
  say "$_ is prime" for @$x;  # WRONG! List keeps values, thus leaking memory
  say "$_ is prime" for $x<>; # RIGHT. Simply decontainerize the Seq
  
  my $y := ^Inf .grep: *.is-prime; # Even better; no Scalars involved at all
  ```
  
  Methods generally don't care whether their invocant is in a scalar, so
  
  ```raku
  my $x = (1, 2, 3);
  $x.map(*.say);              # 3 iterations
  ```
  
  maps over a list of three elements, not of one.
  
  # [Self-referential data](#Containers "go to top of document")[§](#Self-referential_data "direct link")
  
  Container types, including [`Array`](/type/Array) and [`Hash`](/type/Hash), allow you to create self-referential structures.
  
  ```raku
  my @a;
  @a[0] = @a;
  put @a.raku;
  # OUTPUT: «((my @Array_75093712) = [@Array_75093712,])␤»
  ```
  
  Although Raku does not prevent you from creating and using self-referential data, by doing so you may end up in a loop trying to dump the data. As a last resort, you can use Promises to [handle](/type/Promise#method_in) timeouts.
  
  # [Type constraints](#Containers "go to top of document")[§](#Type_constraints "direct link")
  
  Any container can have a type constraint in the form of a [type object](/language/typesystem#Type_objects) or a [subset](/language/typesystem#subset). Both can be placed between a declarator and the variable name or after the trait [of](/type/Variable#trait_of). The constraint is a property of the variable, not the container.
  
  ```raku
  subset Three-letter of Str where .chars == 3;
  my Three-letter $acronym = "ÞFL";
  ```
  
  In this case, the type constraint is the (compile-type defined) subset `Three-letter`.
  
  The default type constraint of a [`Scalar`](/type/Scalar) container is [`Mu`](/type/Mu). Introspection of type constraints on containers is provided by `.VAR.of` method, which for `@` and `%` sigiled variables gives the constraint for values:
  
  ```raku
  my Str $x;
  say $x.VAR.of;  # OUTPUT: «(Str)␤»
  my Num @a;
  say @a.VAR.of;  # OUTPUT: «(Num)␤»
  my Int %h;
  say %h.VAR.of;  # OUTPUT: «(Int)␤»
  ```
  
  ## [Definedness constraints](#Containers "go to top of document")[§](#Definedness_constraints "direct link")
  
  A container can also enforce a variable to be defined. Put a smiley in the declaration:
  
  ```raku
  my Int:D $def = 3;
  say $def;   # OUTPUT: «3␤»
  $def = Int; # Typecheck failure
  ```
  
  You'll also need to initialize the variable in the declaration, it can't be left undefined after all.
  
  It's also possible to have this constraint enforced in all variables declared in a scope with the [default defined variables pragma](/language/variables#Default_defined_variables_pragma). People coming from other languages where variables are always defined will want to have a look.
  
  # [Custom containers](#Containers "go to top of document")[§](#Custom_containers "direct link")
  
  To provide custom containers Raku employs the class [`Proxy`](/type/Proxy). Its constructor takes two arguments, `FETCH` AND `STORE`, that point to methods that are called when values are fetched or stored from the container. Type checks are not done by the container itself and other restrictions like readonlyness can be broken. The returned value must therefore be of the same type as the type of the variable it is bound to. We can use [type captures](/language/signatures#Type_captures) to work with types in Raku.
  
  ```raku
  sub lucky(::T $type) {
      my T $c-value; # closure variable
      return-rw Proxy.new(
          FETCH => method () { $c-value },
          STORE => method (T $new-value) {
              X::OutOfRange.new(what => 'number', got => '13', range => '-∞..12, 14..∞').throw
                  if $new-value == 13;
              $c-value = $new-value;
          }
      );
  }
  
  my Int $a := lucky(Int);
  say $a = 12;    # OUTPUT: «12␤»
  say $a = 'FOO'; # X::TypeCheck::Binding
  say $a = 13;    # X::OutOfRange
  CATCH { default { say .^name, ': ', .Str } };
  ```
  