text:
"""JAX transformations on Modules.

Jax functional transformations operate on pure functions.
Flax extends these transformations to also operate on Module's which
have stateful variables and PRNG sequences. We refer to these extended
versions as "lifted transformations".

A lifted transformation can be applied to a ``Module`` class or a
function that takes a ``Module`` instance as its first argument.
"""
import dataclasses
import functools
import inspect
from typing import (Any, Callable, Dict, Iterable, Mapping, Optional, Sequence,
                    Tuple, Type, TypeVar, Union)

from flax import errors
from flax import struct
from flax import traceback_util
from flax.core import lift
from flax.core import Scope
from flax.core.frozen_dict import FrozenDict
from flax.linen import module as linen_module
from flax.linen.module import Module
from flax.linen.module import Variable
from flax.linen.module import wrap_method_once
from flax.linen.module import _get_unbound_fn
from flax.linen.module import _derive_profiling_name
import jax

traceback_util.register_exclusion(__file__)

# pylint: disable=protected-access

# Utils
# -----------------------------------------------------------------------------
def clean_clone(x):
  """Remove scopes and tracers from children."""
  if isinstance(x, Module):
    object.__setattr__(
        x, 'children',
        {k: clean_clone(v) for k, v in x.children.items()})
    object.__setattr__(x, 'scope', None)
  return x

@struct.dataclass
class VariablePlaceholder:
  """Used to mark Variables in a JAX-compatible way when lifting arguments."""
  collection: str = struct.field(pytree_node=False)
  name: str = struct.field(pytree_node=False)
  id: int = struct.field(pytree_node=False)

@struct.dataclass
class InstancePlaceholder:
  """Marks module instances in a JAX-compatible way when lifting arguments."""
  cls: Type[Any] = struct.field(pytree_node=False)
  attrs: Dict[Any, Any] = struct.field(pytree_node=False)
  id: int = struct.field(pytree_node=False)

def _memoize_by_id(fn, refs):
  """Memoization by module/variable id to handle aliasing in traversal."""
  @functools.wraps(fn)
  def wrapped_fn(x):
    nonlocal refs
    if isinstance(x, (VariablePlaceholder, InstancePlaceholder)):
      x_id = x.id
    elif isinstance(x, (Variable, Module)):
      x_id = x._id
    else:
      return fn(x)
    if x_id not in refs:
      refs[x_id] = fn(x)
    return refs[x_id]
  return wrapped_fn

def get_module_scopes(module, args=None, kwargs=None):
  """Get all scopes on module, including constructor Module arguments.

  To properly functionalize a Module that has other bound Modules passed in
  "from the outside" as dataclass attributes, we need to traverse all dataclass
  fields to find the Scopes associated with the Module.  Additionally, because
  we allow Modules to be passed inside pytrees on the dataclass attributes, we
  must traverse all dataclass attributes as pytrees to find all Modules.  We
  additionally handle lifting Variables (which are just references to data in
  particular scopes) and Module instances that are passed as arguments to
  methods.

  Args:
    module: a bound flax Module.
    args: an *args list possibly containing Variables or Module instances
      referencing a scope.
    kwargs: a **kwargs dict possibly containing Variables or Module instances
      referencing a scope.

  Returns:
    A list of all functional-core Scopes bound on self and inside dataclass
    fields as well as any Scopes passed via argument Variables, an updated args
    list, and an updated kwargs dict that have both had Variables replaced with
    VariablePlaceholders and Module instances replaced with InstancePlaceholders
    that are compatible with jax functions.
  """
  scopes = []
  refs = {}
  # Gather scopes associated with Variables and Module instances passed as
  # positional and keyword arguments.
  @functools.partial(_memoize_by_id, refs=refs)
  def get_arg_scope(x):
    nonlocal scopes
    if isinstance(x, Variable) and isinstance(x.scope, Scope):
      scopes.append(x.scope)
      return VariablePlaceholder(x.collection, x.name, x._id)
    elif isinstance(x, Module) and isinstance(x.scope, Scope):
      x._try_setup(shallow=True)
      scopes.append(x.scope)
      attrs = {
          f.name: getattr(x, f.name)
          for f in dataclasses.fields(x)
          if f.name != 'parent' and f.init
      }
      attrs = jax.tree_util.tree_map(get_arg_scope, attrs)
      return InstancePlaceholder(x.__class__, attrs, x._id)
    return x
  new_args, new_kwargs = jax.tree_util.tree_map(get_arg_scope, (args, kwargs))

  # Gather scopes in Variables and Submodules passed as Module attributes.
  @functools.partial(_memoize_by_id, refs=refs)
  def get_scopes(module):
    nonlocal scopes
    module._try_setup(shallow=True)
    def get_scopes_inner(x):
      nonlocal scopes
      if isinstance(x, Module) and isinstance(x.scope, Scope):
        get_scopes(x)
      elif isinstance(x, Variable) and isinstance(x.scope, Scope):
        scopes.append(x.scope)

    attrs = {
        f.name: getattr(module, f.name)
        for f in dataclasses.fields(module)
        if f.name != 'parent' and f.init
    }
    jax.tree_util.tree_map(get_scopes_inner, attrs)
    scopes.append(module.scope)
  get_scopes(module)
  return scopes, new_args, new_kwargs

def set_module_scopes(module, args, kwargs, scopes):
  """Set all scopes on module, including those on Modules in dataclass fields.

  To properly functionalize a Module we must also "rehydrate" it with Scopes
  from `get_module_scopes`.  We need to set scopes not just on the Module but
  also on any Module living inside dataclass attributes or even pytrees in its
  dataclass attributes.  We additionally handle restoring Variables and Module
  instances from their placeholders in the method positional and keyword
  arguments.  The order of traversal through this method is the same as in
  `get_module_scopes`, guaranteeing the correct Scopes are applied to each
  Module.

  Args:
    module: a flax Module.
    args: an *args list possibly containing VariablePlaceholder or
      InstancePlaceholder members.
    kwargs: a **kwargs dict possibly containing VariablePlaceholder or
      InstancePlaceholder members.
    scopes: a list of Scopes corresponding to this Module and its arguments that
      was created by the `get_module_scopes` function.

  Returns:
    A copy of the module with it and its attributes bound to the scopes passed
    to this function, an updated args list, and an updated kwargs dict with
    updated Variable and Module instance references.
  """
  idx = 0
  refs = {}
  # Set scopes associated with Variables and Module instances passed as
  # positional and keyword arguments.
  @functools.partial(_memoize_by_id, refs=refs)
  def set_arg_scope(x):
    nonlocal idx
    if isinstance(x, VariablePlaceholder):
      new_x = Variable(scope=scopes[idx], collection=x.collection, name=x.name)
      idx += 1
      return new_x
    elif isinstance(x, InstancePlaceholder):
      instance_scope = scopes[idx]
      idx += 1
      instance_attrs = jax.tree_util.tree_map(set_arg_scope, x.attrs)
      return x.cls(parent=instance_scope, **instance_attrs)
    return x

  def is_placeholder(x):
    return isinstance(x, (VariablePlaceholder, InstancePlaceholder))

  new_args, new_kwargs = jax.tree_util.tree_map(
      set_arg_scope, (args, kwargs), is_leaf=is_placeholder)

  # set scopes in Variables and Submodules passed as Module attributes
  @functools.partial(_memoize_by_id, refs=refs)
  def set_scopes(module):
    nonlocal idx
    def set_scopes_inner(x):
      nonlocal idx
      if isinstance(x, Module) and isinstance(x.scope, Scope):
        return set_scopes(x)
      elif isinstance(x, Variable) and isinstance(x.scope, Scope):
        new_x = Variable(scope=scopes[idx],
                         collection=x.collection,
                         name=x.name)
        idx += 1
        return new_x
      else:
        return x

    attrs = {
        f.name: getattr(module, f.name)
        for f in dataclasses.fields(module)
        if f.name != 'parent' and f.init
    }
    new_attrs = jax.tree_util.tree_map(set_scopes_inner, attrs)
    new_module = module.clone(parent=scopes[idx], **new_attrs)
    idx += 1
    return new_module
  new_module = set_scopes(module)
  assert len(scopes) == idx, f'scope list mismatch {len(scopes)} != {idx}'
  return new_module, new_args, new_kwargs

def _test_transformed_return_values(tree, method_name):
  """Tests whether the return value contains any Modules or Variables."""
  impure = any(map(lambda x: isinstance(x, (Module, Variable)),
                   jax.tree_util.tree_leaves(tree)))
  if impure:
    raise errors.TransformedMethodReturnValueError(method_name)

# Class lifting
# -----------------------------------------------------------------------------
def module_class_lift_transform(
    transform,
    module_class,
    *trafo_args,
    methods=None,
    **trafo_kwargs):
  """Module class lift transform."""
  # TODO(marcvanzee): Improve docstrings (#1977).
  # TODO(levskaya): find nicer argument convention for multi-method case?

  # Prepare per-method transform args, kwargs.
  if methods is None:
    # Default case, just transform __call__
    class_trafo_args = {'__call__': (trafo_args, trafo_kwargs)}
  elif isinstance(methods, (list, tuple)):
    # Transform every method in methods with given args, kwargs.
    class_trafo_args = {m: (trafo_args, trafo_kwargs) for m in methods}
  elif isinstance(methods, dict):
    # Pass different trafo args per each method.
    class_trafo_args = {k: ((), v) for k, v in methods.items()}
  else:
    raise ValueError(
        'transform methods argument must be None, tuple, list, or dict.')

  # Handle partially initialized module class constructors.
  if (isinstance(module_class, functools.partial) and
      issubclass(module_class.func, Module)):
    partial_object = module_class
    module_class = module_class.func
  else:
    partial_object = None

  def create_trans_fn(fn_name, fn_trafo_args):
    # get existing unbound method from class
    fn = getattr(module_class, fn_name)
    trafo_args, trafo_kwargs = fn_trafo_args
    # we need to create a scope-function from our class for the given method
    @functools.wraps(fn)
    def wrapped_fn(self, *args, **kwargs):
      state = self._state.export()
      # make a scope-function to transform
      def core_fn(scopes, *args, **kwargs):
        # make a clone of self using its arguments
        attrs = {
            f.name: getattr(self, f.name)
            for f in dataclasses.fields(self)
            if f.name != 'parent' and f.init
        }
        # we reference module_class, not self.__class__ to avoid infinite loop
        cloned = module_class(parent=None, **attrs)
        cloned, args, kwargs = set_module_scopes(cloned, args, kwargs, scopes)
        object.__setattr__(cloned, '_state', state.export())
        res = fn(cloned, *args, **kwargs)
        self._state.reimport(cloned._state)
        _test_transformed_return_values(res, fn_name)
        return res
      # here we apply the given lifting transform to the scope-ingesting fn
      trafo_fn = transform(core_fn, *trafo_args, **trafo_kwargs)
      module_scopes, args, kwargs = get_module_scopes(self, args, kwargs)
      ret = trafo_fn(module_scopes, *args, **kwargs)
      return ret
    return wrapped_fn
  transformed_fns = {fn_name: create_trans_fn(fn_name, fn_trafo_args)
                     for fn_name, fn_trafo_args in class_trafo_args.items()}
  # construct new dynamic class w. transformed methods
  transformed_cls = type(
      transform.__name__.capitalize() + module_class.__name__,
      (module_class,),
      transformed_fns)
  # Handle partially initialized module class constructors.
  if partial_object is not None:
    transformed_cls = functools.partial(transformed_cls,
                                        *partial_object.args,
                                        **partial_object.keywords)
  return transformed_cls

# Function lifting as decorator on methods __inside__ class definition.
# -----------------------------------------------------------------------------
def decorator_lift_transform(transform, class_fn, *trafo_args,
                             multi_scope=True, **trafo_kwargs):
  """Decorator for lifted transform."""
  # TODO(marcvanzee): Improve docstrings (#1977).
  # Due to the ordering of method decorators, we must wrap the class_fn
  # with the module state management wrapper first to maintain Module state
  # correctly.
  if isinstance(class_fn, tuple):
    class_fns = class_fn
  else:
    class_fns = (class_fn,)
  prewrapped_fns = [wrap_method_once(class_fn) for class_fn in class_fns]
  @functools.wraps(prewrapped_fns[0])
  def wrapped_fn(self, *args, **kwargs):
    state = self._state.export()
    # make a scope-function to transform
    def core_fn(prewrapped_fn, class_fn, scopes, *args, **kwargs):
      if not multi_scope:
        scopes = [scopes]
      cloned, args, kwargs = set_module_scopes(self, args, kwargs, scopes)
      object.__setattr__(cloned, '_state', state.export())
      res = prewrapped_fn(cloned, *args, **kwargs)
      self._state.reimport(cloned._state)
      _test_transformed_return_values(res, getattr(class_fn, '__name__', None))
      return res
    core_fns = [functools.partial(core_fn, prewrapped_fn, class_fn)
                for prewrapped_fn, class_fn in zip(prewrapped_fns, class_fns)]
    # here we apply the given lifting transform to the scope-ingesting fn
    trafo_fn = transform(*core_fns, *trafo_args, **trafo_kwargs)
    module_scopes, args, kwargs = get_module_scopes(self, args, kwargs)
    if not multi_scope:
      if len(module_scopes) != 1:
        # TODO(levskaya): transforms like jvp & vjp have args that follow the
        # pytree structure of scopes. The user doesn't explicitly control shared
        # modules passed as arguments to methods or as attributes to Module
        # constructors. Therefore, there is no obvious API for specifying
        # arguments per lifted Module.
        raise NotImplementedError(
            'This transform does not yet support'
            ' Modules that include other Modules passed as arguments.')
      module_scopes = module_scopes[0]
    return trafo_fn(module_scopes, *args, **kwargs)
  return wrapped_fn

# Utility to wrap a class or to use as decorator in def of class method.
# -----------------------------------------------------------------------------

TransformTarget = Union[Type[Module], Callable[..., Any]]
Target = TypeVar('Target', bound=TransformTarget)

def _is_module_class(target: TransformTarget) -> bool:
  return (inspect.isclass(target) and issubclass(target, Module) or
          (isinstance(target, functools.partial)) and
          _is_module_class(target.func))

def lift_transform(transform,
                   target,
                   *trafo_args,
                   methods=None,
                   **trafo_kwargs):
  """Applies to class or as a decorator on class fns."""
  # TODO(marcvanzee): Improve docstrings (#1977).
  if _is_module_class(target):
    return module_class_lift_transform(
        transform, target, *trafo_args, methods=methods, **trafo_kwargs)
  # we presume this is being used as a function decorator in class definition
  elif callable(target) and not isinstance(target, Module):
    return decorator_lift_transform(
        transform, target, *trafo_args, **trafo_kwargs)
  else:
    raise errors.TransformTargetError(target)

def lift_direct_transform(transform: Callable[..., Any],
                          targets: Tuple[Callable[..., Any], ...],
                          mdl: Module,
                          *args, multi_scope=True, **kwargs):
  """Lift direct transform."""
  # TODO(marcvanzee): Improve docstrings (#1977).
  for target in targets:
    if _is_module_class(target):
      raise ValueError(
          f'The {transform.__name__} transform can only be applied on a Module method.'
          ' That is function that takes a Module instance as its first arg.')
    elif not callable(target):
      raise ValueError('transform target must be callable')
  # normalize self.foo bound methods to class.foo unbound methods.
  targets = tuple(_get_unbound_fn(target) for target in targets)
  aug_transform = lambda *fns: functools.partial(transform, *fns)
  return decorator_lift_transform(
      aug_transform, targets, multi_scope=multi_scope)(mdl, *args, **kwargs)

def vmap(target: Target,
         variable_axes: Mapping[lift.CollectionFilter,
                                lift.InOutAxis] = FrozenDict(),
         split_rngs: Mapping[lift.PRNGSequenceFilter, bool] = FrozenDict(),
         in_axes=0,
         out_axes=0,
         axis_size: Optional[int] = None,
         axis_name: Optional[str] = None,
         spmd_axis_name: Optional[str] = None,
         methods=None) -> Target:
  """A lifted version of ``jax.vmap``.

  See ``jax.vmap`` for the unlifted batch transform in Jax.

  ``vmap`` can be used to add a batch axis to a ``Module``.
  For example we could create a version of ``Dense`` with
  a batch axis that does not share parameters::

    BatchDense = nn.vmap(
        nn.Dense,
        in_axes=0, out_axes=0,
        variable_axes={'params': 0},
        split_rngs={'params': True})

  By using ``variable_axes={'params': 0}``, we indicate that the
  parameters themselves are mapped over and therefore not shared along
  the mapped axis. Consequently, we also split the 'params' RNG,
  otherwise the parameters would be initialized identically along
  the mapped axis.

  Similarly, ``vmap`` could be use to add a batch axis with parameter
  sharing::

    BatchFoo = nn.vmap(
        Foo,
        in_axes=0, out_axes=0,
        variable_axes={'params': None},
        split_rngs={'params': False})

  Here we use ``variable_axes={'params': None}`` to indicate the parameter
  variables are shared along the mapped axis. Consequently, the 'params'
  RNG must also be shared.

  Args:
    target: a ``Module`` or a function taking a ``Module``
      as its first argument.
    variable_axes: the variable collections that are lifted into the
      batching transformation. Use `None` to indicate a broadcasted
      collection or an integer to map over an axis.
    split_rngs: Split PRNG sequences will be different for each index
      of the batch dimension. Unsplit PRNGs will be broadcasted.
    in_axes: Specifies the mapping of the input arguments (see `jax.vmap`).
    out_axes: Specifies the mapping of the return value (see `jax.vmap`).
    axis_size: Specifies the size of the batch axis. This only needs
      to be specified if it cannot be derived from the input arguments.
    axis_name: Specifies a name for the batch axis. Can be used together
      with parallel reduction primitives (e.g. `jax.lax.pmean`,
      `jax.lax.ppermute`, etc.)
    methods: If `target` is a `Module`, the methods of `Module` to vmap over.
    spmd_axis_name: Axis name added to any pjit sharding constraints appearing
      in `fn`. See also
      https://github.com/google/flax/blob/main/flax/linen/partitioning.py.

  Returns:
    A batched/vectorized version of ``target``, with the same arguments but with
    extra axes at positions indicated by ``in_axes``, and the same return value,
    but with extra axes at positions indicated by ``out_axes``.
  """
  return lift_transform(
      lift.vmap,
      target,
      variable_axes,
      split_rngs,
      methods=methods,
      in_axes=in_axes,
      out_axes=out_axes,
      axis_size=axis_size,
      axis_name=axis_name,
      spmd_axis_name=spmd_axis_name)

def jit(target: Target,
        variables: lift.CollectionFilter = True,
        rngs: lift.PRNGSequenceFilter = True,
        static_argnums: Union[int, Iterable[int]] = (),
        donate_argnums: Union[int, Iterable[int]] = (),
        device=None,
        backend: Union[str, None] = None,
        methods=None) -> Target:
  """Lifted version of ``jax.jit``.

  Args:
    target: a ``Module`` or a function taking a ``Module``
      as its first argument.
    variables: The variable collections that are lifted. By default all
      collections are lifted.
    rngs: The PRNG sequences that are lifted. By default all PRNG sequences
      are lifted.
    static_argnums: An int or collection of ints specifying which positional
      arguments to treat as static (compile-time constant). Operations that only
      depend on static arguments will be constant-folded in Python (during
      tracing), and so the corresponding argument values can be any Python
      object. Static arguments should be hashable, meaning both ``__hash__`` and
      ``__eq__`` are implemented, and immutable. Calling the jitted function
      with different values for these constants will trigger recompilation. If
      the jitted function is called with fewer positional arguments than
      indicated by ``static_argnums`` then an error is raised. Arguments that
      are not arrays or containers thereof must be marked as static.
      Defaults to ().
    donate_argnums: Specify which arguments are "donated" to the computation.
      It is safe to donate arguments if you no longer need them once the
      computation has finished. In some cases XLA can make use of donated
      buffers to reduce the amount of memory needed to perform a computation,
      for example recycling one of your input buffers to store a result. You
      should not reuse buffers that you donate to a computation, JAX will raise
      an error if you try to.
    device: This is an experimental feature and the API is likely to change.
      Optional, the Device the jitted function will run on. (Available devices
      can be retrieved via :py:func:`jax.devices`.) The default is inherited
      from XLA's DeviceAssignment logic and is usually to use
      ``jax.devices()[0]``.
    backend: a string representing the XLA backend: ``'cpu'``, ``'gpu'``, or
      ``'tpu'``.
    methods: If `target` is a `Module`, the methods of `Module` to jit.

  Returns:
    A wrapped version of target, set up for just-in-time compilation.
  """
  return lift_transform(
      lift.jit, target,
      variables=variables, rngs=rngs,
      static_argnums=static_argnums,
      donate_argnums=donate_argnums,
      device=device,
      backend=backend,
      methods=methods)

def checkpoint(target: Target,
               variables: lift.CollectionFilter = True,
               rngs: lift.PRNGSequenceFilter = True,
               concrete: bool = False,
               prevent_cse: bool = True,
               static_argnums: Union[int, Tuple[int, ...]] = (),
               policy: Optional[Callable[..., bool]] = None,
               methods=None) -> Target:
  """Lifted version of ``jax.checkpoint``.

  This function is aliased to ``lift.remat`` just like ``jax.remat``.

  Args:
    target: a ``Module`` or a function taking a ``Module``
      as its first argument. intermediate computations will be
      re-computed when computing gradients for the target.
    variables: The variable collections that are lifted. By default all
      collections are lifted.
    rngs: The PRNG sequences that are lifted. By default all PRNG sequences
      are lifted.
    concrete: Optional, boolean indicating whether ``fun`` may involve
      value-dependent Python control flow (default False). Support for such
      control flow is optional, and disabled by default, because in some
      edge-case compositions with :func:`jax.jit` it can lead to some extra
      computation.
    prevent_cse: Optional, boolean indicating whether to prevent common
      subexpression elimination (CSE) optimizations in the HLO generated from
      differentiation. This CSE prevention has costs because it can foil other
      optimizations, and because it can incur high overheads on some backends,
      especially GPU. The default is True because otherwise, under a ``jit`` or
      ``pmap``, CSE can defeat the purpose of this decorator. But in some
      settings, like when used inside a ``scan``, this CSE prevention mechanism
      is unnecessary, in which case ``prevent_cse`` should be set to False.
    static_argnums: Optional, int or sequence of ints, indicates which argument
      values on which to specialize for tracing and caching purposes. Specifying
      arguments as static can avoid ConcretizationTypeErrors when tracing, but
      at the cost of more retracing overheads.
    policy: Experimental checkpoint policy, see ``jax.checkpoint``.
    methods: If `target` is a `Module`, the methods of `Module` to checkpoint.

  Returns:
    A wrapped version of ``target``. When computing gradients intermediate
    computations will be re-computed on the backward pass.
  """
  # subtract 1 from each static_argnums because 'self' is not passed to the
  # lifted function
  static_argnums = jax.tree_util.tree_map(lambda x: x - 1, static_argnums)
  return lift_transform(
      lift.checkpoint, target,
      variables=variables, rngs=rngs, concrete=concrete,
      static_argnums=static_argnums,
      prevent_cse=prevent_cse, policy=policy,
      methods=methods)

remat = checkpoint

def remat_scan(
    target: Target,
    lengths: Optional[Sequence[int]] = (),
    policy: Optional[Callable[..., bool]] = None,
    variable_broadcast: lift.CollectionFilter = False,
    variable_carry: lift.CollectionFilter = False,
    variable_axes: Mapping[lift.CollectionFilter,
                           lift.InOutScanAxis] = FrozenDict({True: 0}),
    split_rngs: Mapping[lift.PRNGSequenceFilter, bool] = FrozenDict({True: True})
) -> Target:
  """Combines remat and scan for memory efficiency and constant time compilation.

  ``remat_scan`` allows for constant compile times and sublinear
  memory usage with respect to model depth. At a small constant
  penalty. This is typically beneficial for very deep models.

  Example::

    class BigModel(nn.Module):
      @nn.compact
      def __call__(self, x):
        DenseStack = nn.remat_scan(nn.Dense, lengths=(10, 10))
        # 100x dense with O(sqrt(N)) memory for gradient computation
        return DenseStack(8, name="dense_stack")(x)

  Args:
    target: a ``Module`` or a function taking a ``Module`` as its first
      argument.
    lengths: number of loop iterations at the given level. The total number of
      iterations `n = prod(lengths)`. each loop is rematerialized. This way the
      memory consumption is proportional to `n^(1 / d)` where `d =
      len(lengths)`. Minimal memory consumptions requires tuning the lengths
      such that the same amount of memory is consumed at each level of the
      nested loop.
    policy: Experimental checkpoint policy, see ``jax.checkpoint``.
    variable_broadcast: Specifies the broadcasted variable collections. A
      broadcasted variable should not depend on any computation that cannot be
      lifted out of the loop. This is typically used to define shared parameters
      inside the fn.
    variable_carry: Specifies the variable collections that are carried through
      the loop. Mutations to these variables are carried to the next iteration
      and will be preserved when the scan finishes.
    variable_axes: the variable collections that are scanned over. Defaults to
      ``{True: 0}``.
    split_rngs: Split PRNG sequences will be different for each loop iterations.
      If split is False the PRNGs will be the same across iterations. Defaults
      to ``{True: True}``.

  Returns:
    A wrapped version of ``target`` that repeats itself prod(lengths) times.
  """
  return lift_transform(
      lift.remat_scan, target,
      lengths=lengths,
      variable_broadcast=variable_broadcast,
      variable_carry=variable_carry,
      variable_axes=variable_axes,
      split_rngs=split_rngs,
      policy=policy,
  )

def scan(target: Target,
         variable_axes: Mapping[lift.CollectionFilter,
                                lift.InOutScanAxis] = FrozenDict(),
         variable_broadcast: lift.CollectionFilter = False,
         variable_carry: lift.CollectionFilter = False,
         split_rngs: Mapping[lift.PRNGSequenceFilter, bool] = FrozenDict(),
         in_axes=0, out_axes=0,
         length: Optional[int] = None,
         reverse: bool = False,
         unroll: int = 1,
         data_transform: Optional[Callable[..., Any]] = None,
         methods=None) -> Target:
  """A lifted version of ``jax.lax.scan``.

  See ``jax.lax.scan`` for the unlifted scan in Jax.

  To improve consistency with ``vmap``, this version of scan
  uses ``in_axes`` and ``out_axes`` to determine which arguments
  are scanned over and along which axis.

  ``scan`` distinguishes between 3 different types of values inside the loop:

  #. **scan**: a value that is iterated over in a loop. All scan values must
     have the same size in the axis they are scanned over. Scanned outputs
     will be stacked along the scan axis.

  #. **carry**: A carried value is updated at each loop iteration. It must
     have the same shape and dtype throughout the loop.

  #. **broadcast**: a value that is closed over by the loop. When a variable
     is broadcasted they are typically initialized inside the loop body but
     independent of the loop variables.

  The loop body should have the signature
  ``(scope, body, carry, *xs) -> (carry, ys)``, where ``xs`` and ``ys``
  are the scan values that go in and out of the loop.

  Example::

    import flax
    import flax.linen as nn
    from jax import random

    class SimpleScan(nn.Module):
      @nn.compact
      def __call__(self, c, xs):
        LSTM = nn.scan(nn.LSTMCell,
                       variable_broadcast="params",
                       split_rngs={"params": False},
                       in_axes=1,
                       out_axes=1)
        return LSTM()(c, xs)

    seq_len, batch_size, in_feat, out_feat = 20, 16, 3, 5
    key_1, key_2, key_3 = random.split(random.PRNGKey(0), 3)

    xs = random.uniform(key_1, (batch_size, seq_len, in_feat))
    init_carry = nn.LSTMCell.initialize_carry(key_2, (batch_size,), out_feat)

    model = SimpleScan()
    variables = model.init(key_3, init_carry, xs)
    out_carry, out_val = model.apply(variables, init_carry, xs)

    assert out_val.shape == (batch_size, seq_len, out_feat)

  Note that when providing a function to ``nn.scan``, the scanning happens over
  all arguments starting from the third argument, as specified by ``in_axes``.
  So in the following example, the input that are being scanned over are ``xs``,
  ``*args*``, and ``**kwargs``::

    def body_fn(cls, carry, xs, *args, **kwargs):
      extended_states = cls.some_fn(xs, carry, *args, **kwargs)
      return extended_states

    scan_fn = nn.scan(
        body_fn,
        in_axes=0,  # scan over axis 0 from third arg of body_fn onwards.
        variable_axes=SCAN_VARIABLE_AXES,
        split_rngs=SCAN_SPLIT_RNGS)

  Args:
    target: a ``Module`` or a function taking a ``Module``
      as its first argument.
    variable_axes: the variable collections that are scanned over.
    variable_broadcast: Specifies the broadcasted variable collections. A
      broadcasted variable should not depend on any computation that cannot be
      lifted out of the loop. This is typically used to define shared parameters
      inside the fn.
    variable_carry: Specifies the variable collections that are carried through
      the loop. Mutations to these variables are carried to the next iteration
      and will be preserved when the scan finishes.
    split_rngs: Split PRNG sequences will be different for each loop iterations.
      If split is False the PRNGs will be the same across iterations.
    in_axes: Specifies the axis to scan over for the arguments. Should be a
      prefix tree of the arguments. Use `flax.core.broadcast` to feed an entire
      input to each iteration of the scan body.
    out_axes: Specifies the axis to scan over for the return value. Should be a
      prefix tree of the return value.
    length: Specifies the number of loop iterations. This only needs to be
      specified if it cannot be derivied from the scan arguments.
    reverse: If true, scan from end to start in reverse order.
    unroll: how many scan iterations to unroll within a single
      iteration of a loop (default: 1).
    data_transform: optional function to transform raw functional-core variable
      and rng groups inside lifted scan body_fn, intended for inline SPMD
      annotations.
    methods: If `target` is a `Module`, the methods of `Module` to scan over.

  Returns:
    The scan function with the signature ``(scope, carry, *xxs) -> (carry,
    yys)``, where ``xxs`` and ``yys`` are the scan values that go in and out of
    the loop.
  """
  return lift_transform(
      lift.scan, target,
      variable_axes=variable_axes,
      variable_broadcast=variable_broadcast,
      variable_carry=variable_carry,
      split_rngs=split_rngs,
      in_axes=in_axes, out_axes=out_axes,
      length=length,
      reverse=reverse,
      unroll=unroll,
      data_transform=data_transform,
      methods=methods)

def map_variables(
    target: Target,
    mapped_collections: lift.CollectionFilter = True,
    trans_in_fn: Callable[..., Any] = lift.id_fn,
    trans_out_fn: Callable[..., Any] = lift.id_fn,
    init: bool = False, mutable: bool = False,
    rngs: lift.PRNGSequenceFilter = True,
    variables: lift.CollectionFilter = True,
    methods=None) -> Target:
  """Map Variables inside a module.

  Example::

    class OneBitDense(nn.Module):
      @nn.compact
      def __call__(self, x):
        def sign(x):
          return jax.tree_util.tree_map(jnp.sign, x)
        MapDense = nn.map_variables(nn.Dense, "params", sign, init=True)
        return MapDense(4)(x)

  Args:
    target: the function to be transformed.
    mapped_collections: the collection(s) to be transformed.
    trans_in_fn: creates a view of the target variables.
    trans_out_fn: transforms the updated variables in the view after mutation.
    init: If True, variables are initialized before transformation.
    mutable: If True, the mapped variable collections will be mutable.
    rngs: PRNGSequences added to the transformed scope (default: all).
    variables: Additional Variable collections added to the transformed scope.
      Besides those specified by `target` (default: all).
    methods: If `target` is a `Module`, the methods of `Module` to map variables
      for.
  Returns:
    a wrapped version of ``target`` that will map the specificied collections.
  """

  return lift_transform(
      lift.map_variables, target,
      mapped_collections,
      trans_in_fn, trans_out_fn,
      init, mutable,
      rngs, variables,
      methods=methods,
  )

def vjp(
    fn: Callable[..., Any],
    mdl: Module,
    *primals,
    has_aux: bool = False,
    reduce_axes=(),
    vjp_variables: lift.CollectionFilter = 'params',
    variables: lift.CollectionFilter = True,
    rngs: lift.PRNGSequenceFilter = True,
    ) -> Tuple[Any, Any]:
  """A lifted version of ``jax.vjp``.

  See ``jax.vjp`` for the unlifted vector-Jacobiam product (backward gradient).

  Note that a gradient is returned for all variables in the collections
  specified by `vjp_variables`. However, the backward funtion only expects
  a cotangent for the return value of `fn`. If variables require a co-tangent
  as well they can be returned from `fn` using `Module.variables`.

  Example::

    class LearnScale(nn.Module):
      @nn.compact
      def __call__(self, x, y):
        p = self.param('scale', nn.initializers.zeros, ())
        return p * x * y

    class Foo(nn.Module):
      @nn.compact
      def __call__(self, x, y):
        z, bwd = nn.vjp(lambda mdl, x, y: mdl(x, y), LearnScale(), x, y)
        params_grad, x_grad, y_grad = bwd(jnp.ones(z.shape))
        return z, params_grad, x_grad, y_grad

  Args:
    fn: Function to be differentiated. Its arguments should be arrays, scalars,
      or standard Python containers of arrays or scalars. It should return an
      array, scalar, or standard Python container of arrays or scalars. It will
      receive the scope and primals as arguments.
    mdl: The module of which the variables will be differentiated.
    *primals: A sequence of primal values at which the Jacobian of ``fn``
      should be evaluated. The length of ``primals`` should be equal to the
      number of positional parameters to ``fn``. Each primal value should be a
      tuple of arrays, scalar, or standard Python containers thereof.
    has_aux: Optional, bool. Indicates whether ``fn`` returns a pair where the
     first element is considered the output of the mathematical function to be
     differentiated and the second element is auxiliary data. Default False.
    reduce_axes: Optional, tuple of axis names. If an axis is listed here, and
      ``fn`` implicitly broadcasts a value over that axis, the backward pass
      will perform a ``psum`` of the corresponding gradient. Otherwise, the
      VJP will be per-example over named axes. For example, if ``'batch'``
      is a named batch axis, ``vjp(f, *args, reduce_axes=('batch',))`` will
      create a VJP function that sums over the batch while ``vjp(f, *args)``
      will create a per-example VJP.
    vjp_variables: The vjpfun will return a cotangent vector for all
      variable collections specified by this filter.
    variables: other variables collections that are available inside `fn` but
      do not receive a cotangent.
    rngs: the prngs that are available inside `fn`.

  Returns:
    If ``has_aux`` is ``False``, returns a ``(primals_out, vjpfun)`` pair, where
    ``primals_out`` is ``fn(*primals)``.
    ``vjpfun`` is a function from a cotangent vector with the same shape as
    ``primals_out`` to a tuple of cotangent vectors with the same shape as
    ``primals``, representing the vector-Jacobian product of ``fn`` evaluated at
    ``primals``. If ``has_aux`` is ``True``, returns a
    ``(primals_out, vjpfun, aux)`` tuple where ``aux`` is the auxiliary data
    returned by ``fn``.
  """
  return lift_direct_transform(
      lift.vjp, (fn,), mdl, *primals,
      multi_scope=False,
      has_aux=has_aux, reduce_axes=reduce_axes,
      vjp_variables=vjp_variables,
      variables=variables,
      rngs=rngs)

def jvp(
    fn: Callable[..., Any],
    mdl: Module,
    primals,
    tangents,
    variable_tangents,
    variables: lift.CollectionFilter = True,
    rngs: lift.PRNGSequenceFilter = True,
) -> Union[Tuple[Any, Callable[..., Any]], Tuple[Any, Callable[..., Any], Any]]:
  """A lifted version of ``jax.jvp``.

  See ``jax.jvp`` for the unlifted Jacobian-vector product (forward gradient).

  Note that no tangents are returned for variables. When variable tangents
  are required their value should be returned explicitly by `fn`
  using `Module.variables`::

    class LearnScale(nn.Module):
      @nn.compact
      def __call__(self, x):
        p = self.param('test', nn.initializers.zeros, ())
        return p * x

    class Foo(nn.Module):
      @nn.compact
      def __call__(self, x):
        scale = LearnScale()
        vars_t = jax.tree_util.tree_map(jnp.ones_like,
                                        scale.variables.get('params', {}))
        _, out_t = nn.jvp(
            lambda mdl, x: mdl(x), scale, (x,), (jnp.zeros_like(x),),
            variable_tangents={'params': vars_t})
        return out_t

  Example::

    def learn_scale(scope, x):
      p = scope.param('scale', nn.initializers.zeros, ())
      return p * x

    def f(scope, x):
      vars_t = jax.tree_util.tree_map(jnp.ones_like, scope.variables().get('params', {}))
      x, out_t = lift.jvp(
          learn_scale, scope, (x,), (jnp.zeros_like(x),),
          variable_tangents={'params': vars_t})
      return out_t

  Args:
    fn: Function to be differentiated. Its arguments should be arrays, scalars,
      or standard Python containers of arrays or scalars. It should return an
      array, scalar, or standard Python container of arrays or scalars. It will
      receive the scope and primals as arguments.
    mdl: The module of which the variables will be differentiated.
    primals: The primal values at which the Jacobian of ``fun`` should be
      evaluated. Should be either a tuple or a list of arguments,
      and its length should be equal to the number of positional parameters of
      ``fun``.
    tangents: The tangent vector for which the Jacobian-vector product should be
      evaluated. Should be either a tuple or a list of tangents, with the same
      tree structure and array shapes as ``primals``.
    variable_tangents: A dict or PyTree fo dicts with the same structure as
      scopes. Each entry in the dict specifies the tangents for a variable
      collection. Not specifying a collection in variable_tangents is
      equivalent to passing a zero vector as the tangent.
    variables: other variables collections that are available in `fn` but
      do not receive a tangent.
    rngs: the prngs that are available inside `fn`.

  Returns:
    A ``(primals_out, tangents_out)`` pair, where ``primals_out`` is
    ``fun(*primals)``, and ``tangents_out`` is the Jacobian-vector product of
    ``function`` evaluated at ``primals`` with ``tangents``. The
    ``tangents_out`` value has the same Python tree structure and shapes as
    ``primals_out``.
  """
  return lift_direct_transform(
      lift.jvp, (fn,), mdl, primals, tangents, variable_tangents,
      multi_scope=False,
      variables=variables,
      rngs=rngs)

ModuleT = TypeVar('ModuleT', bound=Module)
C = TypeVar('C')

def while_loop(
    cond_fn: Callable[[ModuleT, C], bool],
    body_fn: Callable[[ModuleT, C], C],
    mdl: ModuleT,
    init: C,
    carry_variables: lift.CollectionFilter = False,
    broadcast_variables: lift.CollectionFilter = True,
    split_rngs: Mapping[lift.PRNGSequenceFilter, bool] = FrozenDict()) -> C:
  """Lifted version of jax.lax.while_loop.

  The lifted scope is passed to `cond_fn` and `body_fn`.
  Broadcasted variables are immutable. The carry variable are
  mutable but cannot change shape and dtype.
  This also means you cannot initialize variables inside
  the body. Consider calling `body_fn` once manually before
  calling `while_loop` if variable initialization is required.

  Example::

    class WhileLoopExample(nn.Module):
      @nn.compact
      def __call__(self, x):
        def cond_fn(mdl, c):
          return mdl.variables['state']['acc'] < 10
        def body_fn(mdl, c):
          acc = mdl.variable('state', 'acc', lambda: jnp.array(0))
          acc.value += 1
          y = nn.Dense(c.shape[-1])(c)
          return y
        c = x
        if self.is_mutable_collection('params'):
          return body_fn(self, c)
        else:
          return nn.while_loop(cond_fn, body_fn, self, c,
                               carry_variables='state')

    k = random.PRNGKey(0)
    x = jnp.ones((2, 2))
    intial_vars = WhileLoopExample().init(k, x)
    result, state = WhileLoopExample().apply(intial_vars, x, mutable=['state'])

  Args:
    cond_fn: Should return True as long as the loop should continue.
    body_fn: The body of the while loop.
    mdl: The Module which should be lifted into the loop.
    init: The initial state passed to the loop
    carry_variables: collections that are carried through the loop
      and are therefore mutable (default: none).
    broadcast_variables: collections that are closed over and are
      therefore read-only (default: all collections)
    split_rngs: Split PRNG sequences will be different for each loop iterations.
      If split is False the PRNGs will be the same across iterations.
  Returns:
    The final state after executing the while loop.
  """
  return lift_direct_transform(
      lift.while_loop, (cond_fn, body_fn), mdl,
      init,
      carry_variables, broadcast_variables,
      split_rngs)

def _cond_wrapper(t_fn, f_fn, scope, pred, *ops, variables, rngs):
  return lift.cond(pred, t_fn, f_fn, scope, *ops, variables=variables, rngs=rngs)

def cond(
    pred: Any,
    true_fun: Callable[..., C], false_fun: Callable[..., C],
    mdl: Module, *operands,
    variables: lift.CollectionFilter = True,
    rngs: lift.PRNGSequenceFilter = True) -> C:
  """Lifted version of ``jax.lax.cond``.

  The returned values from ``true_fun`` and ``false_fun``
  must have the same Pytree structure, shapes, and dtypes.
  The variables created or updated inside the
  branches must also have the same structure.
  Note that this constraint is violated when
  creating variables or submodules in only one branch.
  Because initializing variables in just one branch
  causes the parameter structure to be different.

  Example::

    class CondExample(nn.Module):
      @nn.compact
      def __call__(self, x, pred):
        self.variable('state', 'true_count', lambda: 0)
        self.variable('state', 'false_count', lambda: 0)
        def true_fn(mdl, x):
          mdl.variable('state', 'true_count').value += 1
          return nn.Dense(2, name='dense')(x)
        def false_fn(mdl, x):
          mdl.variable('state', 'false_count').value += 1
          return -nn.Dense(2, name='dense')(x)
        return nn.cond(pred, true_fn, false_fn, self, x)

  Args:
    pred: determines if true_fun or false_fun is evaluated.
    true_fun: The function evalauted when ``pred`` is `True`.
      The signature is (module, *operands) -> T.
    false_fun: The function evalauted when ``pred`` is `False`.
      The signature is (module, *operands) -> T.
    mdl: A Module target to pass.
    *operands: The arguments passed to ``true_fun`` and ``false_fun``
    variables: The variable collections passed to the conditional
      branches (default: all)
    rngs: The PRNG sequences passed to the conditionals (default: all)
  Returns:
    The result of the evaluated branch (``true_fun`` or ``false_fun``).
  """
  return lift_direct_transform(
      _cond_wrapper, (true_fun, false_fun), mdl,
      pred, *operands,
      variables=variables, rngs=rngs)

def _switch_wrapper(*args, variables, rngs, n_branches):
  # first n_branches arguments are branches.
  # then scope, index, and the rest are *operands
  branches = args[:n_branches]
  scope, index, *operands = args[n_branches:]
  return lift.switch(index, branches, scope, *operands, variables=variables, rngs=rngs)

def switch(
    index: Any,
    branches: Sequence[Callable[..., C]],
    mdl: Module, *operands,
    variables: lift.CollectionFilter = True,
    rngs: lift.PRNGSequenceFilter = True) -> C:
  """Lifted version of ``jax.lax.switch``.

  The returned values from ``branches``
  must have the same Pytree structure, shapes, and dtypes.
  The variables created or updated inside the
  branches must also have the same structure.
  Note that this constraint is violated when
  creating variables or submodules in only one branch.
  Because initializing variables in just one branch
  causes the parameter structure to be different.

  Example::

    class SwitchExample(nn.Module):
      @nn.compact
      def __call__(self, x, index):
        self.variable('state', 'a_count', lambda: 0)
        self.variable('state', 'b_count', lambda: 0)
        self.variable('state', 'c_count', lambda: 0)
        def a_fn(mdl, x):
          mdl.variable('state', 'a_count').value += 1
          return nn.Dense(2, name='dense')(x)
        def b_fn(mdl, x):
          mdl.variable('state', 'b_count').value += 1
          return -nn.Dense(2, name='dense')(x)
        def c_fn(mdl, x):
          mdl.variable('state', 'c_count').value += 1
          return nn.Dense(2, name='dense')(x)
        return nn.switch(index, [a_fn, b_fn, c_fn], self, x)

  If you want to have a different parameter structure for each branch
  you should run all branches on initialization before calling switch::

    class MultiHeadSwitchExample(nn.Module):
      def setup(self) -> None:
        self.heads = [
          nn.Sequential([nn.Dense(10), nn.Dense(7), nn.Dense(5)]),
          nn.Sequential([nn.Dense(11), nn.Dense(5)]),
          nn.Dense(5),
        ]

      @nn.compact
      def __call__(self, x, index):
        def head_fn(i):
          return lambda mdl, x: mdl.heads[i](x)
        branches = [head_fn(i) for i in range(len(self.heads))]

        # run all branches on init
        if self.is_mutable_collection('params'):
          for branch in branches:
            _ = branch(self, x)

        return nn.switch(index, branches, self, x)

  Args:
    index: Integer scalar type, indicating which branch function to apply.
    branches: Sequence of functions to be applied based on index.
      The signature of each function is (module, *operands) -> T.
    mdl: A Module target to pass.
    *operands: The arguments passed to the branches.
    variables: The variable collections passed to the conditional
      branches (default: all)
    rngs: The PRNG sequences passed to the conditionals (default: all)
  Returns:
    The result of the evaluated branch.
  """
  return lift_direct_transform(
      _switch_wrapper, tuple(branches), mdl,
      index, *operands,
      variables=variables, rngs=rngs, n_branches=len(branches))

# a version of lift.custom_vjp with a single scope function
# this avoids having to lift multiple functions in
# lift_transform.
def _custom_vjp_single_scope_fn(
    fn: Callable[..., Any],
    backward_fn: Callable[..., Any],
    grad_vars: lift.CollectionFilter = 'params',
    nondiff_argnums=()):
  nodiff_fn = functools.partial(fn, needs_residual=False)
  forward_fn = functools.partial(fn, needs_residual=True)
  return lift.custom_vjp(nodiff_fn, forward_fn, backward_fn, grad_vars,
                         nondiff_argnums)

def custom_vjp(fn: Callable[..., Any],
               forward_fn: Callable[..., Any],
               backward_fn: Callable[..., Any],
               grad_vars: lift.CollectionFilter = 'params',
               nondiff_argnums=()):
  """Lifted version of `jax.custom_vjp`.

  `forward_fn` and `backward_fn` together define a custom vjp for `fn`.
  The original `fn` will run in case a vjp (backward gradient) is not computed.

  The `forward_fn` receives the same arguments as `fn` but is expected to return
  a tuple containing the output of `fn(mdl, *args)` and the residuals that are
  passed to `backward_fn`.

  The `backward_fn` receives the nondiff arguments, residuals, and the output
  tangents. It should return a tuple containing the variable and input tangents.

  Note that the vjp function returned by `nn.vjp` can be passed as residual and
  used in the `backward_fn`. The scope is unavailable during the backward pass.
  If the module is required in `backward_fn`, a snapshot of the variables can
  be taken and returned as a residual in the `forward_fn`.

  Example::

    class Foo(nn.Module):
      @nn.compact
      def __call__(self, x):
        def f(mdl, x):
          return mdl(x)

        def fwd(mdl, x):
          return nn.vjp(f, mdl, x)

        def bwd(vjp_fn, y_t):
          params_t, *inputs_t = vjp_fn(y_t)
          params_t = jax.tree_util.tree_map(jnp.sign, params_t)
          return (params_t, *inputs_t)

        sign_grad = nn.custom_vjp(
            f, forward_fn=fwd, backward_fn=bwd)
        return sign_grad(nn.Dense(1), x).reshape(())

    x = jnp.ones((2,))
    variables = Foo().init(random.PRNGKey(0), x)
    grad = jax.grad(Foo().apply)(variables, x)

  Args:
    fn: The function to define a custom_vjp for.
    forward_fn: A function with the same arguments as ``fn`` returning an tuple
      with the original output and the residuals that will be passsed to
      ``backward_fn``.
    backward_fn: arguments are passed as
      ``(*nondiff_args, residuals, tangents)`` The function should return a
      tuple containing the tangents for the variable in the collections
      specified by `grad_vars` and the input arguments (except the module and
      nondiff args).
    grad_vars: The collections for which a vjp will be computed
      (default: "params").
    nondiff_argnums: arguments for which no vjp is computed.
  Returns:
    A function with the same signature as `fn` with the custom vjp.
  """
  def shared_forward_fn(*args, needs_residual, **kwargs):
    if needs_residual:
      return forward_fn(*args, **kwargs)
    else:
      return fn(*args, ** kwargs)
  return decorator_lift_transform(
      _custom_vjp_single_scope_fn, shared_forward_fn,
      backward_fn=backward_fn, grad_vars=grad_vars,
      nondiff_argnums=nondiff_argnums,
      multi_scope=False)

def named_call(class_fn, force=True):
  """Labels a method for labelled traces in profiles.

  Note that it is better to use the `jax.named_scope` context manager directly
  to add names to JAX's metadata name stack.

  Args:
    class_fn: The class method to label.
    force: If True, the named_call transform is applied even if it is globally
      disabled. (e.g.: by calling `flax.linen.disable_named_call()`)
  Returns:
    A wrapped version of ``class_fn`` that is labeled.
  """
  # We use JAX's dynamic name-stack named_call. No transform boundary needed!
  @functools.wraps(class_fn)
  def wrapped_fn(self, *args, **kwargs):
    if ((not force and not linen_module._use_named_call)  # pylint: disable=protected-access
        or self._state.in_setup):  # pylint: disable=protected-access
      return class_fn(self, *args, **kwargs)
    full_name = _derive_profiling_name(self, class_fn)
    return jax.named_call(class_fn, name=full_name)(self, *args, **kwargs)
  return wrapped_fn


text:
Del Bosque finally proves he's far more than just a player's coach
"'Vicente del Bosque is a lovely man.' Rarely has anyone been quite so damned with faint praise; never has a human strength been so readily and immediately presented as a professional weakness, a quality presumed to be a defect. Del Bosque is indeed a lovely man, a good man — impeccably polite, overwhelmingly decent. Loyal, humble, kindly, he is like a favorite uncle with a bushy 'tache, a portly belly and the warmest of handshakes. But by extension, by definition, he is not much of a coach. After all, nice guys come last." (SI)
Leave a Comment »	| Spain, World Cup 2010	| Tagged: Spain, World Cup	| Permalink
Diego Forlan Deserves the Golden Ball
"World Cup 2010 has been done and dusted, as we have found a champion out of the 32 teams competing for the biggest prize of all in footballing universe, Spain, as well as Thomas Muller, the winner of the Golden Boot award and also for the young German to officially announce his arrival in world football. However, there's still one more award which drew quite a lot of criticisms and that's the Golden Ball award, awarded to the best player in the tournament. In World Cup 2010, the winner is Uruguay's Diego Forlan, and many pundits and fans, especially Internazionale fans out there feel that Forlan is not a deserving recipient of this award, as they feel that Wesley Sneijder, the runners-up for the award, or in other words the Silver Ball winner of this tournament who should have been the recipient of the Golden Ball." (Beopedia)
Leave a Comment »	| Germany, Holland, Spain, Uruguay, World Cup 2010	| Tagged: Germany, Holland, Spain, Uruguay, World Cup 2010	| Permalink
World Cup scouting: The 32 – Conclusions
Antonio Di Natale
"Starting with Nicolás Lodeiro back in December last year, Football Further selected 32 players to watch out for at the 2010 World Cup and then tracked their progress through the tournament via weekly scouting reports. Below is a full compilation of those reports, along with conclusions (and marks out of 10) on how each player performed." (Football Further)
Leave a Comment »	| Argentina, Brazil, Chile, England, Football Manager, France, Germany, Ghana, Holland, Italy, Nigeria, Portugal, South Africa, Spain, Uruguay, USA, World Cup 2010	| Tagged: Argentina, Brazil, Chile, England, Football Manager, France, Germany, Holland, Italy, Nigeria, Portugal, South Africa, Spain, Uruguay, USA, World Cup 2010	| Permalink
From Total to Anti-Football: Why Holland Lost, and I'm Glad
"Over the next couple of weeks, we'll provide you with all the post-WC analysis you can handle, but for now, let's talk about the final. As I was watching the game, I didn't have a strong rooting interest either way, but I expected a great game. Both teams were stocked with All-Stars at virtually every position. The Spanish had won Euro; the Dutch were working on an undefeated tournament. Although the score line of a lot of the Spanish games this tournament were not as impressive as some of the other teams (Germany for example), anyone who watched a Spain game – watched the execution, understood their dominance on the ball, marveled at their ability to play 'keep away' after scoring a goal – knew that they were impressive. Meanwhile, the Dutch had seemingly rolled through the tournament and they managed to defeat mighty and heavily favored Brazil. This game was to be an epic showdown." (Yanks are coming)
The final analysis, part three: brilliant Busquets
"As is customary after a Spain or Barcelona success, the performance of Sergio Busquets (two La Ligas, a Champions League and a World Cup after two seasons of professional football) has largely been ignored. In the World Cup final he was one of the key players for Spain – keeping Wesley Sneijder quiet, providing his usual solid, reliable passing from a deep midfield position, and dropping between his centre-backs to turn Spain's 4-2-3-1 into something more like a 3-3-3-1 or 3-4-3 when in possession." (Zonal Marking)
Leave a Comment »	| Football Manager, Holland, Spain, World Cup 2010	| Tagged: Football Manager, Holland, Spain, World Cup 2010	| Permalink
World Cup 2010 Redux: Links & Videos
"So, yeah, the World Cup has ended. Some hated it. I enjoyed it. Of course, I took into account that no even can realistically live up to the hype as "the greatest and most entertaining" sporting event on Earth. Still, from the fluid counter-attacking Germans to the patient passing Iberians, this tournament was light years ahead of the catenaccio Italians and plodding French of 2006. Yes, Zizou was amazing in the outrounds of that tournament. No, that did not redeem the other 31 teams or other games. But enough of that, onto links and classic videos from South Africa 2010." (futfanatico)
Leave a Comment »	| World Cup 2010	| Tagged: World Cup 2010	| Permalink
Old Firm Haves and Have Nots
Joe Ledley
"Another summer in Scotland, and another period where Celtic possess the money to strengthen their squad and Rangers are forced to sell their most bankable assets. These last few days have seen a whirl of activity surrounding the Old Firm, but once again, as during the past 18 months, the main questions seem to be: where is Rangers' money? And why is there already a sense of deflation around the reigning SPL champions?" (ESPN)
Lennon confident on Juarez
"Celtic manager Neil Lennon believes he is close to making Mexico midfielder Efrain Juarez his latest summer signing. Lennon is confident of landing Pumas UNAM starlet Juarez, 22, and hopes he could even make his Celtic debut against Manchester United while on their pre-season tour of North America." (ESPN)
Leave a Comment »	| Celtic, Scottish Cup	| Tagged: Celtic, Scottish Cup	| Permalink
Champions League is perpetrating big-fish-in-small-pond syndrome
"For half an hour on Tuesday in the Champions League qualifiers, Dinamo Zagreb was worried. Davor Bubanja had given Koper the lead, and the specter of yet more European failure was raised. But then the excellent Milan Mandzukic equalized, and Miroslav Slepicka had the Croatian champion ahead. Second-half goals from Mandzukic and the two Brazilians, Sammir and Etto, gave Dinamo a 5-1 win and should have put the tie to bed." (SI)
Why football can't save the world
"Now the tournament is over, it's a good time to get some perspective on the World Cup. In the latest WSC, Ian Plenderleith looks at some sporting (and marketing) myths." (WSC)
You are currently browsing the Scissors Kick blog archives for the day Friday, July 16th, 2010.

redpajama_set_name: RedPajamaCommonCrawl


text:
"use strict";

var _interopRequireDefault = require("@babel/runtime/helpers/interopRequireDefault");
Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.default = void 0;
var _createSvgIcon = _interopRequireDefault(require("./utils/createSvgIcon"));
var _jsxRuntime = require("react/jsx-runtime");
var _default = (0, _createSvgIcon.default)( /*#__PURE__*/(0, _jsxRuntime.jsx)("path", {
  d: "M22.41 12 12 1.59 1.59 11.99 12 22.41 22.41 12zM14 14.5V12h-4v3H8v-5h6V7.5l3.5 3.5-3.5 3.5z"
}), 'DirectionsSharp');
exports.default = _default;

redpajama_set_name: RedPajamaGithub


text:
Q: In Eclipse what can I use to encapsulate a block of code that I can toggle to view or not? In Visual Studio I remember using some sort of syntax to encapsulate code and be able to toggle it open and close to view.
Is there something similar in Eclipse? (JAVA)

A: Did you mean code fold ?

A: Java doesn't have regions, unlike C#. So there is no user-defined means to define regions that you can collapse.

redpajama_set_name: RedPajamaStackExchange


The top ten seaside destinations for Railcard holders are revealed!
Millions of savvy Railcard holders are saving almost £600million a year – £150 each on average – on train trips up and down Britain, as their favourite summer haunts are revealed.
Four million Railcard customers* - who get up to a third off most train fares including cheaper Advance tickets - made more than 130million journeys last year, 16 return trips each on average.
And Railcard customers do love to be beside the seaside. Brighton, Blackpool and Bournemouth are expected to be the most popular resorts this summer as they pack their Railcards along with the picnics, buckets and spades.


text:
¿¿ AMERICAN GULAG ?? : : INTERVIEW WITH A FUN GUY
July 15, 2018 by MeridaGOround
"Other side of the wall"? Photo by MeridaGOround.com
Excerpted below is a portion of an interview with a Mexican-American author hosted by Krista Tippett, of OnBeing.org who does a weekly podcast. Here they discuss the border wall. (*Liminal* means a transitional stage ; both sides of a boundary.)
MS. TIPPETT: I feel like one thing you do — as much in your fiction as in your nonfiction, and certainly in Into the Beautiful North — is, you work with the idea of a border or a wall, not, in fact, as a hard and fast thing: as a liminal space, as a liminal zone.
MR. URREA: It is a liminal space, absolutely.
MS. TIPPETT: Right, but just to think about it that way opens up a lot of imagination.
MR. URREA: I think liminal space is where all writers go. That place of crossing, that place of pressure, of two things meeting, that's a rich — that's where the plankton wells up and the currents meet. And you can choose to see it in different ways. And either the border is a hideous, festering scar of oppression, horror, and violence, or it's a fraternal space where two cultures meet and can exchange. And honestly, particularly before the narco wars, there was and there still are bastions of friendship along the border. And all you have to do is go to places near Nogales or Yuma, where kids on the Mexican side and kids on the American side play volleyball over the wall with each other.
MS. TIPPETT: Yeah, and see, we don't hear these stories.
MR. URREA: No, you don't. And I recently did a ballet — I didn't. I read poems while they danced.
MS. TIPPETT: I'm imagining it.
MR. URREA: No. Me in a tutu — nah.
But I narrated this ballet. It was the 100th anniversary of a Stravinsky piece, which included a Faustian journey through a wasteland, where the man trying to get to safety has to make a deal with the devil, essentially. That was 100 years ago; this time, it's people dying in the desert, making that terrible deal to survive. But when he did his other piece — his name is Steven Schick.
MS. TIPPETT: Oh, yes.
MR. URREA: He's brilliant. I keep saying, "This guy's…"
MS. TIPPETT: He also spent some time in Berlin, right? And that was a wall with which I had some intimacy. And I remember, still, when Michael Jackson came and did a concert right on the western side of the wall, just as things were falling apart. But the concertgoers gathered on the eastern side. It was exactly that. But one thing you point out is — so in Berlin, on the western side, the wall was painted and raucous and alive and rebellious, on the western side, where people were free. In Mexico, what do you say? In Mexico, it's the Mexican side —
MR. URREA: It's the reverse.
MS. TIPPETT: It's the reverse side that's —
MR. URREA: Well, thank God for Steven Schick. I stole it, when he was telling me it, because it was the perfect wrap-up for that piece for The Times. When you went across to the other side, he said, the Mexican side, the entire fence is an art gallery covered with paintings, sculptures, graffiti. There are ice-cream men and taco stands, and there are mariachis, and there are lovers, and there are people dancing and strolling. The American side: steel, trucks, dogs, helicopters, guns.
MS. TIPPETT: No art, no graffiti.
MR. URREA: No nothin', and he said, "I suddenly realized that that was the Soviet side in Berlin."
MS. TIPPETT: Yeah, it was. And I think you said, "Who was free? Who was free, and who was prisoner?"
MR. URREA: Yeah, what exactly is that wall for, then? Hmm.
This entry was tagged border wall, dead end, Luis Alberto Urrea, OnBeing, prison. Bookmark the permalink.
← TWO BOOKS
¿How Long Can an iPhone Hold its Breath ? →
One thought on "¿¿ AMERICAN GULAG ?? : : INTERVIEW WITH A FUN GUY"
Ron Wible says:
I love stories with a twist ending.


Australia's senior naval officer in charge of border protection has defended the lack of safety surveillance at Christmas Island in the lead up to December's catastrophic boat crash, saying it was not physically possible to prepare for such an "unprecedented" event.
Rear Admiral Timothy Barrett today gave evidence to a Perth coronial Inquest into the resourcing available to him to prevent such a tragedy being repeated in the future.
The inquest is tasked with examining whether customs, navy intelligence or the remote island itself could have done more to prevent the shipwreck, which killed as many as 50 Iranian and Iraqi men, women and children aboard the Indonesian fishing vessel SIEV 221 on December 15 last year.
The 52-year-old seaman and Naval officer for 35 years was put on notice at the start of the inquest that he may have to answer any adverse findings that could result.
Rear Admiral Barrett defended his position to Chief Coroner Alastair Hope, saying that despite increases in the number of illegal boat arrivals to the island in the past three years, it was not up to Border Protection Command to ensure their safe passage to Australia.
"This type of shipwreck was unprecedented at Christmas Island," Rear Admiral Barrett told the inquest.
He said it was the Navy's role to provide law enforcement and to monitor the seaworthiness of vessels once they were intercepted, but they had no intelligence at the time of SIEV 221's arrival.
He said it was up to the master of the boat itself to ensure its own safe passage, which was in keeping with maritime codes.
It would be difficult to provide the necessary surveillance of Australia's entire maritime territory since its sea made up 11 per cent of the Earth's water, he told the court.
He said the priority was to prevent illegal boat arrivals to the Australian mainland even at the expense of undetected arrivals to Christmas Island.
"When weighing up competing priorities the reality is that the SIEV landing to border security... the risk is significantly less for Christmas Island arrival than a mainland arrival," he said.
He said counter measures were also prohibitively costly, with permanent look-outs at the island costing $3.3 million a year to staff and maintain and constant aerial surveillance costing $46,000 an hour due to the distances involved.
Customs relied on and "encouraged" land-based surveillance by Christmas Island residents, he said.
Even if someone had been stationed on the island, the weather and darkness would have prevented anyone from seeing the boat before sunrise (at 5.30am) because it did not have any lights. By this time the boat was already in peril in the heaving seas.
When questioned about the use of radars, he said the technology was sensitive and commercial maritime radars that mostly detected metal would be unlikely to pick up a timber boat.
He deemed it too risky to Indonesian relations to request a military radar for the outpost.
"There is some sensitivity of placing something of that nature, of that type of long radar, close to the Indonesian mainland," he told the inquest.
"I consider that to be highly sensitive so I did not proceed with that suggestion."


text:
Le Camp de Mérignac est un camp d'internement ouvert par le régime de Vichy en 1941 au lieu-dit Beaudésert sur le territoire de la commune de Mérignac (Gironde). Y furent internés des nomades, des Juifs, des étrangers, des communistes et des réfractaires au S.T.O..

Historique
Un camp d'internement fut créé à la demande des Autorités d'Occupation (A.O.) par arrêté préfectoral du . Dans un premier temps, il assura les internements administratifs de populations nomades, mais l'Hôtel des émigrants du 24 quai de Bacalan à Bordeaux s'avérant trop exigu, le camp de Mérignac devint un lieu de détention pour des internés politiques. Dans la continuité du décret du  et du décret du  dit « décret Sérol », instruments d'une répression anticommuniste : après de la signature du Pacte germano-soviétique les autorités françaises poursuivirent à grande échelle la traque des communistes. Ainsi, le ,  et femmes suspectés d'activités syndicalistes et communistes furent arrêtés. La mise en service du camp de Mérignac à Beaudésert permit donc l'augmentation du nombre d'internés fut par la suite pour, l'Occupant, un vivier d'otages.

Enfin, en application des accords Bousquet-Oberg de 1942, le camp de Mérignac servit de lieu de regroupement des juifs internés, en amont de leur déportation.
Des gendarmes français et des Gardes civils furent affectés à la surveillance du camp qui relevait de l'administration française.

Le régime de Vichy interna ainsi à Mérignac des personnes désignées comme « indésirables » : réfugiés espagnols et autres étrangers, juifs, nomades, militants communistes, etc.

Caractéristiques
Le camp d'internement fut installé au début de la guerre sur le site de Beaudésert, où un projet de construction de cité olympique dans les années 1920 n'avait pas abouti.

Il s'agissait d'un petit camp entouré d'une haute palissade de fils barbelés et de miradors. Le camp était gardé par des gendarmes français. Les conditions de vie y étaient déplorables, sans hygiène ni infirmerie.

Y furent détenus pendant quatre ans, tour à tour ou simultanément, militants politiques et syndicaux, condamnés de droit commun, « indésirables » et étrangers, trafiquants du marché noir, juifs, proxénètes, prostituées, réfractaires au S.T.O., raflés sans papiers, « dissidents » et « terroristes ». Cinq à six cents personnes y étaient surveillées par une soixantaine de gendarmes et de gardes civils.

Un lieu d'internement des juifs
À partir du , des juifs des deux sexes âgés de 16 à 45 ans furent détenus au camp, à l'exception des juifs des nationalités : italienne, espagnole, turque, grecque, bulgare, hongroise, finnoise, norvégienne, anglaise, américaine et mexicaine.

Plus de  juifs de Bordeaux et de sa région furent internés au camp de Mérignac. Quinze cents d'entre eux furent transférés au camp de Drancy puis déportés vers les centres de mise à mort de Pologne. Une palissade séparait les baraques des juifs des baraques des autres détenus.

Les nomades et les juifs détenus au camp de Mérignac furent par la suite transférés au Camp de la Lande de Monts ou au camp de La Morellerie en Indre-et-Loire. De juillet à , 459 juifs internés à Mérignac furent transférés à Drancy puis déportés.

Pierre Garat, haut fonctionnaire vichyste, sous la responsabilité de Maurice Papon, secrétaire général de la préfecture de la Gironde, établit les listes, et procède au triage des juifs.

Un lieu d'internement d'opposants politiques
À partir de l'été 1941, une clôture en fils de fer barbelés fut construite pour regrouper les otages.

Le camp de Mérignac fut l'une des annexes de la maison d'arrêt de Bordeaux du Fort du Hâ. Le , des détenus du camp furent désignés comme otages et furent fusillés à Souge et à Bordeaux.

Internés célèbres
Roger Allo, syndicaliste communsiste
Louis de La Bardonnie, résistant français
Charles Nancel-Pénard, médecin communiste
Samy Stourdzé, rabbin résistant

Sur les registres du camp de Beaudésert, on a retrouvé par erreur le nom du neurologue Boris Cyrulnik, dont les parents furent assassinés par la suite au camp d'Auschwitz.

Archives du camp
Les archives du camp de Mérignac sont conservées aux Archives départementales de la Gironde.

Bibliographie
 Florent Leruste, Juifs internés à Bordeaux (1940-1944) : Le camp de Mérignac-Beaudésert, Les Éditions du Net, 2014

Articles connexes
 Liste des camps d'internement de « nomades » en France
 Camp d'internement français
 Fusillés du camp de Souge

Liens externes
 Camp de Mérignac-Beaudésert

 Archives du camp de Mérignac

Notes et références

Notes

Références

Camp de Mérignac
Gironde pendant la Seconde Guerre mondiale
Mérignac
Mémoire de la Seconde Guerre mondiale dans la Gironde
Histoire des Juifs sous le régime de Vichy
Fondation en 1940
Mérignac

redpajama_set_name: RedPajamaWikipedia


text:
Home Streaming News Start Spreading The News…Miranda Sings Returns
Start Spreading The News…Miranda Sings Returns
In an all-new season of Haters Back Off! Her life may be in shambles, but Miranda's finally gone viral! Now she's ready to take on her next big challenge: making it to Broadway. The second season premieres on Friday, October 20, only on Netflix.
About Haters Back Off!: Based on the character created by Colleen Ballinger, Haters Back Off delves into the oddball family life of Miranda Sings. Sings, an incredibly confident, totally untalented star on the rise, continues to fail upward by the power of her belief that she was born famous, it's just no one knows it yet.
The series stars Colleen Ballinger as Miranda Sings, Steve Little (Eastbound & Down) as Miranda's overly confident and utterly misguided Uncle Jim, Angela Kinsey (The Office)as Miranda's mom, Bethany, and Erik Stocklin, as Miranda's neighbor and best friend, Patrick.
Visit Haters Back Off! on Netflix!
Previous articleA Glance At Professor Marston And The Wonder Women
Next articleLong Live The King!
Wanda Vision And The Triumphant Return Of The MCU
Bend Or Break, Watch The Crown Season 4 Trailer
Saved By The Bell Is Streaming November 25th On Peacock.


POLLS OPEN TOMORROW FROM 6 pmto 9 PM $7,497,000 HARTSDALE EXPANSION VOTE… PLEASE PARTICIPATE.
Hartsdale residents can vote tomorrow on the proposed Hartsdale fire district expansion. The cost will be $7,497,000 to expand and renovate Fire district No 1, located on Central Aveand Station No. 2 located on West Hartsdale Ave.


text:
The entrepreneurial culture and resources at Loughborough University London
The Loughborough University London campus is centrally located in an innovation and technology hub, Here East – home to a vibrant mix of innovators, inventors and visionaries looking to challenge the status quo. This enables Loughborough University London to collaborate effectively with organisations from a number of industries and sectors. As well as providing access to primary data from a broad range of sources, these collaborations help to shape the research focus to ensure research is delivering impact for industry and society.
There is an undeniable creative and entrepreneurial culture on Loughborough University's London campus, which has enabled students and alumni to achieve great things, like forge new business plans, grow niche new start-ups and share innovative solutions to business problems. Thought leadership, entrepreneurship and innovation is the forefront of our vibrant London campus and our aim is to influence a narrative by understanding what needs to be done.
Whether you are a visionary, a conscious reflector, or a passionate trail blazer, at Loughborough University London we hone in on all of these qualities aiming for the future generations of our postgraduate students to be leaders in their fields and continuously have forward-thinking approaches and challenge society norms. Together with our unique location, the Future Space and Loughborough Enterprise Network (LEN) teams are here to support your entrepreneurial journeys.
Dr Salman Malik is a resident entrepreneur and business advisor at Loughborough University London helping students put their ideas forward and develop their enterprise skills. Dr Salman Malik (aka Sal) is an ex-scientist, an advisory board member and multi-award winning entrepreneur. Sal has solely founded a biotech start-up and successfully raised funding to work on the research and development of a process used to cost-effectively develop microparticles (i.e. drug delivery carriers). He was also awarded a prestigious Enterprise Fellowship from The Royal Society of Edinburgh (RSE) which allowed him to transition as an academic into an entrepreneur and CEO.
As an ex-scientist, Sal has created a concoction of 8 key mindset features for entrepreneurs and thought leaders that he believes will aid in them becoming a success. Vision, a fearless attitude, problem solving skills, creating value, self development, resourcefulness, the ability to recover quickly from knock backs and most importantly being a DOER. Sal was previously a student entrepreneur and had this to say about his experiences:
"As a (once upon a time) student entrepreneur, I have always been convinced that one of the 'safest' places to commercialise your ideas and start a business has been within a University and, in particular during your time there as either a Undergraduate or Postgraduate student. A university provides a de-risked environment for sole/co-founders to network and connect with other team members. As a student entrepreneur, you have access to multiple sources of funding through mini competitions and enterprise funds giving you admission to endless resources, advice and support while developing your idea. Furthermore, the mentoring available through a university's trusted network of alumni can also open doors for you as a founder or employee within the business.
"One of the most notable things about being a student entrepreneur is your ability to generate innovative ideas. You are young, energetic, motivated, inspired, and are not distracted by some of the 'background noise' or responsibilities that may appear as you leave and enter employment, potentially get married and/or have a family. The appetite for risk is higher, the thirst for success is greater, and the surrounding collegiate ecosystem endorses enterprise making it the ideal scenario to cultivate your ideas/business. However, regardless of all of the above, accessing funds to take your ideas forward will always remain a hurdle and a competitive one too. As a student entrepreneur, there are sources of funding available to you as a student/graduate and are all very realistic opportunities to bring in cash to your start-up."
To find out more about the support available at the London campus for student entrepreneurs, please visit the Loughborough Enterprise Network (LEN) website and our careers and employability support (aka Future Space) pages.
Career Community Staff
Rebecca Davis


text:
Q: Calculating the velocity of a car towards another car I have a Car1 traveling (not directly) towards another Car2, both cars can freely drive away from each other (making the velocity from Car1 to Car2 negative)
What I know of the cars is the velocity, location (Vectors) and their distance in a 3d space.
Both Cars are not traveling at a constant speed, nor in a fixed position (straight line), Car2 could stop. This would obviously result in Car1 approaching Car2 at the velocity of Car1.
What I'm having trouble figuring out is how fast is Car1 approaching Car2?
Would anyone have an ideas where to start, or some formula?

A: I managed to figure this out in due time, I just had to calculate the relative velocity for Car1, by subtracting Car2's velocity from Car1's velocity and get the magnitude from that. I then got the distance from them and from a very small time interval I checked if they are going apart or not.
It's not a perfect system, and the formulas are not hard to figure out, I only have code snippets for it. But it works.


Please share IPs, PORT# of Cassandra database hosts. I want to create & load data through Spark using Sacala context.
Can you please confirm Spark & Scala version? I see Spark version 1.6.3. Don't we have 2.0.0 or upper version? Do you've sample build.sbt and assembly.sbt files for Spark Cassandra connector?
I would need Spark 2.0.0 or upper to to do implicit conversion of RDD to DF to load data into Cassandra.
We don't have any sample build.sbt and assembly.sbt files for Spark Cassandra connector for now.
I am not able to access Cassandra or MongoDB using client tool like DataStax or RoboMongo from my desktop. Please let me know what I need to do?
Can I load data into itversity lab Mongo/Cassandra using my local spark ?
Please open firewall for Cassandra & MongoDB ports. This should solve the issue. No secure information is there in those practice databases.
Otherwise grant access to these server for SSH or provide private key.
Because of security issues, we might not be able to open the ports outside. Please use the procedure "Creating Tunneling to the gateway" and see if you are able to access them.
Its giving me permission denied for both Mongo & Cassandra hosts. Please look into this.
Can you share the steps on how are you trying to connect?
Is the access is not setup properly? Can you please update as soon as you can?
Do you have any updates? MySQL port is open but what is security issue if Cassandra & Mongo ports are open? SSH connection is okay with me but for that I need an access to the box. Please help.
@vinodnerella, can you ask some one to look into issue and close it in timely manner?
Your gateway is gw02 not gw01. You can get the gateway details from your Dashboard once you login to labs.itversity.com.
Can you try connecting to that and let us know?
Is Cassandra & MongoDB available on gw02.itverversity.com? If yes, what ports both DBs are listening?
Nothing is available on gw02. Please resolve the issue with proper access setup or firewall port opening. There should not be anything so sensitive in development/lab environment so that ports can't be open.
Please find the screenshots which will help you to find MongoDB and Cassandra from gw02.


If you have found a lower price on a competitor's site within 10 days of your order (the price include the shipping fee), please Email us the website and tell us the details to request your refund.
(a)You must first purchase the item on Elmontyouthsoccer.ru before requesting your Price Guarantee. You must file your claim within 10 days of purchase.
(b)The item must be currently available in the same size, style, model and color that it is on Elmontyouthsoccer.ru, and be in stock and available for purchase on the competitor's Web site.
(c)The competitor's price must include the shipping fee.
(d)Special promotions such as rebates, gift certificates and "buy one, get one" offers are not eligible. Price matches will not be honored on clearance, close out, inventory reduction or timed sales.
(e)Guarantee applies to Web sites only, must not be an auction site, a discount site (including but not limited to www.overstock.com, www.tjmax.com, www.qvc.com, www.woot.com, www.marshallsonline.com), a search engine or a membership club. The Price Guarantee means we will refund you 100% of the difference between the Elmontyouthsoccer.ru price (less any promotional discounts or coupons and excluding shipping and tax) and the competitor's price.
There is a limit of one guarantee claim per item, per customer.
Guarantee only applies to purchases for personal use. Guarantee does not apply to purchases for resale or commercial purposes. Emontyouthsoccer reserves the right to terminate this price guarantee policy or deny a price match at any time due to abuse or any other unforeseen circumstances.


text:
The Attachments Energy Rating Council to Partner with Baltimore Gas and Electric to Launch Nation's First Smart Shade Pilot Program
The Attachments Energy Rating Council to Partner with Baltimore Gas and Electric to Launch Nation's First Smart Shade Pilot Program
NEW YORK (December 1, 2020) – The Attachments Energy Rating Council (AERC) is partnering with Baltimore Gas and Electric (BGE), industry-leading custom window treatment manufacturer, Hunter Douglas, and roller shade manufacturer, Rollease Acmeda and fabricator partner Illinois Window Shades, on the nation's first smart shade pilot program. The BGE Smart Shade Pilot Program offers participating customers up to ten AERC rated and certified automated (smart) shades from Hunter Douglas in ten homes and a max of ten, soon-to-be AERC rated and certified, Rollease Acmeda automated roller shades in up to five homes. The overall goal is to be able to test the energy savings that customers can realize from automated window shades. AERC developed the automated shades' operation schedule, which will serve as the guiding force behind the program so that participating customers may maximize the energy savings opportunity of the shades in their homes. The shades will be installed on west- and south-facing windows, and programmed to a summer and winter schedule for energy savings.
"AERC is excited to lend our research and knowledge about automation and certified shading toward this partnership with BGE," said Ralph Vasami, Executive Director of AERC. "AERC and our members look forward to working with other utilities to continue promoting the energy-saving benefits of window attachments and smart-home systems."
Further, research from AERC conducted in partnership with the Department of Energy, Lawrence Berkeley National Lab, Pacific Northwest National Lab and Oak Ridge National Lab have demonstrated that window attachment products can improve energy performance at the window. Automated shades that operate according to a specific operation schedule may further improve energy savings.
AERC has rated and certified the Hunter Douglas cellular shades and cellular roller shades that participants in the pilot will receive, and the roller shades from Rollease Acmeda are in the AERC certification pipeline, likely to be fully rated and certified by early 2021. Cellular shades and cellular roller shades are designed to help keep homes warmer in the winter and cooler in the summer, and these automated shades can integrate with smart home automation systems. In addition to providing the automated shades, professional installation and automation scheduling is provided by a Hunter Douglas installer at no additional cost to the consumer. Similarly, the Rollease Acmeda roller shades operate in a comparable fashion; offering improved energy performance to windows and integrating with smart home systems to ensure complete automation.
"The pilot program's main goal is to build on the research from AERC and to provide critical real-world data on the energy impact of automated shades on home heating and cooling systems," said Amey Bayes, a representative from BGE. "We want to measure the energy savings that smart shades can capture, while also helping customers enhance the look and comfort of their home environments."
BGE expects to release results at the conclusion of this one-year study.
For more information on the energy savings of window attachments contact Danielle Meyer, [email protected].


text:
Setting The Record Straight: Debunking The Myth Of The "Kill Zone"
Anti-tech crusaders too often cry "kill zone" — despite clear evidence that today's tech climate encourages startup activity, rather than stifling it. The myth of the "kill zone" alleges that due to an overly concentrated tech market, venture capital funding is stifled for startups and small businesses. The theory holds that leading tech services often replicate the products of their smaller competitors in an effort to shut them out from their respective industries. The reality is that the global venture-investing market is at record levels: U.S. companies attracted more than $130 billion in financing in 2018 — the highest sum in a decade. The facts are clear: it has never been easier for new firms to autonomously enter and compete. Check out the research below:
—For startups entering new arenas, "kill zones" are not a concern — and acquisition is often a goal.
—Arguments that leading tech services create "kill zones" through data accumulation are unfounded.
—The current venture capital investment climate is thriving, encouraging innovation among startups and ultimately benefiting consumers.
As noted by Jim Pethokoukis of the American Enterprise Institute, future acquisition is often a goal for founders, as it allows them to continue expanding on other growth opportunities. "For founders, future acquisition is often "the goal. Then the entrepreneur can go on to start another firm or become an investor in other aspirational startups working on risky new ideas. Same goes for the investors in the acquired firm. What's more, these purchases are often "acquisition–by–hire" situations where the prize is talent rather than the Next Big Thing. And when an upstart firm has a valuable idea, acquisition can be the fastest way for it to get to users."
"Killer acquisitions" are often beneficial for both the startup and the consumer, highlights Will Rinehart, Senior Research Fellow at the Center for Growth and Opportunity. "The evidence of a kill zone in the tech industry is thin." "The vast majority of acquisitions by large tech companies are made either for the tech or the talent of the target company, not to stifle a future competitor. Both parties benefit from these deals. Startups often have great ideas but lack the technical and marketing resources to bring a product to a wider audience. Large companies, on the other hand, have the resources and consumer base for a new product but often lack innovative ideas. Yet, the true benefactors of these deals are consumers who can now choose an innovative product that may not have existed otherwise."
Rapid growth in the tech sector directly defies the "kill zone" argument and benefits consumers, as noted by Keith Hylton of Boston University. "Apple's App Store contains roughly 2 million apps at present. Between 2008 and 2018, the number of apps in Apple's store increased from 500 to 2 million. This huge rate of growth, 4,000 percent over ten years, is hard to square with the assertion that entry is difficult to encourage within a kill zone."
"When the platform owner replicates a particular innovative function and integrates the function into the platform, it enhances the entire platform, which is itself a form of innovation with benefits to consumers. The benefits to consumers are necessarily greater with replication and integration of a productive functionality across the entire platform than with the functionality residing exclusively within a stand-alone application residing on the platform."
Data alone does not lead to a sustainable competitive moat, meaning that incumbents are not untouchable simply due to data possession, notes Greg Back, Free Sky Capital Managing Member. "Data is not destiny. Particularly if you're on the early-stage side, company building is messy. You've got to get so many things right, whether it be people or infrastructure or hiring or fundraising. So what I think what we saw there was a missed opportunity, from an execution standpoint, for MySpace to have evolved into Facebook. Again, it's easy for us to say 'this company has this pool of data, they're going to dominate this space,' but often times this doesn't happen because there are so many other things required to be successful."
Due to data's diminishing returns, firm defensibility stems from innovation rather than volume, note Martin Casado and Peter Lauten of Andreessen Horowitz. "Yet even with scale effects, our observation is that data is rarely a strong enough moat. Unlike traditional economies of scale, where the economics of fixed, upfront investment can get increasingly favorable with scale over time, the exact opposite dynamic often plays out with data scale effects: The cost of adding unique data to your corpus may actually go up, while the value of incremental data goes down!"
"Of course, the point beyond which the data scale effect diminishes varies by domain. But regardless of exactly when this happens, the ultimate outcome is often the same: the ability to stay ahead of the pack tends to slow down, not speed up, with data scale. Instead of getting stronger, the defensible moat erodes as the data corpus grows and the competition races to catch up."
"Startups are nourished" in today's tech sector, notes Ed Black in RealClearMarkets. "The mythical 'kill-zone,' where investors won't invest because of successful incumbent companies, doesn't exist; startups are nourished, not suffocated, in the current venture capital climate. Actually, VC investment in the tech sector is at a 20-year high, performing exceptionally well compared to venture capital investment in other industries. Investor appetite for these high-risk, high-reward companies clearly shows the dynamism of the tech sector."
Today's thriving venture capital investment rejects the conventional "kill zone" wisdom, according to Jim Pethokoukis of the American Enterprise Institute. "Indeed, given all the many areas of interest to Big Tech, one might think there would also be so many kill zones that few startups would get funded. Yet VC investment in the US reached $136.5 billion last year, second only to 2018's $140 billion."
acquisitionsCompetition In TechConcentrationinnovation