text:
\chapter{Basic Guide to DASHMM}
\label{ch:basic}

In this chapter, the basic interface to DASHMM will be
covered. Generally speaking, the basic user interface to DASHMM is
anything needed to employ the provided methods and kernels in
applications. For instructions on how to define and use new methods and
kernels, please see chapter~\ref{ch:advanced}.

One of the goals of DASHMM is to make it easy to use multipole
methods, and so the basic interface targets ease-of-use. Additionally,
the library aims to make it easy to perform \emph{parallel} multiple
method computations. However, the advanced dynamic techniques
that DASHMM employs are not simple to use directly, so the library is
constructed in a way that allows the user to get the benefits of
parallel execution without having to write the parallel code
themselves. That being said, it can be useful to have a sense of what
underlies the conceptual framework of DAHSMM. And so, the following
section will cover these concepts at a level that might be relevant
for basic use of the library. More details can be found in
Chapter~\ref{ch:advanced}.

\section{DASHMM concepts}

This section covers both the conceptual framework of the multipole
methods supported by DASHMM and the parallelization of those
methods. More explanation of DASHMM's conceptual framework can be
found in the following chapter, or in the code paper.

\subsection{Multipole method abstractions}

DASHMM is a templated library allowing for the description of a
general set of multipole methods with a small set of template
parameters. To use DASHMM, one must specify four types: the {\it Source}
data type, the {\it Target} data type, the {\it Expansion} type and
the {\it Method} type. Within certain limits, each of these can be
varied independently, for example, a given expansion might be used
with a number of methods, allowing users to easily experiment and
select the method that most meets their needs.

DASHMM includes a number of built-in expansion and method types, which are
described in sections \ref{sec:bi-met} and \ref{sec:bi-exp}.

\subsubsection{Source}

The Source type gives the structure of the source point data. There
are few requirements on the Source type. Typically the minimum
requirements are the position and charge of the source. The \texttt{position}
is of type \texttt{Point} and the \texttt{charge} is of type
\texttt{double}. For example, the following is a minimal Source type that
works with every expansion provided with DASHMM.

\begin{lstlisting}[frame=]
struct SourceData {
  Point position;
  double charge;
};
\end{lstlisting}3

\noindent Beyond these required members, anything might be added to a Source
type. This allows the user to associate application specific data to
the sources in the evaluation.

The Source records will be copied across the network, and so if the type is
non-trivial, the user of DASHMM must provide a subclass of \texttt{Serializer}
to handle the serialization and deserialization of Source objects. See
section \ref{sec:serializer} for details.

\subsubsection{Target}

The Target type gives the structure of the target point data. There
are similarly few requirements on the Target type. Each Target type
will need a \texttt{position} of type \texttt{Point} and a member to store
the result. The details on the exact requirements can be found in the
documentation of the individual expansions below. Typically this is a
member \texttt{phi} of type \texttt{std::complex<double>}, which has been
aliased as \texttt{dcomplex\_t} in DASHMM. So the following would work
for many of the included DASHMM expansions:

\begin{lstlisting}[frame=]
struct TargetData {
  Point position;
  std::complex<double> phi;
};
\end{lstlisting}

\noindent Beyond the required members, anything might be added to the Target
type. A typical choice is an identifier of some kind because DASHMM
evaluations will sort the input data to suit the parallel computation,
and points can then be identified after the computation.

The Target records will be copied across the network, and so if the type is
non-trivial, the user of DASHMM must provide a subclass of \texttt{Serializer}
to handle the serialization and deserialization of Target objects. See
section \ref{sec:serializer} for details.

\subsubsection{Expansion}

The particular potential or interaction that is being computed with
the multipole moment is called the kernel. For example, the Laplace
kernel is the traditional potential from electrostatics or Newtonian
gravitation. In DASHMM, kernels are not represented directly. Instead,
Expansions are created that implement the needed operations for the
particular kernel. The distinction is that there are often multiple
ways to expand a given kernel to be used in a multipole method
computation. Each Expansion represents a way (or a closely related set
of ways) that the potential is expanded into an approximation.

The Expansion type is a template type over two parameters, the Source
and Target types being employed. It is the Expansion type that places
restrictions on the Source and Target types; the Expansion requires
certain data to compute from (taken from the Sources), and it will
produce certain data (into the Targets). This allows the Expansion to
operate on any data that meets its requirements, meaning that a very
general set of source and target data types can be supported.

For details on creating user-defined Expansions, please see
chapter~\ref{ch:advanced}.

\subsubsection{Method}

The final major abstraction in DASHMM is the Method. This type
specifies how the Expansion is used on the provided Source data to
compute the interaction at the locations specified in the Target
data. The Method is responsible for connecting the various Expansions
representing the hierarchically subdivided set of Source and Target
locations with the appropriate operations provided by the
Expansion. It is the Method that allows one to perform both a
Barnes-Hut computation as well as the Fast Multipole Method.

The Method type is a template over three types: the Source, the Target
and the Expansion. For the Method, it is not important exactly how the
various operations are implemented, but only that they are
implemented. The details of the Expansion, and thus the central
details of the particular interaction being studied, are hidden and
unimportant to the method.

Despite this generality, it is possible to create a Method that only
works for certain expansions. Indeed, included in DASHMM is the
\texttt{LaplaceCOM} expansion, which does not provide implementations for
all of the operations needed by the \texttt{FMM} or \texttt{FMM97}
methods. This is intentional, as the style of expansion in \texttt{LaplaceCOM}
is not terribly well suited to the Fast Multipole
Method. Nevertheless, great flexibility and generality is possible
with DASHMM.

For details on creating user-defined Methods, please see
chapter~\ref{ch:advanced}.

\subsection{Parallelization abstractions}

DASHMM uses the advanced runtime system HPX-5 for its
parallelization. HPX-5 provides a number of features that allow DASHMM
to naturally express the parallelism and data dependence of the
computation directly in programming constructs. However, the
flexibility of HPX-5 comes with a significant amount of effort to
learn the system. One major goal of DAHSMM is to get the benefits of
the dynamic adaptive techniques enabled by HPX-5 without the end-user
having to write directly to HPX-5 constructs, and DASHMM is successful
in this regard. It is, nevertheless, useful to have some notion of a
few concepts from HPX-5 for the basic use of DASHMM. For more details
on HPX-5, please visit \url{https://hpx.crest.iu.edu/}. For more
details on the use of HPX-5 in DASHMM, please see the advanced use
guide in the next chapter, or the code paper.

\subsubsection{PGAS}

HPX-5 provides a partitioned global address space (PGAS) that DASHMM uses for
the data during the computation.

That the address space is partitioned means that though the addresses are
unified into a global space, each byte of the global address space is served
by the physical memory on one particular locality of the system. Further,
the mapping from global address to physical address is fixed. A locality
is similar to the concept of a rank in an MPI program. That the address space
is global means that there is a single virtual address space allowing any
locality to refer to data, even if that data is not stored in the same physical
memory of the referring locality.

Practically speaking for users of DASHMM, the important part of the global
address space provided by HPX-5, as used by DASHMM, is that it is partitioned.
Each locality will have direct access to a portion of the data, and
indirect access to all of it.

\subsubsection{Execution model}

The execution model for a DASHMM program is one very similar to the SPMD model.
The program written to use DASHMM will be run on every locality in the allocated
resources. And at certain points, the execution will be handed off to DASHMM and
ultimately HPX-5 by making DASHMM library calls. Inside DASHMM, the execution is
very dynamic and is formed of a large number of small, interdependent tasks.
This complication, however, is hidden from the user. Instead, DASHMM presents
an interface where each locality participates in collective calls, with each
presenting possibly different data to the DASHMM library. So in many ways,
using basic DASHMM will be very similar to using MPI.

\section{Basic types}

DASHMM defines a number of basic types that are used throughout the system,
and which might be needed by users of the library.

\subsection{\texttt{ReturnCode}}

DASHMM calls will return values of the type \texttt{ReturnCode} where it is
reasonable to do so. The possible values are: \texttt{kSuccess}, \texttt{kRuntimeError}, \texttt{kIncompatible},
\texttt{kAllocationError}, \texttt{kInitError}, \texttt{kFiniError} and
\texttt{kDomainError}. See specific
library calls for cases where each might be returned.

\subsection{\texttt{dcomplex\_t}}
Many kernels return potential values that are complex numbers. DASHMM provides
\texttt{dcomplex\_t} as an alias to \texttt{std::complex<double>}.

\subsection{\texttt{Point}}
The \texttt{Point} class is used to represent locations in three dimensional
space. \texttt{Point} is expected for giving the locations of sources and
targets. It has the following members and relevant non-member operations:

\begin{lstlisting}
Point::Point(double x = 0, double y = 0, double z = 0)
\end{lstlisting}

\noindent Construct a point from a given set of coordinates. This can also be
used to default construct a point.

\begin{lstlisting}
Point::Point(double *arr)
\end{lstlisting}

\noindent  Construct a point from a C-style array. It is important that
\texttt{arr} should contain at least three members.

\begin{lstlisting}
Point::Point(const Point &pt)
\end{lstlisting}

\noindent Copy construct a point.

\begin{lstlisting}
Point Point::scale(double c) const
\end{lstlisting}

\noindent Return a point whose coordinates have  all been scaled by the factor
\texttt{c}.

\begin{lstlisting}
double Point::operator[](size_t i) const
\end{lstlisting}

\noindent Indexing access to the coordinates of the point. \texttt{i} must be
in the range $[0,2]$.

\begin{lstlisting}
double Point::x() const
\end{lstlisting}

\noindent Return the x coordinate of the point.

\begin{lstlisting}
double Point::y() const
\end{lstlisting}

\noindent Return the y coordinate of the point.

\begin{lstlisting}
double Point::z() const
\end{lstlisting}

\noindent Return the z coordinate of the point.

\begin{lstlisting}
double Point::norm() const
\end{lstlisting}

\noindent Return the 2-norm of the point.

\begin{lstlisting}
void Point::lower_bound(const Point &other)
\end{lstlisting}

\noindent This computes the lowest coordinate in each direction of this point
and \texttt{other} and sets this point's coordinate to that value.

\begin{lstlisting}
void Point::upper_bound(const Point &other)
\end{lstlisting}

\noindent This computes the highest coordinate in each direction of this point
and \texttt{other} and sets this point's coordinates to that value.

\begin{lstlisting}
double point_dot(const Point &left, const Point &right)
\end{lstlisting}

\noindent Treat the points as if they are vectors and take their dot product.

\begin{lstlisting}
Point point_add(const Point &left, const Point &right)
\end{lstlisting}

\noindent Perform a  component-wise addition of \texttt{left} and \texttt{right}
and return a point with the result.

\begin{lstlisting}
Point point_sub(const Point &left, const Point &right)
\end{lstlisting}

\noindent Perform a component-wise subtraction of \texttt{right} from
\texttt{left} and return a point with the result.

\section{Initializing DASHMM}

DASHMM must be initialized and finalized to be used. There are some DASHMM
operations that must only occur before initialization, and some that can only
occur after initialization. All DASHMM operations must occur before the
library is finalized.

\begin{lstlisting}
ReturnCode init(int *argc, char ***argv)
\end{lstlisting}

\noindent Initialize the runtime system supporting DASHMM and allocate any
resources needed by DASHMM. The addresses of the command line arguments must be
provided as the behavior of HPX-5 can be controlled by these arguments. Any
arguments dealing with HPX-5 directly will be removed and \texttt{argc} and
\texttt{argv} will be updated accordingly.

\texttt{init()} returns \texttt{kSuccess} if the system is successfully
started, and \texttt{kRuntimeError} otherwise. If \texttt{init()} returns
\texttt{kRuntimeError} all subsequent calls to DASHMM will have undefined
behavior.

All other DASHMM library calls must occur after \texttt{init()}. However, some
DASHMM related objects must be constructed before the call to \texttt{init()}.
See below for details.

This is a collective call; all localities must participate.

\begin{lstlisting}
ReturnCode finalize()
\end{lstlisting}

\noindent This will free any DASHMM specific resources and shut down the
runtime system. No other calls to DASHMM must occur after the call to
\texttt{finalize()}. This is a collective call; all localities must
participate.

\section{SPMD utilities}

DASHMM provides a small number of traditional SPMD utilities to make certain
things simpler.

\begin{lstlisting}
int get_my_rank()
\end{lstlisting}

\noindent This returns the rank of the calling locality.

\begin{lstlisting}
int get_num_ranks()
\end{lstlisting}

\noindent This returns the number of ranks available.

\begin{lstlisting}
void broadcast(T *value)
\end{lstlisting}

\noindent This performs a broadcast of the given value at rank 0 to all
other ranks. This is a template over the type \texttt{T}. This is a
collective operation. Each rank must provide the address of a type
\texttt{T} object. For rank 0, this will provide  the address of the value
to share; for all other ranks this provides the address into which the
value broadcast from rank 0 will be stored. The type \texttt{T} must be
trivially copyable.

\section{Evaluation}

The central object in any DASHMM evaluation is the \texttt{Evaluator} object.
This object not only manages the registration of certain actions with the
runtime system, but also provides the interface to performing the multipole
method evaluation.

The \texttt{Evaluator} object is a template over four types: the source type,
the target type, the expansion type and the method type. The \texttt{Evaluator}
for a given set of types must be declared before the call to \texttt{init()}.
For example:

\begin{lstlisting}[frame=]
dashmm::Evaluator<Source, Target, dashmm::Laplace, dashmm::FMM97> eval{};
\end{lstlisting}

\noindent would be an \texttt{Evaluator} for the Laplace kernel using the
advanced FMM method for two user-defined types
\texttt{Source} and \texttt{Target} implementing the data that the user
requires of the source and target points.

Specifying the full type of the evaluator will cause the template to expand
out all of the needed actions to actually implement the evaluation using
HPX-5, and will also register those actions with HPX-5.

\begin{lstlisting}
ReturnCode Evaluator::evaluate(
    const Array<Source> &sources,
    const Array<Target> &targets,
    int refinement_limit,
    const Method<Source, Target, Expansion<Source, Target>> *method,
    int n_digits,
    const std::vector<double> *kernelparams)
\end{lstlisting}

\noindent Perform a multipole method evaluation. The arguments to this method
are as follows:

\begin{itemize}
\item \texttt{const Array<Source> \&sources}: the \texttt{Array} containing
  the source data.
\item \texttt{const Array<Target> \&targets}: the \texttt{Array} containing
  the target data.
\item \texttt{int refinement\_limit}: the refinement limit of the tree. The
  sources and  targets will be placed into a hierarchical partitioning of
  space. This partitioning will end when there are fewer sources or targets than
  the supplied refinement limit in the region under consideration.
\item \texttt{const Method<Source, Target, Expansion<Source, Target>> *method}:
  an instance of the method to use for the evaluation. A few methods require
  parameters at construction, so this is passed in to provide those parameters
  to the evaluation.
\item \texttt{int n\_digits}: the accuracy parameter for the \texttt{Expansion}
  in use. This is often the number of digits of accuracy required.
\item \texttt{const std::vector<double> *kernelparams}: the parameters for the
  kernel evaluation. These are those quantities that are constant for each use
  of the particular expansion. See the individual expansions for details about
  what needs to be provided.
\end{itemize}

Note that during evaluation, the records in the source and target arrays may be
sorted. So a separate identifier should be added to the source and target types
if the identity of the sources or targets needs to be tracked. Other than the
sorting, the only change to the data in the target array will be the output
potential or other field value (as specified by the chosen expansion). The
only change to the source array beyond the sorting will only occur in the case
that the source and target arrays are the same, in which case the previous
comment about the targets also applies to the source.

This is a collective call; all localities must participate.

The possible return values are \texttt{kSuccess} when there is no problem, and
\texttt{kRuntimeError} when there is a problem with the execution.

\section{DASHMM Array}

DASHMM provides an array construct that represents a distributed collection of
records. The \texttt{Array} object is a template
type over the record. For basic use of \texttt{Array} the record type should
be trivially copyable. For more advanced record types, see
section~\ref{sec:serializer}. When an \texttt{Array} is created, as in

\begin{lstlisting}[frame=]
Array<T> source_data{};
\end{lstlisting}

\noindent no memory is yet allocated for the array. To allocate the
memory that will serve the array, one must use \texttt{allocate()}.
Array objects can be thought of as a traditional array,
but which is broken into one part for each locality in the system. The parts,
or \emph{segments} of the array, could have different lengths. Some segments can
even have no records, meaning that a given locality does not have a share of
the data.

\texttt{Array} objects are intended to be employed in user code, and so most
members of the interface are SPMD in nature, and are collective operations.

\begin{lstlisting}
Array<T>::Array()
\end{lstlisting}

\noindent \texttt{Array} objects have a single constructor, which does not
allocate any
memory. When default constructed, an \texttt{Array} will be invalid (see
\texttt{valid()} below).

This is not a collective operation. To use an array, each locality must create
an \texttt{Array} object. The individual \texttt{Array} objects will be bound
into a unified object using \texttt{allocate()}.

\begin{lstlisting}
bool Array<T>::valid() const
\end{lstlisting}

\noindent This returns if the array is valid, that is, it refers to some global
memory.
When initially constructed, an \texttt{Array} will be invalid, and can only be
made valid by allocating memory for the records (see \texttt{allocate()}).

This method is not collective.

\begin{lstlisting}
size_t Array<T>::count() const
\end{lstlisting}

\noindent This returns the number of records in the segment of the array on the
calling
locality. This is a collective call, and it is an error to call this method
on an invalid array. Each calling rank will receive a different result from
this method.

\begin{lstlisting}
size_t Array<T>::length() const
\end{lstlisting}

\noindent This returns the total length of the entire array. The result of
\texttt{length()} is equal to the sum of the results of \texttt{count()} from
each rank. This is a collective call, and it is an error to call this method
on an invalid array.

\begin{lstlisting}
ReturnCode Array<T>::allocate(size_t record_count, T *segment = nullptr)
\end{lstlisting}

\noindent This allocates the memory to serve an array with the given
counts on each locality. After this call the array will be valid unless
there is an error, which will be indicated by the return code. Possible
return values are: \texttt{kSuccess} if the array is successfully allocated;
\texttt{kDomainError} is the object already has an allocation;
\texttt{kAllocationError} is the global memory cannot be allocated; or
\texttt{kRuntimeError} if there is some error in the runtime.

This is a collective call. Each locality can provided a different number of
records to allocate via the \texttt{record\_count} parameter. The resulting
allocation will match the input of \texttt{allocate()}. A locality can
provide \texttt{0} as an argument, so long as at least one rank asks for a
non-zero number of records. For instance, one locality might allocate all of
the records for an array, or each locality might own a portion of the overall
records.

The optional second argument allows users to provide the data that will make
up the segments of the array. If this argument is \texttt{nullptr} (the
default), then DASHMM will allocate the memory for each segment. If this
argument is not null, then DASHMM will assume ownership of the provided
memory, and will build the resulting array with the provided data.
Providing an incorrect \texttt{record\_count} for this use case will cause
undefined behavior.

This will return \texttt{kDomainError} if the object already has an allocation,
\texttt{kAllocationError} if memory is unable to be allocated,
\texttt{kRuntimeError} if there is an error with the runtime system, or
\texttt{kSuccess} if the method completes without a problem.

\begin{lstlisting}
ReturnCode Array<T>::destroy()
\end{lstlisting}

\noindent This will destroy the memory allocated for this array object.
It is
an error to call this on an invalid array. This is a collective call. The
result of the method will be either \texttt{kRuntimeError} if there is an error
in the runtime, or \texttt{kSuccess} otherwise.

\begin{lstlisting}
ReturnCode Array<T>::get(size_t first, size_t last, T *out)
\end{lstlisting}

\noindent This method gets data from an array object, and places the requested
records into the buffer provided by \texttt{out}. The range of records that is
retrieved is specified by \texttt{first} (inclusive) and \texttt{last}
(exclusive).

This is a collective call. Each locality will provide a different range of
records, and a different local buffer into which the retrieved data will be
placed. \texttt{first} and \texttt{last} are given in terms of the segment of
the array on this locality. To discover the number of records on the calling
locality, use \texttt{count()}.

Note that this is a copy of the data; changes to the retrieved values
will not be reflected in the array object.

This method will return one of the following: \texttt{kRuntimeError} if there
is an error with the runtime; \texttt{kDomainError} if the provided index range
is inconsistent with the array object; or \texttt{kSuccess} otherwise.

{\bf NOTE: Are we going to deprecate this?}

\begin{lstlisting}
ReturnCode Array<T>::put(size_t first, size_t last, T *in)
\end{lstlisting}

\noindent This method puts data into an array object, copying the specified
records from the buffer provided by \texttt{in}. The range of records that is
copied is specified by \texttt{first} (inclusive) and \texttt{last} (exclusive).

This is a collective call. Each locality will provide a different range of
records, and a different local buffer from which the retrieved data will be
placed. \texttt{first} and \texttt{last} are given in terms of the segment of
the array on this locality. To discover the capacity of the array on the
calling locality, use \texttt{count()}.

Note that this places a copy of the records into the address space;
subsequent changes in the local data will not be reflected in the array object.

This method will return one of the following: \texttt{kRuntimeError} if there
is an error with the runtime; \texttt{kDomainError} if the provided index
range is inconsistent with the array object; or \texttt{kSuccess} otherwise.

{\bf NOTE: Are we going to deprecate this?}

\begin{lstlisting}
std::unique_ptr<T[]> Array<T>::collect()
\end{lstlisting}

\noindent This method collects all of the records in the array and returns a
new local allocation containing the records at locality 0. This is, largely
speaking, a convenience feature. Note that this will allocate a copy of the
entirety of the array on one locality.

This is a collective call. The returned smart pointer will only be valid on
locality zero. All other ranks will receive a null pointer.

Note that this provides a copy of the data; changes to the returned data will
not be reflected in the array object.

\begin{lstlisting}
T *Array<T>::segment(size_t &count)
\end{lstlisting}

\noindent This method provides direct access to the local segment of an array.
Because some DASHMM routines will modify the segments of arrays, the length
of the returned array data cannot be known ahead of time. So, in addition to
the segment's address, the number of records is returned via the \texttt{count}
parameter. It is possible that this will return \texttt{nullptr}, but this will
occur only when \texttt{count} is zero.

This is a collective call. Each rank will receive the address and count of its
own segment.

Note that this provides direct access to the data; changes to the data will
persist in the object.

Note that calls to some other DASHMM routines will invalidate the address
returned by this method.

\section{Array For Each Actions}

To avoid the round trip from and to the global address space via
\texttt{Array}'s \texttt{get()} and \texttt{put()} methods, one can make use
of the \texttt{ArrayForEachAction} type. This type specifies an action to be
performed on the records of an array. Then, together with a method of the
\texttt{Array} object, this allows for some computation to occur on the
records of an array.

This class is a template requiring two parameters: the type of records for
the array to which the action will be applied (hereafter \texttt{T}), and a
type specifying an environment to provide to the action (hereafter \texttt{E}).

The action is specified during construction of an object of type
\texttt{ArrayForEachAction}. This is provided as a function pointer, for a
function with a particular signature. So that DASHMM can use the action on
every locality, like the \texttt{Evaluator} object, any
\texttt{ArrayForEachAction} objects must be defined before \texttt{init()}
is called.

The function implementing the action must take three arguments, the first is
a \texttt{T *} giving the data on which to act, the second is a
\texttt{const size\_t} giving the number of records on which to act, and the
third is \texttt{const E *}, where \texttt{E} is the environment type of the
\texttt{ArrayForEachAction}. For example:

\begin{lstlisting}[frame=]
void update_position(T *data, const size_t count, const E *env) {
  for (size_t i = 0; i < count; ++i) {
    data[i].position += data[i].velocity * env->delta_t;
  }
}
\end{lstlisting}

\noindent is an action that might perform a position update for a
time-stepping code. It is important to note that the action implementation can
place requirements on the array record type \texttt{T}. In the previous
example, type \texttt{T} needs to have a member called \texttt{velocity}.
The first argument to the action will be a correctly offset pointer into a
contiguous chunk of records. The action should only assume that
\texttt{data[0]} through \texttt{data[count - 1]} are available for use.

\begin{lstlisting}
ArrayForEachAction<T, E>::map_function_t
\end{lstlisting}

\noindent This class aliases the type of function that can implement the
action. It is:

\begin{lstlisting}[frame=]
void (*)(T *, const size_t, const E *)
\end{lstlisting}

\begin{lstlisting}
ArrayForEachAction<T, E>::ArrayForEachAction(map_function_t f)
\end{lstlisting}

\noindent This constructs an array map action object. The provided function
pointer
will give the action that is performed on the array. To use an action,
the associated \texttt{ArrayForEachAction} must be defined before
\texttt{init()} is called. This object will register the needed actions with
the runtime system.

For any given action \texttt{f} only a single \texttt{ArrayForEachAction}
should be defined.

\begin{lstlisting}
ReturnCode Array<T>::forEach(ArrayForEachAction<T, E> &act, const E *env)
\end{lstlisting}

\noindent Once an \texttt{ArrayForEachAction} is defined, it can be used on a
specific array by calling that array's \texttt{map()} method. This will cause
the action represented by \texttt{act} to be applied on all entries of this
array. The action ultimately works on segments of the array. The environment,
\texttt{env} is provided unmodified to each segment.

This is a collective call. It is an error to call \texttt{forEach()} on an
invalid array.

\section{Built-in methods}
\label{sec:bi-met}

DASHMM includes a number of built-in methods that are ready to use for
problems: the Barnes-Hut method, two forms of the Fast Multipole Method and
a Direct summation method. These will be covered in detail below. To
successfully compile, all operations in the Expansion concept need to be
implemented for a given expansion. However, some methods only require a
subset of the full complement of operations. The built-in methods will be
covered below, in order of increasing complexity.

\subsection{\texttt{Direct}}

The \texttt{Direct} method is primarily intended for use as a comparison for
computing the exact result for a given set of sources. There is some
parallelism implemented for this method, so the execution time is reduced
somewhat. However, for realistic problem sizes, this method should be avoided.

The only operation that is used by the \texttt{Direct} method is
\texttt{StoT}, so any Expansion implementing that operator can be used with the
\texttt{Direct} method.

Please see the demo program \texttt{demo/basic} included with DASHMM for an
example use case of \texttt{Direct} and note that it is only applied to a very
small subset of the target locations.

Construction of \texttt{Direct} is simple, as it can only be default
constructed.

\subsection{\texttt{BH}}

The \texttt{BH} method implements the Barnes-Hut algorithm in the framework of
DASHMM. The implemented method uses the simple multipole acceptance criterion
parameterized by a single angle. If a multipole expansion is centered a
distance $R$ away from a given point and the multipole expansion is
associated with a region of size $D$ then the multipole expansion is used
only if $D/R < \theta_C$ for some critical angle $\theta_C$.

In practice, since the
hierarchical partitioning of the source and target locations in DASHMM produces
leaves that can contain multiple particles, the multipole acceptance criterion
is evaluated pessimistically: the radius is computed from the nearest possible
location in a given target tree node to the multipole in question. This will
tend to produce results with a slightly smaller error, with a slightly larger
execution time.

When creating a \texttt{BH} object, the opening angle $\theta_C$ is specified:

\begin{lstlisting}[frame=]
BH bh_method(0.6);
\end{lstlisting}

\noindent Generally the opening angle should be less than one. As the
angle approaches
zero, the method approaches the direct summation method. The critical angle
for a particular \texttt{BH} object can be accessed with
\texttt{double BH::theta()}, which returns the angle specified at creation
time. A default constructed \texttt{BH} uses a critical angle of \texttt{0}.

In addition to the direct contribution, \texttt{StoT}, to use \texttt{BH} an
expansion must also implement fully the following operations: \texttt{StoM},
\texttt{MtoM} and \texttt{MtoT}.

\subsection{\texttt{FMM}}

The \texttt{FMM} method implements the Fast Multipole Method in its original
form. To create an \texttt{FMM} method, no arguments are needed as the
decisions about which expansions to use in which situations are all made based
on fixed geometric considerations.

The following operations must have a full implementation in an expansion to be
used with \texttt{FMM}: \texttt{StoT}, \texttt{StoM}, \texttt{StoL},
\texttt{MtoM}, \texttt{MtoL}, \texttt{MtoT}, \texttt{LtoL}, and \texttt{LtoT}.

\subsection{\texttt{FMM97}}

The \texttt{FMM97} method implements the Fast Multipole Method in the form that
uses exponential expansions and the merge-and-shift technique. No parameters
are needed to construct an \texttt{FMM97} method.

The following operations must have a full implementation in an expansion to be
used with \texttt{FMM97}: \texttt{StoT}, \texttt{StoM}, \texttt{StoL},
\texttt{MtoM}, \texttt{MtoL}, \texttt{MtoT}, \texttt{LtoL},
\texttt{LtoT}, \texttt{MtoI}, \texttt{ItoI} and \texttt{ItoL}.

\section{Built-in expansions}
\label{sec:bi-exp}

DASHMM includes a number of built-in expansion that are ready to use for
applications. Each expansion will have a different set of implemented
operations which will restrict their use for certain methods. Further, each
expansion will place requirements on the source and target types used during
a DASHMM evaluation. These details will be covered for each expansion
below.

\subsection{\texttt{Laplace}}

The \texttt{Laplace} expansion is a spherical harmonic expansion of the Laplace
potential that is designed to handle sources with both signs of charge without
losing accuracy, unlike \texttt{LaplaceCOM} and \texttt{LaplaceCOMAcc} below.
This potential is scale-invariant, so there are no kernel parameters that are
needed in the call to \texttt{evaluate()}. However, calls to evaluate must
supply an accuracy parameter giving the number of digits of accuracy that are
requested.

Though this expansion is in principle compatible with every method included
with DASHMM, it is designed for the \texttt{FMM} and \texttt{FMM97} methods.

This expansion imposes the following restrictions on the source type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member of type \texttt{double} with the name \texttt{charge} must be
provided.

This expansion imposes the following restrictions on the target type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member of type \texttt{dcomplex\_t} with the name \texttt{potential}
must be provided.

\subsection{\texttt{Yukawa}}

The \texttt{Yukawa} expansion is a spherical harmonic expansion of the Yukawa
potential. This potential is scaling variant, so a single kernel parameter
must be provided to \texttt{evaluate()}. Calls to evaluate must also supply an
accuracy parameter that gives the number of digits of accuracy required.

Though this expansion is in principle compatible with every method included
with DASHMM, it is designed for the \texttt{FMM97} method.

This expansion imposes the following restrictions on the source type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member of
type \texttt{double} with the name \texttt{charge} must be provided.

This expansion imposes the following restrictions on the target type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member of type \texttt{dcomplex\_t} with the name \texttt{potential}
 must be provided.

\subsection{\texttt{Helmholtz}}

The \texttt{Helmholtz} expansion expands the Helmholtz potential in the
low-frequency regime using spherical harmonics, spherical Bessel and Hankel
functions. This potential is scaling variant, so a single kernel parameter
must be provided to \texttt{evaluate()}. Calls to evaluate must also supply an
accuracy parameter that gives the number of digits of accuracy required.

Though this expansion is designed for use with the \texttt{FMM97} method.

This expansion imposes the following restrictions on the source type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member of
type \texttt{double} with the name \texttt{charge} must be provided.

This expansion imposes the following restrictions on the target type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member of type \texttt{dcomplex\_t} with the name \texttt{potential}
 must be provided.

\subsection{\texttt{LaplaceCOM}}

The \texttt{LaplaceCOM} expansion is a center of mass expansion of the Laplace
potential. This form of the expansion extends to the quadrupole term, and
because of the choice of center has an identically zero dipole term. This
expansion can be used to compute the potential. See \texttt{LaplaceCOMAcc} for
an equivalent expansion that computes the acceleration. This potential is
scale-invariant so no kernel parameters are needed in the call to
\texttt{evaluate()}.
Further, the number of terms in the expansion is fixed, so the accuracy
parameter to \texttt{evaluate()} is ignored.

This expansion is only compatible with the \texttt{BH} or \texttt{Direct}
methods; it does not implement all the needed operations for use with
\texttt{FMM} or \texttt{FMM97}.

This expansion imposes the following restrictions on the source type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member of type \texttt{double} with the name \texttt{charge} must be
provided.

This expansion imposes the following restrictions on the target type: a
member of type \texttt{Point} with the name `position` must be provided; a
member of type \texttt{dcomplex\_t} with the name \texttt{potential} must be
provided.

\subsection{\texttt{LaplaceCOMAcc}}

The \texttt{LaplaceCOM} expansion is a center of mass expansion of the Laplace
potential. This form of the expansion extends to the quadrupole term, and
because of the choice of center has an identically zero dipole term. This
expansion can be used to compute the acceleration. See \texttt{LaplaceCOM} for
an equivalent expansion that computes the potential. This potential is
scale-invariant so no kernel parameters are needed in the call to
\texttt{evaluate()}. Further, the number of terms in the expansion is fixed,
so the accuracy parameter to \texttt{evaluate()} is ignored.

This expansion is only compatible with the \texttt{BH} or \texttt{Direct}
methods; it does not implement all the needed operations for use with
\texttt{FMM} or \texttt{FMM97}.

This expansion imposes the following restrictions on the source type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member of  type \texttt{double} with the name \texttt{charge} must be
provided.

This expansion imposes the following restrictions on the target type: a
member of type \texttt{Point} with the name \texttt{position} must be provided;
a member \texttt{double acceleration[3]} must be provided.

redpajama_set_name: RedPajamaGithub


text:
#include <ruby/ruby.h>

#include "rb_grpc_imports.generated.h"
#include "rb_server.h"

#include <grpc/grpc.h>
#include <grpc/grpc_security.h>
#include <grpc/support/atm.h>
#include <grpc/support/log.h>
#include "rb_byte_buffer.h"
#include "rb_call.h"
#include "rb_channel_args.h"
#include "rb_completion_queue.h"
#include "rb_grpc.h"
#include "rb_server_credentials.h"

/* grpc_rb_cServer is the ruby class that proxies grpc_server. */
static VALUE grpc_rb_cServer = Qnil;

/* id_at is the constructor method of the ruby standard Time class. */
static ID id_at;

/* id_insecure_server is used to indicate that a server is insecure */
static VALUE id_insecure_server;

/* grpc_rb_server wraps a grpc_server. */
typedef struct grpc_rb_server {
  /* The actual server */
  grpc_server* wrapped;
  grpc_completion_queue* queue;
  int shutdown_and_notify_done;
  int destroy_done;
} grpc_rb_server;

static void grpc_rb_server_maybe_shutdown_and_notify(grpc_rb_server* server,
                                                     gpr_timespec deadline) {
  grpc_event ev;
  void* tag = &ev;
  if (!server->shutdown_and_notify_done) {
    server->shutdown_and_notify_done = 1;
    if (server->wrapped != NULL) {
      grpc_server_shutdown_and_notify(server->wrapped, server->queue, tag);
      ev = rb_completion_queue_pluck(server->queue, tag, deadline, NULL);
      if (ev.type == GRPC_QUEUE_TIMEOUT) {
        grpc_server_cancel_all_calls(server->wrapped);
        ev = rb_completion_queue_pluck(
            server->queue, tag, gpr_inf_future(GPR_CLOCK_REALTIME), NULL);
      }
      if (ev.type != GRPC_OP_COMPLETE) {
        gpr_log(GPR_INFO,
                "GRPC_RUBY: bad grpc_server_shutdown_and_notify result:%d",
                ev.type);
      }
    }
  }
}

static void grpc_rb_server_maybe_destroy(grpc_rb_server* server) {
  // This can be started by app or implicitly by GC. Avoid a race between these.
  if (!server->destroy_done) {
    server->destroy_done = 1;
    if (server->wrapped != NULL) {
      grpc_server_destroy(server->wrapped);
      grpc_rb_completion_queue_destroy(server->queue);
      server->wrapped = NULL;
      server->queue = NULL;
    }
  }
}

/* Destroys server instances. */
static void grpc_rb_server_free(void* p) {
  grpc_rb_server* svr = NULL;
  gpr_timespec deadline;
  if (p == NULL) {
    return;
  };
  svr = (grpc_rb_server*)p;

  deadline = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME),
                          gpr_time_from_seconds(2, GPR_TIMESPAN));

  grpc_rb_server_maybe_shutdown_and_notify(svr, deadline);
  grpc_rb_server_maybe_destroy(svr);

  xfree(p);
}

static const rb_data_type_t grpc_rb_server_data_type = {
    "grpc_server",
    {GRPC_RB_GC_NOT_MARKED,
     grpc_rb_server_free,
     GRPC_RB_MEMSIZE_UNAVAILABLE,
     {NULL, NULL}},
    NULL,
    NULL,
#ifdef RUBY_TYPED_FREE_IMMEDIATELY
    /* It is unsafe to specify RUBY_TYPED_FREE_IMMEDIATELY because the free
     * function would block and we might want to unlock GVL
     * TODO(yugui) Unlock GVL?
     */
    0,
#endif
};

/* Allocates grpc_rb_server instances. */
static VALUE grpc_rb_server_alloc(VALUE cls) {
  grpc_rb_server* wrapper = ALLOC(grpc_rb_server);
  wrapper->wrapped = NULL;
  wrapper->destroy_done = 0;
  wrapper->shutdown_and_notify_done = 0;
  return TypedData_Wrap_Struct(cls, &grpc_rb_server_data_type, wrapper);
}

/*
  call-seq:
    server = Server.new({'arg1': 'value1'})

  Initializes server instances. */
static VALUE grpc_rb_server_init(VALUE self, VALUE channel_args) {
  grpc_completion_queue* cq = NULL;
  grpc_rb_server* wrapper = NULL;
  grpc_server* srv = NULL;
  grpc_channel_args args;
  MEMZERO(&args, grpc_channel_args, 1);

  grpc_ruby_once_init();

  cq = grpc_completion_queue_create_for_pluck(NULL);
  TypedData_Get_Struct(self, grpc_rb_server, &grpc_rb_server_data_type,
                       wrapper);
  grpc_rb_hash_convert_to_channel_args(channel_args, &args);
  srv = grpc_server_create(&args, NULL);

  if (args.args != NULL) {
    xfree(args.args); /* Allocated by grpc_rb_hash_convert_to_channel_args */
  }
  if (srv == NULL) {
    rb_raise(rb_eRuntimeError, "could not create a gRPC server, not sure why");
  }
  grpc_server_register_completion_queue(srv, cq, NULL);
  wrapper->wrapped = srv;
  wrapper->queue = cq;

  return self;
}

/* request_call_stack holds various values used by the
 * grpc_rb_server_request_call function */
typedef struct request_call_stack {
  grpc_call_details details;
  grpc_metadata_array md_ary;
} request_call_stack;

/* grpc_request_call_stack_init ensures the request_call_stack is properly
 * initialized */
static void grpc_request_call_stack_init(request_call_stack* st) {
  MEMZERO(st, request_call_stack, 1);
  grpc_metadata_array_init(&st->md_ary);
  grpc_call_details_init(&st->details);
}

/* grpc_request_call_stack_cleanup ensures the request_call_stack is properly
 * cleaned up */
static void grpc_request_call_stack_cleanup(request_call_stack* st) {
  grpc_metadata_array_destroy(&st->md_ary);
  grpc_call_details_destroy(&st->details);
}

/* call-seq:
   server.request_call

   Requests notification of a new call on a server. */
static VALUE grpc_rb_server_request_call(VALUE self) {
  grpc_rb_server* s = NULL;
  grpc_call* call = NULL;
  grpc_event ev;
  grpc_call_error err;
  request_call_stack st;
  VALUE result;
  void* tag = (void*)&st;
  grpc_completion_queue* call_queue =
      grpc_completion_queue_create_for_pluck(NULL);
  gpr_timespec deadline;

  TypedData_Get_Struct(self, grpc_rb_server, &grpc_rb_server_data_type, s);
  if (s->wrapped == NULL) {
    rb_raise(rb_eRuntimeError, "destroyed!");
    return Qnil;
  }
  grpc_request_call_stack_init(&st);
  /* call grpc_server_request_call, then wait for it to complete using
   * pluck_event */
  err = grpc_server_request_call(s->wrapped, &call, &st.details, &st.md_ary,
                                 call_queue, s->queue, tag);
  if (err != GRPC_CALL_OK) {
    grpc_request_call_stack_cleanup(&st);
    rb_raise(grpc_rb_eCallError,
             "grpc_server_request_call failed: %s (code=%d)",
             grpc_call_error_detail_of(err), err);
    return Qnil;
  }

  ev = rb_completion_queue_pluck(s->queue, tag,
                                 gpr_inf_future(GPR_CLOCK_REALTIME), NULL);
  if (!ev.success) {
    grpc_request_call_stack_cleanup(&st);
    rb_raise(grpc_rb_eCallError, "request_call completion failed");
    return Qnil;
  }

  /* build the NewServerRpc struct result */
  deadline = gpr_convert_clock_type(st.details.deadline, GPR_CLOCK_REALTIME);
  result = rb_struct_new(
      grpc_rb_sNewServerRpc, grpc_rb_slice_to_ruby_string(st.details.method),
      grpc_rb_slice_to_ruby_string(st.details.host),
      rb_funcall(rb_cTime, id_at, 2, INT2NUM(deadline.tv_sec),
                 INT2NUM(deadline.tv_nsec / 1000)),
      grpc_rb_md_ary_to_h(&st.md_ary), grpc_rb_wrap_call(call, call_queue),
      NULL);
  grpc_request_call_stack_cleanup(&st);
  return result;
}

static VALUE grpc_rb_server_start(VALUE self) {
  grpc_rb_server* s = NULL;
  TypedData_Get_Struct(self, grpc_rb_server, &grpc_rb_server_data_type, s);
  if (s->wrapped == NULL) {
    rb_raise(rb_eRuntimeError, "destroyed!");
  } else {
    grpc_server_start(s->wrapped);
  }
  return Qnil;
}

static VALUE grpc_rb_server_shutdown_and_notify(VALUE self, VALUE timeout) {
  gpr_timespec deadline;
  grpc_rb_server* s = NULL;

  TypedData_Get_Struct(self, grpc_rb_server, &grpc_rb_server_data_type, s);
  if (TYPE(timeout) == T_NIL) {
    deadline = gpr_inf_future(GPR_CLOCK_REALTIME);
  } else {
    deadline = grpc_rb_time_timeval(timeout, /* absolute time*/ 0);
  }

  grpc_rb_server_maybe_shutdown_and_notify(s, deadline);

  return Qnil;
}

/*
  call-seq:
    server = Server.new({'arg1': 'value1'})
    ... // do stuff with server
    ...
    ... // initiate server shutdown
    server.shutdown_and_notify(timeout)
    ... // to shutdown the server
    server.destroy()

  Destroys server instances. */
static VALUE grpc_rb_server_destroy(VALUE self) {
  grpc_rb_server* s = NULL;
  TypedData_Get_Struct(self, grpc_rb_server, &grpc_rb_server_data_type, s);
  grpc_rb_server_maybe_destroy(s);
  return Qnil;
}

/*
  call-seq:
    // insecure port
    insecure_server = Server.new(cq, {'arg1': 'value1'})
    insecure_server.add_http2_port('mydomain:50051', :this_port_is_insecure)

    // secure port
    server_creds = ...
    secure_server = Server.new(cq, {'arg1': 'value1'})
    secure_server.add_http_port('mydomain:50051', server_creds)

    Adds a http2 port to server */
static VALUE grpc_rb_server_add_http2_port(VALUE self, VALUE port,
                                           VALUE rb_creds) {
  grpc_rb_server* s = NULL;
  grpc_server_credentials* creds = NULL;
  int recvd_port = 0;

  TypedData_Get_Struct(self, grpc_rb_server, &grpc_rb_server_data_type, s);
  if (s->wrapped == NULL) {
    rb_raise(rb_eRuntimeError, "destroyed!");
    return Qnil;
  } else if (TYPE(rb_creds) == T_SYMBOL) {
    if (id_insecure_server != SYM2ID(rb_creds)) {
      rb_raise(rb_eTypeError, "bad creds symbol, want :this_port_is_insecure");
      return Qnil;
    }
    recvd_port =
        grpc_server_add_insecure_http2_port(s->wrapped, StringValueCStr(port));
    if (recvd_port == 0) {
      rb_raise(rb_eRuntimeError,
               "could not add port %s to server, not sure why",
               StringValueCStr(port));
    }
  } else {
    creds = grpc_rb_get_wrapped_server_credentials(rb_creds);
    recvd_port = grpc_server_add_secure_http2_port(
        s->wrapped, StringValueCStr(port), creds);
    if (recvd_port == 0) {
      rb_raise(rb_eRuntimeError,
               "could not add secure port %s to server, not sure why",
               StringValueCStr(port));
    }
  }
  return INT2NUM(recvd_port);
}

void Init_grpc_server() {
  grpc_rb_cServer =
      rb_define_class_under(grpc_rb_mGrpcCore, "Server", rb_cObject);

  /* Allocates an object managed by the ruby runtime */
  rb_define_alloc_func(grpc_rb_cServer, grpc_rb_server_alloc);

  /* Provides a ruby constructor and support for dup/clone. */
  rb_define_method(grpc_rb_cServer, "initialize", grpc_rb_server_init, 1);
  rb_define_method(grpc_rb_cServer, "initialize_copy", grpc_rb_cannot_init_copy,
                   1);

  /* Add the server methods. */
  rb_define_method(grpc_rb_cServer, "request_call", grpc_rb_server_request_call,
                   0);
  rb_define_method(grpc_rb_cServer, "start", grpc_rb_server_start, 0);
  rb_define_method(grpc_rb_cServer, "shutdown_and_notify",
                   grpc_rb_server_shutdown_and_notify, 1);
  rb_define_method(grpc_rb_cServer, "destroy", grpc_rb_server_destroy, 0);
  rb_define_alias(grpc_rb_cServer, "close", "destroy");
  rb_define_method(grpc_rb_cServer, "add_http2_port",
                   grpc_rb_server_add_http2_port, 2);
  id_at = rb_intern("at");
  id_insecure_server = rb_intern("this_port_is_insecure");
}

/* Gets the wrapped server from the ruby wrapper */
grpc_server* grpc_rb_get_wrapped_server(VALUE v) {
  grpc_rb_server* wrapper = NULL;
  TypedData_Get_Struct(v, grpc_rb_server, &grpc_rb_server_data_type, wrapper);
  return wrapper->wrapped;
}


text:
<?php

/**
 * PluginProfile_photo form.
 *
 * @package    ##PROJECT_NAME##
 * @subpackage form
 * @author     ##AUTHOR_NAME##
 * @version    SVN: $Id: sfDoctrineFormPluginTemplate.php 23810 2009-11-12 11:07:44Z Kris.Wallsmith $
 */
abstract class PluginProfile_photoForm extends BaseProfile_photoForm
{
}


text:
package descriptor

import (
	"io/ioutil"
	"net"
	"path/filepath"
	"strconv"
	"strings"

	"github.com/vishvananda/netlink"
	"go.ligato.io/vpp-agent/v3/pkg/models"
	"golang.org/x/sys/unix"

	"github.com/golang/protobuf/proto"
	prototypes "github.com/golang/protobuf/ptypes/empty"
	"github.com/pkg/errors"

	"github.com/ligato/cn-infra/idxmap"
	"github.com/ligato/cn-infra/logging"
	"github.com/ligato/cn-infra/logging/logrus"
	"github.com/ligato/cn-infra/servicelabel"

	kvs "go.ligato.io/vpp-agent/v3/plugins/kvscheduler/api"
	"go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin/descriptor/adapter"
	"go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin/ifaceidx"
	iflinuxcalls "go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin/linuxcalls"
	"go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin"
	nsdescriptor "go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin/descriptor"
	nslinuxcalls "go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin/linuxcalls"
	"go.ligato.io/vpp-agent/v3/plugins/netalloc"
	netalloc_descr "go.ligato.io/vpp-agent/v3/plugins/netalloc/descriptor"
	vpp_ifaceidx "go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin/ifaceidx"
	interfaces "go.ligato.io/vpp-agent/v3/proto/ligato/linux/interfaces"
	namespace "go.ligato.io/vpp-agent/v3/proto/ligato/linux/namespace"
	netalloc_api "go.ligato.io/vpp-agent/v3/proto/ligato/netalloc"
	vpp_intf "go.ligato.io/vpp-agent/v3/proto/ligato/vpp/interfaces"
)

const (
	// InterfaceDescriptorName is the name of the descriptor for Linux interfaces.
	InterfaceDescriptorName = "linux-interface"

	// default MTU - expected when MTU is not specified in the config.
	defaultEthernetMTU = 1500
	defaultLoopbackMTU = 65536

	// dependency labels
	existingHostInterfaceDep = "host-interface-exists"
	tapInterfaceDep          = "vpp-tap-interface-exists"
	vethPeerDep              = "veth-peer-exists"
	microserviceDep          = "microservice-available"

	// suffix attached to logical names of duplicate VETH interfaces
	vethDuplicateSuffix = "-DUPLICATE"

	// suffix attached to logical names of VETH interfaces with peers not found by Retrieve
	vethMissingPeerSuffix = "-MISSING_PEER"
)

// A list of non-retriable errors:
var (
	// ErrUnsupportedLinuxInterfaceType is returned for Linux interfaces of unknown type.
	ErrUnsupportedLinuxInterfaceType = errors.New("unsupported Linux interface type")

	// ErrInterfaceWithoutName is returned when Linux interface configuration has undefined
	// Name attribute.
	ErrInterfaceWithoutName = errors.New("Linux interface defined without logical name")

	// ErrInterfaceWithoutType is returned when Linux interface configuration has undefined
	// Type attribute.
	ErrInterfaceWithoutType = errors.New("Linux interface defined without type")

	// ErrNamespaceWithoutReference is returned when namespace is missing reference.
	ErrInterfaceReferenceMismatch = errors.New("Linux interface reference does not match the interface type")

	// ErrVETHWithoutPeer is returned when VETH interface is missing peer interface
	// reference.
	ErrVETHWithoutPeer = errors.New("VETH interface defined without peer reference")

	// ErrTAPWithoutVPPReference is returned when TAP_TO_VPP interface is missing reference to VPP TAP.
	ErrTAPWithoutVPPReference = errors.New("TAP_TO_VPP interface defined without reference to VPP TAP")

	// ErrTAPRequiresVPPIfPlugin is returned when TAP_TO_VPP is supposed to be configured but VPP ifplugin
	// is not loaded.
	ErrTAPRequiresVPPIfPlugin = errors.New("TAP_TO_VPP interface requires VPP interface plugin to be loaded")

	// ErrNamespaceWithoutReference is returned when namespace is missing reference.
	ErrNamespaceWithoutReference = errors.New("namespace defined without name")

	// ErrExistingWithNamespace is returned when namespace is specified for
	// EXISTING interface.
	ErrExistingWithNamespace = errors.New("EXISTING interface defined with namespace")

	// ErrInvalidIPWithMask is returned when address is invalid or mask is missing
	ErrInvalidIPWithMask = errors.New("IP with mask is not valid")

	// ErrLoopbackAlreadyConfigured is returned when multiple logical NB interfaces tries to configure the same loopback
	ErrLoopbackAlreadyConfigured = errors.New("loopback already configured")

	// ErrLoopbackNotFound is returned if loopback interface can not be found
	ErrLoopbackNotFound = errors.New("loopback not found")
)

// InterfaceDescriptor teaches KVScheduler how to configure Linux interfaces.
type InterfaceDescriptor struct {
	log          logging.Logger
	serviceLabel servicelabel.ReaderAPI
	ifHandler    iflinuxcalls.NetlinkAPI
	nsPlugin     nsplugin.API
	vppIfPlugin  VPPIfPluginAPI
	addrAlloc    netalloc.AddressAllocator

	// runtime
	intfIndex ifaceidx.LinuxIfMetadataIndex
}

// VPPIfPluginAPI is defined here to avoid import cycles.
type VPPIfPluginAPI interface {
	// GetInterfaceIndex gives read-only access to map with metadata of all configured
	// VPP interfaces.
	GetInterfaceIndex() vpp_ifaceidx.IfaceMetadataIndex
}

// NewInterfaceDescriptor creates a new instance of the Interface descriptor.
func NewInterfaceDescriptor(
	serviceLabel servicelabel.ReaderAPI, nsPlugin nsplugin.API, vppIfPlugin VPPIfPluginAPI,
	addrAlloc netalloc.AddressAllocator, log logging.PluginLogger) (descr *kvs.KVDescriptor,
	ctx *InterfaceDescriptor) {

	// descriptor context
	ctx = &InterfaceDescriptor{
		nsPlugin:     nsPlugin,
		vppIfPlugin:  vppIfPlugin,
		addrAlloc:    addrAlloc,
		serviceLabel: serviceLabel,
		log:          log.NewLogger("if-descriptor"),
	}

	typedDescr := &adapter.InterfaceDescriptor{
		Name:               InterfaceDescriptorName,
		NBKeyPrefix:        interfaces.ModelInterface.KeyPrefix(),
		ValueTypeName:      interfaces.ModelInterface.ProtoName(),
		KeySelector:        interfaces.ModelInterface.IsKeyValid,
		KeyLabel:           interfaces.ModelInterface.StripKeyPrefix,
		ValueComparator:    ctx.EquivalentInterfaces,
		WithMetadata:       true,
		MetadataMapFactory: ctx.MetadataFactory,
		Validate:           ctx.Validate,
		Create:             ctx.Create,
		Delete:             ctx.Delete,
		Update:             ctx.Update,
		UpdateWithRecreate: ctx.UpdateWithRecreate,
		Retrieve:           ctx.Retrieve,
		IsRetriableFailure: ctx.IsRetriableFailure,
		DerivedValues:      ctx.DerivedValues,
		Dependencies:       ctx.Dependencies,
		RetrieveDependencies: []string{
			// refresh the pool of allocated IP addresses first
			netalloc_descr.IPAllocDescriptorName,
			nsdescriptor.MicroserviceDescriptorName},
	}
	descr = adapter.NewInterfaceDescriptor(typedDescr)
	return
}

// SetInterfaceIndex should be used to provide interface index immediately after
// the descriptor registration.
func (d *InterfaceDescriptor) SetInterfaceIndex(intfIndex ifaceidx.LinuxIfMetadataIndex) {
	d.intfIndex = intfIndex
}

// SetInterfaceHandler provides interface handler to the descriptor immediately after
// the registration.
func (d *InterfaceDescriptor) SetInterfaceHandler(ifHandler iflinuxcalls.NetlinkAPI) {
	d.ifHandler = ifHandler
}

// EquivalentInterfaces is case-insensitive comparison function for
// interfaces.LinuxInterface, also ignoring the order of assigned IP addresses.
func (d *InterfaceDescriptor) EquivalentInterfaces(key string, oldIntf, newIntf *interfaces.Interface) bool {
	// attributes compared as usually:
	if oldIntf.Name != newIntf.Name ||
		oldIntf.Type != newIntf.Type ||
		oldIntf.Enabled != newIntf.Enabled ||
		oldIntf.LinkOnly != newIntf.LinkOnly ||
		getHostIfName(oldIntf) != getHostIfName(newIntf) {
		return false
	}
	if oldIntf.Type == interfaces.Interface_VETH {
		if oldIntf.GetVeth().GetPeerIfName() != newIntf.GetVeth().GetPeerIfName() {
			return false
		}
		// handle default config for checksum offloading
		if getRxChksmOffloading(oldIntf) != getRxChksmOffloading(newIntf) ||
			getTxChksmOffloading(oldIntf) != getTxChksmOffloading(newIntf) {
			return false
		}
	}
	if oldIntf.Type == interfaces.Interface_TAP_TO_VPP &&
		oldIntf.GetTap().GetVppTapIfName() != newIntf.GetTap().GetVppTapIfName() {
		return false
	}
	if !proto.Equal(oldIntf.Namespace, newIntf.Namespace) {
		return false
	}

	// handle default MTU
	if getInterfaceMTU(oldIntf) != getInterfaceMTU(newIntf) {
		return false
	}

	// for link-only everything else is ignored
	if oldIntf.LinkOnly {
		return true
	}

	// compare MAC addresses case-insensitively (also handle unspecified MAC address)
	if newIntf.PhysAddress != "" &&
		strings.ToLower(oldIntf.PhysAddress) != strings.ToLower(newIntf.PhysAddress) {
		return false
	}

	return true
}

// MetadataFactory is a factory for index-map customized for Linux interfaces.
func (d *InterfaceDescriptor) MetadataFactory() idxmap.NamedMappingRW {
	return ifaceidx.NewLinuxIfIndex(logrus.DefaultLogger(), "linux-interface-index")
}

// Validate validates Linux interface configuration.
func (d *InterfaceDescriptor) Validate(key string, linuxIf *interfaces.Interface) error {
	// validate name (this should never happen, since key is derived from name)
	if linuxIf.GetName() == "" {
		return kvs.NewInvalidValueError(ErrInterfaceWithoutName, "name")
	}

	// validate namespace
	if ns := linuxIf.GetNamespace(); ns != nil {
		if ns.GetType() == namespace.NetNamespace_UNDEFINED || ns.GetReference() == "" {
			return kvs.NewInvalidValueError(ErrNamespaceWithoutReference, "namespace")
		}
	}

	// validate type
	switch linuxIf.GetType() {
	case interfaces.Interface_EXISTING:
		if linuxIf.GetLink() != nil {
			return kvs.NewInvalidValueError(ErrInterfaceReferenceMismatch, "link")
		}
		// For now support only the same namespace as the agent.
		if linuxIf.GetNamespace() != nil {
			return kvs.NewInvalidValueError(ErrExistingWithNamespace, "namespace")
		}
	case interfaces.Interface_LOOPBACK:
		if linuxIf.GetLink() != nil {
			return kvs.NewInvalidValueError(ErrInterfaceReferenceMismatch, "link")
		}
	case interfaces.Interface_TAP_TO_VPP:
		if d.vppIfPlugin == nil {
			return ErrTAPRequiresVPPIfPlugin
		}
	case interfaces.Interface_UNDEFINED:
		return kvs.NewInvalidValueError(ErrInterfaceWithoutType, "type")
	}

	// validate link
	switch linuxIf.GetLink().(type) {
	case *interfaces.Interface_Tap:
		if linuxIf.GetType() != interfaces.Interface_TAP_TO_VPP {
			return kvs.NewInvalidValueError(ErrInterfaceReferenceMismatch, "link")
		}
		if linuxIf.GetTap().GetVppTapIfName() == "" {
			return kvs.NewInvalidValueError(ErrTAPWithoutVPPReference, "vpp_tap_if_name")
		}
	case *interfaces.Interface_Veth:
		if linuxIf.GetType() != interfaces.Interface_VETH {
			return kvs.NewInvalidValueError(ErrInterfaceReferenceMismatch, "link")
		}
		if linuxIf.GetVeth().GetPeerIfName() == "" {
			return kvs.NewInvalidValueError(ErrVETHWithoutPeer, "peer_if_name")
		}
	}

	return nil
}

// Create creates VETH or configures TAP interface.
func (d *InterfaceDescriptor) Create(key string, linuxIf *interfaces.Interface) (metadata *ifaceidx.LinuxIfMetadata, err error) {
	// move to the default namespace
	nsCtx := nslinuxcalls.NewNamespaceMgmtCtx()
	revert1, err := d.nsPlugin.SwitchToNamespace(nsCtx, nil)
	if err != nil {
		d.log.Error(err)
		return nil, err
	}
	defer revert1()

	// create interface based on its type
	switch linuxIf.Type {
	case interfaces.Interface_VETH:
		metadata, err = d.createVETH(nsCtx, key, linuxIf)
	case interfaces.Interface_TAP_TO_VPP:
		metadata, err = d.createTAPToVPP(nsCtx, key, linuxIf)
	case interfaces.Interface_LOOPBACK:
		metadata, err = d.createLoopback(nsCtx, linuxIf)
	case interfaces.Interface_EXISTING:
		// We expect that the interface already exists, therefore nothing needs to be done.
		// We just get the metadata for the interface.
		getMetadata := func(linuxIf *interfaces.Interface) (*ifaceidx.LinuxIfMetadata, error) {
			link, err := d.ifHandler.GetLinkByName(getHostIfName(linuxIf))
			if err != nil {
				d.log.Error(err)
				return nil, err
			}
			return &ifaceidx.LinuxIfMetadata{
				Namespace:    linuxIf.GetNamespace(),
				LinuxIfIndex: link.Attrs().Index,
			}, nil
		}
		metadata, err = getMetadata(linuxIf)
	default:
		return nil, ErrUnsupportedLinuxInterfaceType
	}
	if err != nil {
		d.log.Errorf("creating %v interface failed: %+v", linuxIf.GetType(), err)
		return nil, err
	}

	metadata.HostIfName = getHostIfName(linuxIf)

	// move to the namespace with the interface
	revert2, err := d.nsPlugin.SwitchToNamespace(nsCtx, linuxIf.Namespace)
	if err != nil {
		d.log.Error(err)
		return nil, err
	}
	defer revert2()

	// set interface up
	hostName := getHostIfName(linuxIf)
	if linuxIf.Enabled {
		err = d.ifHandler.SetInterfaceUp(hostName)
		if nil != err {
			err = errors.Errorf("failed to set linux interface %s up: %v", linuxIf.Name, err)
			d.log.Error(err)
			return nil, err
		}
	}

	// set checksum offloading
	if linuxIf.Type == interfaces.Interface_VETH {
		rxOn := getRxChksmOffloading(linuxIf)
		txOn := getTxChksmOffloading(linuxIf)
		err = d.ifHandler.SetChecksumOffloading(hostName, rxOn, txOn)
		if err != nil {
			err = errors.Errorf("failed to configure checksum offloading (rx=%t,tx=%t) for linux interface %s: %v",
				rxOn, txOn, linuxIf.Name, err)
			d.log.Error(err)
			return nil, err
		}
	}

	// set interface MTU
	if linuxIf.Mtu != 0 {
		mtu := int(linuxIf.Mtu)
		err = d.ifHandler.SetInterfaceMTU(hostName, mtu)
		if err != nil {
			err = errors.Errorf("failed to set MTU %d to linux interface %s: %v",
				mtu, linuxIf.Name, err)
			d.log.Error(err)
			return nil, err
		}
	}

	if linuxIf.GetLinkOnly() {
		// addresses are configured externally
		return metadata, nil
	}

	// set interface MAC address
	if linuxIf.PhysAddress != "" {
		err = d.ifHandler.SetInterfaceMac(hostName, linuxIf.PhysAddress)
		if err != nil {
			err = errors.Errorf("failed to set MAC address %s to linux interface %s: %v",
				linuxIf.PhysAddress, linuxIf.Name, err)
			d.log.Error(err)
			return nil, err
		}
	}

	return metadata, nil
}

// Delete removes VETH or unconfigures TAP interface.
func (d *InterfaceDescriptor) Delete(key string, linuxIf *interfaces.Interface, metadata *ifaceidx.LinuxIfMetadata) error {
	// move to the namespace with the interface
	nsCtx := nslinuxcalls.NewNamespaceMgmtCtx()
	revert, err := d.nsPlugin.SwitchToNamespace(nsCtx, linuxIf.Namespace)
	if err != nil {
		d.log.Error("switch to namespace failed:", err)
		return err
	}
	defer revert()

	switch linuxIf.Type {
	case interfaces.Interface_VETH:
		return d.deleteVETH(nsCtx, key, linuxIf, metadata)
	case interfaces.Interface_TAP_TO_VPP:
		return d.deleteAutoTAP(nsCtx, key, linuxIf, metadata)
	case interfaces.Interface_LOOPBACK:
		return d.deleteLoopback(nsCtx, linuxIf)
	case interfaces.Interface_EXISTING:
		// We only need to unconfigure the interface.
		// Nothing else needs to be done.
		return nil
	}

	err = ErrUnsupportedLinuxInterfaceType
	d.log.Error(err)
	return err
}

// Update is able to change Type-unspecific attributes.
func (d *InterfaceDescriptor) Update(key string, oldLinuxIf, newLinuxIf *interfaces.Interface, oldMetadata *ifaceidx.LinuxIfMetadata) (newMetadata *ifaceidx.LinuxIfMetadata, err error) {
	oldHostName := getHostIfName(oldLinuxIf)
	newHostName := getHostIfName(newLinuxIf)

	// move to the namespace with the interface
	nsCtx := nslinuxcalls.NewNamespaceMgmtCtx()
	revert, err := d.nsPlugin.SwitchToNamespace(nsCtx, oldLinuxIf.Namespace)
	if err != nil {
		d.log.Error(err)
		return nil, err
	}
	defer revert()

	// update host name
	if oldHostName != newHostName {
		if err := d.ifHandler.RenameInterface(oldHostName, newHostName); err != nil {
			d.log.Error("renaming interface failed:", err)
			return nil, err
		}
	}

	// update admin status
	if oldLinuxIf.Enabled != newLinuxIf.Enabled {
		if newLinuxIf.Enabled {
			err = d.ifHandler.SetInterfaceUp(newHostName)
			if nil != err {
				err = errors.Errorf("failed to set linux interface %s UP: %v", newHostName, err)
				d.log.Error(err)
				return nil, err
			}
		} else {
			err = d.ifHandler.SetInterfaceDown(newHostName)
			if nil != err {
				err = errors.Errorf("failed to set linux interface %s DOWN: %v", newHostName, err)
				d.log.Error(err)
				return nil, err
			}
		}
	}

	// update MAC address
	if !newLinuxIf.GetLinkOnly() {
		if newLinuxIf.PhysAddress != "" && newLinuxIf.PhysAddress != oldLinuxIf.PhysAddress {
			err := d.ifHandler.SetInterfaceMac(newHostName, newLinuxIf.PhysAddress)
			if err != nil {
				err = errors.Errorf("failed to reconfigure MAC address for linux interface %s: %v",
					newLinuxIf.Name, err)
				d.log.Error(err)
				return nil, err
			}
		}
	}

	// MTU
	if getInterfaceMTU(newLinuxIf) != getInterfaceMTU(oldLinuxIf) {
		mtu := getInterfaceMTU(newLinuxIf)
		err := d.ifHandler.SetInterfaceMTU(newHostName, mtu)
		if nil != err {
			err = errors.Errorf("failed to reconfigure MTU for the linux interface %s: %v",
				newLinuxIf.Name, err)
			d.log.Error(err)
			return nil, err
		}
	}

	// update checksum offloading
	if newLinuxIf.Type == interfaces.Interface_VETH {
		rxOn := getRxChksmOffloading(newLinuxIf)
		txOn := getTxChksmOffloading(newLinuxIf)
		if rxOn != getRxChksmOffloading(oldLinuxIf) || txOn != getTxChksmOffloading(oldLinuxIf) {
			err = d.ifHandler.SetChecksumOffloading(newHostName, rxOn, txOn)
			if err != nil {
				err = errors.Errorf("failed to reconfigure checksum offloading (rx=%t,tx=%t) for linux interface %s: %v",
					rxOn, txOn, newLinuxIf.Name, err)
				d.log.Error(err)
				return nil, err
			}
		}
	}

	// update metadata
	link, err := d.ifHandler.GetLinkByName(newHostName)
	if err != nil {
		d.log.Error(err)
		return nil, err
	}
	oldMetadata.LinuxIfIndex = link.Attrs().Index
	oldMetadata.HostIfName = newHostName
	return oldMetadata, nil
}

// UpdateWithRecreate returns true if Type or Type-specific attributes are different.
func (d *InterfaceDescriptor) UpdateWithRecreate(key string, oldLinuxIf, newLinuxIf *interfaces.Interface, metadata *ifaceidx.LinuxIfMetadata) bool {
	if oldLinuxIf.Type != newLinuxIf.Type {
		return true
	}
	if oldLinuxIf.LinkOnly != newLinuxIf.LinkOnly {
		return true
	}
	if !proto.Equal(oldLinuxIf.Namespace, newLinuxIf.Namespace) {
		// anything attached to the interface (ARPs, routes, ...) will be re-created as well
		return true
	}
	switch oldLinuxIf.Type {
	case interfaces.Interface_VETH:
		return oldLinuxIf.GetVeth().GetPeerIfName() != newLinuxIf.GetVeth().GetPeerIfName()
	case interfaces.Interface_TAP_TO_VPP:
		return oldLinuxIf.GetTap().GetVppTapIfName() != newLinuxIf.GetTap().GetVppTapIfName()
	}
	return false
}

// Dependencies lists dependencies for a Linux interface.
func (d *InterfaceDescriptor) Dependencies(key string, linuxIf *interfaces.Interface) []kvs.Dependency {
	var dependencies []kvs.Dependency

	// EXISTING depends on a referenced Linux interface in the default namespace
	if linuxIf.Type == interfaces.Interface_EXISTING {
		dependencies = append(dependencies, kvs.Dependency{
			Label: existingHostInterfaceDep,
			Key:   interfaces.InterfaceHostNameKey(getHostIfName(linuxIf)),
		})
	}

	if linuxIf.Type == interfaces.Interface_TAP_TO_VPP {
		// dependency on VPP TAP
		dependencies = append(dependencies, kvs.Dependency{
			Label: tapInterfaceDep,
			Key:   vpp_intf.InterfaceKey(linuxIf.GetTap().GetVppTapIfName()),
		})
	}

	// circular dependency between VETH ends
	if linuxIf.Type == interfaces.Interface_VETH {
		peerName := linuxIf.GetVeth().GetPeerIfName()
		if peerName != "" {
			dependencies = append(dependencies, kvs.Dependency{
				Label: vethPeerDep,
				Key:   interfaces.InterfaceKey(peerName),
			})
		}
	}

	if linuxIf.GetNamespace().GetType() == namespace.NetNamespace_MICROSERVICE {
		dependencies = append(dependencies, kvs.Dependency{
			Label: microserviceDep,
			Key:   namespace.MicroserviceKey(linuxIf.Namespace.Reference),
		})
	}

	return dependencies
}

// DerivedValues derives one empty value to represent interface state and also
// one empty value for every IP address assigned to the interface.
func (d *InterfaceDescriptor) DerivedValues(key string, linuxIf *interfaces.Interface) (derValues []kvs.KeyValuePair) {
	// interface state
	derValues = append(derValues, kvs.KeyValuePair{
		Key:   interfaces.InterfaceStateKey(linuxIf.Name, linuxIf.Enabled),
		Value: &prototypes.Empty{},
	})
	if !linuxIf.GetLinkOnly() {
		// IP addresses
		for _, ipAddr := range linuxIf.IpAddresses {
			derValues = append(derValues, kvs.KeyValuePair{
				Key:   interfaces.InterfaceAddressKey(linuxIf.Name, ipAddr, netalloc_api.IPAddressSource_STATIC),
				Value: &prototypes.Empty{},
			})
		}
	}
	return derValues
}

// retrievedIfaces is used as the return value sent via channel by retrieveInterfaces().
type retrievedIfaces struct {
	interfaces []adapter.InterfaceKVWithMetadata
	err        error
}

func (d *InterfaceDescriptor) IsRetriableFailure(err error) bool {
	if err == ErrLoopbackAlreadyConfigured {
		return false
	}
	return true
}

// Retrieve returns all Linux interfaces managed by this agent, attached to the default namespace
// or to one of the configured non-default namespaces.
func (d *InterfaceDescriptor) Retrieve(correlate []adapter.InterfaceKVWithMetadata) ([]adapter.InterfaceKVWithMetadata, error) {
	nsList := []*namespace.NetNamespace{nil}              // nil = default namespace, which always should be listed for interfaces
	ifCfg := make(map[string]*interfaces.Interface)       // interface logical name -> interface config (as expected by correlate)
	expExisting := make(map[string]*interfaces.Interface) // EXISTING interface host name -> expected interface config

	// process interfaces for correlation to get:
	//  - the set of namespaces to list for interfaces
	//  - mapping between interface name and the configuration for correlation
	// beware: the same namespace can have multiple different references (e.g. integration of Contiv with SFC)
	for _, kv := range correlate {
		nsListed := false
		for _, ns := range nsList {
			if proto.Equal(ns, kv.Value.Namespace) {
				nsListed = true
				break
			}
		}
		if !nsListed {
			nsList = append(nsList, kv.Value.Namespace)
		}
		ifCfg[kv.Value.Name] = kv.Value
		if kv.Value.Type == interfaces.Interface_EXISTING {
			expExisting[getHostIfName(kv.Value)] = kv.Value
		}
	}

	// Obtain interface details - all interfaces with metadata
	ifDetails, err := d.ifHandler.DumpInterfacesFromNamespaces(nsList)
	if err != nil {
		return nil, errors.Errorf("Failed to retrieve linux interfaces: %v", err)
	}
	// interface logical name -> interface data
	ifaces := make(map[string]adapter.InterfaceKVWithMetadata)
	// already retrieved interfaces by their Linux indexes
	indexes := make(map[int]struct{})

	for _, ifDetail := range ifDetails {
		// Transform linux interface details to the type-safe value with metadata
		kv := adapter.InterfaceKVWithMetadata{
			Origin: kvs.FromNB,
			Value:  ifDetail.Interface,
			Metadata: &ifaceidx.LinuxIfMetadata{
				LinuxIfIndex: ifDetail.Meta.LinuxIfIndex,
				Namespace:    ifDetail.Interface.GetNamespace(),
				VPPTapName:   ifDetail.Interface.GetTap().GetVppTapIfName(),
				HostIfName:   ifDetail.Interface.HostIfName,
			},
			Key: interfaces.InterfaceKey(ifDetail.Interface.Name),
		}

		// skip if this interface was already retrieved and this is not the expected
		// namespace from correlation - remember, the same namespace may have
		// multiple different references
		var rewrite bool
		if _, alreadyRetrieved := indexes[kv.Metadata.LinuxIfIndex]; alreadyRetrieved {
			if expCfg, hasExpCfg := ifCfg[kv.Value.Name]; hasExpCfg {
				if proto.Equal(expCfg.Namespace, kv.Value.Namespace) {
					rewrite = true
				}
			}
			if !rewrite {
				continue
			}
		}
		indexes[kv.Metadata.LinuxIfIndex] = struct{}{}

		// test for duplicity of VETH logical names
		if kv.Value.Type == interfaces.Interface_VETH {
			if _, duplicate := ifaces[kv.Value.Name]; duplicate && !rewrite {
				// add suffix to the duplicate to make its logical name unique
				// (and not configured by NB so that it will get removed)
				dupIndex := 1
				for intf2 := range ifaces {
					if strings.HasPrefix(intf2, kv.Value.Name+vethDuplicateSuffix) {
						dupIndex++
					}
				}
				kv.Value.Name = kv.Value.Name + vethDuplicateSuffix + strconv.Itoa(dupIndex)
				kv.Key = interfaces.InterfaceKey(kv.Value.Name)
			}
		}
		// correlate link_only attribute
		if expCfg, hasExpCfg := ifCfg[kv.Value.Name]; hasExpCfg {
			kv.Value.LinkOnly = expCfg.GetLinkOnly()
		}
		ifaces[kv.Value.Name] = kv
	}

	// first collect VETHs with duplicate logical names
	var values []adapter.InterfaceKVWithMetadata
	for ifName, kv := range ifaces {
		if kv.Value.Type == interfaces.Interface_VETH {
			isDuplicate := strings.Contains(ifName, vethDuplicateSuffix)
			// first interface retrieved from the set of duplicate VETHs still
			// does not have the vethDuplicateSuffix appended to the name
			_, hasDuplicate := ifaces[ifName+vethDuplicateSuffix+"1"]
			if hasDuplicate {
				kv.Value.Name = ifName + vethDuplicateSuffix + "0"
				kv.Key = interfaces.InterfaceKey(kv.Value.Name)
			}
			if isDuplicate || hasDuplicate {
				// clear peer reference so that Delete removes the VETH-end
				// as standalone
				kv.Value.Link = &interfaces.Interface_Veth{}
				delete(ifaces, ifName)
				values = append(values, kv)
			}
		}
	}

	// next collect VETHs with missing peer
	for ifName, kv := range ifaces {
		if kv.Value.Type == interfaces.Interface_VETH {
			peer, retrieved := ifaces[kv.Value.GetVeth().GetPeerIfName()]
			if !retrieved || peer.Value.GetVeth().GetPeerIfName() != kv.Value.Name {
				// append vethMissingPeerSuffix to the logical name so that VETH
				// will get removed during resync
				kv.Value.Name = ifName + vethMissingPeerSuffix
				kv.Key = interfaces.InterfaceKey(kv.Value.Name)
				// clear peer reference so that Delete removes the VETH-end
				// as standalone
				kv.Value.Link = &interfaces.Interface_Veth{}
				delete(ifaces, ifName)
				values = append(values, kv)
			}
		}
	}

	// collect AUTO-TAPs and valid VETHs
	for _, kv := range ifaces {
		values = append(values, kv)
	}

	// retrieve EXISTING interfaces
	existingIfaces, err := d.retrieveExistingInterfaces(expExisting)
	if err != nil {
		return nil, err
	}
	for _, kv := range existingIfaces {
		values = append(values, kv)
	}

	// correlate IP addresses with netalloc references from the expected config
	for _, kv := range values {
		if expCfg, hasExpCfg := ifCfg[kv.Value.Name]; hasExpCfg {
			kv.Value.IpAddresses = d.addrAlloc.CorrelateRetrievedIPs(
				expCfg.IpAddresses, kv.Value.IpAddresses,
				kv.Value.Name, netalloc_api.IPAddressForm_ADDR_WITH_MASK)
		}
	}

	return values, nil
}

// retrieveExistingInterfaces retrieves already created Linux interface - i.e. not created
// by this agent = type EXISTING.
func (d *InterfaceDescriptor) retrieveExistingInterfaces(expected map[string]*interfaces.Interface) ([]adapter.InterfaceKVWithMetadata, error) {
	var retrieved []adapter.InterfaceKVWithMetadata

	// get all links in the default namespace
	links, err := d.ifHandler.GetLinkList()
	if err != nil {
		d.log.Error("Failed to get link list:", err)
		return nil, err
	}
	for _, link := range links {
		expCfg, isExp := expected[link.Attrs().Name]
		if !isExp {
			// do not touch existing interfaces which are not configured by the agent
			continue
		}
		iface := &interfaces.Interface{
			Name:        expCfg.GetName(),
			Type:        interfaces.Interface_EXISTING,
			HostIfName:  link.Attrs().Name,
			PhysAddress: link.Attrs().HardwareAddr.String(),
			Mtu:         uint32(link.Attrs().MTU),
			LinkOnly:    expCfg.LinkOnly,
		}

		// retrieve addresses, MTU, etc.
		d.retrieveLinkDetails(link, iface, nil)

		// build key-value pair for the retrieved interface
		retrieved = append(retrieved, adapter.InterfaceKVWithMetadata{
			Key:    models.Key(iface),
			Value:  iface,
			Origin: kvs.FromNB,
			Metadata: &ifaceidx.LinuxIfMetadata{
				LinuxIfIndex: link.Attrs().Index,
				HostIfName:   link.Attrs().Name,
			},
		})
	}

	return retrieved, nil
}

// retrieveLinkDetails retrieves link details common to all interface types (e.g. addresses).
func (d *InterfaceDescriptor) retrieveLinkDetails(link netlink.Link, iface *interfaces.Interface, nsRef *namespace.NetNamespace) {
	var err error
	// read interface status
	iface.Enabled, err = d.ifHandler.IsInterfaceUp(link.Attrs().Name)
	if err != nil {
		d.log.WithFields(logging.Fields{
			"if-host-name": link.Attrs().Name,
			"namespace":    nsRef,
		}).Warn("Failed to read interface status:", err)
	}

	// read assigned IP addresses
	addressList, err := d.ifHandler.GetAddressList(link.Attrs().Name)
	if err != nil {
		d.log.WithFields(logging.Fields{
			"if-host-name": link.Attrs().Name,
			"namespace":    nsRef,
		}).Warn("Failed to read address list:", err)
	}
	for _, address := range addressList {
		if address.Scope == unix.RT_SCOPE_LINK {
			// ignore link-local IPv6 addresses
			continue
		}
		mask, _ := address.Mask.Size()
		addrStr := address.IP.String() + "/" + strconv.Itoa(mask)
		iface.IpAddresses = append(iface.IpAddresses, addrStr)
	}

	// read checksum offloading
	if iface.Type == interfaces.Interface_VETH {
		rxOn, txOn, err := d.ifHandler.GetChecksumOffloading(link.Attrs().Name)
		if err != nil {
			d.log.WithFields(logging.Fields{
				"if-host-name": link.Attrs().Name,
				"namespace":    nsRef,
			}).Warn("Failed to read checksum offloading:", err)
		} else {
			if !rxOn {
				iface.GetVeth().RxChecksumOffloading = interfaces.VethLink_CHKSM_OFFLOAD_DISABLED
			}
			if !txOn {
				iface.GetVeth().TxChecksumOffloading = interfaces.VethLink_CHKSM_OFFLOAD_DISABLED
			}
		}
	}
}

// setInterfaceNamespace moves linux interface from the current to the desired
// namespace.
func (d *InterfaceDescriptor) setInterfaceNamespace(ctx nslinuxcalls.NamespaceMgmtCtx, ifName string, namespace *namespace.NetNamespace) error {
	// Get namespace handle.
	ns, err := d.nsPlugin.GetNamespaceHandle(ctx, namespace)
	if err != nil {
		return err
	}
	defer ns.Close()

	// Get the interface link handle.
	link, err := d.ifHandler.GetLinkByName(ifName)
	if err != nil {
		return errors.Errorf("failed to get link for interface %s: %v", ifName, err)
	}

	// When interface moves from one namespace to another, it loses all its IP addresses, admin status
	// and MTU configuration -- we need to remember the interface configuration before the move
	// and re-configure the interface in the new namespace.
	addresses, isIPv6, err := d.getInterfaceAddresses(link.Attrs().Name)
	if err != nil {
		return errors.Errorf("failed to get IP address list from interface %s: %v", link.Attrs().Name, err)
	}
	enabled, err := d.ifHandler.IsInterfaceUp(ifName)
	if err != nil {
		return errors.Errorf("failed to get admin status of the interface %s: %v", link.Attrs().Name, err)
	}

	// Move the interface into the namespace.
	if err := d.ifHandler.SetLinkNamespace(link, ns); err != nil {
		return errors.Errorf("failed to set interface %s file descriptor: %v", link.Attrs().Name, err)
	}

	// Re-configure interface in its new namespace
	revertNs, err := d.nsPlugin.SwitchToNamespace(ctx, namespace)
	if err != nil {
		return errors.Errorf("failed to switch namespace: %v", err)
	}
	defer revertNs()

	if enabled {
		// Re-enable interface
		err = d.ifHandler.SetInterfaceUp(ifName)
		if nil != err {
			return errors.Errorf("failed to re-enable Linux interface `%s`: %v", ifName, err)
		}
	}

	// Re-add IP addresses
	for _, address := range addresses {
		// Skip IPv6 link local address if there is no other IPv6 address
		if !isIPv6 && address.IP.IsLinkLocalUnicast() {
			continue
		}
		if err := d.ifHandler.AddInterfaceIP(ifName, address); err != nil {
			if err.Error() == "file exists" {
				continue
			}
			return errors.Errorf("failed to re-assign IP address to a Linux interface `%s`: %v", ifName, err)
		}
	}

	// Revert back the MTU config
	err = d.ifHandler.SetInterfaceMTU(ifName, link.Attrs().MTU)
	if nil != err {
		return errors.Errorf("failed to re-assign MTU of a Linux interface `%s`: %v", ifName, err)
	}

	return nil
}

// getInterfaceAddresses returns a list of IP addresses assigned to the given linux interface.
// <hasIPv6> is returned as true if a non link-local IPv6 address is among them.
func (d *InterfaceDescriptor) getInterfaceAddresses(ifName string) (addresses []*net.IPNet, hasIPv6 bool, err error) {
	// get all assigned IP addresses
	ipAddrs, err := d.ifHandler.GetAddressList(ifName)
	if err != nil {
		return nil, false, err
	}

	// iterate over IP addresses to see if there is IPv6 among them
	for _, ipAddr := range ipAddrs {
		if ipAddr.IP.To4() == nil && !ipAddr.IP.IsLinkLocalUnicast() {
			// IP address is version 6 and not a link local address
			hasIPv6 = true
		}
		addresses = append(addresses, ipAddr.IPNet)
	}
	return addresses, hasIPv6, nil
}

// getHostIfName returns the interface host name.
func getHostIfName(linuxIf *interfaces.Interface) string {
	if linuxIf.Type == interfaces.Interface_LOOPBACK {
		return iflinuxcalls.DefaultLoopbackName
	}
	hostIfName := linuxIf.HostIfName
	if hostIfName == "" {
		hostIfName = linuxIf.Name
	}
	return hostIfName
}

// getInterfaceMTU returns the interface MTU.
func getInterfaceMTU(linuxIntf *interfaces.Interface) int {
	mtu := int(linuxIntf.Mtu)
	if mtu == 0 {
		if linuxIntf.Type == interfaces.Interface_LOOPBACK {
			return defaultLoopbackMTU
		}
		return defaultEthernetMTU
	}
	return mtu
}

func getRxChksmOffloading(linuxIntf *interfaces.Interface) (rxOn bool) {
	return isChksmOffloadingOn(linuxIntf.GetVeth().GetRxChecksumOffloading())
}

func getTxChksmOffloading(linuxIntf *interfaces.Interface) (txOn bool) {
	return isChksmOffloadingOn(linuxIntf.GetVeth().GetTxChecksumOffloading())
}

func isChksmOffloadingOn(offloading interfaces.VethLink_ChecksumOffloading) bool {
	switch offloading {
	case interfaces.VethLink_CHKSM_OFFLOAD_DEFAULT:
		return true // enabled by default
	case interfaces.VethLink_CHKSM_OFFLOAD_ENABLED:
		return true
	case interfaces.VethLink_CHKSM_OFFLOAD_DISABLED:
		return false
	}
	return true
}

func getSysctl(name string) (string, error) {
	fullName := filepath.Join("/proc/sys", strings.Replace(name, ".", "/", -1))
	fullName = filepath.Clean(fullName)
	data, err := ioutil.ReadFile(fullName)
	if err != nil {
		return "", err
	}
	return string(data[:len(data)-1]), nil
}

func setSysctl(name, value string) (string, error) {
	fullName := filepath.Join("/proc/sys", strings.Replace(name, ".", "/", -1))
	fullName = filepath.Clean(fullName)
	if err := ioutil.WriteFile(fullName, []byte(value), 0644); err != nil {
		return "", err
	}
	return getSysctl(name)
}


text:
from django.apps import AppConfig

class AlgoliaConfig(AppConfig):
    """Simple AppConfig which does not do automatic discovery."""

    name = 'algoliasearch_django'

    def ready(self):
        super(AlgoliaConfig, self).ready()
        self.module.autodiscover()


text:
<?php

namespace Elcodi\Component\StateTransitionMachine\Tests\UnitTest\Machine;

use Doctrine\Common\Collections\ArrayCollection;

use Elcodi\Component\StateTransitionMachine\Entity\StateLine;
use Elcodi\Component\StateTransitionMachine\Entity\StateLineStack;
use Elcodi\Component\StateTransitionMachine\Tests\UnitTest\Fixtures\AbstractStateTransitionTest;
use Elcodi\Component\StateTransitionMachine\Tests\UnitTest\Fixtures\Order;

/**
 * Class MachineManagerTest
 */
class MachineManagerTest extends AbstractStateTransitionTest
{
    /**
     * Test initialize
     *
     * @expectedException \Elcodi\Component\StateTransitionMachine\Exception\ObjectAlreadyInitializedException
     */
    public function testInitializeNotEmpty()
    {
        $machineManager = $this->getMachineManager(0);

        $order = new Order();
        $stateLineStack = StateLineStack::create(
            new ArrayCollection(),
            null
        );
        $order->setStateLineStack($stateLineStack);

        $stateLineStack->addStateLine(new StateLine());
        $machineManager->initialize(
            $order,
            $stateLineStack,
            ''
        );
    }

    /**
     * Test initialize
     */
    public function testInitializeEmpty()
    {
        $machineManager = $this->getMachineManager(1);

        $order = new Order();
        $stateLineStack = StateLineStack::create(
            new ArrayCollection(),
            null
        );
        $order->setStateLineStack($stateLineStack);

        $stateLineStack = $machineManager->initialize(
            $order,
            $stateLineStack,
            ''
        );

        $this->assertEquals('unpaid', $stateLineStack->getLastStateLine()->getName());
        $this->assertCount(1, $stateLineStack->getStateLines());
    }

    /**
     * Test make transition with a non-initialized object
     *
     * @expectedException \Elcodi\Component\StateTransitionMachine\Exception\ObjectNotInitializedException
     */
    public function testMakeTransitionNonInitialized()
    {
        $machineManager = $this->getMachineManager(0);

        $order = new Order();
        $stateLineStack = StateLineStack::create(
            new ArrayCollection(),
            null
        );
        $order->setStateLineStack($stateLineStack);

        $machineManager->transition(
            $order,
            $stateLineStack,
            'pay',
            ''
        );
    }

    /**
     * Test make transition with an initialized object
     */
    public function testMakeTransitionInitialized()
    {
        $machineManager = $this->getMachineManager(5);

        $order = new Order();
        $stateLineStack = StateLineStack::create(
            new ArrayCollection(),
            null
        );
        $order->setStateLineStack($stateLineStack);

        $stateLineStack = $machineManager->initialize(
            $order,
            $stateLineStack,
            ''
        );

        $stateLineStack = $machineManager->transition(
            $order,
            $stateLineStack,
            'pay',
            ''
        );

        $this->assertEquals('paid', $stateLineStack->getLastStateLine()->getName());
        $this->assertEquals('paid', $stateLineStack->getStateLines()->last()->getName());
    }

    /**
     * Test make transition with a non-initialized object
     *
     * @expectedException \Elcodi\Component\StateTransitionMachine\Exception\ObjectNotInitializedException
     */
    public function testReachStateNonInitialized()
    {
        $machineManager = $this->getMachineManager(0);

        $order = new Order();
        $stateLineStack = StateLineStack::create(
            new ArrayCollection(),
            null
        );
        $order->setStateLineStack($stateLineStack);

        $machineManager->reachState(
            $order,
            $stateLineStack,
            'paid',
            ''
        );
    }

    /**
     * Test make transition with an initialized object
     */
    public function testReachStateInitialized()
    {
        $machineManager = $this->getMachineManager(5);

        $order = new Order();
        $stateLineStack = StateLineStack::create(
            new ArrayCollection(),
            null
        );
        $order->setStateLineStack($stateLineStack);
        $stateLineStack = $machineManager->initialize(
            $order,
            $stateLineStack,
            ''
        );
        $order->setStateLineStack($stateLineStack);

        $stateLineStack = $machineManager->reachState(
            $order,
            $stateLineStack,
            'paid',
            ''
        );

        $this->assertEquals('paid', $stateLineStack->getLastStateLine()->getName());
        $this->assertEquals('paid', $stateLineStack->getStateLines()->last()->getName());
    }
}


text:
package openmarker.trove.classfile;

import java.io.*;

/**
 * This class corresponds to the CONSTANT_NameAndType_info structure as defined
 * in section 4.4.6 of <i>The Java Virtual Machine Specification</i>.
 *
 * @author Brian S O'Neill
 */
public class ConstantNameAndTypeInfo extends ConstantInfo {
    private String mName;
    private Descriptor mType;

    private ConstantUTFInfo mNameConstant;
    private ConstantUTFInfo mDescriptorConstant;

    /**
     * Will return either a new ConstantNameAndTypeInfo object or one
     * already in the constant pool.
     * If it is a new ConstantNameAndTypeInfo, it will be inserted
     * into the pool.
     */
    static ConstantNameAndTypeInfo make(ConstantPool cp,
                                        String name,
                                        Descriptor type) {
        ConstantInfo ci = new ConstantNameAndTypeInfo(cp, name, type);
        return (ConstantNameAndTypeInfo)cp.addConstant(ci);
    }

    ConstantNameAndTypeInfo(ConstantUTFInfo nameConstant,
                            ConstantUTFInfo descConstant) {
        super(TAG_NAME_AND_TYPE);
        mNameConstant = nameConstant;
        mDescriptorConstant = descConstant;

        mName = nameConstant.getValue();
        mType = Descriptor.parse(descConstant.getValue());
    }

    private ConstantNameAndTypeInfo(ConstantPool cp,
                                    String name,
                                    Descriptor type) {
        super(TAG_NAME_AND_TYPE);
        mName = name;
        mType = type;

        mNameConstant = ConstantUTFInfo.make(cp, name);
        mDescriptorConstant = ConstantUTFInfo.make(cp, mType.toString());
    }

    public String getName() {
        return mName;
    }

    public Descriptor getType() {
        return mType;
    }

    public int hashCode() {
        return mName.hashCode();
    }

    public boolean equals(Object obj) {
        if (obj instanceof ConstantNameAndTypeInfo) {
            ConstantNameAndTypeInfo other = (ConstantNameAndTypeInfo)obj;
            return mName.equals(other.mName) &&
                mType.toString().equals(other.mType.toString());
        }

        return false;
    }

    public void writeTo(DataOutput dout) throws IOException {
        super.writeTo(dout);
        dout.writeShort(mNameConstant.getIndex());
        dout.writeShort(mDescriptorConstant.getIndex());
    }

    public String toString() {
        return "CONSTANT_NameAndType_info: " + getName() + ", " + getType();
    }
}


text:
module Postman
  class MailServer < ActiveRecord::Base
    attr_accessible :name, :delivery_method, :address, :port, :domain, :user_name, :password, :authentication, :enable_starttls_auto
    store :settings, accessors: [
                                 :address, :port, :domain,
                                 :user_name, :password, :authentication, :enable_starttls_auto
                                ]

    validates :name, presence: true, uniqueness: true
    validates :name, :delivery_method, :settings, presence: true
    validates :user_name, :password, :address, :port, :domain,
              :authentication, :enable_starttls_auto, presence: true, :if => proc{ |t| t.delivery_method.to_s == 'smtp' }

    def test(to_address)
      Pony.mail({
                  :from => "postman@postman.com",
                  :subject => "[ POSTMAN ] test",
                  :body => "TEST: text body",
                  :html_body => "
TEST
:
 html body",
                  :to => to_address,
                  :via => delivery_method.to_sym,
                  :via_options => settings
                })
    end

  end
end