text:
.. File automatically generated using DocOnce (https://github.com/doconce/doconce/):

.. doconce format sphinx kleingordon.do.txt --sphinx_preserve_bib_keys

.. Document title:

Demo - Cubic nonlinear Klein-Gordon equation
============================================

:Authors: Mikael Mortensen (mikaem at math.uio.no)
:Date: April 13, 2018

*Summary.* This is a demonstration of how the Python module `shenfun <https://github.com/spectralDNS/shenfun>`__ can be used to solve the time-dependent,
nonlinear Klein-Gordon equation, in a triply periodic domain. The demo is implemented in
a single Python file `KleinGordon.py <https://github.com/spectralDNS/shenfun/blob/master/demo/KleinGordon.py>`__, and it may be run
in parallel using MPI. The Klein-Gordon equation is solved using a mixed
formulation. The discretization, and some background on the spectral Galerkin
method is given first, before we turn to the actual details of the ``shenfun``
implementation.

.. raw:: html

        <embed src="https://rawgit.com/spectralDNS/spectralutilities/master/movies/KleinGordon.gif"  autoplay="false" loop="true"></embed>

Movie showing the evolution of the solution :math:`u` from Eq. :eq:`eq:kg`, in a slice through the center of the domain, computed with the code described in this demo.

The nonlinear Klein-Gordon equation
-----------------------------------

The cubic nonlinear Klein-Gordon equation is a wave equation important for many
scientific applications such as solid state physics, nonlinear optics and
quantum field theory :cite:`abdul08`. The equation is given as

.. math::
   :label: eq:kg

        \frac{\partial^2 u}{\partial t^2} = \nabla^2 u - \gamma(u - u|u|^2) \quad
        \text{for} \, u \in
        \Omega,

with initial conditions

.. math::
   :label: eq:init

        u(\boldsymbol{x}, t=0) = u^0 \quad \text{and} \quad \frac{\partial u(\boldsymbol{x},
        t=0)}{\partial t} = u_t^0.

The spatial coordinates are here denoted as :math:`\boldsymbol{x} = (x, y, z)`, and
:math:`t` is time. The parameter :math:`\gamma=\pm 1` determines whether the equations are focusing
(:math:`+1`) or defocusing (:math:`-1`) (in the movie we have used :math:`\gamma=1`). The domain :math:`\Omega=[-2\pi, 2\pi]^3` is triply
periodic and initial conditions will here be set as

.. math::
   :label: _auto1

        u^0 = 0.1 \exp \left( -\boldsymbol{x} \cdot \boldsymbol{x} \right),

.. math::
   :label: _auto2

        u_t^0 = 0.

We will solve these equations using a mixed formulation and a spectral Galerkin
method. The mixed formulation reads

.. math::
   :label: eq:df

        \frac{\partial f}{\partial t} = \nabla^2 u - \gamma (u - u|u|^2),

.. math::
   :label: eq:du

        \frac{\partial u}{\partial t} = f.

The energy of the solution can be computed as

.. math::
   :label: _auto3

        E(u) = \int_{\Omega} \left( \frac{1}{2} f^2 + \frac{1}{2}|\nabla u|^2 + \gamma(\frac{1}{2}u^2 - \frac{1}{4}u^4) \right) dx

and it is crucial that this energy remains constant in time.

The movie (:ref:`mov:kleingordon`) is showing the solution :math:`u`, computed with the
code shown below.

.. _sec:specgal:

Spectral Galerkin formulation
-----------------------------
The PDEs in :eq:`eq:df` and :eq:`eq:du` can be solved with many different
numerical methods. We will here use the `shenfun <https://github.com/spectralDNS/shenfun>`__ software and this software makes use of
the spectral Galerkin method. Being a Galerkin method, we need to reshape the
governing equations into proper variational forms, and this is done by
multiplying  :eq:`eq:df` and :eq:`eq:du` with the complex conjugate of proper
test functions and then integrating
over the domain. To this end we use testfunctions :math:`g\in V(\Omega)`
with Eq. :eq:`eq:df`  and  :math:`v \in V(\Omega)` with Eq. :eq:`eq:du`, where
:math:`V(\omega)` is a suitable function space, and obtain

.. math::
   :label: eq:df_var

        \frac{\partial}{\partial t} \int_{\Omega} f\, \overline{g}\, w \,dx = \int_{\Omega}
        \left(\nabla^2 u - \gamma( u\, - u|u|^2) \right) \overline{g} \, w \,dx,

.. math::
   :label: eq:kg:du_var

        \frac{\partial }{\partial t} \int_{\Omega} u\, \overline{v}\, w \, dx =
        \int_{\Omega} f\, \overline{v} \, w \, dx.

Note that the overline is used to indicate a complex conjugate, and
:math:`w` is a weight function associated with the test functions. The functions
:math:`f` and :math:`u` are now
to be considered as trial functions, and the integrals over the
domain are often referred to as inner products. With inner product notation

.. math::

        \left(u, v\right) = \int_{\Omega} u \, \overline{v} \, w\, dx.

and an integration by parts on the Laplacian, the variational problem can be
formulated as:

.. math::
   :label: eq:df_var2

        \frac{\partial}{\partial t} (f, g) = -(\nabla u, \nabla g)
        -\gamma \left( u - u|u|^2, g \right),

.. math::
   :label: eq:kg:du_var2

        \frac{\partial }{\partial t} (u, v) = (f, v).

The time and space discretizations are
still left open. There are numerous different approaches that one could take for
discretizing in time, and the first two terms on the right hand side of
:eq:`eq:df_var2` can easily be treated implicitly as well as explicitly. However,
the approach we will follow in Sec. (:ref:`sec:rk`) is a fully explicit 4th order `Runge-Kutta <https://en.wikipedia.org/wiki/Runge-Kutta_methods>`__ method.

Discretization
--------------
To find a numerical solution we need to discretize the continuous problem
:eq:`eq:df_var2` and :eq:`eq:kg:du_var2` in space as well as time. Since the
problem is triply periodic, Fourier exponentials are normally the best choice
for trial and test functions, and as such we use basis functions

.. math::
   :label: _auto4

        \phi_l(x) = e^{\imath \underline{l} x}, \quad -\infty < l < \infty,

where :math:`l` is the wavenumber, and
:math:`\underline{l}=\frac{2\pi}{L}l` is the scaled wavenumber, scaled with domain
length :math:`L` (here :math:`4\pi`). Since we want to solve these equations on a computer, we need to choose
a finite number of test functions. A function space :math:`V^N` can be defined as

.. math::
   :label: eq:kg:Vn

        V^N(x) = \text{span} \{\phi_l(x)\}_{l\in \boldsymbol{l}},

where :math:`N` is chosen as an even positive integer and :math:`\boldsymbol{l} = (-N/2,
-N/2+1, \ldots, N/2-1)`. And now, since :math:`\Omega` is a
three-dimensional domain, we can create tensor products of such bases to get,
e.g., for three dimensions

.. math::
   :label: eq:kg:Wn

        W^{\boldsymbol{N}}(x, y, z) = V^N(x) \otimes V^N(y) \otimes V^N(z),

where :math:`\boldsymbol{N} = (N, N, N)`. Obviously, it is not necessary to use the
same number (:math:`N`) of basis functions for each direction, but it is done here
for simplicity. A 3D tensor product basis function is now defined as

.. math::
   :label: _auto5

        \Phi_{lmn}(x,y,z) = e^{\imath \underline{l} x} e^{\imath \underline{m} y}
        e^{\imath \underline{n} z} = e^{\imath
        (\underline{l}x + \underline{m}y + \underline{n}z)}

where the indices for :math:`y`- and :math:`z`-direction are :math:`\underline{m}=\frac{2\pi}{L}m,
\underline{n}=\frac{2\pi}{L}n`, and :math:`\boldsymbol{m}` and :math:`\boldsymbol{n}` are the same as
:math:`\boldsymbol{l}` due to using the same number of basis functions for each direction. One
distinction, though, is that for the :math:`z`-direction expansion coefficients are only stored for
:math:`n=(0, 1, \ldots, N/2)` due to Hermitian symmetry (real input data).

We now look for solutions of the form

.. math::
   :label: eq:usg

        u(x, y, z, t) = \sum_{n=-N/2}^{N/2-1}\sum_{m=-N/2}^{N/2-1}\sum_{l=-N/2}^{N/2-1}
        \hat{u}_{lmn} (t)\Phi_{lmn}(x,y,z).

The expansion coefficients :math:`\hat{\boldsymbol{u}} = \{\hat{u}_{lmn}(t)\}_{(l,m,n) \in \boldsymbol{l} \times \boldsymbol{m} \times \boldsymbol{n}}`
can be related directly to the solution :math:`u(x, y, z, t)` using Fast
Fourier Transforms (FFTs) if we are satisfied with obtaining
the solution in quadrature points corresponding to

.. math::
   :label: _auto6

         x_i = \frac{4 \pi i}{N}-2\pi \quad \forall \, i \in \boldsymbol{i},
        \text{where}\, \boldsymbol{i}=(0,1,\ldots,N-1),

.. math::
   :label: _auto7

         y_j = \frac{4 \pi j}{N}-2\pi \quad \forall \, j \in \boldsymbol{j},
        \text{where}\, \boldsymbol{j}=(0,1,\ldots,N-1),

.. math::
   :label: _auto8

         z_k = \frac{4 \pi k}{N}-2\pi \quad \forall \, k \in \boldsymbol{k},
        \text{where}\, \boldsymbol{k}=(0,1,\ldots,N-1).

Note that these points are different from the standard (like :math:`2\pi j/N`) since
the domain
is set to :math:`[-2\pi, 2\pi]^3` and not the more common :math:`[0, 2\pi]^3`. We have

.. math::
   :label: eq:uxyz

        \boldsymbol{u} = \mathcal{F}_k^{-1}\left(\mathcal{F}_j^{-1}\left(\mathcal{F}_i^{-1}\left(\hat{\boldsymbol{u}}\right)\right)\right)

with :math:`\boldsymbol{u} = \{u(x_i, y_j, z_k)\}_{(i,j,k)\in \boldsymbol{i} \times \boldsymbol{j} \times \boldsymbol{k}}`
and where :math:`\mathcal{F}_i^{-1}` is the inverse Fourier transform along the direction
of index :math:`i`, for
all :math:`(j, k) \in \boldsymbol{j} \times \boldsymbol{k}`. Note that the three
inverse FFTs are performed sequentially, one direction at the time, and that there is no
scaling factor due to
the definition used for the inverse `Fourier transform <https://mpi4py-fft.readthedocs.io/en/latest/dft.html>`__

.. math::
   :label: _auto9

        u(x_j) = \sum_{l=-N/2}^{N/2-1} \hat{u}_l e^{\imath \underline{l}
        x_j}, \quad \,\, \forall \, j \in \, \boldsymbol{j}.

Note that this differs from the definition used by, e.g.,
`Numpy <https://docs.scipy.org/doc/numpy-1.13.0/reference/routines.fft.html>`__.

The inner products used in Eqs. :eq:`eq:df_var2`, :eq:`eq:kg:du_var2` may be
computed using forward FFTs. However, there is a tiny detail that deserves
a comment. The regular Fourier inner product is given as

.. math::
        \int_{0}^{L} e^{\imath \underline{k}x} e^{- \imath \underline{l}x} dx = L\, \delta_{kl}

where a weight function is chosen as :math:`w(x) = 1` and :math:`\delta_{kl}` equals unity
for :math:`k=l` and zero otherwise. In Shenfun we choose instead to use a weight
function :math:`w(x)=1/L`, such that the weighted inner product integrates to
unity:

.. math::
        \int_{0}^{L} e^{\imath \underline{k}x} e^{- \imath \underline{l}x} \frac{1}{L} dx = \delta_{kl}.

With this weight function the scalar product and the forward transform
are the same and we obtain:

.. math::
   :label: _auto10

        \left(u, \Phi_{lmn}\right) = \hat{u}_{lmn} =
        \left(\frac{1}{N}\right)^3
        \mathcal{F}_l\left(\mathcal{F}_m\left(\mathcal{F}_n\left(\boldsymbol{u}\right)\right)\right)
        \quad \forall (l,m,n) \in \boldsymbol{l} \times \boldsymbol{m} \times
        \boldsymbol{n},

From this we see that the variational forms :eq:`eq:df_var2` and :eq:`eq:kg:du_var2`
may be written in terms of the Fourier transformed quantities :math:`\hat{\boldsymbol{u}}` and
:math:`\hat{\boldsymbol{f}}`. Expanding the exact derivatives of the nabla operator, we have

.. math::
   :label: _auto11

        (\nabla u, \nabla v) =
        (\underline{l}^2+\underline{m}^2+\underline{n}^2)\hat{u}_{lmn},

.. math::
   :label: _auto12

        (u, v) = \hat{u}_{lmn},

.. math::
   :label: _auto13

        (u|u|^2, v) = \widehat{u|u|^2}_{lmn}

and as such the equations to be solved for each wavenumber can be found directly as

.. math::
   :label: eq:df_var3

        \frac{\partial \hat{f}_{lmn}}{\partial t}  =
        \left(-(\underline{l}^2+\underline{m}^2+\underline{n}^2+\gamma)\hat{u}_{lnm} + \gamma \widehat{u|u|^2}_{lnm}\right),

.. math::
   :label: eq:kg:du_var3

        \frac{\partial \hat{u}_{lnm}}{\partial t} = \hat{f}_{lnm}.

There is more than one way to arrive at these equations. Taking the 3D Fourier
transform of both equations  :eq:`eq:df` and :eq:`eq:du` is one obvious way.
With the Python module `shenfun <https://github.com/spectralDNS/shenfun>`__, one can work with the
inner products as seen in :eq:`eq:df_var2` and :eq:`eq:kg:du_var2`, or the Fourier
transforms directly. See for example Sec. :ref:`sec:rk` for how :math:`(\nabla u, \nabla
v)` can be
implemented.  In short, :mod:`.shenfun` contains all the tools required to work with
the spectral Galerkin method, and we will now see how :mod:`.shenfun` can be used to solve
the Klein-Gordon equation.

For completion, we note that the discretized problem to solve can be formulated
with the Galerkin method as:
for all :math:`t>0`, find :math:`(f, u) \in W^N \times W^N`  such that

.. math::
   :label: eq:dff

        \frac{\partial}{\partial t} (f, g) = -(\nabla u, \nabla g)
        -\gamma \left( u - u|u|^2, g \right),

.. math::
   :label: eq:kg:duu

        \frac{\partial }{\partial t} (u, v) = (f, v) \quad \forall \, (g, v) \in W^N \times W^N.

where :math:`u(x, y, z, 0)` and :math:`f(x, y, z, 0)` are given as the initial conditions
according to Eq. :eq:`eq:init`.

Implementation
--------------

To solve the Klein-Gordon equations we need to make use of the Fourier function
spaces defined in
:mod:`.shenfun`, and these are found in submodule
:mod:`shenfun.fourier.bases`.
The triply periodic domain allows for Fourier in all three directions, and we
can as such create one instance of this space using :func:`.FunctionSpace` with
family ``Fourier``
for each direction. However, since the initial data are real, we
can take advantage of Hermitian symmetries and thus make use of a
real to complex class for one (but only one) of the directions, by specifying
``dtype='d'``. We can only make use of the
real-to-complex class for the direction that we choose to transform first with the forward
FFT, and the reason is obviously that the output from a forward transform of
real data is now complex. We may start implementing the solver as follows

.. code-block:: python

    from shenfun import *
    import numpy as np
    import sympy as sp

    # Set size of discretization
    N = (32, 32, 32)

    # Defocusing or focusing
    gamma = 1

    rank = comm.Get_rank()

    # Create function spaces
    K0 = FunctionSpace(N[0], 'F', domain=(-2*np.pi, 2*np.pi), dtype='D')
    K1 = FunctionSpace(N[1], 'F', domain=(-2*np.pi, 2*np.pi), dtype='D')
    K2 = FunctionSpace(N[2], 'F', domain=(-2*np.pi, 2*np.pi), dtype='d')

We now have three instances ``K0``, ``K1`` and ``K2``, corresponding to the space
:eq:`eq:kg:Vn`, that each can be used to solve
one-dimensional problems. However, we want to solve a 3D problem, and for this
we need a tensor product space, like :eq:`eq:kg:Wn`, created as a tensor
product of these three spaces

.. code-block:: python

    T = TensorProductSpace(comm, (K0, K1, K2), **{'planner_effort':
                                                  'FFTW_MEASURE'})

Here the ``planner_effort``, which is a flag used by `FFTW <http://www.fftw.org>`__, is optional. Possibel choices are from the list
(``FFTW_ESTIMATE``, ``FFTW_MEASURE``, ``FFTW_PATIENT``, ``FFTW_EXHAUSTIVE``), and the
flag determines how much effort FFTW puts in looking for an optimal algorithm
for the current platform. Note that it is also possible to use FFTW `wisdom <http://www.fftw.org/fftw3_doc/Wisdom.html#Wisdom>`__ with
``shenfun``, and as such, for production, one may perform exhaustive planning once
and then simply import the result of that planning later, as wisdom.

The :class:`.TensorProductSpace` instance ``T`` contains pretty much all we need for
computing inner products or fast transforms between real and wavenumber space.
However, since we are going to solve for a mixed system, it is convenient to also use the
:class:`.CompositeSpace` class

.. code-block:: python

    TT = CompositeSpace([T, T])
    TV = VectorSpace(T)

Here the space ``TV`` will be used to compute gradients, which
explains why it is a vector.

We need containers for the solution as well as intermediate work arrays for,
e.g., the Runge-Kutta method. Arrays are created using
`Sympy <http://www.sympy.org/en/index.html>`__ for
initialization. Below ``f`` is initialized to 0,
whereas ``u = 0.1*sp.exp(-(x**2 + y**2 + z**2))``.

.. code-block:: python

    # Use sympy to set up initial condition
    x, y, z = sp.symbols("x,y,z", real=True)
    ue = 0.1*sp.exp(-(x**2 + y**2 + z**2))

    fu = Array(TT, buffer=(0, ue)) # Solution array in physical space
    f, u = fu                # Split solution array by creating two views u and f
    dfu = Function(TT)       # Array for right hand sides
    df, du = dfu             # Split into views
    Tp = T.get_dealiased((1.5, 1.5, 1.5))
    up = Array(Tp)           # Work array

    fu_hat = Function(TT)    # Solution in spectral space
    fu_hat = fu.forward()
    f_hat, u_hat = fu_hat

    gradu = Array(TV)        # Solution array for gradient

The :class:`.Array` class is a subclass of Numpy's `ndarray <https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.html>`__,
without much more functionality than constructors that return arrays of the
correct shape according to the basis used in the construction. The
:class:`.Array` represents the left hand side of :eq:`eq:usg`,
evaluated on the quadrature mesh. A different type
of array is returned by the :class:`.Function`
class, that subclasses both Nympy's ndarray as well as an internal
:class:`.BasisFunction`
class. An instance of the :class:`.Function` represents the entire
spectral Galerkin function :eq:`eq:usg`.

.. _sec:rk:

Runge-Kutta integrator
~~~~~~~~~~~~~~~~~~~~~~

We use an explicit fourth order Runge-Kutta integrator,
imported from :mod:`shenfun.utilities.integrators`. The solver
requires one function to compute nonlinear terms,
and one to compute linear. But here we will make
just one function that computes both, and call it
``NonlinearRHS``:

.. code-block:: python

    uh = TrialFunction(T)
    vh = TestFunction(T)
    L = inner(grad(vh), -grad(uh)) + [inner(vh, -gamma*uh)]
    L = la.SolverDiagonal(L).mat.scale

    def NonlinearRHS(self, fu, fu_hat, dfu_hat, **par):
        global count, up
        dfu_hat.fill(0)
        f_hat, u_hat = fu_hat
        df_hat, du_hat = dfu_hat
        up = Tp.backward(u_hat, up)
        df_hat = Tp.forward(gamma*up**3, df_hat)
        df_hat += L*u_hat
        du_hat[:] = f_hat
        return dfu_hat

Note that ``L`` now is an array that represents the linear
coefficients in :eq:`eq:kg:du_var3`.

All that is left is to write a function that is called
on each time step, which will allow us to store intermediate
solutions, compute intermediate energies, and plot
intermediate solutions. Since we will plot the same plot
many times, we create the figure first, and then simply update
the plotted arrays in the ``update`` function.

.. code-block:: python

    import matplotlib.pyplot as plt
    X = T.local_mesh(True)
    if rank == 0:
        plt.figure()
        image = plt.contourf(X[1][..., 0], X[0][..., 0], u[..., N[2]//2], 100)
        plt.draw()
        plt.pause(1e-6)

The actual ``update`` function is

.. code-block:: python

    # Get wavenumbers
    K = np.array(T.local_wavenumbers(True, True, True))

    def update(self, fu, fu_hat, t, tstep, **params):
        global gradu

        transformed = False
        if rank == 0 and tstep % params['plot_tstep'] == 0 and params['plot_tstep'] > 0:
            fu = fu_hat.backward(fu)
            f, u = fu[:]
            image.axes.clear()
            image.axes.contourf(X[1][..., 0], X[0][..., 0], u[..., N[2]//2], 100)
            plt.pause(1e-6)
            transformed = True

        if tstep % params['Compute_energy'] == 0:
            if transformed is False:
                fu = fu_hat.backward(fu)
            f, u = fu
            f_hat, u_hat = fu_hat
            ekin = 0.5*energy_fourier(f_hat, T)
            es = 0.5*energy_fourier(1j*(K*u_hat), T)
            eg = gamma*np.sum(0.5*u**2 - 0.25*u**4)/np.prod(np.array(N))
            eg = comm.allreduce(eg)
            gradu = TV.backward(1j*(K[0]*u_hat[0]+K[1]*u_hat[1]+K[2]*u_hat[2]), gradu)
            ep = comm.allreduce(np.sum(f*gradu)/np.prod(np.array(N)))
            ea = comm.allreduce(np.sum(np.array(X)*(0.5*f**2 + 0.5*gradu**2 - (0.5*u**2 - 0.25*u**4)*f))/np.prod(np.array(N)))
            if rank == 0:
                print("Time = %2.2f Total energy = %2.8e Linear momentum %2.8e Angular momentum %2.8e" %(t, ekin+es+eg, ep, ea))
            comm.barrier()

With all functions in place, the actual integrator
can be created and called as

.. code-block:: python

    par = {'Compute_energy': 10,
           'plot_tstep': 10,
           'end_time': 1}
    dt = 0.005
    integrator = RK4(TT, N=NonlinearRHS, update=update, **par)
    integrator.setup(dt)
    fu_hat = integrator.solve(fu, fu_hat, dt, (0, par['end_time']))

A complete solver is found `here <https://github.com/spectralDNS/shenfun/blob/master/demo/KleinGordon.py>`__.

.. ======= Bibliography =======


text:
package eu.modernmt.api.framework.routing;

import eu.modernmt.api.framework.HttpMethod;

import java.util.ArrayList;
import java.util.HashMap;

public class RouteTree {

    private Node root = new Node();

    public RouteTree() {
        root = new Node();
    }

    public void add(RouteTemplate route) {
        this.add(root, route, 0);
    }

    private void add(Node parent, RouteTemplate template, int depth) {
        String token = template.getTokenAt(depth);

        if (token == null) {
            parent.setValue(template);
            return;
        }

        boolean isVar = template.isTokenVariable(token);

        String key = isVar ? "*" : token;

        Node child = parent.childs.computeIfAbsent(key, k -> new Node());

        if (template.size() - 1 == depth)
            child.setValue(template);
        else
            this.add(child, template, depth + 1);
    }

    public RouteTemplate get(HttpMethod method, String path) {
        ArrayList<Node> nodes = new ArrayList<>();
        ArrayList<Node> newNodes = new ArrayList<>();
        nodes.add(root);

        for (String token : RouteTemplate.tokenize(path)) {
            newNodes.clear();
            for (Node node : nodes) {
                Node child = node.childs.get(token);
                if (child != null)
                    newNodes.add(child);
                child = node.childs.get("*");
                if (child != null)
                    newNodes.add(child);
            }

            nodes.clear();
            nodes.addAll(newNodes);
        }

        if (nodes.isEmpty())
            return null;
        else
            return nodes.get(0).getValue(method);
    }

    protected static class Node {

        public final HashMap<String, Node> childs = new HashMap<>();
        private HashMap<HttpMethod, RouteTemplate> values = new HashMap<>();

        public void setValue(RouteTemplate value) {
            this.values.put(value.getMethod(), value);
        }

        public boolean hasValue(HttpMethod method) {
            return values.containsKey(method);
        }

        public RouteTemplate getValue(HttpMethod method) {
            return values.get(method);
        }

    }

    @Override
    public String toString() {
        return this.root.toString();
    }

}

redpajama_set_name: RedPajamaGithub


text:
'use strict';

import { EventEmitter } from 'events';
import * as path from 'path';
import { StringDecoder, NodeStringDecoder } from 'string_decoder';

import * as cp from 'child_process';
import { rgPath } from 'vscode-ripgrep';

import objects = require('vs/base/common/objects');
import platform = require('vs/base/common/platform');
import * as strings from 'vs/base/common/strings';
import * as paths from 'vs/base/common/paths';
import * as extfs from 'vs/base/node/extfs';
import * as encoding from 'vs/base/node/encoding';
import * as glob from 'vs/base/common/glob';
import { ISearchLog } from 'vs/platform/search/common/search';
import { TPromise } from 'vs/base/common/winjs.base';

import { ISerializedFileMatch, ISerializedSearchComplete, IRawSearch, IFolderSearch, LineMatch, FileMatch } from './search';

export class RipgrepEngine {
	private isDone = false;
	private rgProc: cp.ChildProcess;
	private killRgProcFn: Function;
	private postProcessExclusions: glob.ParsedExpression;

	private ripgrepParser: RipgrepParser;

	private resultsHandledP: TPromise<any> = TPromise.wrap(null);

	constructor(private config: IRawSearch) {
		this.killRgProcFn = () => this.rgProc && this.rgProc.kill();
	}

	cancel(): void {
		this.isDone = true;
		this.ripgrepParser.cancel();
		this.rgProc.kill();
	}

	// TODO@Rob - make promise-based once the old search is gone, and I don't need them to have matching interfaces anymore
	search(onResult: (match: ISerializedFileMatch) => void, onMessage: (message: ISearchLog) => void, done: (error: Error, complete: ISerializedSearchComplete) => void): void {
		if (!this.config.folderQueries.length && !this.config.extraFiles.length) {
			process.removeListener('exit', this.killRgProcFn);
			done(null, {
				limitHit: false,
				stats: null
			});
			return;
		}

		const rgArgs = getRgArgs(this.config);
		if (rgArgs.siblingClauses) {
			this.postProcessExclusions = glob.parseToAsync(rgArgs.siblingClauses, { trimForExclusions: true });
		}

		const cwd = platform.isWindows ? 'c:/' : '/';
		process.nextTick(() => { // Allow caller to register progress callback
			const escapedArgs = rgArgs.globArgs
				.map(arg => arg.match(/^-/) ? arg : `'${arg}'`)
				.join(' ');

			const rgCmd = `rg ${escapedArgs}\n - cwd: ${cwd}\n`;
			onMessage({ message: rgCmd });
			if (rgArgs.siblingClauses) {
				onMessage({ message: ` - Sibling clauses: ${JSON.stringify(rgArgs.siblingClauses)}\n` });
			}
		});
		this.rgProc = cp.spawn(rgPath, rgArgs.globArgs, { cwd });
		process.once('exit', this.killRgProcFn);

		this.ripgrepParser = new RipgrepParser(this.config.maxResults, cwd, this.config.extraFiles);
		this.ripgrepParser.on('result', (match: ISerializedFileMatch) => {
			if (this.postProcessExclusions) {
				const handleResultP = (<TPromise<string>>this.postProcessExclusions(match.path, undefined, () => getSiblings(match.path)))
					.then(globMatch => {
						if (!globMatch) {
							onResult(match);
						}
					});

				this.resultsHandledP = TPromise.join([this.resultsHandledP, handleResultP]);
			} else {
				onResult(match);
			}
		});
		this.ripgrepParser.on('hitLimit', () => {
			this.cancel();
			process.removeListener('exit', this.killRgProcFn);
			done(null, {
				limitHit: true,
				stats: null
			});
		});

		this.rgProc.stdout.on('data', data => {
			this.ripgrepParser.handleData(data);
		});

		let gotData = false;
		this.rgProc.stdout.once('data', () => gotData = true);

		let stderr = '';
		this.rgProc.stderr.on('data', data => {
			const message = data.toString();
			onMessage({ message });
			stderr += message;
		});

		this.rgProc.on('close', code => {
			// Trigger last result, then wait on async result handling
			this.ripgrepParser.flush();
			this.resultsHandledP.then(() => {
				this.rgProc = null;
				if (!this.isDone) {
					this.isDone = true;
					let displayMsg: string;
					process.removeListener('exit', this.killRgProcFn);
					if (stderr && !gotData && (displayMsg = rgErrorMsgForDisplay(stderr))) {
						done(new Error(displayMsg), {
							limitHit: false,
							stats: null
						});
					} else {
						done(null, {
							limitHit: false,
							stats: null
						});
					}
				}
			});
		});
	}
}

/**
 * Read the first line of stderr and return an error for display or undefined, based on a whitelist.
 * Ripgrep produces stderr output which is not from a fatal error, and we only want the search to be
 * "failed" when a fatal error was produced.
 */
export function rgErrorMsgForDisplay(msg: string): string | undefined {
	const firstLine = msg.split('\n')[0];

	if (strings.startsWith(firstLine, 'Error parsing regex')) {
		return firstLine;
	}

	if (strings.startsWith(firstLine, 'error parsing glob') ||
		strings.startsWith(firstLine, 'unsupported encoding')) {
		// Uppercase first letter
		return firstLine.charAt(0).toUpperCase() + firstLine.substr(1);
	}

	return undefined;
}

export class RipgrepParser extends EventEmitter {
	private static readonly RESULT_REGEX = /^\u001b\[m(\d+)\u001b\[m:(.*)(\r?)/;
	private static readonly FILE_REGEX = /^\u001b\[m(.+)\u001b\[m$/;

	public static readonly MATCH_START_MARKER = '\u001b[m\u001b[31m';
	public static readonly MATCH_END_MARKER = '\u001b[m';

	private fileMatch: FileMatch;
	private remainder: string;
	private isDone: boolean;
	private stringDecoder: NodeStringDecoder;
	private extraSearchFiles: string[];

	private numResults = 0;

	constructor(private maxResults: number, private rootFolder: string, extraFiles?: string[]) {
		super();
		this.stringDecoder = new StringDecoder();

		this.extraSearchFiles = extraFiles || [];
	}

	public cancel(): void {
		this.isDone = true;
	}

	public flush(): void {
		this.handleDecodedData(this.stringDecoder.end());

		if (this.fileMatch) {
			this.onResult();
		}
	}

	public handleData(data: Buffer | string): void {
		const dataStr = typeof data === 'string' ? data : this.stringDecoder.write(data);
		this.handleDecodedData(dataStr);
	}

	private handleDecodedData(decodedData: string): void {
		// If the previous data chunk didn't end in a newline, prepend it to this chunk
		const dataStr = this.remainder ?
			this.remainder + decodedData :
			decodedData;

		const dataLines: string[] = dataStr.split(/\r\n|\n/);
		this.remainder = dataLines[dataLines.length - 1] ? dataLines.pop() : null;

		for (let l = 0; l < dataLines.length; l++) {
			const outputLine = dataLines[l].trim();
			if (this.isDone) {
				break;
			}

			let r: RegExpMatchArray;
			if (r = outputLine.match(RipgrepParser.RESULT_REGEX)) {
				const lineNum = parseInt(r[1]) - 1;
				let matchText = r[2];

				// workaround https://github.com/BurntSushi/ripgrep/issues/416
				// If the match line ended with \r, append a match end marker so the match isn't lost
				if (r[3]) {
					matchText += RipgrepParser.MATCH_END_MARKER;
				}

				// Line is a result - add to collected results for the current file path
				this.handleMatchLine(outputLine, lineNum, matchText);
			} else if (r = outputLine.match(RipgrepParser.FILE_REGEX)) {
				// Line is a file path - send all collected results for the previous file path
				if (this.fileMatch) {
					this.onResult();
				}

				this.fileMatch = this.getFileMatch(r[1]);
			} else {
				// Line is empty (or malformed)
			}
		}
	}

	private getFileMatch(relativeOrAbsolutePath: string): FileMatch {
		const absPath = path.isAbsolute(relativeOrAbsolutePath) ?
			relativeOrAbsolutePath :
			path.join(this.rootFolder, relativeOrAbsolutePath);

		return new FileMatch(absPath);
	}

	private handleMatchLine(outputLine: string, lineNum: number, text: string): void {
		if (lineNum === 0) {
			text = strings.stripUTF8BOM(text);
		}

		const lineMatch = new LineMatch(text, lineNum);
		if (!this.fileMatch) {
			// When searching a single file and no folderQueries, rg does not print the file line, so create it here
			const singleFile = this.extraSearchFiles[0];
			if (!singleFile) {
				throw new Error('Got match line for unknown file');
			}

			this.fileMatch = this.getFileMatch(singleFile);
		}

		this.fileMatch.addMatch(lineMatch);

		let lastMatchEndPos = 0;
		let matchTextStartPos = -1;

		// Track positions with color codes subtracted - offsets in the final text preview result
		let matchTextStartRealIdx = -1;
		let textRealIdx = 0;
		let hitLimit = false;

		const realTextParts: string[] = [];

		for (let i = 0; i < text.length - (RipgrepParser.MATCH_END_MARKER.length - 1);) {
			if (text.substr(i, RipgrepParser.MATCH_START_MARKER.length) === RipgrepParser.MATCH_START_MARKER) {
				// Match start
				const chunk = text.slice(lastMatchEndPos, i);
				realTextParts.push(chunk);
				i += RipgrepParser.MATCH_START_MARKER.length;
				matchTextStartPos = i;
				matchTextStartRealIdx = textRealIdx;
			} else if (text.substr(i, RipgrepParser.MATCH_END_MARKER.length) === RipgrepParser.MATCH_END_MARKER) {
				// Match end
				const chunk = text.slice(matchTextStartPos, i);
				realTextParts.push(chunk);
				if (!hitLimit) {
					lineMatch.addMatch(matchTextStartRealIdx, textRealIdx - matchTextStartRealIdx);
				}

				matchTextStartPos = -1;
				matchTextStartRealIdx = -1;
				i += RipgrepParser.MATCH_END_MARKER.length;
				lastMatchEndPos = i;
				this.numResults++;

				// Check hit maxResults limit
				if (this.numResults >= this.maxResults) {
					// Finish the line, then report the result below
					hitLimit = true;
				}
			} else {
				i++;
				textRealIdx++;
			}
		}

		const chunk = text.slice(lastMatchEndPos);
		realTextParts.push(chunk);

		// Replace preview with version without color codes
		const preview = realTextParts.join('');
		lineMatch.preview = preview;

		if (hitLimit) {
			this.cancel();
			this.onResult();
			this.emit('hitLimit');
		}
	}

	private onResult(): void {
		this.emit('result', this.fileMatch.serialize());
		this.fileMatch = null;
	}
}

export interface IRgGlobResult {
	globArgs: string[];
	siblingClauses: glob.IExpression;
}

export function foldersToRgExcludeGlobs(folderQueries: IFolderSearch[], globalExclude: glob.IExpression, excludesToSkip?: Set<string>, absoluteGlobs = true): IRgGlobResult {
	const globArgs: string[] = [];
	let siblingClauses: glob.IExpression = {};
	folderQueries.forEach(folderQuery => {
		const totalExcludePattern = objects.assign({}, folderQuery.excludePattern || {}, globalExclude || {});
		const result = globExprsToRgGlobs(totalExcludePattern, absoluteGlobs && folderQuery.folder, excludesToSkip);
		globArgs.push(...result.globArgs);
		if (result.siblingClauses) {
			siblingClauses = objects.assign(siblingClauses, result.siblingClauses);
		}
	});

	return { globArgs, siblingClauses };
}

export function foldersToIncludeGlobs(folderQueries: IFolderSearch[], globalInclude: glob.IExpression, absoluteGlobs = true): string[] {
	const globArgs: string[] = [];
	folderQueries.forEach(folderQuery => {
		const totalIncludePattern = objects.assign({}, globalInclude || {}, folderQuery.includePattern || {});
		const result = globExprsToRgGlobs(totalIncludePattern, absoluteGlobs && folderQuery.folder);
		globArgs.push(...result.globArgs);
	});

	return globArgs;
}

function globExprsToRgGlobs(patterns: glob.IExpression, folder?: string, excludesToSkip?: Set<string>): IRgGlobResult {
	const globArgs: string[] = [];
	let siblingClauses: glob.IExpression = null;
	Object.keys(patterns)
		.forEach(key => {
			if (excludesToSkip && excludesToSkip.has(key)) {
				return;
			}

			if (!key) {
				return;
			}

			const value = patterns[key];
			key = trimTrailingSlash(folder ? getAbsoluteGlob(folder, key) : key);

			// glob.ts requires forward slashes, but a UNC path still must start with \\
			// #38165 and #38151
			if (strings.startsWith(key, '\\\\')) {
				key = '\\\\' + key.substr(2).replace(/\\/g, '/');
			} else {
				key = key.replace(/\\/g, '/');
			}

			if (typeof value === 'boolean' && value) {
				globArgs.push(fixDriveC(key));
			} else if (value && value.when) {
				if (!siblingClauses) {
					siblingClauses = {};
				}

				siblingClauses[key] = value;
			}
		});

	return { globArgs, siblingClauses };
}

/**
 * Resolves a glob like "node_modules/**" in "/foo/bar" to "/foo/bar/node_modules/**".
 * Special cases C:/foo paths to write the glob like /foo instead - see https://github.com/BurntSushi/ripgrep/issues/530.
 *
 * Exported for testing
 */
export function getAbsoluteGlob(folder: string, key: string): string {
	return paths.isAbsolute(key) ?
		key :
		path.join(folder, key);
}

function trimTrailingSlash(str: string): string {
	str = strings.rtrim(str, '\\');
	return strings.rtrim(str, '/');
}

export function fixDriveC(path: string): string {
	const root = paths.getRoot(path);
	return root.toLowerCase() === 'c:/' ?
		path.replace(/^c:[/\\]/i, '/') :
		path;
}

function getRgArgs(config: IRawSearch): IRgGlobResult {
	const args = ['--hidden', '--heading', '--line-number', '--color', 'ansi', '--colors', 'path:none', '--colors', 'line:none', '--colors', 'match:fg:red', '--colors', 'match:style:nobold'];
	if (config.contentPattern.isSmartCase) {
		args.push('--smart-case');
	} else {
		args.push(config.contentPattern.isCaseSensitive ? '--case-sensitive' : '--ignore-case');
	}

	// includePattern can't have siblingClauses
	foldersToIncludeGlobs(config.folderQueries, config.includePattern).forEach(globArg => {
		args.push('-g', globArg);
	});

	let siblingClauses: glob.IExpression;

	// Find excludes that are exactly the same in all folderQueries - e.g. from user settings, and that start with `**`.
	// To make the command shorter, don't resolve these against every folderQuery path - see #33189.
	const universalExcludes = findUniversalExcludes(config.folderQueries);
	const rgGlobs = foldersToRgExcludeGlobs(config.folderQueries, config.excludePattern, universalExcludes);
	rgGlobs.globArgs
		.forEach(rgGlob => args.push('-g', `!${rgGlob}`));
	if (universalExcludes) {
		universalExcludes
			.forEach(exclude => args.push('-g', `!${trimTrailingSlash(exclude)}`));
	}
	siblingClauses = rgGlobs.siblingClauses;

	if (config.maxFilesize) {
		args.push('--max-filesize', config.maxFilesize + '');
	}

	if (config.disregardIgnoreFiles) {
		// Don't use .gitignore or .ignore
		args.push('--no-ignore');
	} else {
		args.push('--no-ignore-parent');
	}

	// Follow symlinks
	if (!config.ignoreSymlinks) {
		args.push('--follow');
	}

	// Set default encoding if only one folder is opened
	if (config.folderQueries.length === 1 && config.folderQueries[0].fileEncoding && config.folderQueries[0].fileEncoding !== 'utf8') {
		args.push('--encoding', encoding.toCanonicalName(config.folderQueries[0].fileEncoding));
	}

	// Ripgrep handles -- as a -- arg separator. Only --.
	// - is ok, --- is ok, --some-flag is handled as query text. Need to special case.
	if (config.contentPattern.pattern === '--') {
		config.contentPattern.isRegExp = true;
		config.contentPattern.pattern = '\\-\\-';
	}

	let searchPatternAfterDoubleDashes: string;
	if (config.contentPattern.isWordMatch) {
		const regexp = strings.createRegExp(config.contentPattern.pattern, config.contentPattern.isRegExp, { wholeWord: config.contentPattern.isWordMatch });
		const regexpStr = regexp.source.replace(/\\\//g, '/'); // RegExp.source arbitrarily returns escaped slashes. Search and destroy.
		args.push('--regexp', regexpStr);
	} else if (config.contentPattern.isRegExp) {
		args.push('--regexp', config.contentPattern.pattern);
	} else {
		searchPatternAfterDoubleDashes = config.contentPattern.pattern;
		args.push('--fixed-strings');
	}

	// Folder to search
	args.push('--');

	if (searchPatternAfterDoubleDashes) {
		// Put the query after --, in case the query starts with a dash
		args.push(searchPatternAfterDoubleDashes);
	}

	args.push(...config.folderQueries.map(q => q.folder));
	args.push(...config.extraFiles);

	return { globArgs: args, siblingClauses };
}

function getSiblings(file: string): TPromise<string[]> {
	return new TPromise<string[]>((resolve, reject) => {
		extfs.readdir(path.dirname(file), (error: Error, files: string[]) => {
			if (error) {
				reject(error);
			}

			resolve(files);
		});
	});
}

function findUniversalExcludes(folderQueries: IFolderSearch[]): Set<string> {
	if (folderQueries.length < 2) {
		// Nothing to simplify
		return null;
	}

	const firstFolder = folderQueries[0];
	if (!firstFolder.excludePattern) {
		return null;
	}

	const universalExcludes = new Set<string>();
	Object.keys(firstFolder.excludePattern).forEach(key => {
		if (strings.startsWith(key, '**') && folderQueries.every(q => q.excludePattern && q.excludePattern[key] === true)) {
			universalExcludes.add(key);
		}
	});

	return universalExcludes;
}


text:
#import "NSData.h"

@interface NSData (HSAdditions)
+ (id)mappedDataWithContentsOfTemporaryFile:(id)arg1 error:(id *)arg2;
- (id)decompressedGzipDataUsingTemporaryFile;
- (id)decompressedGzipData;
@end


text:
uuid: ec688aff-7c8d-431b-8e6b-debbfc47e477
layout: speaking
title: "Let's play with Node.js streams"
slug: 'lets-play-with-nodejs-streams-codemotion'
date: 2021-03-23T20:30:00.000Z
author: Luciano Mammino
author_slug: luciano-mammino
header_img: null
status: published
language: it_IT
presentation_language: it_IT
meta_title: null
meta_description: null
event_name: 'Codemotion'
event_link: 'https://events.codemotion.com/conferences/online/2021/online-tech-conference-italian-edition-spring/workshops'
event_location: 'Remote'
event_location_gps: null
event_city: 'remote'
event_days: 'March 23-25 2021'
is_workshop: true
slides_link: 'https://loige.link/streams-cm'
video_link: null
with: []
---

Gli streams Node.js rappresentano un argomento ancora piuttosto sconosciuto e talvolta addirittura temuto, nonostante costituiscano uno dei concetti alla base dell'ecosistema Node.js. In questo workshop, cercheremo di esplorare gli stream Node.js in modo semplice cosi da riuscire ad impararne i costrutti di base, utili per le situazioni piú comuni, fino ad affrontare esempi piú avanzati come la creazione di stream personalizzati. Alla fine di questo workshop riuscirai a sfruttare gli streams Node.js e sarai in grado di risolvere problematiche comuni in modo idiomatico ed efficiente.

## Contenuti

 - Streams: quando e come
 - Diversi tipi di stream
 - Streams predefiniti e streams personalizzati
 - Componibilitá
 - Utility per gli stream
 - Streams e async iterators


text:
<?php

namespace Magento\Theme\Controller\Adminhtml\System\Design\Wysiwyg\Files;

class NewFolder extends \Magento\Theme\Controller\Adminhtml\System\Design\Wysiwyg\Files
{
    /**
     * New folder action
     *
     * @return void
     */
    public function execute()
    {
        $name = $this->getRequest()->getPost('name');
        try {
            $path = $this->storage->getCurrentPath();
            $result = $this->_getStorage()->createFolder($name, $path);
        } catch (\Magento\Framework\Exception\LocalizedException $e) {
            $result = ['error' => true, 'message' => $e->getMessage()];
        } catch (\Exception $e) {
            $result = ['error' => true, 'message' => __('Sorry, something went wrong. That\'s all we know.')];
            $this->_objectManager->get('Psr\Log\LoggerInterface')->critical($e);
        }
        $this->getResponse()->representJson(
            $this->_objectManager->get('Magento\Framework\Json\Helper\Data')->jsonEncode($result)
        );
    }
}


text:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "//www.w3.org/TR/html4/strict.dtd">

<meta name="generator" content="JDiff v1.1.0">

<meta name="description" content="JDiff is a Javadoc doclet which generates an HTML report of all the packages, classes, constructors, methods, and fields which have been removed, added or changed in any way, including their documentation, when two APIs are compared.">
<meta name="keywords" content="diff, jdiff, javadiff, java diff, java difference, API difference, difference between two APIs, API diff, Javadoc, doclet">
<TITLE>
All Changes Index
</TITLE>
<link href="../../../../assets/android-developer-docs.css" rel="stylesheet" type="text/css" />
<link href="../stylesheet-jdiff.css" rel="stylesheet" type="text/css" />

    Filter the Index:

All Differences

Removals

Additions

Changes

Listed as: Added,  Removed,  Changed

A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

<nobr>A_8
</nobr>

AbsListView

AccessibilityEvent

AccessibilityManager

AccessibilityNodeInfo

AccessibilityNodeProvider

Activity

ActivityManager

AlarmClock

AlarmManager

Allocation

android

android.animation

android.app

android.bluetooth

android.content

android.content.pm

android.content.res

android.database

android.graphics

android.graphics.drawable

android.hardware

android.hardware.display

android.location

android.media

android.media.audiofx

android.net

android.net.wifi

android.nfc

android.opengl

android.os

android.preference

android.provider

android.renderscript

android.security

android.speech.tts

android.telephony

android.test.mock

android.text.method

android.util

android.view

android.view.accessibility

android.view.inputmethod

android.webkit

android.widget

Animator

AnimatorListenerAdapter

AssertionError

AssetFileDescriptor

AudioEffect

AudioManager

AudioTrack

B
A
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

Bitmap

BitmapDrawable

BitmapFactory.Options

BitSet

BluetoothAssignedNumbers

BluetoothDevice

BluetoothGatt

BluetoothHeadset

Boolean

build

  <nobr>type
() in android.app.Notification.BigPictureStyle
</nobr>

  <nobr>type
() in android.app.Notification.BigTextStyle
</nobr>

  <nobr>type
() in android.app.Notification.InboxStyle
</nobr>

  <nobr>type
() in android.app.Notification.Style
</nobr>

Build.VERSION_CODES

Byte

C
A
B
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

CallLog.Calls

<nobr>capturePicture
()</nobr>

Character

Character.UnicodeBlock

CharBuffer

Cipher

CipherSpi

ClassNotFoundException

<nobr>clear
()</nobr>

Closeable

Collections

ColorDrawable

ComponentInfo

ConcurrentModificationException

Configuration

Connection

ContentProvider

ContentProviderClient

ContentResolver

Context

ContextWrapper

<nobr>create
(RenderScript, Element)</nobr>

Currency

Cursor

CursorWrapper

D
A
B
C
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

DatagramSocket

Debug.MemoryInfo

Deflater

DeflaterOutputStream

DhcpInfo

Display

DisplayManager

DisplayMetrics

Drawable

DrawableContainer.DrawableContainerState

E
A
B
C
D
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

Element

Environment

<nobr>EXTRA_RINGTONE_INCLUDE_DRM
</nobr>

F
A
B
C
D
E
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

FileInputStream

FileLock

FileOutputStream

FrameLayout.LayoutParams

<nobr>freeMemory
()</nobr>

G
A
B
C
D
E
F
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

<nobr>getAlpha
()</nobr>

<nobr>getDatabasePath
()</nobr>

<nobr>getDefaultZoom
()</nobr>

<nobr>getDhcpInfo
()</nobr>

<nobr>getIncludeDrm
()</nobr>

<nobr>getNativeFrameCount
()</nobr>

GZIPOutputStream

H
A
B
C
D
E
F
G
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

HttpURLConnection

I
A
B
C
D
E
F
G
H
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

IllegalAccessException

ImageFormat

InetAddress

InetSocketAddress

InputDevice

InputMethodManager

InputMethodSubtype

  android.view.inputmethod

  <nobr>InputMethodSubtype
(int, int, String, String, String, boolean, boolean)</nobr> constructor

  <nobr>InputMethodSubtype
(int, int, String, String, String, boolean, boolean, int)</nobr> constructor

InsetDrawable

InstantiationException

Integer

Intent

IntentFilter

InvocationTargetException

<nobr>isLocationProviderEnabled
(ContentResolver, String)</nobr>

java.io

java.lang

java.lang.reflect

java.net

java.nio

java.nio.channels

java.nio.charset

java.sql

java.util

java.util.logging

java.util.zip

javax.crypto

javax.crypto.spec

JSONArray

JSONObject

K
A
B
C
D
E
F
G
H
I
J
L
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

KeyEvent

KeyPairGeneratorSpec

KeyPairGeneratorSpec.Builder

L
A
B
C
D
E
F
G
H
I
J
K
M
N
O
P
R
S
T
U
V
W
X
Z
 TOP

<nobr>L_8
</nobr>

LinearLayout.LayoutParams

LinkageError

ListPopupWindow

ListView

LocalSocket

<nobr>LOCATION_PROVIDERS_ALLOWED
</nobr>

LocationManager

Logger

Long

M
A
B
C
D
E
F
G
H
I
J
K
L
N
O
P
R
S
T
U
V
W
X
Z
 TOP

Manifest.permission

Matrix

  android.opengl

  <nobr>Matrix
()</nobr> constructor

MatrixCursor.RowBuilder

MediaCodec

MediaCodec.CryptoException

MediaCodecInfo.CodecCapabilities

MediaDrm

MediaFormat

MediaMuxer

MediaPlayer

MediaPlayer.TrackInfo

MediaRecorder.AudioSource

MetaKeyKeyListener

MockContext

MockCursor

MockPackageManager

<nobr>MODE_WORLD_READABLE
</nobr>

<nobr>MODE_WORLD_WRITEABLE
</nobr>

Modifier

MotionEvent

N
A
B
C
D
E
F
G
H
I
J
K
L
M
O
P
R
S
T
U
V
W
X
Z
 TOP

NetworkInterface

NfcAdapter

NinePatch

NoSuchFieldException

NoSuchMethodException

Notification

Notification.BigPictureStyle

Notification.BigTextStyle

Notification.Builder

Notification.InboxStyle

Notification.Style

O
A
B
C
D
E
F
G
H
I
J
K
L
M
N
P
R
S
T
U
V
W
X
Z
 TOP

ObjectInput

ObjectOutput

<nobr>onExceededDatabaseQuota
(String, String, long, long, long, QuotaUpdater)</nobr>

<nobr>onReachedMaxAppCacheSize
(long, long, QuotaUpdater)</nobr>

org.json

P
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
R
S
T
U
V
W
X
Z
 TOP

PackageManager

Paint

ParcelFileDescriptor

Path

PixelFormat

PopupMenu

PopupWindow

Preference

PreferenceActivity

ProviderInfo

<nobr>putBitmap
(int, Bitmap)</nobr>

<nobr>putLong
(int, long)</nobr>

<nobr>putString
(int, String)</nobr>

R
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
S
T
U
V
W
X
Z
 TOP

R.attr

R.style

RelativeLayout.LayoutParams

RemoteControlClient

RemoteControlClient.MetadataEditor

ResolveInfo

ResultSet

RingtoneManager

S
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
T
U
V
W
X
Z
 TOP

ScaleGestureDetector

Scanner

ScriptIntrinsicColorMatrix

Selector

Sensor

SensorManager

ServerSocket

<nobr>setDatabasePath
(String)</nobr>

<nobr>setDefaultZoom
(ZoomDensity)</nobr>

<nobr>setIncludeDrm
(boolean)</nobr>

<nobr>setLocationProviderEnabled
(ContentResolver, String, boolean)</nobr>

<nobr>setState
(int)</nobr>

Settings

Settings.Secure

Short

Socket

SparseArray

Statement

<nobr>subSequence
(int, int)</nobr>

Surface.OutOfResourcesException

SurfaceTexture

SurfaceTexture.OutOfResourcesException

SynthesisRequest

System

T
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
U
V
W
X
Z
 TOP

TelephonyManager

Throwable

U
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
V
W
X
Z
 TOP

UserManager

V
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
W
X
Z
 TOP

VideoView

View

ViewGroup

ViewParent

ViewPropertyAnimator

Visualizer

W
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
X
Z
 TOP

WallpaperManager

WebChromeClient

WebSettings

WebStorage.QuotaUpdater

WebView

WifiManager

Window

WindowManager.LayoutParams

X
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
Z
 TOP

XmlResourceParser

Z
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
 TOP

ZipFile


text:
<?php

namespace Tests\Feature;

use App\User;
use Tests\TestCase;
use Illuminate\Foundation\Testing\DatabaseMigrations;

class GetUserFromUsertypeTest extends TestCase
{
    use DatabaseMigrations;

    public function testUserCanFoundByUsertype()
    {
        $createdUser = factory(User::class)->create(['usertype' => 'superadmin']);
        $foundUser = User::getUserFromUsertype('superadmin');
        $this->assertEquals($createdUser->id, $foundUser->id);
        $this->assertEquals($createdUser->usertype, $foundUser->usertype);
    }
}


text:
package com.sequenceiq.cloudbreak.cmtemplate.configproviders.kafka;

import static com.sequenceiq.cloudbreak.cmtemplate.CMRepositoryVersionUtil.CLOUDERA_STACK_VERSION_7_2_14;
import static com.sequenceiq.cloudbreak.cmtemplate.CMRepositoryVersionUtil.isVersionNewerOrEqualThanLimited;
import static com.sequenceiq.cloudbreak.cmtemplate.configproviders.ConfigUtils.config;
import static com.sequenceiq.cloudbreak.cmtemplate.configproviders.kafka.KafkaConfigProviderUtils.getCdhVersion;
import static com.sequenceiq.cloudbreak.cmtemplate.configproviders.kafka.KafkaConfigs.DEFAULT_REPLICATION_FACTOR;
import static com.sequenceiq.cloudbreak.cmtemplate.configproviders.kafka.KafkaConfigs.ENABLE_RACK_AWARENESS;

import java.util.ArrayList;
import java.util.List;

import org.springframework.stereotype.Component;

import com.cloudera.api.swagger.model.ApiClusterTemplateConfig;
import com.google.common.collect.Lists;
import com.sequenceiq.cloudbreak.api.endpoint.v4.common.StackType;
import com.sequenceiq.cloudbreak.cmtemplate.CmTemplateProcessor;
import com.sequenceiq.cloudbreak.cmtemplate.configproviders.AbstractRoleConfigProvider;
import com.sequenceiq.cloudbreak.template.TemplatePreparationObject;

@Component
public class KafkaMultiAzConfigProvider extends AbstractRoleConfigProvider {

    @Override
    public List<ApiClusterTemplateConfig> getServiceConfigs(CmTemplateProcessor templateProcessor, TemplatePreparationObject source) {
        ArrayList<ApiClusterTemplateConfig> configs = Lists.newArrayList();
        String cdhVersion = getCdhVersion(source);
        if (isVersionNewerOrEqualThanLimited(cdhVersion, CLOUDERA_STACK_VERSION_7_2_14)) {
            configs.add(config(DEFAULT_REPLICATION_FACTOR, "3"));
        }
        return configs;
    }

    @Override
    protected List<ApiClusterTemplateConfig> getRoleConfigs(String roleType, TemplatePreparationObject source) {
        ArrayList<ApiClusterTemplateConfig> configs = Lists.newArrayList();
        String cdhVersion = getCdhVersion(source);
        if (isVersionNewerOrEqualThanLimited(cdhVersion, CLOUDERA_STACK_VERSION_7_2_14)) {
            configs.add(config(ENABLE_RACK_AWARENESS, "true"));
            return configs;
        }
        return List.of();
    }

    @Override
    public List<String> getRoleTypes() {
        return List.of(KafkaRoles.KAFKA_BROKER);
    }

    @Override
    public String getServiceType() {
        return KafkaRoles.KAFKA_SERVICE;
    }

    @Override
    public boolean isConfigurationNeeded(CmTemplateProcessor cmTemplateProcessor, TemplatePreparationObject source) {
        return (StackType.DATALAKE.equals(source.getStackType()) || StackType.WORKLOAD.equals(source.getStackType()))
                && source.getGeneralClusterConfigs().isMultiAzEnabled();
    }
}


text:
title: "half"
content: "A Fraction of the Cost"
class: "intro-item"
weight: 3
---


text:
package com.questionanswer.model;

import java.security.Principal;
import java.util.HashSet;
import java.util.Set;

import javax.persistence.CascadeType;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.FetchType;
import javax.persistence.ManyToMany;
import javax.persistence.OneToMany;

import org.hibernate.validator.constraints.Email;
import org.hibernate.validator.constraints.NotEmpty;

import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;

@Entity
public class User extends BaseEntityAudit implements Principal {

	private static final long serialVersionUID = -1525848083627501220L;

	@OneToMany(mappedBy = "author", cascade = {CascadeType.ALL})
	private Set<Answer> answers = new HashSet<>();

	@Email(message = "Please provide a valid email address.")
	@NotEmpty(message = "Email is required.")
	@Column(unique = true, nullable = false)
	private String email;

	@NotEmpty(message = "First name is required.")
	private String firstName;

	@NotEmpty(message = "Last name is required.")
	private String lastName;

	@NotEmpty(message = "Password is required.")
	private String password;

	@OneToMany(mappedBy = "user", cascade = {CascadeType.ALL})
	private Set<Question> questions = new HashSet<>();

	@ManyToMany(mappedBy = "userroles", fetch = FetchType.EAGER)
	private Set<Role> roles = new HashSet<>();

	@ManyToMany(cascade = {CascadeType.ALL}, fetch = FetchType.EAGER)
	private Set<Answer> answersvotes = new HashSet<Answer>();

	public User() {
	}

	public User(String firstName, String lastName, String email, String password) {
		this.setFirstName(firstName);
		this.setLastName(lastName);
		this.setEmail(email);
		this.setPassword(password);
	}

	public User(User user) {
		this.setId(user.getId());
		this.setFirstName(user.getFirstName());
		this.setLastName(user.getLastName());
		this.setEmail(user.getEmail());
		this.setPassword(user.getPassword());
		this.setRoles(user.getRoles());
	}

	@Override
	public boolean equals(Object object) {
		if (!(object instanceof User)) {
			return false;
		}

		User other = (User) object;

		return other.getEmail().equals(this.getEmail());
	}

	@Override
	public String getName() {
		return this.getEmail();
	}

	public Set<Answer> getAnswers() {
		return answers;
	}

	public void setAnswers(Set<Answer> answers) {
		this.answers = answers;
	}

	public String getEmail() {
		return email;
	}

	public void setEmail(String email) {
		this.email = email;
	}

	public String getFirstName() {
		return firstName;
	}

	public void setFirstName(String firstName) {
		this.firstName = firstName;
	}

	public String getLastName() {
		return lastName;
	}

	public void setLastName(String lastName) {
		this.lastName = lastName;
	}

	// Not a good idea to expose the password even to administrators
	@JsonIgnore
	public String getPassword() {
		return password;
	}

	@JsonProperty
	public void setPassword(String password) {
		this.password = password;
	}

	public Set<Question> getQuestions() {
		return questions;
	}

	public void setQuestions(Set<Question> questions) {
		this.questions = questions;
	}

	public Set<Role> getRoles() {
		return roles;
	}

	public void setRoles(Set<Role> roles) {
		this.roles = roles;
	}

	public Set<Answer> getAnswersvotes() {
		return answersvotes;
	}

	public void setAnswersvotes(Set<Answer> answersvotes) {
		this.answersvotes = answersvotes;
	}
}


text:
MAKEFLAGS += -L
SHELL=/bin/bash -o pipefail
BUILD_DIR=build
SOURCE_DIR=src
UTILS_DIR=utils
CFLAGS=-Wall -Werror -O2

# Check that given variables are set and all have non-empty values,
# die with an error otherwise.
#
# Params:
#   1. Variable name(s) to test.
#   2. (optional) Error message to print.
#
# Source:
# https://stackoverflow.com/a/10858332 by Eldar Abusalimov
check_defined = \
    $(strip $(foreach 1,$1, \
        $(call __check_defined,$1,$(strip $(value 2)))))
__check_defined = \
    $(if $(value $1),, \
      $(error Undefined $1$(if $2, ($2))))

HOME_DIR := $(shell sed -n -e 's/^HOME=\(.*\)/\1/p' user.cfg)
$(call check_defined, HOME_DIR)

LOCAL_PREFIX := $(shell sed -n -e 's/^LOCAL_PREFIX=\(.*\)/\1/p' user.cfg)
$(call check_defined, LOCAL_PREFIX)
BIN_DIR := $(LOCAL_PREFIX)/bin

CONFIG_DIR := $(shell sed -n -e 's/^XDG_CONFIG_HOME=\(.*\)/\1/p' user.cfg)
$(call check_defined, CONFIG_DIR)

DATA_DIR := $(shell sed -n -e 's/^XDG_DATA_HOME=\(.*\)/\1/p' user.cfg)
$(call check_defined, DATA_DIR)

CACHE_DIR := $(shell sed -n -e 's/^XDG_CACHE_HOME=\(.*\)/\1/p' user.cfg)
$(call check_defined, CACHE_DIR)

SYSTEM_PREFIX := $(shell sed -n -e 's/^SYSTEM_PREFIX=\(.*\)/\1/p' user.cfg)
$(call check_defined, SYSTEM_PREFIX)

# Meant to be used by env/ scripts ONLY
# Echo to force ~ expansion
export HOME_DIR := $(shell echo $(HOME_DIR))
export LOCAL_PREFIX  := $(shell echo $(LOCAL_PREFIX))
BIN_DIR := $(shell echo $(BIN_DIR))
export CONFIG_DIR := $(shell echo $(CONFIG_DIR))
export DATA_DIR := $(shell echo $(DATA_DIR))
export CACHE_DIR := $(shell echo $(CACHE_DIR))
export SYSTEM_PREFIX := $(shell echo $(SYSTEM_PREFIX))

I3BLOCKS_SCRIPT_DIR := $(CONFIG_DIR)/i3blocks/scripts

# Incomplete files can be created when pipes / scripts fail
# so delete the target when the rule fails.
.DELETE_ON_ERROR:

WARNING_PREFIX:=$(shell echo "$$(tput setaf 172)WARNING$$(tput sgr0):")

# The following variables need to be defined
# for each TYPE = bin | config | data | home | system
#
# <TYPE>_FIRST_BUILD
# 	- Names % where BUILD_DIR/<TYPE>/% does not depend on any other files in the
# 		same directory.
# <TYPE>_BUILD
# 	- Names % where BUILD_DIR/<TYPE>/% is required for install
# <TYPE>_INSTALL
# 	- Names % where <TYPE>_INSTALL_PREFIX/% is an install target
# <TYPE>_INSTALL_ONCE
# 	- Names % where <TYPE>_INSTALL_PREFIX/% is an install target that does not
# 		need to be updated once installed.
#
# These variables are used to define the make targets (build and install)
# and to create templated rules that depend on the target directory existing.
#
# Useful intermediate variables:
# <TYPE>_FBI (or a subset of FBI)
# 	- Intersections of <TYPE>_FIRST_BUILD, <TYPE>_BUILD, <TYPE>_INSTALL
# <TYPE>_LINKS
# 	- Names % where <TYPE>_INSTALL_PREFIX/% is a symbolic link to a file
# <TYPE>_DLINKS
# 	- Names % where <TYPE>_INSTALL_PREFIX/% is a symbolic link to a directory

# Files can be installed directly from SOURCE_DIR or from BUILD_DIR
# Files in BUILD_DIR can depend on other build files.

# Bin
# ---
BIN_FIRST_BUILD:=
BIN_BUILD:=
BIN_LINKS:=
BIN_DLINKS:=
BIN_INSTALL_ONCE:=$(BIN_LINKS) $(BIN_DLINKS)
BIN_INSTALL:=\
	backtrace\
	check-home\
	combinediff-careful\
	csv2yaml\
	dblp-makebib\
	decrypt-private\
	dfix\
	geolocate\
	get-gitignore\
	git-w\
	gr\
	low-battery-action\
	mdlynx\
	pip-deps\
	pip-update-all\
	plot\
	pplayerctl\
	print24bitcolours\
	print256colours\
	pyproject\
	tmuxm\
	xopen\

ifneq ($(strip $(shell command -v dwarffortress)),)
BIN_INSTALL+=dwarffortress
endif

# Config
# ------
CONFIG_FBI_SYSTEMD:=\
	systemd/user/auto-suspend.service\
	systemd/user/auto-suspend.timer\
	systemd/user/duplicacy-backup.service\
	systemd/user/duplicacy-backup.timer\
	systemd/user/duplicacy-prune.service\
	systemd/user/duplicacy-prune.timer\
	systemd/user/low-battery.service\
	systemd/user/low-battery.timer\
	systemd/user/redshift.service\
	systemd/user/ssh-agent.service\
	systemd/user/xsession.target\

CONFIG_FBI_VIM:=\
	vim/plugin/settings/airline.vim\
	vim/plugin/settings/tmuxline.vim\
	vim/vimrc\
	vim/ycm_extra_conf.py\

CONFIG_I3BLOCKS_CONTRIB:=\
	memory/memory\
	temperature/temperature\
	volume/volume\

CONFIG_FBI:=\
	$(CONFIG_FBI_SYSTEMD)\
	$(CONFIG_FBI_VIM)\
	$(addprefix i3blocks/scripts/,$(notdir $(CONFIG_I3BLOCKS_CONTRIB)))\
	alacritty/alacritty.yml\
	bash/aliases\
	bash/bashrc\
	bash/profile\
	conky/default-popup.lua\
	dircolors/database\
	duplicacy-web/settings.json\
	duplicacy/filters\
	env_profile\
	fish/config.fish\
	fish/functions/ccopy.fish\
	fish/functions/cpaste.fish\
	fish/functions/fish_prompt.fish\
	fish/functions/hostname-icon.fish\
	fish/functions/ip.fish\
	fish/functions/R.fish\
	fish/functions/tmux.fish\
	fish/functions/torch-activate.fish\
	flake8\
	gem/config.yaml\
	git/config\
	git/ignore\
	git/template/hooks/pre-commit\
	gsimplecal/config\
	gtk-3.0/settings.ini\
	hg/hgrc\
	htop/htoprc\
	i3/config\
	i3blocks/config\
	i3blocks/scripts/cpu_usage2\
	imwheel/config\
	isort.cfg\
	latexmk/latexmkrc\
	locale.conf\
	matplotlib/matplotlibrc\
	mimeapps.list\
	npm/config\
	pacman/makepkg.conf\
	procps/toprc\
	profile\
	pylint/config\
	redshift/redshift.conf\
	security/pam_env.conf\
	task/config\
	terminator/config\
	termite/config\
	theano/config\
	tmux/tmux.conf\
	user-dirs.dirs\
	wgetrc\
	xbindkeys/config\
	xinit/startx.fish\
	xinit/xinitrc\
	xinit/Xmodmap\
	xinit/Xresources\
	xinit/xserverrc\
	xprofile\
	yapf/style\
	yay/config.json\

CONFIG_LINKS:=
CONFIG_DLINKS:=\
	chromium\
	duplicacy-web/bin\
	duplicacy-web/stats\
	duplicacy/cache\
	launcher-main\
	libreoffice\
	Slack\

CONFIG_INSTALL_ONCE:=$(CONFIG_LINKS) $(CONFIG_DLINKS)

CONFIG_FB:=\
	$(addsuffix .link,$(CONFIG_LINKS))\
	$(addsuffix .dlink,$(CONFIG_DLINKS))\

CONFIG_FIRST_BUILD:=\
	$(CONFIG_FBI)\
	$(CONFIG_FB)\

CONFIG_BUILD:=\
	$(CONFIG_FBI)\
	$(CONFIG_FB)\

CONFIG_INSTALL_FISH_FOREIGN_ENV:=$(addprefix fish/plugins/foreign-env/,\
	functions/fenv.apply.fish\
	functions/fenv.fish\
	functions/fenv.main.fish\
	functions/fenv.parse.after.fish\
	functions/fenv.parse.before.fish\
	functions/fenv.parse.diff.fish\
	functions/fenv.parse.divider.fish\
	LICENSE\
	README.md\
)

CONFIG_INSTALL_VIM_DIRECT:=\
	$(shell cd "$(SOURCE_DIR)/config" && find vim -type f -not -name '*.m4')

CONFIG_INSTALL:=\
	$(CONFIG_INSTALL_FISH_FOREIGN_ENV)\
	$(CONFIG_INSTALL_VIM_DIRECT)\
	$(CONFIG_FBI)\
	fontconfig/fonts.conf\
	gkrellm/user-config-cpu\
	gkrellm/user-config-memory\
	i3blocks/scripts/battery\
	i3blocks/scripts/conky-toggle\
	i3blocks/scripts/date-calendar\
	i3blocks/scripts/gkrellm-toggle\
	i3blocks/scripts/gpu-usage\
	i3blocks/scripts/music\
	i3blocks/scripts/ssid\
	i3blocks/scripts/weather\
	i3blocks/scripts/wifi\
	keepassxc/keepassxc.ini\
	playerctl/players\
	pudb/pudb.cfg\
	pyproject/templates/license/MIT.txt\
	pyproject/templates/pyroot/__init__.py\
	pyproject/templates/pyroot/version.py\
	pyproject/templates/root/.gitignore\
	pyproject/templates/root/.pre-commit-config.yaml\
	pyproject/templates/root/MANIFEST.in\
	pyproject/templates/root/pyproject.toml\
	pyproject/templates/root/README.md\
	pyproject/templates/root/setup.cfg\
	pyproject/templates/root/setup.py\
	python/startup.py\
	zathura/zathurarc\

# Data
# ----
DATA_LINKS:=
DATA_DLINKS:=\
	libreoffice/4/cache\
	Slack/Cache\
	Steam/appcache\

# These directories need to exist
# In some cases, if there is a environment variable telling the
# program to use this directory then the program expects it to exist.
DATA_DIRS:=\
	gnupg/.\

DATA_INSTALL_ONCE=$(DATA LINKS) $(DATA_DLINKS) $(DATA_DIRS)

DATA_FB:=\
	$(addsuffix .link,$(DATA_LINKS))\
	$(addsuffix .dlink,$(DATA_DLINKS))\

# Custom install for Steam Desktop
SOURCE_STEAM_DESKTOP:=/usr/share/applications/steam.desktop
DATA_STEAM_DESKTOP:=applications/steam.desktop

DATA_FIRST_BUILD:=\
	$(DATA_FB)\
	$(DATA_STEAM_DESKTOP).sed\

DATA_BI:=
ifneq ("$(wildcard $(SOURCE_STEAM_DESKTOP))","")
DATA_BI+=$(DATA_STEAM_DESKTOP)
endif

DATA_BUILD:=\
	$(DATA_FB)\
	$(DATA_BI)\

DATA_FONTS:=\
	fonts/PowerlineSymbols.otf\

DATA_VUNDLE_DIR:=vim/bundle/Vundle.vim

DATA_INSTALL:=\
	$(DATA_FONTS)\
	$(DATA_BI)\
	$(DATA_VUNDLE_DIR)\
	tig/.\
	wget/.\

# Home
# ----
HOME_FBI:=\
	.ssh/config\

# To avoid putting files here, only include if the relevant program exists.
HOME_LINKS:=\
	.bash_profile\
	.bashrc\
	.pam_environment\
	.profile\
	.xprofile\

HOME_DLINKS:=

ifneq ($(strip $(shell command -v duplicacy-web)),)
HOME_DLINKS += .duplicacy-web
endif
ifneq ($(strip $(shell command -v factorio)),)
HOME_DLINKS += .factorio
endif
ifneq ($(strip $(shell command -v gkrellm)),)
HOME_DLINKS += .gkrellm2
endif
ifneq ($(strip $(shell command -v imwheel)),)
HOME_LINKS += .imwheelrc
endif
ifneq ($(strip $(shell command -v firefox)),)
HOME_DLINKS += .mozilla
endif
ifneq ($(strip $(shell command -v steam)),)
HOME_DLINKS += .paradoxlauncher
endif
ifneq ($(strip $(shell command -v thunderbird)),)
HOME_DLINKS += .thunderbird
endif
ifneq ($(strip $(shell command -v lightdm || command -v gdm)),)
HOME_LINKS += .Xresources
endif
ifneq ($(strip $(shell command -v zotero)),)
HOME_DLINKS += .zotero
endif

HOME_INSTALL_ONCE=$(HOME_LINKS) $(HOME_DLINKS)

HOME_FB:=\
	$(addsuffix .link,$(HOME_LINKS))\
	$(addsuffix .dlink,$(HOME_DLINKS))\

HOME_FIRST_BUILD:=\
	$(HOME_FBI)\
	$(HOME_FB)\

HOME_BUILD:=\
	$(HOME_FBI)\
	$(HOME_FB)\

HOME_INSTALL:=\
	$(HOME_FBI)\
	$(HOME_INSTALL)\

# System
# ------

SYSTEM_FBI:=\
	etc/systemd/system/getty@tty1.service.d/override.conf\
	etc/X11/xorg.conf.d/30-touchpad.conf\
	etc/X11/xorg.conf.d/80-monitor.conf\

SYSTEM_FIRST_BUILD:=\
	$(SYSTEM_FBI)\

SYSTEM_BUILD:=\
	$(SYSTEM_FBI)\

SYSTEM_INSTALL:=\
	$(SYSTEM_FBI)\
	etc/udev/rules.d/90-backlight.rules\
	etc/X11/xorg.conf.d/90-keyboard.conf\

# Environment
# -----------
ENV_FIRST_BUILD:=\
	applications\
	battery\
	colours\
	cpu\
	cuda\
	display\
	fonts\
	git\
	github_id\
	go\
	lock\
	modules\
	mujoco\
	netuser\
	nvidia-smi\
	osx\
	perl\
	programs\
	python\
	root\
	ruby\
	rust\
	sys-monitor\
	terminfo\
	tmux\
	torch\
	touchpad\
	user\
	virtualfish\
	wifi\

ENV_CONFIG_TARGETS=$(addsuffix .m4,\
	$(addprefix $(BUILD_DIR)/env/,$(ENV_FIRST_BUILD)))

# Combined
# --------
FIRST_BUILD_TARGETS:=$(addprefix $(BUILD_DIR)/,\
	$(addprefix bin/,$(BIN_FIRST_BUILD))\
	$(addprefix config/,$(CONFIG_FIRST_BUILD))\
	$(addprefix data/,$(DATA_FIRST_BUILD))\
	$(addprefix env/,$(ENV_FIRST_BUILD))\
	$(addprefix home/,$(HOME_FIRST_BUILD))\
	$(addprefix system/,$(SYSTEM_FIRST_BUILD))\
)

BUILD_TARGETS:=$(addprefix $(BUILD_DIR)/,\
	$(addprefix bin/,$(BIN_BUILD))\
	$(addprefix config/,$(CONFIG_BUILD))\
	$(addprefix data/,$(DATA_BUILD))\
	$(addprefix home/,$(HOME_BUILD))\
	$(addprefix system/,$(SYSTEM_BUILD))\
)

INSTALL_TARGETS:=\
	$(addprefix $(BIN_DIR)/,$(BIN_INSTALL))\
	$(addprefix $(CONFIG_DIR)/,$(CONFIG_INSTALL))\
	$(addprefix $(DATA_DIR)/,$(DATA_INSTALL))\
	$(addprefix $(HOME_DIR)/,$(HOME_INSTALL))\

INSTALL_ONCE_TARGETS:=\
	$(addprefix $(BIN_DIR)/,$(BIN_INSTALL_ONCE))\
	$(addprefix $(CONFIG_DIR)/,$(CONFIG_INSTALL_ONCE))\
	$(addprefix $(DATA_DIR)/,$(DATA_INSTALL_ONCE))\
	$(addprefix $(HOME_DIR)/,$(HOME_INSTALL_ONCE))\

INSTALL_SYSTEM_TARGETS:=$(addprefix $(SYSTEM_PREFIX)/,$(SYSTEM_INSTALL))

INSTALLED_SYSTEMD_CONFIGS:=$(addprefix $(CONFIG_DIR)/,$(CONFIG_FBI_SYSTEMD))
INSTALLED_FONTS:=$(addprefix $(DATA_DIR)/,$(DATA_FONTS))

############
# Commands #
############

.PHONY: build install install-user install-system clean help

build: $(BUILD_TARGETS)

install: install-user systemd-reload font-cache

install-user: $(INSTALL_TARGETS) | $(INSTALL_ONCE_TARGETS)

install-system: $(INSTALL_SYSTEM_TARGETS)

clean:
	rm -rf $(BUILD_DIR)

define HELP_MESSAGE
Makefile Commands
=================
build [default]    : Build the dotfiles into $(BUILD_DIR)/
install            : Install user dotfiles.
                     Destinations are specified in user.cfg.
install-system     : Install system dotfiles.
                     Likely requires root permissions.
clean              : Remove build products.
help               : Print this help message

show               : Run most `show-*` commands.
show-dirs          : Print the install directories.
show-config        : Print the user and environment configuration.
show-wants         : Print a list of desirable programs that are not
                     checked by the environment configuration (show-config).
show-links         : Print information about files that should be links.

vim                : Run all `vim-*` commands.
                     Uses the installed configurations so install first.
vim-vundle         : Install the Vundle vim plugin.
vim-plugins        : Install and update vim plugins.
vim-ycm            : Compile YouCompleteMe for vim.
vim-tmuxline       : Create the tmuxline configuration.

systemd-reload     : Reload user systemd units if any changed.
font-cache         : Create the font cache.
persistent-configs : Set persistent settings that are not file-based.
endef

help:
	@: $(info $(HELP_MESSAGE))

################
##  Build M4  ##
################

USER_CONFIG_PREFIX:=m4_user_config_
ENV_CONFIG_PREFIX:=m4_env_config_
# Hopefully unlikely to appear in the dotfiles.
QUOTE_START:=??[[<<
QUOTE_END:=>>]]??

# TODO: Support %.local dependencies
# Add a list of files that might have .local dependencies and template the
# dependence

$(BUILD_DIR)/%: $(SOURCE_DIR)/%.m4 \
		$(BUILD_DIR)/user_config.m4 $(BUILD_DIR)/env_config.m4
	echo "m4_changecom()m4_changequote($(QUOTE_START),$(QUOTE_END))m4_dnl" | \
		cat - "$<" | \
		m4 --prefix-builtins -I "$(BUILD_DIR)" > "$@"
	! grep m4_ "$@" || (echo "Undefined m4 variables" && rm "$@" && false)

##############################
##  Build I3Blocks Contrib  ##
##############################

define BUILD_I3BLOCKS_CONTRIB_TEMPLATE
$(BUILD_DIR)/config/i3blocks/scripts/$(notdir $1):\
		$(SOURCE_DIR)/config/i3blocks/i3blocks-contrib/$1\
		| $(BUILD_DIR)/config/i3blocks/scripts/.
	cp -v "$$<" "$$@"
endef

$(foreach CONTRIB_SCRIPT,$(CONFIG_I3BLOCKS_CONTRIB),\
	$(eval $(call BUILD_I3BLOCKS_CONTRIB_TEMPLATE,$(CONTRIB_SCRIPT))))

CPU_USAGE_PATCH:=$(SOURCE_DIR)/config/i3blocks/patches/cpu_usage2.c.patch
$(BUILD_DIR)/config/i3blocks/scripts/cpu_usage2.c: \
		$(SOURCE_DIR)/config/i3blocks/i3blocks-contrib/cpu_usage2/cpu_usage2.c \
		$(CPU_USAGE_PATCH) \
		| $(BUILD_DIR)/config/i3blocks/scripts/.
	patch -o "$@" "$<" "$(CPU_USAGE_PATCH)"

$(BUILD_DIR)/config/i3blocks/scripts/cpu_usage2: \
	$(BUILD_DIR)/config/i3blocks/scripts/cpu_usage2.c
	$(CC) $(CFLAGS) -o "$@" "$<"

###################
##  Build Steam  ##
###################

$(BUILD_DIR)/data/$(DATA_STEAM_DESKTOP):\
		$(BUILD_DIR)/data/$(DATA_STEAM_DESKTOP).sed\
		$(SOURCE_STEAM_DESKTOP)
	sed -f "$<" "$(SOURCE_STEAM_DESKTOP)" > "$@"

#########################
##  Build Environment  ##
#########################

$(BUILD_DIR)/env/%.m4: $(SOURCE_DIR)/env/% \
		$(SOURCE_DIR)/env/env_utils\
		$(BUILD_DIR)/env/paths.sh\
		$(UTILS_DIR)/config_replace.sh
	source "$(BUILD_DIR)/env/paths.sh" && \
		"$<" | \
		$(UTILS_DIR)/config_replace.sh \
			"$(ENV_CONFIG_PREFIX)" "$(QUOTE_START)" "$(QUOTE_END)" | \
		(echo "m4_dnl $<" && cat) > "$@"

$(BUILD_DIR)/env/colours.m4: $(SOURCE_DIR)/env/colours.toml

$(BUILD_DIR)/user_config.m4: user.cfg\
		$(UTILS_DIR)/config_replace.sh\
		| $(BUILD_DIR)/.
	sed -e 's/\s*#.*$$//' -e '/^\s*$$/d' $< | \
		$(UTILS_DIR)/config_replace.sh \
			"${USER_CONFIG_PREFIX}" "${QUOTE_START}" "${QUOTE_END}" > $@

$(BUILD_DIR)/env/paths.sh: user.cfg | $(BUILD_DIR)/env/.
	grep -e '^ *[A-Z_]*\(HOME\|PREFIX\) *=' user.cfg | \
		sed -e "s/^/export /" > "$@"

$(BUILD_DIR)/env/absolute_paths.m4: user.cfg\
		$(UTILS_DIR)/config_replace.sh\
		| $(BUILD_DIR)/env/.
	# Will fail if $HOME contains | characters
	sed user.cfg -n -e "s|=~|=$(HOME)|" -e "/\(HOME\|PREFIX\)=/p" |\
		$(UTILS_DIR)/config_replace.sh \
			"${ENV_CONFIG_PREFIX}" "${QUOTE_START}" "${QUOTE_END}" |\
		(echo "m4_dnl $<" && cat) > "$@"

$(BUILD_DIR)/env_config.m4: $(ENV_CONFIG_TARGETS) \
		$(BUILD_DIR)/env/absolute_paths.m4 |\
		$(BUILD_DIR)/.
	cat $^ > "$@"

########################
##  Show Information  ##
########################
.PHONY: show show-dirs show-config show-wants show-links

show: show-dirs show-config show-wants

show-dirs:
	@echo "HOME_DIR:      $(HOME_DIR)"
	@echo "BIN_DIR:       $(BIN_DIR)"
	@echo "CONFIG_DIR:    $(CONFIG_DIR)"
	@echo "DATA_DIR:      $(DATA_DIR)"
	@echo "CACHE_DIR:     $(CACHE_DIR)"
	@echo "SYSTEM_PREFIX: $(SYSTEM_PREFIX)"

PYGMENTIZE:=$(shell command -v pygmentize)
ifdef PYGMENTIZE
# pygmentize does not apply the right colours for dark background on
# vim style so use a different style that does work.
COLORIZE_CONFIG:=pygmentize -l 'cfg' -O style=native -f terminal256
else
COLORIZE_CONFIG:=cat
endif

ESCAPED_QUOTE_START=$(subst [,\[,$(subst ],\],$(QUOTE_START)))
ESCAPED_QUOTE_END=$(subst [,\[,$(subst ],\],$(QUOTE_END)))

show-config: user.cfg $(BUILD_DIR)/env_config.m4
ifndef PYGMENTIZE
	@echo "Install pygmentize (python pygments) to view coloured output."
	@echo
endif
	@echo '# User Config' | cat - user.cfg | $(COLORIZE_CONFIG)
	@echo
	@echo '# Environment Config' | cat - $(BUILD_DIR)/env_config.m4 | \
		sed -e "s/^m4_dnl/#/" \
			-e "s/$(ESCAPED_QUOTE_START)//g" \
			-e "s/$(ESCAPED_QUOTE_END)//g" \
			-e "s/^m4_define(m4_env_config_//" \
			-e "s/)m4_dnl$$//" \
			-e "s/,/=/" | \
		$(COLORIZE_CONFIG)

show-wants: $(SOURCE_DIR)/env/wants $(SOURCE_DIR)/env/env_utils
	@"$<" | ( echo "# wants" && cat ) | $(COLORIZE_CONFIG)

show-links: build $(UTILS_DIR)/check-links
	@$(UTILS_DIR)/check-links

###############
##  Install  ##
###############

# Args: INSTALL_DIR SOURCE_TYPE
define INSTALL_TEMPLATE
$1/%: $(SOURCE_DIR)/$2/%
	@install -v "$$<" "$$@"

$1/%: $(BUILD_DIR)/$2/%
	@install -v "$$<" "$$@"

$1/%: | $(BUILD_DIR)/$2/%.link
	DEST="$$$$(grep -m 1 "[^[:space:]]" "$$(patsubst $1/%,$(BUILD_DIR)/$2/%.link,$$@)")"; \
	[ -n "$$$$DEST" ] && ! [ -d "$$@" ] && \
	ln -s -f "$$$$DEST" "$$@"

$1/%: | $(BUILD_DIR)/$2/%.dlink
	DEST="$$$$(grep -m 1 "[^[:space:]]" "$$(patsubst $1/%,$(BUILD_DIR)/$2/%.dlink,$$@)")"; \
	[ -n "$$$$DEST" ] && ! [ -d "$$@" ] && \
	ln -s -f "$$$$DEST" "$$@" && \
	mkdir -p "$$$$DEST"
endef

$(eval $(call INSTALL_TEMPLATE,$(BIN_DIR),bin))
$(eval $(call INSTALL_TEMPLATE,$(CONFIG_DIR),config))
$(eval $(call INSTALL_TEMPLATE,$(DATA_DIR),data))
$(eval $(call INSTALL_TEMPLATE,$(HOME_DIR),home))
$(eval $(call INSTALL_TEMPLATE,$(SYSTEM_PREFIX),system))

###########
##  Vim  ##
###########
.PHONY: vim vim-vundle vim-plugins vim-ycm vim-tmuxline

vim: vim-vundle vim-plugins vim-ycm vim-tmuxline

VUNDLE_TARGET:=$(DATA_DIR)/$(DATA_VUNDLE_DIR)
vim-vundle: | $(VUNDLE_TARGET)

$(VUNDLE_TARGET): | $(dir $(VUNDLE_TARGET)).
	cd $(dir $@) && git clone https://github.com/VundleVim/Vundle.vim.git

vim-plugins: vim-vundle
	vim +PluginInstall +PluginUpdate +qall

YCM_DIR:=$(DATA_DIR)/vim/bundle/YouCompleteMe
YCM_CORE:=$(YCM_DIR)/third_party/ycmd/ycm_core.so
YCM_GIT_CHECKOUT:=$(YCM_DIR)/.git/logs/HEAD

PYTHON := $(shell command -v python3 || command -v python)

vim-ycm: $(YCM_CORE)

$(YCM_CORE): $(YCM_GIT_CHECKOUT)
	cd $(YCM_DIR) && \
		$(PYTHON) ./install.py --clang-completer \
		$$(if [[ "$$(uname -r)" == *ARCH* ]]; then echo --system-libclang; fi)

TMUXLINE_CONFIG:=tmux/tmuxline.conf
TMUXLINE_TARGET:=$(CONFIG_DIR)/$(TMUXLINE_CONFIG)
RANDOM_ID:=$(shell echo $$RANDOM)

vim-tmuxline: $(TMUXLINE_TARGET)

$(TMUXLINE_TARGET): \
	| $(dir $(TMUXLINE_TARGET)).
	# Start a new temporary tmux session and in that tmux session run vim
	# and in vim call TmuxlineSnapshot to save the tmuxline configuration to
	# tmuxline.conf
	tmux new-session -d -s 'tmuxline-$(RANDOM_ID)' 'vim -E -c "TmuxlineSnapshot! $@" -c "q"'
	while tmux list-sessions 2>/dev/null | grep 'tmuxline-$(RANDOM_ID)' >/dev/null ; do \
		sleep 0.05; \
	done
	@if [ ! -f "$@" ]; then \
		echo "$(WARNING_PREFIX) Unable to generate tmuxline snapshot. Install tmuxline vim plugin and remake."; \
		false; \
	fi

#########################
##  Persisent Configs  ##
#########################
.PHONY: persistent-configs

persistent-configs: $(BUILD_DIR)/make/persistent-configs

$(BUILD_DIR)/make/persistent-configs: \
		$(UTILS_DIR)/set-persistent-configs.sh \
		$(INSTALLED_SYSTEMD_FILES) \
		| $(BUILD_DIR)/make/
	$(UTILS_DIR)/set-persistent-configs.sh
	touch "$@"

######################
##  Systemd Reload  ##
######################
.PHONY: systemd-reload

systemd-reload: $(BUILD_DIR)/make/systemd-reload

SYSTEMCTL := $(shell command -v systemctl 2>/dev/null)
ifdef SYSTEMCTL
$(BUILD_DIR)/make/systemd-reload: \
		$(INSTALLED_SYSTEMD_CONFIGS) \
		| $(BUILD_DIR)/make/.
	systemctl --user daemon-reload
else
$(BUILD_DIR)/make/systemd-reload: | $(BUILD_DIR)/make/.
endif
	touch "$@"

##################
##  Font Cache  ##
##################
.PHONY: font-cache

font-cache: $(BUILD_DIR)/make/font-cache

$(BUILD_DIR)/make/font-cache: $(INSTALLED_FONTS) | $(BUILD_DIR)/make/.
	fc-cache -v $(DATA_DIR)/fonts/
	touch "$@"

########################
##  Make Directories  ##
########################
define MKDIR_DEPENDENCY_TEMPLATE
$1: | $(dir $1).
endef

# Targets that might be the first in their directory
ORIGINAL_TARGETS:=\
	$(FIRST_BUILD_TARGETS)\
	$(INSTALL_TARGETS)\
	$(INSTALL_ONCE_TARGETS)\
	$(INSTALL_SYSTEM_TARGETS)\
	$(TMUXLINE_TARGET)\
	$(BUILD_DIR)/.\
	$(BUILD_DIR)/make/.\
# Exclude directories
ORIGINAL_FILE_TARGETS:=$(filter-out %/.,$(ORIGINAL_TARGETS))

$(foreach TARGET,$(ORIGINAL_FILE_TARGETS),\
	$(eval $(call MKDIR_DEPENDENCY_TEMPLATE,$(TARGET))))

# Use a template so that this rule is preferred over the generic install copy
# rule.
.PRECIOUS: %/.

define MKDIR_TEMPLATE

$1.:
	mkdir -p "$$@"
endef
$(foreach TARGET_DIR,\
	$(sort $(BUILD_DIR)/ $(BUILD_DIR)/make/ $(dir $(ORIGINAL_TARGETS))),\
	$(eval $(call MKDIR_TEMPLATE,$(TARGET_DIR))))


text:
namespace chilon { namespace parser {

/// Indicates that a type is a node in the tree.
template <class T>
struct node : parse_base<T> {
    typedef T node_type;
};

} }
#endif


text:
/*!
 *  Copyright (c) 2016 by Contributors
 * \file ordering_op.cc
 * \brief CPU Implementation of the ordering operations
 */
// this will be invoked by gcc and compile CPU version
#include "./ordering_op-inl.h"
#include "./elemwise_unary_op.h"

namespace mxnet {
namespace op {
DMLC_REGISTER_PARAMETER(TopKParam);
DMLC_REGISTER_PARAMETER(SortParam);
DMLC_REGISTER_PARAMETER(ArgSortParam);

NNVM_REGISTER_OP(topk)
.add_alias("_npx_topk")
.describe(R"code(Returns the indices of the top *k* elements in an input array along the given
 axis (by default).
 If ret_type is set to 'value' returns the value of top *k* elements (instead of indices).
 In case of ret_type = 'both', both value and index would be returned.
 The returned elements will be sorted.

Examples::

  x = [[ 0.3,  0.2,  0.4],
       [ 0.1,  0.3,  0.2]]

  // returns an index of the largest element on last axis
  topk(x) = [[ 2.],
             [ 1.]]

  // returns the value of top-2 largest elements on last axis
  topk(x, ret_typ='value', k=2) = [[ 0.4,  0.3],
                                   [ 0.3,  0.2]]

  // returns the value of top-2 smallest elements on last axis
  topk(x, ret_typ='value', k=2, is_ascend=1) = [[ 0.2 ,  0.3],
                                               [ 0.1 ,  0.2]]

  // returns the value of top-2 largest elements on axis 0
  topk(x, axis=0, ret_typ='value', k=2) = [[ 0.3,  0.3,  0.4],
                                           [ 0.1,  0.2,  0.2]]

  // flattens and then returns list of both values and indices
  topk(x, ret_typ='both', k=2) = [[[ 0.4,  0.3], [ 0.3,  0.2]] ,  [[ 2.,  0.], [ 1.,  2.]]]

)code" ADD_FILELINE)
.set_num_inputs(1)
.set_num_outputs(TopKNumOutputs)
.set_attr_parser(ParamParser<TopKParam>)
.set_attr<mxnet::FInferShape>("FInferShape", TopKShape)
.set_attr<nnvm::FInferType>("FInferType", TopKType)
.set_attr<nnvm::FNumVisibleOutputs>("FNumVisibleOutputs", TopKNumVisibleOutputs)
.set_attr<FCompute>("FCompute<cpu>", TopK<cpu>)
.set_attr<nnvm::FGradient>("FGradient",
  [](const nnvm::ObjectPtr& n, const std::vector<nnvm::NodeEntry>& ograds) {
    const TopKParam& param = nnvm::get<TopKParam>(n->attrs.parsed);
    if (param.ret_typ == topk_enum::kReturnValue || param.ret_typ == topk_enum::kReturnBoth) {
      std::vector<nnvm::NodeEntry> inputs;
      uint32_t n_out = n->num_outputs();
      for (uint32_t i = 0; i < n_out; ++i) {
        inputs.emplace_back(n, i, 0);
      }
      return MakeNonlossGradNode("_backward_topk", n, {ograds[0]}, inputs, n->attrs.dict);
    } else {
      return MakeZeroGradNodes(n, ograds);
    }
  })
.set_attr<FResourceRequest>("FResourceRequest",
  [](const NodeAttrs& attrs) {
    return std::vector<ResourceRequest>{ResourceRequest::kTempSpace};
  })
.set_attr<THasDeterministicOutput>("THasDeterministicOutput", true)
.add_argument("data", "NDArray-or-Symbol", "The input array")
.add_arguments(TopKParam::__FIELDS__());

NNVM_REGISTER_OP(_backward_topk)
.set_num_inputs(3)
.set_num_outputs(1)
.set_attr_parser(ParamParser<TopKParam>)
.set_attr<nnvm::TIsBackward>("TIsBackward", true)
.set_attr<FCompute>("FCompute<cpu>", TopKBackward_<cpu>)
.set_attr<FResourceRequest>("FResourceRequest",
  [](const NodeAttrs& attrs) {
  return std::vector<ResourceRequest>{ResourceRequest::kTempSpace};
});

NNVM_REGISTER_OP(sort)
.add_alias("_npi_sort")
.describe(R"code(Returns a sorted copy of an input array along the given axis.

Examples::

  x = [[ 1, 4],
       [ 3, 1]]

  // sorts along the last axis
  sort(x) = [[ 1.,  4.],
             [ 1.,  3.]]

  // flattens and then sorts
  sort(x, axis=None) = [ 1.,  1.,  3.,  4.]

  // sorts along the first axis
  sort(x, axis=0) = [[ 1.,  1.],
                     [ 3.,  4.]]

  // in a descend order
  sort(x, is_ascend=0) = [[ 4.,  1.],
                          [ 3.,  1.]]

)code" ADD_FILELINE)
.set_num_inputs(1)
.set_num_outputs(2)
.set_attr_parser(ParamParser<SortParam>)
.set_attr<mxnet::FInferShape>("FInferShape", SortShape)
.set_attr<nnvm::FInferType>("FInferType", SortType)
.set_attr<nnvm::FNumVisibleOutputs>("FNumVisibleOutputs", [](const NodeAttrs& attrs) { return 1; })
.set_attr<FCompute>("FCompute<cpu>", Sort<cpu>)
.set_attr<nnvm::FGradient>("FGradient",
  [](const nnvm::ObjectPtr& n, const std::vector<nnvm::NodeEntry>& ograds) {
    const SortParam& param = nnvm::get<SortParam>(n->attrs.parsed);
    std::vector<nnvm::NodeEntry> inputs;
    uint32_t n_out = n->num_outputs();
    for (uint32_t i = 0; i < n_out; ++i) {
      inputs.emplace_back(n, i, 0);
    }
    return MakeNonlossGradNode("_backward_topk", n, {ograds[0]}, inputs,
                               {{"axis", n->attrs.dict["axis"]},
                                {"k", "0"},
                                {"ret_typ", "value"},
                                {"is_ascend", std::to_string(param.is_ascend)}});
  })
.set_attr<FResourceRequest>("FResourceRequest",
  [](const NodeAttrs& attrs) {
    return std::vector<ResourceRequest>{ResourceRequest::kTempSpace};
  })
.set_attr<THasDeterministicOutput>("THasDeterministicOutput", true)
.add_argument("data", "NDArray-or-Symbol", "The input array")
.add_arguments(SortParam::__FIELDS__());

NNVM_REGISTER_OP(argsort)
.add_alias("_npi_argsort")
.describe(R"code(Returns the indices that would sort an input array along the given axis.

This function performs sorting along the given axis and returns an array of indices having same shape
as an input array that index data in sorted order.

Examples::

  x = [[ 0.3,  0.2,  0.4],
       [ 0.1,  0.3,  0.2]]

  // sort along axis -1
  argsort(x) = [[ 1.,  0.,  2.],
                [ 0.,  2.,  1.]]

  // sort along axis 0
  argsort(x, axis=0) = [[ 1.,  0.,  1.]
                        [ 0.,  1.,  0.]]

  // flatten and then sort
  argsort(x, axis=None) = [ 3.,  1.,  5.,  0.,  4.,  2.]
)code" ADD_FILELINE)
.set_num_inputs(1)
.set_num_outputs(1)
.set_attr_parser(ParamParser<ArgSortParam>)
.set_attr<mxnet::FInferShape>("FInferShape", ArgSortShape)
.set_attr<nnvm::FInferType>("FInferType", ArgSortType)
.set_attr<FCompute>("FCompute<cpu>", ArgSort<cpu>)
.set_attr<nnvm::FGradient>("FGradient", MakeZeroGradNodes)
.set_attr<FResourceRequest>("FResourceRequest",
  [](const NodeAttrs& attrs) {
    return std::vector<ResourceRequest>{ResourceRequest::kTempSpace};
  })
.set_attr<THasDeterministicOutput>("THasDeterministicOutput", true)
.add_argument("data", "NDArray-or-Symbol", "The input array")
.add_arguments(ArgSortParam::__FIELDS__());
}  // namespace op
}  // namespace mxnet


text:
import os
import string
import os.path as op
import sys
import shutil
from collections import namedtuple

try:
    from seqcluster import prepare_data as prepare
    from seqcluster import templates as template_seqcluster
    from seqcluster.seqbuster import _create_counts, _read_miraligner, _tab_output
except ImportError:
    pass

from bcbio.utils import file_exists, safe_makedir, move_safe, append_stem
from bcbio.provenance import do
from bcbio.distributed.transaction import file_transaction
from bcbio.log import logger
from bcbio.pipeline import datadict as dd
from bcbio.pipeline.sample import process_alignment
from bcbio.srna import mirdeep

def run_prepare(*data):
    """
    Run seqcluster prepare to merge all samples in one file
    """
    out_dir = os.path.join(dd.get_work_dir(data[0][0]), "seqcluster", "prepare")
    out_dir = os.path.abspath(safe_makedir(out_dir))
    prepare_dir = os.path.join(out_dir, "prepare")
    fn = []
    for sample in data:
        name = sample[0]["rgnames"]['sample']
        fn.append("%s\t%s" % (sample[0]['collapse'], name))
    args = namedtuple('args', 'debug print_debug minc minl maxl out')
    args = args(False, False, 2, 17, 40, out_dir)
    ma_out = op.join(out_dir, "seqs.ma")
    seq_out = op.join(out_dir, "seqs.fastq")
    min_shared = max(int(len(fn) / 10.0), 1)
    if not file_exists(ma_out):
        seq_l, sample_l = prepare._read_fastq_files(fn, args)
        with file_transaction(ma_out) as ma_tx:
            with open(ma_tx, 'w') as ma_handle:
                with open(seq_out, 'w') as seq_handle:
                    prepare._create_matrix_uniq_seq(sample_l, seq_l, ma_handle, seq_handle, min_shared)

    return data

def run_align(*data):
    """
    Prepare data to run alignment step, only once for each project
    """
    work_dir = dd.get_work_dir(data[0][0])
    out_dir = op.join(work_dir, "seqcluster", "prepare")
    seq_out = op.join(out_dir, "seqs.fastq")
    bam_dir = op.join(work_dir, "align")
    new_bam_file = op.join(bam_dir, "seqs.bam")
    if not file_exists(new_bam_file):
        sample = process_alignment(data[0][0], [seq_out, None])
        bam_file = dd.get_work_bam(sample[0][0])
        shutil.move(bam_file, new_bam_file)
        shutil.move(bam_file + ".bai", new_bam_file + ".bai")
        shutil.rmtree(op.join(bam_dir, sample[0][0]["rgnames"]['sample']))
    return data

def run_cluster(*data):
    """
    Run seqcluster cluster to detect smallRNA clusters
    """
    sample = data[0][0]
    work_dir = dd.get_work_dir(sample)
    out_dir = op.join(work_dir, "seqcluster", "cluster")
    out_dir = op.abspath(safe_makedir(out_dir))
    prepare_dir = op.join(work_dir, "seqcluster", "prepare")
    bam_file = op.join(work_dir, "align", "seqs.bam")
    cluster_dir = _cluster(bam_file, prepare_dir, out_dir, dd.get_ref_file(sample), dd.get_srna_gtf_file(sample))
    sample["report"] = _report(sample, dd.get_ref_file(sample))
    sample["seqcluster"] = out_dir

    out_mirna = _make_isomir_counts(data, out_dir=op.join(work_dir, "mirbase"))
    if out_mirna:
        sample = dd.set_mirna_counts(sample, out_mirna[0])
        sample = dd.set_isomir_counts(sample, out_mirna[1])

    out_novel = _make_isomir_counts(data, "seqbuster_novel", op.join(work_dir, "mirdeep2"), "_novel")
    novel_db = mirdeep.run(data)
    if out_novel:
        sample = dd.set_novel_mirna_counts(sample, out_novel[0])
        sample = dd.set_novel_isomir_counts(sample, out_novel[1])
    data[0][0] = sample
    return data

def _cluster(bam_file, prepare_dir, out_dir, reference, annotation_file=None):
    """
    Connect to seqcluster to run cluster with python directly
    """
    seqcluster = op.join(os.path.dirname(sys.executable), "seqcluster")
    ma_file = op.join(prepare_dir, "seqs.ma")
    # cl = ["cluster", "-o", out_dir, "-m", ma_file, "-a", bam_file, "-r", reference]
    if annotation_file:
        annotation_file = "-g " + annotation_file
    else:
        annotation_file = ""

    if not file_exists(op.join(out_dir, "counts.tsv")):
        cmd = ("{seqcluster} cluster -o {out_dir} -m {ma_file} -a {bam_file} -r {reference} {annotation_file}")
        do.run(cmd.format(**locals()), "Running seqcluster.")
    return out_dir

def _report(data, reference):
    """
    Run report of seqcluster to get browser options for results
    """
    seqcluster = op.join(os.path.dirname(sys.executable), "seqcluster")
    work_dir = dd.get_work_dir(data)
    out_dir = safe_makedir(os.path.join(work_dir, "seqcluster", "report"))
    out_file = op.join(out_dir, "seqcluster.db")
    json = op.join(work_dir, "seqcluster", "cluster", "seqcluster.json")
    cmd = ("{seqcluster} report -o {out_dir} -r {reference} -j {json}")
    if not file_exists(out_file):
        do.run(cmd.format(**locals()), "Run report on clusters")
    return out_file

def report(data):
    """Create a Rmd report for small RNAseq analysis"""
    work_dir = dd.get_work_dir(data[0][0])
    out_dir = op.join(work_dir, "report")
    safe_makedir(out_dir)
    summary_file = op.join(out_dir, "summary.csv")
    with file_transaction(summary_file) as out_tx:
        with open(out_tx, 'w') as out_handle:
            print >>out_handle, "sample_id,size_stats,miraligner,group"
            for sample in data:
                info = sample[0]
                group = _guess_group(info)
                files = info["seqbuster"] if "seqbuster" in info else "None"
                print >>out_handle, ",".join([dd.get_sample_name(info),
                                              info["size_stats"],
                                              files, group])
    _create_rmd(summary_file)
    return summary_file

def _guess_group(info):
    """Add the first group to get report with some factor"""
    value = "fake"
    if "metadata" in info:
        if info["metadata"]:
            key, value = info['metadata'].popitem()
    return value

def _create_rmd(summary_fn):
    """Create relatie path files for Rmd report"""
    root_path, fn = op.split(os.path.abspath(summary_fn))
    out_file = op.join(root_path, fn.replace(".csv", "_re.csv"))
    with open(summary_fn) as in_handle:
        with open(out_file, 'w') as out_handle:
            for line in in_handle:
                cols = line.strip().split(",")
                fix_line = ",".join([op.relpath(c, root_path) if op.exists(c) else c for c in cols])
                print >>out_handle, fix_line
    report_file = _modify_report(root_path, out_file)

    return out_file, report_file

def _modify_report(summary_path, summary_fn):
    """Read Rmd template and dump with project path."""
    summary_path = op.abspath(summary_path)
    template = op.normpath(op.join(op.dirname(op.realpath(template_seqcluster.__file__)), "report.rmd"))
    content = open(template).read()
    out_content = string.Template(content).safe_substitute({'path_abs': summary_path,
                                                            'path_summary': os.path.join(summary_path, summary_fn)})
    out_file = op.join(op.dirname(summary_fn), "srna_report.rmd")
    with open(out_file, 'w') as out_handle:
        print >>out_handle, out_content

    return out_file

def _make_isomir_counts(data, srna_type="seqbuster", out_dir=None, stem=""):
    """
    Parse miraligner files to create count matrix.
    """
    work_dir = dd.get_work_dir(data[0][0])
    if not out_dir:
        out_dir = op.join(work_dir, "mirbase")
    out_novel_isomir = append_stem(op.join(out_dir, "counts.tsv"), stem)
    out_novel_mirna = append_stem(op.join(out_dir, "counts_mirna.tsv"), stem)
    logger.debug("Create %s count data at %s." % (srna_type, out_dir))
    if file_exists(out_novel_mirna):
        return [out_novel_mirna, out_novel_isomir]
    out_dts = []
    for sample in data:
        if sample[0].get(srna_type):
            miraligner_fn = sample[0][srna_type]
            reads = _read_miraligner(miraligner_fn)
            if reads:
                out_file, dt, dt_pre = _tab_output(reads, miraligner_fn + ".back", dd.get_sample_name(sample[0]))
                out_dts.append(dt)
            else:
                logger.debug("WARNING::%s has NOT miRNA annotated for %s. Check if fasta files is small or species value." % (dd.get_sample_name(sample[0]), srna_type))
    if out_dts:
        out_files = _create_counts(out_dts, out_dir)
        out_files = [move_safe(out_files[0], out_novel_isomir), move_safe(out_files[1], out_novel_mirna)]
        return out_files
    else:
        logger.debug("WARNING::any samples have miRNA annotated for %s. Check if fasta files is small or species value." % srna_type)


text:
package org.renjin.compiler.ir.tac.functions;

import org.renjin.compiler.ir.exception.InvalidSyntaxException;
import org.renjin.compiler.ir.tac.IRBodyBuilder;
import org.renjin.compiler.ir.tac.expressions.Expression;
import org.renjin.compiler.ir.tac.statements.ExprStatement;
import org.renjin.sexp.FunctionCall;
import org.renjin.sexp.SEXP;
import org.renjin.sexp.Symbol;

public class InternalCallTranslator extends FunctionCallTranslator {

  @Override
  public Symbol getName() {
    return Symbol.get(".Internal");
  }

  @Override
  public Expression translateToExpression(IRBodyBuilder builder,
      TranslationContext context, FunctionCall call) {
    SEXP argument = call.getArgument(0);
    if(!(argument instanceof FunctionCall)) {
      throw new InvalidSyntaxException(".Internal() expects a language object as its only argument");
    }
    FunctionCall primitiveCall = (FunctionCall) argument;

    return builder.translatePrimitiveCall(context, primitiveCall);
  }

  @Override
  public void addStatement(IRBodyBuilder builder, TranslationContext context,
      FunctionCall call) {
    builder.addStatement(
        new ExprStatement(translateToExpression(builder, context, call)));
  }
}


text:
require 'irb'

$LOAD_PATH.unshift(File.dirname(__FILE__))

require 'helper'

IRB.start


text:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<title>FeedbackSettings</title>
<meta name="date" content="2013-11-30">
<link rel="stylesheet" type="text/css" href="../../../stylesheet.css" title="Style">

JavaScript is disabled on your browser.

Package

Class

Use

Tree

Deprecated

Index

Help

Prev Class

Next Class

Frames

No Frames

All Classes

Summary:

Nested |

Field |

Constr |

Method

Detail:

Field |

Constr |

Method

com.suredigit.inappfeedback

Class FeedbackSettings

java.lang.Object

com.suredigit.inappfeedback.FeedbackSettings

public class FeedbackSettings
extends java.lang.Object

Constructor Summary

Constructors

Constructor and Description

FeedbackSettings()

Method Summary

Methods

Modifier and Type
Method and Description

java.lang.String
getBugLabel()

java.lang.String
getCancelButtonText()

int
getGravity()

java.lang.String
getIdeaLabel()

int
getOrientation()

java.lang.String
getQuestionLabel()

java.lang.String
getReplyCloseButtonText()

java.lang.String
getReplyTitle()

java.lang.String
getSendButtonText()

java.lang.String
getText()

java.lang.String
getTitle()

java.lang.String
getToast()

java.lang.String
getYourComments()

boolean
isEnableRadio()

boolean
isModal()

void
setBugLabel(java.lang.String bugLabel)

void
setCancelButtonText(java.lang.String cancelButton)

void
setGravity(int gravity)

void
setIdeaLabel(java.lang.String ideaLabel)

void
setModal(boolean isModal)

void
setOrientation(int orientation)

void
setQuestionLabel(java.lang.String questionLabel)

void
setRadioButtons(boolean enableRadio)

void
setReplyCloseButtonText(java.lang.String replyCloseButtonText)

void
setReplyTitle(java.lang.String replyTitle)

void
setSendButtonText(java.lang.String sendButton)

void
setText(java.lang.String text)

void
setTitle(java.lang.String title)

void
setToast(java.lang.String toast)

void
setYourComments(java.lang.String yourComments)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FeedbackSettings

public FeedbackSettings()

Method Detail

getTitle

public java.lang.String getTitle()

setTitle

public void setTitle(java.lang.String title)

getText

public java.lang.String getText()

setText

public void setText(java.lang.String text)

getToast

public java.lang.String getToast()

setToast

public void setToast(java.lang.String toast)

getSendButtonText

public java.lang.String getSendButtonText()

setSendButtonText

public void setSendButtonText(java.lang.String sendButton)

getCancelButtonText

public java.lang.String getCancelButtonText()

setCancelButtonText

public void setCancelButtonText(java.lang.String cancelButton)

getYourComments

public java.lang.String getYourComments()

setYourComments

public void setYourComments(java.lang.String yourComments)

getBugLabel

public java.lang.String getBugLabel()

setBugLabel

public void setBugLabel(java.lang.String bugLabel)

getIdeaLabel

public java.lang.String getIdeaLabel()

setIdeaLabel

public void setIdeaLabel(java.lang.String ideaLabel)

getQuestionLabel

public java.lang.String getQuestionLabel()

setQuestionLabel

public void setQuestionLabel(java.lang.String questionLabel)

isModal

public boolean isModal()

setModal

public void setModal(boolean isModal)

isEnableRadio

public boolean isEnableRadio()

setRadioButtons

public void setRadioButtons(boolean enableRadio)

getReplyTitle

public java.lang.String getReplyTitle()

setReplyTitle

public void setReplyTitle(java.lang.String replyTitle)

getReplyCloseButtonText

public java.lang.String getReplyCloseButtonText()

setReplyCloseButtonText

public void setReplyCloseButtonText(java.lang.String replyCloseButtonText)

getOrientation

public int getOrientation()

setOrientation

public void setOrientation(int orientation)

getGravity

public int getGravity()

setGravity

public void setGravity(int gravity)

Package

Class

Use

Tree

Deprecated

Index

Help

Prev Class

Next Class

Frames

No Frames

All Classes

Summary:

Nested |

Field |

Constr |

Method

Detail:

Field |

Constr |

Method