text:
Makahiki Development Environment
================================
This chapter describes how to setup the Makahiki development environment.

Using the Makahiki VirtualBox Virtual Machine
---------------------------------------------
To facilitate the development environment setup, we create an VirtualBox virtual machine which has the necessary dependencies of the Makahiki and the development tools pre-installed. The advantage of developing using the Makahiki VM is the quick set up without the need of installing all the dependency of Makahiki. The shortcoming is of cause the performance penalty of running everything inside a virtual machine.

If you decide not to use the Makahiki virtual machine, you can skip this step. We assume that you have already gone through the local Makahiki installation
successfully, especially, the python virtual environment has been set up.

To use the Makahiki virtual machine, first download it from the following url:

    https://www.dropbox.com/s/4563cxd2rxf1hdm/Makahiki_Ubuntu1204LTS_V1.zip

The download may take a while since the VM image is quite large as about 1.2G. In the mean time, you can download and install the Oracle VirtualBox software (https://www.virtualbox.org/wiki/Downloads) if you have not use VirtualBox before.

.. note:: You may want to install the corresponding version of the VirtualBox Extension Pack after you install the virtualbox software. This provides support for USB 2.0 etc in your VMs.

Once the download completes, unzip the VM into the same directory of the other virtualbox virtual machines. For example, in the Mac OS host, the default virtual machines directory is ~/VirtualBox VMs. If you have not use virtualbox before, you can create the default virtualbox vm directory and unzip the Makahiki vm there.

Once the virtualbox software is installed and the vm files unzipped, locate the Makahiki VM .vbox file, i.e., Makahiki_Ubuntu1204LTS_V1.vbox, and double click, the virtualbox will start and add the makahiki vm in the list of the virtualbox managed vms, as shown in the screenshot here:

.. figure:: figs/development/virtualbox.png
   :width: 600 px
   :align: center

The Makahiki VM is based on Ubuntu 12.04LTS and has the makahiki dependency pre-installed. See the "readme.txt" file in the unzipped directory for details of this VM release and also the login info for the VM.

.. note:: By default, the Makahiki VM is allocated with 2G RAM, which is the recommended configuration for development in Makahiki. If your host machine has limitted memory, you can decrease the RAM by changing the "Base Memory" under the System of the settings of the VM.

Double click on the Makahiki VM in the virtualbox VM manager to start the Makahiki VM, login using the info in the readme.txt file.

Using an IDE
------------
Although you can develop without an IDE, but we recommend using an IDE to program effeciently and effectively. Pydev and Pycharm are two excellent IDEs for
Python/Django development. The following describes the steps involved in
installing and using the Pydev IDE:

1. Install Eclipse and PyDev.

  .. note:: If you use the Makahiki VirtualBox VM, the Eclipse ad PyDev are already installed in the VM image, you can skip this step.

  You want to install a version of Eclipse that includes eGit and WebTools components. The comparison page shows which components are in which versions of Eclipse.  You can choose a version with a superset of these components if you like. After you download the install the Eclipse, you want to install the PyDev following the instructions in "Installing with the Update Site".

2. Configure the Python Interpreter for PyDev.

  .. note:: If you use the Makahiki VM, you can start the Eclipse by invoking "eclipse" from any path in a shell terminal.

  To configure the interpreter for PyDev in Eclipse, start Eclipse, go to the Eclipse menu::

    Window -> Preferences -> PyDev -> Interpreter - Python

  Choose the interpreter in the virtual environment you have created during the Makahiki Installation. To do that, Click on the "New..." button, and browse and select the python executable under the virtual environment's bin directory, see screenshot below.

.. figure:: figs/development/pydev-intepreter.png
   :width: 600 px
   :align: center

On the next screen, click on the "Select All" then "OK".

Fork the Makahiki source
------------------------

You may want to fork the Makahiki source to your own Git repository and manage the
version control in your own repository. To fork the Makahiki source, go to
https://github.com/csdl/makahiki/ and click the "Fork" button. Go to your fork,
you will see your repository has the git url like this:

git@github.com:<your-name>/makahiki.git

This is the Git url you will use for your development in Makahiki.

Now, in the Terminal, cd to ~/workspace (you may want to create this directory
if it does not exist. This is the default directory where Eclipse will place
all the projects.), run the following command to download the forked
Makahiki source::

  % git clone git@github.com/<your-name>/makahiki.git

This will create a directory called "makahiki" containing the source code
for the system.

Workon makahiki
---------------

Next, you need to activate the makahiki virtual environment. To do this, type the following in the Terminal::

  % workon makahiki

This will set up the python environment, as well as the necessary environment
variables in your current shell for developing and running Makahiki. You can examine the environment variables set by the virtual environemnt by looking at the script located at:

    ~/.virtualenvs/makahiki/bin/postactivate

  .. note:: Always activate the virtual environment by running the workon command before doing any Makahiki development or running Makahiki instance.

Initialize or update the Makahiki instance
------------------------------------------

If you had not initialized the Makahiki instance, you need to invoke the initialize_instance script, passing it an argument to specify what kind of initial data to load.  If you had already initialized the instance, you can run the update_instance script to apply any changes from the newly download makahiki source.

To initialize the instance with default dataset, run::

  % scripts/initialize_instance.py -t default

To update the instance from the latest downloaded source, run::

   % ./scripts/update_instance.py

Run Makahiki outsite of Eclipse
-------------------------------
You can start the Makahiki server using::

  % ./manage.py runserver

It will start the makahiki server at http://127.0.0.1:8000.

Import the Makahiki Project into Eclipse
----------------------------------------

Start Eclipse, go to menu::

    File -> Import -> General -> Existing Projects into Workspace

as shown in the following screenshot:

.. figure:: figs/development/import.png
   :width: 600 px
   :align: center

Next, browse and select the Makahiki source you just clone from github in the previous
step. Make sure you select the second makahiki directory which is root directory
of the makahiki Django application. It is also where the Makahiki project definition
file is located. See screenshot below.

.. figure:: figs/development/import-makahiki.png
   :width: 600 px
   :align: center

Create a Run Confiugration in Eclipse
-------------------------------------

To help you start the Makahiki server conveniently, you can create a "Run Configuration" in Eclipse. To do so, select the makahiki project in the "PyDev Package Explorer", right click to bring up the context menu, select "Run As" and "PyDev Django", see the screenshot below:

.. figure:: figs/development/runconfig.png
   :width: 600 px
   :align: center

It will run the makahiki as a Django project and start the makahiki server at http://127.0.0.1:8000. You can also find the output of the run in the Eclipse run console.

You can verify the running server by going to the above url in a brower. Once verify, go to Eclipse menu::

    Run -> Run Configurations -> PyDev Django -> makahiki makahiki

You will see a run configuration already created for you from the previous "Run as" command, as shown here:

.. figure:: figs/development/runconfig-makahiki.png
   :width: 600 px
   :align: center

You just need to change the name of the run configuration to the one you like, such as "Local Makahiki".

Now you can stop the server from previous "Run As", and select the newly created run configuration from the menu to start the makahiki server.


text:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

    <title>nova-volume — nova 2012.2.1 documentation</title>

    <link rel="stylesheet" href="../_static/nature.css" type="text/css" />
    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
    <link rel="stylesheet" href="../_static/tweaks.css" type="text/css" />

    <link rel="top" title="nova 2012.2.1 documentation" href="../index.html" />

OpenStack

Home

Projects

User Stories

Community

Blog

Wiki

Documentation

nova-volume¶

Nova Volume Server¶

<col class="field-name" />
<col class="field-body" />

Author:	openstack@lists.launchpad.net

Date:	2012-09-27

Copyright:	OpenStack LLC

Version:	2012.1

Manual section:	1

Manual group:	cloud computing

SYNOPSIS¶

nova-volume [options]

DESCRIPTION¶

nova-volume manages creating, attaching, detaching, and persistent storage.

Persistent storage volumes keep their state independent of instances.  You can
attach to an instance, terminate the instance, spawn a new instance (even
one from a different image) and re-attach the volume with the same data
intact.

OPTIONS¶

General options

FILES¶

/etc/nova/nova.conf

/etc/nova/policy.json

/etc/nova/rootwrap.conf

/etc/nova/rootwrap.d/

SEE ALSO¶

OpenStack Nova

OpenStack Nova

BUGS¶

Nova is sourced in Launchpad so you can view current bugs at OpenStack Nova

Table Of Contents

nova-volume

Nova Volume Server

SYNOPSIS

DESCRIPTION

OPTIONS

FILES

SEE ALSO

BUGS

This Page

Show Source

Quick search

              <form class="search" action="../search.html" method="get">
                <input type="text" name="q" size="18" />
                <input type="submit" value="Go" />
                <input type="hidden" name="check_keywords" value="yes" />
                <input type="hidden" name="area" value="default" />
              </form>

              Enter search terms or a module, class or function name.

Navigation

          index

          modules |

nova 2012.2.1 documentation »

        © Copyright 2010-present, OpenStack, LLC.
      Last updated on Thu Nov 1 16:11:51 2012, commit 4ed8d3e.
      Created using Sphinx 1.1.3.

redpajama_set_name: RedPajamaGithub


text:
namespace ktt
{

class TimeConfiguration : public DisableCopyMove
{
public:
    static TimeConfiguration& GetInstance();

    void SetTimeUnit(const TimeUnit unit);

    TimeUnit GetTimeUnit() const;
    std::string GetUnitTag() const;

    uint64_t ConvertFromNanoseconds(const Nanoseconds duration) const;
    double ConvertFromNanosecondsDouble(const Nanoseconds duration) const;

    Nanoseconds ConvertToNanoseconds(const uint64_t duration) const;
    Nanoseconds ConvertToNanosecondsDouble(const double duration) const;

private:
    TimeUnit m_TimeUnit;

    TimeConfiguration();
};

} // namespace ktt


text:
function dpvl = palm_vtxlabel(dpv,fac)
% Label a DPV file (vertexwise data).
%
% Usage:
% dpvl = palm_vtxlabel(dpv,fac)
%
% - dpv  : Binary data per vertex to be labelled. The non-zero
%          regions receive an unique identifier.
% - fac  : Face indices (see palm_srfread for details).
% - dpvl : Labelled data per vertex.
%
% _____________________________________
% Anderson M. Winkler
% FMRIB / Univ. of Oxford
% Feb/2012 (1st version)
% Sep/2013 (this version)
% http://brainder.org

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
% PALM -- Permutation Analysis of Linear Models
% Copyright (C) 2015 Anderson M. Winkler
%
% This program is free software: you can redistribute it and/or modify
% it under the terms of the GNU General Public License as published by
% the Free Software Foundation, either version 3 of the License, or
% any later version.
%
% This program is distributed in the hope that it will be useful,
% but WITHOUT ANY WARRANTY; without even the implied warranty of
% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
% GNU General Public License for more details.
%
% You should have received a copy of the GNU General Public License
% along with this program.  If not, see <http://www.gnu.org/licenses/>.
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

% For the labelled data
dpvl = zeros(size(dpv));

% Identify faces that are entirely (3 vertices)
% within the supra-threshold vertices. These are
% the only that will be labelled.
facm = reshape(dpv(fac(:)),size(fac));
faci = all(facm,2);

% The strategy below is as follows: at each iteration of the
% loop, it sorts the labels assigned to the 3 vertices of a
% face. If the last vertex, after sorting is zero, then all 3
% are zero, so none was yet labelled. All three then receive
% the current label "k". If, however, the 3rd is non-zero,
% then all three should stay with that label that was assigned
% in an earlier iteration. But before changing the assignment,
% change also all other vertices that have the same label as
% the 1st and 2nd vertices to have the same as the 3rd.
% Then increment the counter.

% For each face
k = 1; % label indices (to be incremented)
for f = find(faci)',

    % Sort the labels assigned for each face
    flab = sort(dpvl(fac(f,:)));
    if flab(3) == 0,

        % If the last is zero, then this is the first time all these
        % vertices are seen, so label them with the current "k".
        dpvl(fac(f,:)) = k;

        % Increment the label counter here, as otherwise the labels
        % are reused.
        k = k + 1;
    else
        % Otherwise, this means that the last (3rd) was seen already
        % and is part of some other label. Assign then this higher label
        % to all other vertices in the surface that may have received,
        % in earlier iterations of the loop, indices that are assigned
        % to the 1st or 2nd vertices of the current face.
        if flab(2) ~= 0;
            dpvl(dpvl == flab(2)) = flab(3);
        end
        if flab(1) ~= 0;
            dpvl(dpvl == flab(1)) = flab(3);
        end

        % Finally, mark all other vertices of this face with the
        % same lavel as the 3rd one
        dpvl(fac(f,:)) = flab(3);
    end
end


text:
<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
<title>Signal Handling</title>
<link rel="stylesheet" href="../../../../doc/src/boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.78.1">
<link rel="home" href="../../boost_asio.html" title="Boost.Asio">
<link rel="up" href="../overview.html" title="Overview">
<link rel="prev" href="serial_ports.html" title="Serial Ports">
<link rel="next" href="posix.html" title="POSIX-Specific Functionality">

Home
Libraries
People
FAQ
More

Signal Handling

        Boost.Asio supports signal handling using a class called signal_set.
        Programs may add one or more signals to the set, and then perform an async_wait()
        operation. The specified handler will be called when one of the signals occurs.
        The same signal number may be registered with multiple signal_set
        objects, however the signal number must be used only with Boost.Asio.

void handler(
    const boost::system::error_code& error,
    int signal_number)
{
  if (!error)
  {
    // A signal occurred.
  }
}

...

// Construct a signal set registered for process termination.
boost::asio::signal_set signals(io_service, SIGINT, SIGTERM);

// Start an asynchronous wait for one of the signals to occur.
signals.async_wait(handler);

        Signal handling also works on Windows, as the Microsoft Visual C++ runtime
        library maps console events like Ctrl+C to the equivalent signal.

        See
        Also

        signal_set, HTTP server example
        (C++03), HTTP
        server example (C++11).

Copyright © 2003-2015 Christopher M.
      Kohlhoff

        Distributed under the Boost Software License, Version 1.0. (See accompanying
        file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)


text:
/*
 * SnrUltraSonic.h
 *
 * Created: 06/07/2012 17:29:08
 *  Author: berryer
 */

#ifndef CMPULTRASONIC_H
#define CMPULTRASONIC_H

#include "Conf/ConfHard.h"
#include "Drv/DrvTick.h"
#include "Drv/DrvEeprom.h"

////////////////////////////////////////INCLUDES//////////////////////////////////////////////////

////////////////////////////////////////PUBLIC DEFINES////////////////////////////////////////////

////////////////////////////////////////PUBLIC STRUCTURES/////////////////////////////////////////
typedef struct
{
	Int16U distance;
} UltraSonicData;

typedef Boolean (*CmpUltraSonicPtrFctInit)		( void );
typedef Boolean (*CmpUltraSonicPtrFctReadData)	( void );

typedef struct
{
	const CmpUltraSonicPtrFctInit init;
	const CmpUltraSonicPtrFctReadData readData;
}UltraSonicFunctions;

typedef struct
{
	UltraSonicFunctions *functions;
	UltraSonicData *data;
}UltraSonic;

////////////////////////////////////////PUBLIC FUNCTIONS//////////////////////////////////////////

Boolean SnrUltraSonicInit				( void );
Boolean SnrUltraSonicGetDistance     	( void );
UltraSonicData* SnrUltraSonicGetStruct	( void );

#endif //CMPULTRASONIC_H


text:
package org.apache.tamaya;

import java.util.Collection;
import java.util.Map;

/**
 * Stage which should be supported by all layers.
 */
public interface Stage {
    /**
     * Get the stage's name. A stage's name is unique for a whole system.
     * @return the stage's name, never null.
     */
    public String getName();

    /**
     * Get all the stage's (unmodifiable) properties.
     * @return the stage's (unmodifiable) properties, never null.
     */
    public Map<String,String> getProperties();

    /**
     * Get the default stage for develpment.
     * @return the default stage, never null.
     */
    public static Stage development(){
        return Stages.getDevelopmentStage();
    }

    /**
     * Get the default stage for (component) testing.
     * @return the default stage, never null.
     */
    public static Stage test(){
        return Stages.getTestStage();
    }

    /**
     * Get the default stage for integration (testing).
     * @return the default stage, never null.
     */
    public static Stage integration(){
        return Stages.getIntegrationStage();
    }

    /**
     * Get the default stage for staging.
     * @return the default stage, never null.
     */
    public static Stage staging(){
        return Stages.getStagingStage();
    }

    /**
     * Get the default stage for production.
     * @return the default stage, never null.
     */
    public static Stage production(){
        return Stages.getProductionStage();
    }

    /**
     * Get a stage by name. If not present, create a new stage.
     *
     * @param name the stage's name.
     * @return tge stage instance, never null.
     */
    public static Stage of(String name){
        return Stages.getStage(name);
    }

    /**
     * Adds a new stage.
     *
     * @param stage the new stage instance.
     * @throws IllegalStateException if a stage with the same name is already existing.
     */
    public static void add(Stage stage){
        Stages.addStage(stage);
    }

    /**
     * Access all the stages currently defined.
     *
     * @return the current stages, never null.
     */
    public static Collection<Stage> getStages(){
        return Stages.getStages();
    }
}


text:
package pathfs

import (
	"fmt"
	"syscall"
	"time"

	"github.com/hanwen/go-fuse/fuse"
)

func (fs *loopbackFileSystem) StatFs(name string) *fuse.StatfsOut {
	s := syscall.Statfs_t{}
	err := syscall.Statfs(fs.GetPath(name), &s)
	if err == nil {
		return &fuse.StatfsOut{
			Blocks:  s.Blocks,
			Bsize:   uint32(s.Bsize),
			Bfree:   s.Bfree,
			Bavail:  s.Bavail,
			Files:   s.Files,
			Ffree:   s.Ffree,
			Frsize:  uint32(s.Frsize),
			NameLen: uint32(s.Namelen),
		}
	}
	return nil
}

func (fs *loopbackFileSystem) ListXAttr(name string, context *fuse.Context) ([]string, fuse.Status) {
	data, err := listXAttr(fs.GetPath(name))

	return data, fuse.ToStatus(err)
}

func (fs *loopbackFileSystem) RemoveXAttr(name string, attr string, context *fuse.Context) fuse.Status {
	err := sysRemovexattr(fs.GetPath(name), attr)
	return fuse.ToStatus(err)
}

func (fs *loopbackFileSystem) String() string {
	return fmt.Sprintf("LoopbackFs(%s)", fs.Root)
}

func (fs *loopbackFileSystem) GetXAttr(name string, attr string, context *fuse.Context) ([]byte, fuse.Status) {
	data := make([]byte, 1024)
	data, err := getXAttr(fs.GetPath(name), attr, data)

	return data, fuse.ToStatus(err)
}

func (fs *loopbackFileSystem) SetXAttr(name string, attr string, data []byte, flags int, context *fuse.Context) fuse.Status {
	err := syscall.Setxattr(fs.GetPath(name), attr, data, flags)
	return fuse.ToStatus(err)
}

const _UTIME_NOW = ((1 << 30) - 1)
const _UTIME_OMIT = ((1 << 30) - 2)

// Utimens - path based version of loopbackFile.Utimens()
func (fs *loopbackFileSystem) Utimens(path string, a *time.Time, m *time.Time, context *fuse.Context) (code fuse.Status) {
	var ts [2]syscall.Timespec

	if a == nil {
		ts[0].Nsec = _UTIME_OMIT
	} else {
		ts[0] = syscall.NsecToTimespec(a.UnixNano())
		ts[0].Nsec = 0
	}

	if m == nil {
		ts[1].Nsec = _UTIME_OMIT
	} else {
		ts[1] = syscall.NsecToTimespec(m.UnixNano())
		ts[1].Nsec = 0
	}

	err := sysUtimensat(0, fs.GetPath(path), &ts, _AT_SYMLINK_NOFOLLOW)
	return fuse.ToStatus(err)
}


text:
<?php

namespace Briz\tests\Base;

use Briz\Base\Collection;

class CollectionTest extends \PHPUnit_Framework_TestCase
{

    /**
     * @var Collection
     */
    private $collection;

    /**
     * @var ReflectionProperty
     */
    private $property;

    public function setUp()
    {
        $this->collection = new Collection();
        $this->property = new \ReflectionProperty($this->collection, 'items');
        $this->property->setAccessible(true);
    }

    public function testConstruct()
    {
        $data = ['foo' => 'bar'];
        $bag = new Collection($data);
        $bagProperty = new \ReflectionProperty($bag, 'items');
        $bagProperty->setAccessible(true);

        $this->assertEquals($data, $bagProperty->getValue($bag));
    }

    public function testAll()
    {
        $data = ['foo'=>'bar','data1'=>'data2'];
        $this->property->setValue($this->collection, $data);
        $this->assertEquals($data, $this->collection->all());
    }

    public function testClear()
    {
        $data = ['foo'=>'bar','data1'=>'data2'];
        $this->property->setValue($this->collection, $data);
        $this->collection->clear();
        $this->assertEquals([], $this->collection->all());
    }

    public function testCount()
    {
        $data = ['foo'=>'bar','data1'=>'data2'];
        $this->property->setValue($this->collection, $data);
        $count = $this->collection->count();
        $this->assertEquals($count, 2);
    }

    public function testGet()
    {
        $data = ['foo'=>'bar','data1'=>'data2','user'=>'abc1'];
        $this->property->setValue($this->collection, $data);
        $this->assertEquals($data['data1'], $this->collection->get('data1'));
    }

    public function testHas()
    {
        $data = ['foo'=>'bar','data1'=>'data2','user'=>'abc1'];
        $this->property->setValue($this->collection, $data);
        $this->assertFalse($this->collection->has('none'));
        $this->assertTrue($this->collection->has('data1'));
    }

    public function testIsempty()
    {
        $data = ['foo'=>'bar'];
        $this->property->setValue($this->collection, $data);
        $this->assertFalse($this->collection->isempty());
        $data = [];
        $this->property->setValue($this->collection, $data);
        $this->assertTrue($this->collection->isempty());
    }

    public function testRemove()
    {
        $data = ['foo'=>'bar','data1'=>'data2','user'=>'abc1'];
        $this->property->setValue($this->collection, $data);
        $this->collection->remove('user');
        $array = $this->property->getValue($this->collection);
        $this->assertArrayNotHasKey('user', $array);
    }

    public function testReplace()
    {
        $data1 = ['foo'=>'bar','data1'=>'data2','user'=>'abc1'];
        $data2 = ['foo1'=>'bar','data'=>'data2','user1'=>'abc1'];
        $this->property->setValue($this->collection, $data2);
        $this->collection->replace($data1);
        $this->assertArrayHasKey('data1', $this->property->getValue($this->collection));
        $this->assertArrayNotHasKey('data', $this->property->getValue($this->collection));
    }

    public function testSet()
    {
        $this->collection->set('abcd', 'a');
        $this->assertArrayHasKey('abcd', $this->property->getValue($this->collection));
    }
}


text:
package App::Happyman::Test;
use v5.14;
use warnings;

use parent 'Test::Builder::Module';
our @EXPORT
    = qw(make_happyman_with_plugin make_test_client wait_on_event_or_timeout disconnect_and_wait wait_on_message_or_timeout wait_on_action_or_timeout load_local_config async_sleep);

use AnyEvent;
use AnyEvent::IRC::Client;
use File::Slurp;
use Mojo::JSON;

sub load_local_config {
    my $json = Mojo::JSON->new;
    my $conf = $json->decode( scalar read_file('happyman.conf') );
    if ( !$conf ) {
        __PACKAGE__->builder->BAIL_OUT('Failed to load local happyman.conf');
    }

    return $conf;
}

sub make_happyman_with_plugin {
    my ( $plugin_name, $plugin_params ) = @_;
    my $conf     = load_local_config();
    my $happyman = App::Happyman::Connection->new(
        nick    => $conf->{connection}{nick}    // 'happyman',
        host    => $conf->{connection}{host}    // 'localhost',
        port    => $conf->{connection}{port}    // 6667,
        channel => $conf->{connection}{channel} // '#happyman',
        debug => $ENV{HAPPYMAN_TEST_DEBUG} ? 1 : 0,
    );
    $happyman->load_plugin( $plugin_name, $plugin_params );
    return $happyman;
}

sub make_test_client {
    my ($nick) = @_;
    my $conf = load_local_config();
    $nick ||= 'HMTest';
    my $irc = AnyEvent::IRC::Client->new();
    my $cv  = AE::cv;
    $irc->reg_cb( connect => $cv );
    $irc->connect(
        $conf->{connection}{host} // 'localhost',
        $conf->{connection}{port} // 6667,
        { nick => $nick }
    );
    my ( undef, $error ) = $cv->recv();

    if ($error) {
        __PACKAGE__->builder->BAIL_OUT(
            "Failed to connect to test IRC server!: $error");
    }
    $irc->send_srv( 'JOIN', '#happyman' );
    return $irc;
}

sub wait_on_event_or_timeout {
    my ( $irc, $event, $timeout ) = @_;
    $timeout //= 10;
    my $cv = AE::cv;
    $irc->reg_cb( $event => $cv );
    my $timer = AE::timer( $timeout, 0, $cv );
    return $cv->recv();
}

sub wait_on_message_or_timeout {
    my ( $irc, $timeout ) = @_;
    my ( undef, undef, $ircmsg )
        = wait_on_event_or_timeout( $irc, 'publicmsg', $timeout );
    return $ircmsg ? $ircmsg->{params}->[1] : undef;
}

sub wait_on_action_or_timeout {
    my ( $irc, $timeout ) = @_;
    my ( undef, undef, undef, $msg )
        = wait_on_event_or_timeout( $irc, 'ctcp_action', $timeout );
    return $msg;
}

sub disconnect_and_wait {
    my ( $irc, $timeout ) = @_;
    my $cv = AE::cv;
    $irc->reg_cb( disconnect => $cv );
    $irc->send_srv('QUIT');
    $cv->recv();
    return;
}

sub async_sleep {
    my ($seconds) = @_;
    my $cv = AE::cv;
    my $timer = AE::timer $seconds, 0, $cv;
    $cv->recv();
    return;
}

1;


text:
<?php

namespace Cartalyst\Stripe\Api\Terminal;

use Cartalyst\Stripe\Api\Api;

class Readers extends Api
{
    /**
     * Creates a new terminal reader.
     *
     * @param  array  $parameters
     * @return array
     */
    public function create(array $parameters = [])
    {
        return $this->_post('terminal/readers', $parameters);
    }

    /**
     * Retrieves an existing terminal reader.
     *
     * @param  string  $readerId
     * @return array
     */
    public function find($readerId)
    {
        return $this->_get("terminal/readers/{$readerId}");
    }

    /**
     * Updates an existing terminal reader.
     *
     * @param  string  $readerId
     * @param  array  $parameters
     * @return array
     */
    public function update($readerId, array $parameters = [])
    {
        return $this->_post("terminal/readers/{$readerId}", $parameters);
    }

    /**
     * Deletes an existing terminal reader.
     *
     * @param  string  $readerId
     * @return array
     */
    public function delete($readerId)
    {
        return $this->_delete("terminal/readers/{$readerId}");
    }

    /**
     * Lists all terminal readers.
     *
     * @param  array  $parameters
     * @return array
     */
    public function all(array $parameters = [])
    {
        return $this->_get('terminal/readers', $parameters);
    }
}


text:
/* Alexander Heinecke (Intel Corp.)
******************************************************************************/

#ifndef GENERATOR_GEMM_AVX512_MICROKERNEL_H
#define GENERATOR_GEMM_AVX512_MICROKERNEL_H

#include "generator_common.h"
#include "generator_gemm_common.h"

LIBXSMM_API_INTERN void libxsmm_generator_gemm_avx512_microkernel_nofsdbcst( libxsmm_generated_code*            io_generated_code,
                                                                             const libxsmm_gp_reg_mapping*      i_gp_reg_mapping,
                                                                             const libxsmm_micro_kernel_config* i_micro_kernel_config,
                                                                             const libxsmm_gemm_descriptor*     i_xgemm_desc,
                                                                             const unsigned int                 i_m_blocking,
                                                                             const unsigned int                 i_n_blocking,
                                                                             const int                          i_offset );

LIBXSMM_API_INTERN void libxsmm_generator_gemm_avx512_microkernel_bf16_emu_nofsdbcst( libxsmm_generated_code*            io_generated_code,
                                                                                const libxsmm_gp_reg_mapping*      i_gp_reg_mapping,
                                                                                const libxsmm_micro_kernel_config* i_micro_kernel_config,
                                                                                const libxsmm_gemm_descriptor*     i_xgemm_desc,
                                                                                const unsigned int                 i_m_blocking,
                                                                                const unsigned int                 i_n_blocking,
                                                                                const int                          i_offset );

LIBXSMM_API_INTERN void libxsmm_generator_gemm_avx512_microkernel_m8_bf16_emu_nofsdbcst( libxsmm_generated_code*            io_generated_code,
                                                                                const libxsmm_gp_reg_mapping*      i_gp_reg_mapping,
                                                                                const libxsmm_micro_kernel_config* i_micro_kernel_config,
                                                                                const libxsmm_gemm_descriptor*     i_xgemm_desc,
                                                                                const unsigned int                 i_m_blocking,
                                                                                const unsigned int                 i_n_blocking,
                                                                                const int                          i_offset );

LIBXSMM_API_INTERN void libxsmm_generator_gemm_avx512_microkernel_m8_nofsdbcst( libxsmm_generated_code*            io_generated_code,
                                                                                const libxsmm_gp_reg_mapping*      i_gp_reg_mapping,
                                                                                const libxsmm_micro_kernel_config* i_micro_kernel_config,
                                                                                const libxsmm_gemm_descriptor*     i_xgemm_desc,
                                                                                const unsigned int                 i_m_blocking,
                                                                                const unsigned int                 i_n_blocking,
                                                                                const int                          i_offset );

LIBXSMM_API_INTERN void libxsmm_generator_gemm_avx512_microkernel_fsdbcst( libxsmm_generated_code*            io_generated_code,
                                                                           const libxsmm_gp_reg_mapping*      i_gp_reg_mapping,
                                                                           const libxsmm_micro_kernel_config* i_micro_kernel_config,
                                                                           const libxsmm_gemm_descriptor*     i_xgemm_desc,
                                                                           const unsigned int                 i_n_blocking,
                                                                           const unsigned int                 i_k_blocking );

LIBXSMM_API_INTERN void libxsmm_generator_gemm_avx512_microkernel_fsdbcst_qfma( libxsmm_generated_code*            io_generated_code,
                                                                                const libxsmm_gp_reg_mapping*      i_gp_reg_mapping,
                                                                                const libxsmm_micro_kernel_config* i_micro_kernel_config,
                                                                                const libxsmm_gemm_descriptor*     i_xgemm_desc,
                                                                                const unsigned int                 i_n_blocking,
                                                                                const unsigned int                 i_k_blocking );

#endif /* GENERATOR_GEMM_AVX512_MICROKERNEL_H */


text:
package domainsocket

import "v2ray.com/core/common/errors"

func newError(values ...interface{}) *errors.Error {
	return errors.New(values...).Path("Transport", "Internet", "DomainSocket")
}


text:
<?php
use yii\widgets\Breadcrumbs;
use dmstr\widgets\Alert;
?>

        <?=
        Breadcrumbs::widget(
            [
                'links' => isset($this->params['breadcrumbs']) ? $this->params['breadcrumbs'] : [],
            ]
        ) ?>

        <?= Alert::widget() ?>
        <?= $content ?>


text:
<?php
$route['dashboard'] = 'dashboard';


text:
package com.google.cloud.video.livestream.v1;

import com.google.api.pathtemplate.PathTemplate;
import com.google.api.resourcenames.ResourceName;
import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableMap;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import javax.annotation.Generated;

// AUTO-GENERATED DOCUMENTATION AND CLASS.
@Generated("by gapic-generator-java")
public class InputName implements ResourceName {
  private static final PathTemplate PROJECT_LOCATION_INPUT =
      PathTemplate.createWithoutUrlEncoding(
          "projects/{project}/locations/{location}/inputs/{input}");
  private volatile Map<String, String> fieldValuesMap;
  private final String project;
  private final String location;
  private final String input;

  @Deprecated
  protected InputName() {
    project = null;
    location = null;
    input = null;
  }

  private InputName(Builder builder) {
    project = Preconditions.checkNotNull(builder.getProject());
    location = Preconditions.checkNotNull(builder.getLocation());
    input = Preconditions.checkNotNull(builder.getInput());
  }

  public String getProject() {
    return project;
  }

  public String getLocation() {
    return location;
  }

  public String getInput() {
    return input;
  }

  public static Builder newBuilder() {
    return new Builder();
  }

  public Builder toBuilder() {
    return new Builder(this);
  }

  public static InputName of(String project, String location, String input) {
    return newBuilder().setProject(project).setLocation(location).setInput(input).build();
  }

  public static String format(String project, String location, String input) {
    return newBuilder()
        .setProject(project)
        .setLocation(location)
        .setInput(input)
        .build()
        .toString();
  }

  public static InputName parse(String formattedString) {
    if (formattedString.isEmpty()) {
      return null;
    }
    Map<String, String> matchMap =
        PROJECT_LOCATION_INPUT.validatedMatch(
            formattedString, "InputName.parse: formattedString not in valid format");
    return of(matchMap.get("project"), matchMap.get("location"), matchMap.get("input"));
  }

  public static List<InputName> parseList(List<String> formattedStrings) {
    List<InputName> list = new ArrayList<>(formattedStrings.size());
    for (String formattedString : formattedStrings) {
      list.add(parse(formattedString));
    }
    return list;
  }

  public static List<String> toStringList(List<InputName> values) {
    List<String> list = new ArrayList<>(values.size());
    for (InputName value : values) {
      if (value == null) {
        list.add("");
      } else {
        list.add(value.toString());
      }
    }
    return list;
  }

  public static boolean isParsableFrom(String formattedString) {
    return PROJECT_LOCATION_INPUT.matches(formattedString);
  }

  @Override
  public Map<String, String> getFieldValuesMap() {
    if (fieldValuesMap == null) {
      synchronized (this) {
        if (fieldValuesMap == null) {
          ImmutableMap.Builder<String, String> fieldMapBuilder = ImmutableMap.builder();
          if (project != null) {
            fieldMapBuilder.put("project", project);
          }
          if (location != null) {
            fieldMapBuilder.put("location", location);
          }
          if (input != null) {
            fieldMapBuilder.put("input", input);
          }
          fieldValuesMap = fieldMapBuilder.build();
        }
      }
    }
    return fieldValuesMap;
  }

  public String getFieldValue(String fieldName) {
    return getFieldValuesMap().get(fieldName);
  }

  @Override
  public String toString() {
    return PROJECT_LOCATION_INPUT.instantiate(
        "project", project, "location", location, "input", input);
  }

  @Override
  public boolean equals(Object o) {
    if (o == this) {
      return true;
    }
    if (o != null || getClass() == o.getClass()) {
      InputName that = ((InputName) o);
      return Objects.equals(this.project, that.project)
          && Objects.equals(this.location, that.location)
          && Objects.equals(this.input, that.input);
    }
    return false;
  }

  @Override
  public int hashCode() {
    int h = 1;
    h *= 1000003;
    h ^= Objects.hashCode(project);
    h *= 1000003;
    h ^= Objects.hashCode(location);
    h *= 1000003;
    h ^= Objects.hashCode(input);
    return h;
  }

  /** Builder for projects/{project}/locations/{location}/inputs/{input}. */
  public static class Builder {
    private String project;
    private String location;
    private String input;

    protected Builder() {}

    public String getProject() {
      return project;
    }

    public String getLocation() {
      return location;
    }

    public String getInput() {
      return input;
    }

    public Builder setProject(String project) {
      this.project = project;
      return this;
    }

    public Builder setLocation(String location) {
      this.location = location;
      return this;
    }

    public Builder setInput(String input) {
      this.input = input;
      return this;
    }

    private Builder(InputName inputName) {
      this.project = inputName.project;
      this.location = inputName.location;
      this.input = inputName.input;
    }

    public InputName build() {
      return new InputName(this);
    }
  }
}


text:
<Page xmlns="http://schemas.nativescript.org/tns.xsd" loaded="pageLoaded">
    <Page.actionBar>
        <ActionBar title="NativeScript-FacebookRebound" backgroundColor="#336699" color="#fff"/>
    </Page.actionBar>
    <StackLayout>
        <Label text="Tap the picture or button" textWrap="true" class="message"/>
        <Image tap="springThis" loaded="picLoaded" width="200" id="pic" height="240" src="~/images/brokenGlass.jpg" stretch="aspectFill" />
        <label text="{{ movedUp }}" textWrap="true" />
        <button text="Spring Time!!!" tap="springThis" />
    </StackLayout>
</Page>


text:
<?php

namespace Magento\Sales\Test\TestCase;

use Magento\Sales\Test\Fixture\OrderInjectable;
use Magento\Mtf\Fixture\FixtureFactory;
use Magento\Mtf\Fixture\FixtureInterface;
use Magento\Mtf\TestCase\Injectable;

/**
 * Preconditions:
 * 1. Enable payment method "Check/Money Order".
 * 2. Enable shipping method one of "Flat Rate/Free Shipping".
 * 3. Create order.
 * 4. Create Invoice.
 *
 * Steps:
 * 1. Go to Sales > Orders > find out placed order and open.
 * 2. Click 'Credit Memo' button.
 * 3. Fill data from dataset.
 * 4. On order's page click 'Refund offline' button.
 * 5. Perform all assertions.
 *
 * @group Order_Management_(CS)
 * @ZephyrId MAGETWO-29116
 */
class CreateCreditMemoEntityTest extends Injectable
{
    /* tags */
    const MVP = 'yes';
    const DOMAIN = 'CS';
    /* end tags */

    /**
     * Fixture factory.
     *
     * @var FixtureFactory
     */
    protected $fixtureFactory;

    /**
     * Skip fields for create product fixture.
     *
     * @var array
     */
    protected $skipFields = [
        'attribute_set_id',
        'website_ids',
        'checkout_data',
        'type_id',
        'price',
    ];

    /**
     * Set up configuration.
     *
     * @param FixtureFactory $fixtureFactory
     * @return void
     */
    public function __prepare(FixtureFactory $fixtureFactory)
    {
        $this->fixtureFactory = $fixtureFactory;

        $setupConfigurationStep = $this->objectManager->create(
            'Magento\Config\Test\TestStep\SetupConfigurationStep',
            ['configData' => 'checkmo, flatrate']
        );
        $setupConfigurationStep->run();
    }

    /**
     * Create credit memo.
     *
     * @param OrderInjectable $order
     * @param array $data
     * @return array
     */
    public function test(OrderInjectable $order, array $data)
    {
        // Preconditions
        $order->persist();
        $this->objectManager->create('Magento\Sales\Test\TestStep\CreateInvoiceStep', ['order' => $order])->run();

        // Steps
        $createCreditMemoStep = $this->objectManager->create(
            'Magento\Sales\Test\TestStep\CreateCreditMemoStep',
            ['order' => $order, 'data' => $data]
        );
        $result = $createCreditMemoStep->run();

        return [
            'ids' => ['creditMemoIds' => $result['creditMemoIds']],
            'product' => $this->getProduct($order, $data),
            'customer' => $order->getDataFieldConfig('customer_id')['source']->getCustomer()
        ];
    }

    /**
     * Get product's fixture.
     *
     * @param OrderInjectable $order
     * @param array $data
     * @param int $index [optional]
     * @return FixtureInterface
     */
    protected function getProduct(OrderInjectable $order, array $data, $index = 0)
    {
        if (!isset($data['items_data'][$index]['back_to_stock'])
            || $data['items_data'][$index]['back_to_stock'] != 'Yes'
        ) {
            return $order->getEntityId()['products'][$index];
        }
        $product = $order->getEntityId()['products'][$index];
        $productData = $product->getData();
        $checkoutDataQty = $productData['checkout_data']['qty'];
        $productData['quantity_and_stock_status']['qty'] -= ($checkoutDataQty - $data['items_data'][$index]['qty']);
        $productData = array_diff_key($productData, array_flip($this->skipFields));

        return $this->fixtureFactory->create(get_class($product), ['data' => $productData]);
    }
}


text:
package org.apache.flink.test.operators.util;

import org.apache.flink.api.common.typeinfo.BasicTypeInfo;
import org.apache.flink.api.common.typeinfo.PrimitiveArrayTypeInfo;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.api.java.tuple.Tuple3;
import org.apache.flink.api.java.tuple.Tuple5;
import org.apache.flink.api.java.tuple.Tuple7;
import org.apache.flink.api.java.typeutils.TupleTypeInfo;

import org.apache.hadoop.io.IntWritable;

import java.io.File;
import java.io.Serializable;
import java.math.BigDecimal;
import java.math.BigInteger;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.Date;
import java.util.GregorianCalendar;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import scala.math.BigInt;

/**
 * #######################################################################################################
 * BE AWARE THAT OTHER TESTS DEPEND ON THIS TEST DATA. IF YOU MODIFY THE DATA MAKE SURE YOU CHECK
 * THAT ALL TESTS ARE STILL WORKING!
 * #######################################################################################################
 */
public class CollectionDataSets {

    public static DataSet<Tuple3<Integer, Long, String>> get3TupleDataSet(
            ExecutionEnvironment env) {

        List<Tuple3<Integer, Long, String>> data = new ArrayList<>();
        data.add(new Tuple3<>(1, 1L, "Hi"));
        data.add(new Tuple3<>(2, 2L, "Hello"));
        data.add(new Tuple3<>(3, 2L, "Hello world"));
        data.add(new Tuple3<>(4, 3L, "Hello world, how are you?"));
        data.add(new Tuple3<>(5, 3L, "I am fine."));
        data.add(new Tuple3<>(6, 3L, "Luke Skywalker"));
        data.add(new Tuple3<>(7, 4L, "Comment#1"));
        data.add(new Tuple3<>(8, 4L, "Comment#2"));
        data.add(new Tuple3<>(9, 4L, "Comment#3"));
        data.add(new Tuple3<>(10, 4L, "Comment#4"));
        data.add(new Tuple3<>(11, 5L, "Comment#5"));
        data.add(new Tuple3<>(12, 5L, "Comment#6"));
        data.add(new Tuple3<>(13, 5L, "Comment#7"));
        data.add(new Tuple3<>(14, 5L, "Comment#8"));
        data.add(new Tuple3<>(15, 5L, "Comment#9"));
        data.add(new Tuple3<>(16, 6L, "Comment#10"));
        data.add(new Tuple3<>(17, 6L, "Comment#11"));
        data.add(new Tuple3<>(18, 6L, "Comment#12"));
        data.add(new Tuple3<>(19, 6L, "Comment#13"));
        data.add(new Tuple3<>(20, 6L, "Comment#14"));
        data.add(new Tuple3<>(21, 6L, "Comment#15"));

        Collections.shuffle(data);

        return env.fromCollection(data);
    }

    public static DataSet<Tuple3<Integer, Long, String>> getSmall3TupleDataSet(
            ExecutionEnvironment env) {

        List<Tuple3<Integer, Long, String>> data = new ArrayList<>();
        data.add(new Tuple3<>(1, 1L, "Hi"));
        data.add(new Tuple3<>(2, 2L, "Hello"));
        data.add(new Tuple3<>(3, 2L, "Hello world"));

        Collections.shuffle(data);

        return env.fromCollection(data);
    }

    public static DataSet<Tuple5<Integer, Long, Integer, String, Long>> get5TupleDataSet(
            ExecutionEnvironment env) {

        List<Tuple5<Integer, Long, Integer, String, Long>> data = new ArrayList<>();
        data.add(new Tuple5<>(1, 1L, 0, "Hallo", 1L));
        data.add(new Tuple5<>(2, 2L, 1, "Hallo Welt", 2L));
        data.add(new Tuple5<>(2, 3L, 2, "Hallo Welt wie", 1L));
        data.add(new Tuple5<>(3, 4L, 3, "Hallo Welt wie gehts?", 2L));
        data.add(new Tuple5<>(3, 5L, 4, "ABC", 2L));
        data.add(new Tuple5<>(3, 6L, 5, "BCD", 3L));
        data.add(new Tuple5<>(4, 7L, 6, "CDE", 2L));
        data.add(new Tuple5<>(4, 8L, 7, "DEF", 1L));
        data.add(new Tuple5<>(4, 9L, 8, "EFG", 1L));
        data.add(new Tuple5<>(4, 10L, 9, "FGH", 2L));
        data.add(new Tuple5<>(5, 11L, 10, "GHI", 1L));
        data.add(new Tuple5<>(5, 12L, 11, "HIJ", 3L));
        data.add(new Tuple5<>(5, 13L, 12, "IJK", 3L));
        data.add(new Tuple5<>(5, 14L, 13, "JKL", 2L));
        data.add(new Tuple5<>(5, 15L, 14, "KLM", 2L));

        Collections.shuffle(data);

        TupleTypeInfo<Tuple5<Integer, Long, Integer, String, Long>> type =
                new TupleTypeInfo<>(
                        BasicTypeInfo.INT_TYPE_INFO,
                        BasicTypeInfo.LONG_TYPE_INFO,
                        BasicTypeInfo.INT_TYPE_INFO,
                        BasicTypeInfo.STRING_TYPE_INFO,
                        BasicTypeInfo.LONG_TYPE_INFO);

        return env.fromCollection(data, type);
    }

    public static DataSet<Tuple5<Integer, Long, Integer, String, Long>> getSmall5TupleDataSet(
            ExecutionEnvironment env) {

        List<Tuple5<Integer, Long, Integer, String, Long>> data = new ArrayList<>();
        data.add(new Tuple5<>(1, 1L, 0, "Hallo", 1L));
        data.add(new Tuple5<>(2, 2L, 1, "Hallo Welt", 2L));
        data.add(new Tuple5<>(2, 3L, 2, "Hallo Welt wie", 1L));

        Collections.shuffle(data);

        TupleTypeInfo<Tuple5<Integer, Long, Integer, String, Long>> type =
                new TupleTypeInfo<>(
                        BasicTypeInfo.INT_TYPE_INFO,
                        BasicTypeInfo.LONG_TYPE_INFO,
                        BasicTypeInfo.INT_TYPE_INFO,
                        BasicTypeInfo.STRING_TYPE_INFO,
                        BasicTypeInfo.LONG_TYPE_INFO);

        return env.fromCollection(data, type);
    }

    public static DataSet<Tuple2<Tuple2<Integer, Integer>, String>> getSmallNestedTupleDataSet(
            ExecutionEnvironment env) {

        List<Tuple2<Tuple2<Integer, Integer>, String>> data = new ArrayList<>();
        data.add(new Tuple2<>(new Tuple2<>(1, 1), "one"));
        data.add(new Tuple2<>(new Tuple2<>(2, 2), "two"));
        data.add(new Tuple2<>(new Tuple2<>(3, 3), "three"));

        TupleTypeInfo<Tuple2<Tuple2<Integer, Integer>, String>> type =
                new TupleTypeInfo<>(
                        new TupleTypeInfo<Tuple2<Integer, Integer>>(
                                BasicTypeInfo.INT_TYPE_INFO, BasicTypeInfo.INT_TYPE_INFO),
                        BasicTypeInfo.STRING_TYPE_INFO);

        return env.fromCollection(data, type);
    }

    public static DataSet<Tuple2<Tuple2<Integer, Integer>, String>>
            getGroupSortedNestedTupleDataSet(ExecutionEnvironment env) {

        List<Tuple2<Tuple2<Integer, Integer>, String>> data = new ArrayList<>();
        data.add(new Tuple2<>(new Tuple2<>(1, 3), "a"));
        data.add(new Tuple2<>(new Tuple2<>(1, 2), "a"));
        data.add(new Tuple2<>(new Tuple2<>(2, 1), "a"));
        data.add(new Tuple2<>(new Tuple2<>(2, 2), "b"));
        data.add(new Tuple2<>(new Tuple2<>(3, 3), "c"));
        data.add(new Tuple2<>(new Tuple2<>(3, 6), "c"));
        data.add(new Tuple2<>(new Tuple2<>(4, 9), "c"));

        TupleTypeInfo<Tuple2<Tuple2<Integer, Integer>, String>> type =
                new TupleTypeInfo<>(
                        new TupleTypeInfo<Tuple2<Integer, Integer>>(
                                BasicTypeInfo.INT_TYPE_INFO, BasicTypeInfo.INT_TYPE_INFO),
                        BasicTypeInfo.STRING_TYPE_INFO);

        return env.fromCollection(data, type);
    }

    public static DataSet<Tuple3<Tuple2<Integer, Integer>, String, Integer>>
            getGroupSortedNestedTupleDataSet2(ExecutionEnvironment env) {

        List<Tuple3<Tuple2<Integer, Integer>, String, Integer>> data = new ArrayList<>();
        data.add(new Tuple3<>(new Tuple2<>(1, 3), "a", 2));
        data.add(new Tuple3<>(new Tuple2<>(1, 2), "a", 1));
        data.add(new Tuple3<>(new Tuple2<>(2, 1), "a", 3));
        data.add(new Tuple3<>(new Tuple2<>(2, 2), "b", 4));
        data.add(new Tuple3<>(new Tuple2<>(3, 3), "c", 5));
        data.add(new Tuple3<>(new Tuple2<>(3, 6), "c", 6));
        data.add(new Tuple3<>(new Tuple2<>(4, 9), "c", 7));

        TupleTypeInfo<Tuple3<Tuple2<Integer, Integer>, String, Integer>> type =
                new TupleTypeInfo<>(
                        new TupleTypeInfo<Tuple2<Integer, Integer>>(
                                BasicTypeInfo.INT_TYPE_INFO, BasicTypeInfo.INT_TYPE_INFO),
                        BasicTypeInfo.STRING_TYPE_INFO,
                        BasicTypeInfo.INT_TYPE_INFO);

        return env.fromCollection(data, type);
    }

    public static DataSet<Tuple2<byte[], Integer>> getTuple2WithByteArrayDataSet(
            ExecutionEnvironment env) {
        List<Tuple2<byte[], Integer>> data = new ArrayList<>();
        data.add(new Tuple2<>(new byte[] {0, 4}, 1));
        data.add(new Tuple2<>(new byte[] {2, 0}, 1));
        data.add(new Tuple2<>(new byte[] {2, 0, 4}, 4));
        data.add(new Tuple2<>(new byte[] {2, 1}, 3));
        data.add(new Tuple2<>(new byte[] {0}, 0));
        data.add(new Tuple2<>(new byte[] {2, 0}, 1));

        TupleTypeInfo<Tuple2<byte[], Integer>> type =
                new TupleTypeInfo<>(
                        PrimitiveArrayTypeInfo.BYTE_PRIMITIVE_ARRAY_TYPE_INFO,
                        BasicTypeInfo.INT_TYPE_INFO);

        return env.fromCollection(data, type);
    }

    public static DataSet<String> getStringDataSet(ExecutionEnvironment env) {

        List<String> data = new ArrayList<>();
        data.add("Hi");
        data.add("Hello");
        data.add("Hello world");
        data.add("Hello world, how are you?");
        data.add("I am fine.");
        data.add("Luke Skywalker");
        data.add("Random comment");
        data.add("LOL");

        Collections.shuffle(data);

        return env.fromCollection(data);
    }

    public static DataSet<Integer> getIntegerDataSet(ExecutionEnvironment env) {

        List<Integer> data = new ArrayList<>();
        data.add(1);
        data.add(2);
        data.add(2);
        data.add(3);
        data.add(3);
        data.add(3);
        data.add(4);
        data.add(4);
        data.add(4);
        data.add(4);
        data.add(5);
        data.add(5);
        data.add(5);
        data.add(5);
        data.add(5);

        Collections.shuffle(data);

        return env.fromCollection(data);
    }

    public static DataSet<CustomType> getCustomTypeDataSet(ExecutionEnvironment env) {

        List<CustomType> data = new ArrayList<>();
        data.add(new CustomType(1, 0L, "Hi"));
        data.add(new CustomType(2, 1L, "Hello"));
        data.add(new CustomType(2, 2L, "Hello world"));
        data.add(new CustomType(3, 3L, "Hello world, how are you?"));
        data.add(new CustomType(3, 4L, "I am fine."));
        data.add(new CustomType(3, 5L, "Luke Skywalker"));
        data.add(new CustomType(4, 6L, "Comment#1"));
        data.add(new CustomType(4, 7L, "Comment#2"));
        data.add(new CustomType(4, 8L, "Comment#3"));
        data.add(new CustomType(4, 9L, "Comment#4"));
        data.add(new CustomType(5, 10L, "Comment#5"));
        data.add(new CustomType(5, 11L, "Comment#6"));
        data.add(new CustomType(5, 12L, "Comment#7"));
        data.add(new CustomType(5, 13L, "Comment#8"));
        data.add(new CustomType(5, 14L, "Comment#9"));
        data.add(new CustomType(6, 15L, "Comment#10"));
        data.add(new CustomType(6, 16L, "Comment#11"));
        data.add(new CustomType(6, 17L, "Comment#12"));
        data.add(new CustomType(6, 18L, "Comment#13"));
        data.add(new CustomType(6, 19L, "Comment#14"));
        data.add(new CustomType(6, 20L, "Comment#15"));

        Collections.shuffle(data);

        return env.fromCollection(data);
    }

    public static DataSet<CustomType> getSmallCustomTypeDataSet(ExecutionEnvironment env) {

        List<CustomType> data = new ArrayList<>();
        data.add(new CustomType(1, 0L, "Hi"));
        data.add(new CustomType(2, 1L, "Hello"));
        data.add(new CustomType(2, 2L, "Hello world"));

        Collections.shuffle(data);

        return env.fromCollection(data);
    }

    /** POJO. */
    public static class CustomType implements Serializable {

        private static final long serialVersionUID = 1L;

        public int myInt;
        public long myLong;
        public String myString;

        public CustomType() {}

        public CustomType(int i, long l, String s) {
            myInt = i;
            myLong = l;
            myString = s;
        }

        @Override
        public String toString() {
            return myInt + "," + myLong + "," + myString;
        }
    }

    private static class CustomTypeComparator implements Comparator<CustomType> {
        @Override
        public int compare(CustomType o1, CustomType o2) {
            int diff = o1.myInt - o2.myInt;
            if (diff != 0) {
                return diff;
            }
            diff = (int) (o1.myLong - o2.myLong);
            return diff != 0 ? diff : o1.myString.compareTo(o2.myString);
        }
    }

    public static DataSet<Tuple7<Integer, String, Integer, Integer, Long, String, Long>>
            getSmallTuplebasedDataSet(ExecutionEnvironment env) {
        List<Tuple7<Integer, String, Integer, Integer, Long, String, Long>> data =
                new ArrayList<>();
        data.add(new Tuple7<>(1, "First", 10, 100, 1000L, "One", 10000L));
        data.add(new Tuple7<>(2, "Second", 20, 200, 2000L, "Two", 20000L));
        data.add(new Tuple7<>(3, "Third", 30, 300, 3000L, "Three", 30000L));
        return env.fromCollection(data);
    }

    public static DataSet<Tuple7<Long, Integer, Integer, Long, String, Integer, String>>
            getSmallTuplebasedDataSetMatchingPojo(ExecutionEnvironment env) {
        List<Tuple7<Long, Integer, Integer, Long, String, Integer, String>> data =
                new ArrayList<>();
        data.add(new Tuple7<>(10000L, 10, 100, 1000L, "One", 1, "First"));
        data.add(new Tuple7<>(20000L, 20, 200, 2000L, "Two", 2, "Second"));
        data.add(new Tuple7<>(30000L, 30, 300, 3000L, "Three", 3, "Third"));

        return env.fromCollection(data);
    }

    public static DataSet<POJO> getSmallPojoDataSet(ExecutionEnvironment env) {
        List<POJO> data = new ArrayList<>();
        data.add(
                new POJO(
                        1 /*number*/,
                        "First" /*str*/,
                        10 /*f0*/,
                        100 /*f1.myInt*/,
                        1000L /*f1.myLong*/,
                        "One" /*f1.myString*/,
                        10000L /*nestedPojo.longNumber*/));
        data.add(new POJO(2, "Second", 20, 200, 2000L, "Two", 20000L));
        data.add(new POJO(3, "Third", 30, 300, 3000L, "Three", 30000L));
        return env.fromCollection(data);
    }

    public static DataSet<POJO> getDuplicatePojoDataSet(ExecutionEnvironment env) {
        List<POJO> data = new ArrayList<>();
        data.add(new POJO(1, "First", 10, 100, 1000L, "One", 10000L)); // 5x
        data.add(new POJO(1, "First", 10, 100, 1000L, "One", 10000L));
        data.add(new POJO(1, "First", 10, 100, 1000L, "One", 10000L));
        data.add(new POJO(1, "First", 10, 100, 1000L, "One", 10000L));
        data.add(new POJO(1, "First", 10, 100, 1000L, "One", 10000L));
        data.add(new POJO(2, "Second", 20, 200, 2000L, "Two", 20000L));
        data.add(new POJO(3, "Third", 30, 300, 3000L, "Three", 30000L)); // 2x
        data.add(new POJO(3, "Third", 30, 300, 3000L, "Three", 30000L));
        return env.fromCollection(data);
    }

    public static DataSet<POJO> getMixedPojoDataSet(ExecutionEnvironment env) {
        List<POJO> data = new ArrayList<>();
        data.add(new POJO(1, "First", 10, 100, 1000L, "One", 10100L)); // 5x
        data.add(new POJO(2, "First_", 10, 105, 1000L, "One", 10200L));
        data.add(new POJO(3, "First", 11, 102, 3000L, "One", 10200L));
        data.add(new POJO(4, "First_", 11, 106, 1000L, "One", 10300L));
        data.add(new POJO(5, "First", 11, 102, 2000L, "One", 10100L));
        data.add(new POJO(6, "Second_", 20, 200, 2000L, "Two", 10100L));
        data.add(new POJO(7, "Third", 31, 301, 2000L, "Three", 10200L)); // 2x
        data.add(new POJO(8, "Third_", 30, 300, 1000L, "Three", 10100L));
        return env.fromCollection(data);
    }

    /** POJO. */
    public static class POJO {
        public int number;
        public String str;
        public Tuple2<Integer, CustomType> nestedTupleWithCustom;
        public NestedPojo nestedPojo;
        public transient Long ignoreMe;

        public POJO(int i0, String s0, int i1, int i2, long l0, String s1, long l1) {
            this.number = i0;
            this.str = s0;
            this.nestedTupleWithCustom = new Tuple2<>(i1, new CustomType(i2, l0, s1));
            this.nestedPojo = new NestedPojo();
            this.nestedPojo.longNumber = l1;
        }

        public POJO() {}

        @Override
        public String toString() {
            return number + " " + str + " " + nestedTupleWithCustom + " " + nestedPojo.longNumber;
        }
    }

    /** Nested POJO. */
    public static class NestedPojo {
        public static Object ignoreMe;
        public long longNumber;

        public NestedPojo() {}
    }

    public static DataSet<CrazyNested> getCrazyNestedDataSet(ExecutionEnvironment env) {
        List<CrazyNested> data = new ArrayList<>();
        data.add(new CrazyNested("aa"));
        data.add(new CrazyNested("bb"));
        data.add(new CrazyNested("bb"));
        data.add(new CrazyNested("cc"));
        data.add(new CrazyNested("cc"));
        data.add(new CrazyNested("cc"));
        return env.fromCollection(data);
    }

    /** Deeply nested POJO. */
    public static class CrazyNested {
        public CrazyNestedL1 nestLvl1;
        public Long something; // test proper null-value handling

        public CrazyNested() {}

        public CrazyNested(
                String set,
                String second,
                long s) { // additional CTor to set all fields to non-null values
            this(set);
            something = s;
            nestLvl1.a = second;
        }

        public CrazyNested(String set) {
            nestLvl1 = new CrazyNestedL1();
            nestLvl1.nestLvl2 = new CrazyNestedL2();
            nestLvl1.nestLvl2.nestLvl3 = new CrazyNestedL3();
            nestLvl1.nestLvl2.nestLvl3.nestLvl4 = new CrazyNestedL4();
            nestLvl1.nestLvl2.nestLvl3.nestLvl4.f1nal = set;
        }
    }

    /** Nested POJO level 2. */
    public static class CrazyNestedL1 {
        public String a;
        public int b;
        public CrazyNestedL2 nestLvl2;
    }

    /** Nested POJO level 3. */
    public static class CrazyNestedL2 {
        public CrazyNestedL3 nestLvl3;
    }

    /** Nested POJO level 4. */
    public static class CrazyNestedL3 {
        public CrazyNestedL4 nestLvl4;
    }

    /** Nested POJO level 5. */
    public static class CrazyNestedL4 {
        public String f1nal;
    }

    // Copied from TypeExtractorTest
    private static class FromTuple extends Tuple3<String, String, Long> {
        private static final long serialVersionUID = 1L;
        public int special;
    }

    /** Pojo extending from tuple WITH custom fields. */
    public static class FromTupleWithCTor extends FromTuple {

        private static final long serialVersionUID = 1L;

        public FromTupleWithCTor() {}

        public FromTupleWithCTor(int special, long tupleField) {
            this.special = special;
            this.setField(tupleField, 2);
        }
    }

    public static DataSet<FromTupleWithCTor> getPojoExtendingFromTuple(ExecutionEnvironment env) {
        List<FromTupleWithCTor> data = new ArrayList<>();
        data.add(new FromTupleWithCTor(1, 10L)); // 3x
        data.add(new FromTupleWithCTor(1, 10L));
        data.add(new FromTupleWithCTor(1, 10L));
        data.add(new FromTupleWithCTor(2, 20L)); // 2x
        data.add(new FromTupleWithCTor(2, 20L));
        return env.fromCollection(data);
    }

    /** POJO with Tuple and Writable. */
    public static class PojoContainingTupleAndWritable {
        public int someInt;
        public String someString;
        public IntWritable hadoopFan;
        public Tuple2<Long, Long> theTuple;

        public PojoContainingTupleAndWritable() {}

        public PojoContainingTupleAndWritable(int i, long l1, long l2) {
            hadoopFan = new IntWritable(i);
            someInt = i;
            theTuple = new Tuple2<>(l1, l2);
        }
    }

    public static DataSet<PojoContainingTupleAndWritable> getPojoContainingTupleAndWritable(
            ExecutionEnvironment env) {
        List<PojoContainingTupleAndWritable> data = new ArrayList<>();
        data.add(new PojoContainingTupleAndWritable(1, 10L, 100L)); // 1x
        data.add(new PojoContainingTupleAndWritable(2, 20L, 200L)); // 5x
        data.add(new PojoContainingTupleAndWritable(2, 20L, 200L));
        data.add(new PojoContainingTupleAndWritable(2, 20L, 200L));
        data.add(new PojoContainingTupleAndWritable(2, 20L, 200L));
        data.add(new PojoContainingTupleAndWritable(2, 20L, 200L));
        return env.fromCollection(data);
    }

    public static DataSet<PojoContainingTupleAndWritable>
            getGroupSortedPojoContainingTupleAndWritable(ExecutionEnvironment env) {
        List<PojoContainingTupleAndWritable> data = new ArrayList<>();
        data.add(new PojoContainingTupleAndWritable(1, 10L, 100L)); // 1x
        data.add(new PojoContainingTupleAndWritable(2, 20L, 200L)); // 5x
        data.add(new PojoContainingTupleAndWritable(2, 20L, 201L));
        data.add(new PojoContainingTupleAndWritable(2, 30L, 200L));
        data.add(new PojoContainingTupleAndWritable(2, 30L, 600L));
        data.add(new PojoContainingTupleAndWritable(2, 30L, 400L));
        return env.fromCollection(data);
    }

    public static DataSet<Tuple3<Integer, CrazyNested, POJO>> getTupleContainingPojos(
            ExecutionEnvironment env) {
        List<Tuple3<Integer, CrazyNested, POJO>> data = new ArrayList<>();
        data.add(
                new Tuple3<>(
                        1,
                        new CrazyNested("one", "uno", 1L),
                        new POJO(1, "First", 10, 100, 1000L, "One", 10000L))); // 3x
        data.add(
                new Tuple3<>(
                        1,
                        new CrazyNested("one", "uno", 1L),
                        new POJO(1, "First", 10, 100, 1000L, "One", 10000L)));
        data.add(
                new Tuple3<>(
                        1,
                        new CrazyNested("one", "uno", 1L),
                        new POJO(1, "First", 10, 100, 1000L, "One", 10000L)));
        // POJO is not initialized according to the first two fields.
        data.add(
                new Tuple3<>(
                        2,
                        new CrazyNested("two", "duo", 2L),
                        new POJO(1, "First", 10, 100, 1000L, "One", 10000L))); // 1x
        return env.fromCollection(data);
    }

    /** POJO. */
    public static class Pojo1 {
        public String a;
        public String b;

        public Pojo1() {}

        public Pojo1(String a, String b) {
            this.a = a;
            this.b = b;
        }
    }

    /** Another POJO. */
    public static class Pojo2 {
        public String a2;
        public String b2;
    }

    /** Nested POJO. */
    public static class PojoWithMultiplePojos {
        public Pojo1 p1;
        public Pojo2 p2;
        public Integer i0;

        public PojoWithMultiplePojos() {}

        public PojoWithMultiplePojos(String a, String b, String a1, String b1, Integer i0) {
            p1 = new Pojo1();
            p1.a = a;
            p1.b = b;
            p2 = new Pojo2();
            p2.a2 = a1;
            p2.b2 = b1;
            this.i0 = i0;
        }
    }

    public static DataSet<PojoWithMultiplePojos> getPojoWithMultiplePojos(
            ExecutionEnvironment env) {
        List<PojoWithMultiplePojos> data = new ArrayList<>();
        data.add(new PojoWithMultiplePojos("a", "aa", "b", "bb", 1));
        data.add(new PojoWithMultiplePojos("b", "bb", "c", "cc", 2));
        data.add(new PojoWithMultiplePojos("b", "bb", "c", "cc", 2));
        data.add(new PojoWithMultiplePojos("b", "bb", "c", "cc", 2));
        data.add(new PojoWithMultiplePojos("d", "dd", "e", "ee", 3));
        data.add(new PojoWithMultiplePojos("d", "dd", "e", "ee", 3));
        return env.fromCollection(data);
    }

    /** Custom enum. */
    public enum Category {
        CAT_A,
        CAT_B
    }

    /** POJO with Date and enum. */
    public static class PojoWithDateAndEnum {
        public String group;
        public Date date;
        public Category cat;
    }

    public static DataSet<PojoWithDateAndEnum> getPojoWithDateAndEnum(ExecutionEnvironment env) {
        List<PojoWithDateAndEnum> data = new ArrayList<>();

        PojoWithDateAndEnum one = new PojoWithDateAndEnum();
        one.group = "a";
        one.date = new Date(666);
        one.cat = Category.CAT_A;
        data.add(one);

        PojoWithDateAndEnum two = new PojoWithDateAndEnum();
        two.group = "a";
        two.date = new Date(666);
        two.cat = Category.CAT_A;
        data.add(two);

        PojoWithDateAndEnum three = new PojoWithDateAndEnum();
        three.group = "b";
        three.date = new Date(666);
        three.cat = Category.CAT_B;
        data.add(three);

        return env.fromCollection(data);
    }

    /** POJO with collection. */
    public static class PojoWithCollection {
        public List<Pojo1> pojos;
        public int key;
        public java.sql.Date sqlDate;
        public BigInteger bigInt;
        public BigDecimal bigDecimalKeepItNull;
        public BigInt scalaBigInt;
        public List<Object> mixed;

        @Override
        public String toString() {
            return "PojoWithCollection{"
                    + "pojos.size()="
                    + pojos.size()
                    + ", key="
                    + key
                    + ", sqlDate="
                    + sqlDate
                    + ", bigInt="
                    + bigInt
                    + ", bigDecimalKeepItNull="
                    + bigDecimalKeepItNull
                    + ", scalaBigInt="
                    + scalaBigInt
                    + ", mixed="
                    + mixed
                    + '}';
        }
    }

    /** POJO with generic collection. */
    public static class PojoWithCollectionGeneric {
        public List<Pojo1> pojos;
        public int key;
        public java.sql.Date sqlDate;
        public BigInteger bigInt;
        public BigDecimal bigDecimalKeepItNull;
        public BigInt scalaBigInt;
        public List<Object> mixed;
        private PojoWithDateAndEnum makeMeGeneric;

        @Override
        public String toString() {
            return "PojoWithCollection{"
                    + "pojos.size()="
                    + pojos.size()
                    + ", key="
                    + key
                    + ", sqlDate="
                    + sqlDate
                    + ", bigInt="
                    + bigInt
                    + ", bigDecimalKeepItNull="
                    + bigDecimalKeepItNull
                    + ", scalaBigInt="
                    + scalaBigInt
                    + ", mixed="
                    + mixed
                    + '}';
        }
    }

    public static DataSet<PojoWithCollection> getPojoWithCollection(ExecutionEnvironment env) {
        List<PojoWithCollection> data = new ArrayList<>();

        List<Pojo1> pojosList1 = new ArrayList<>();
        pojosList1.add(new Pojo1("a", "aa"));
        pojosList1.add(new Pojo1("b", "bb"));

        List<Pojo1> pojosList2 = new ArrayList<>();
        pojosList2.add(new Pojo1("a2", "aa2"));
        pojosList2.add(new Pojo1("b2", "bb2"));

        PojoWithCollection pwc1 = new PojoWithCollection();
        pwc1.pojos = pojosList1;
        pwc1.key = 0;
        pwc1.bigInt = BigInteger.valueOf(Long.MAX_VALUE).multiply(BigInteger.TEN);
        pwc1.scalaBigInt = BigInt.int2bigInt(10);
        pwc1.bigDecimalKeepItNull = null;

        // use calendar to make it stable across time zones
        GregorianCalendar gcl1 = new GregorianCalendar(2033, 4, 18);
        pwc1.sqlDate = new java.sql.Date(gcl1.getTimeInMillis());
        pwc1.mixed = new ArrayList<>();
        Map<String, Integer> map = new HashMap<>();
        map.put("someKey", 1); // map.put("anotherKey", 2); map.put("third", 3);
        pwc1.mixed.add(map);
        pwc1.mixed.add(new File("/this/is/wrong"));
        pwc1.mixed.add("uhlala");

        PojoWithCollection pwc2 = new PojoWithCollection();
        pwc2.pojos = pojosList2;
        pwc2.key = 0;
        pwc2.bigInt = BigInteger.valueOf(Long.MAX_VALUE).multiply(BigInteger.TEN);
        pwc2.scalaBigInt = BigInt.int2bigInt(31104000);
        pwc2.bigDecimalKeepItNull = null;

        GregorianCalendar gcl2 = new GregorianCalendar(1976, 4, 3);
        pwc2.sqlDate = new java.sql.Date(gcl2.getTimeInMillis()); // 1976

        data.add(pwc1);
        data.add(pwc2);

        return env.fromCollection(data);
    }
}


text:
/*
 public domain BASE64 code

 modified for cocos2d-iphone: http://www.cocos2d-iphone.org
 */

#ifndef __CC_BASE64_DECODE_H
#define __CC_BASE64_DECODE_H

#ifdef __cplusplus
extern "C" {
#endif

/** @file
 base64 helper functions
 */

/**
 * Decodes a 64base encoded memory. The decoded memory is
 * expected to be freed by the caller.
 *
 * @returns the length of the out buffer
 *
 */
int base64Decode(unsigned char *in, unsigned int inLength, unsigned char **out);

#ifdef __cplusplus
}
#endif

#endif // __CC_BASE64_DECODE_H


text:
"""Tests for the experimental input pipeline ops."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import math

import numpy as np

from tensorflow.contrib.data.python.kernel_tests import dataset_serialization_test_base
from tensorflow.contrib.data.python.ops import batching
from tensorflow.python.data.ops import dataset_ops
from tensorflow.python.framework import constant_op
from tensorflow.python.framework import dtypes
from tensorflow.python.framework import errors
from tensorflow.python.framework import sparse_tensor
from tensorflow.python.ops import array_ops
from tensorflow.python.ops import math_ops
from tensorflow.python.ops import string_ops
from tensorflow.python.platform import test

class BatchDatasetTest(test.TestCase):

  def assertSparseValuesEqual(self, a, b):
    self.assertAllEqual(a.indices, b.indices)
    self.assertAllEqual(a.values, b.values)
    self.assertAllEqual(a.dense_shape, b.dense_shape)

  def testDenseToSparseBatchDataset(self):
    components = np.random.randint(12, size=(100,)).astype(np.int32)
    iterator = (
        dataset_ops.Dataset.from_tensor_slices(components)
        .map(lambda x: array_ops.fill([x], x)).apply(
            batching.dense_to_sparse_batch(4, [12]))
        .make_initializable_iterator())
    init_op = iterator.initializer
    get_next = iterator.get_next()

    with self.test_session() as sess:
      sess.run(init_op)

      for start in range(0, len(components), 4):
        results = sess.run(get_next)
        self.assertAllEqual([[i, j]
                             for i, c in enumerate(components[start:start + 4])
                             for j in range(c)], results.indices)
        self.assertAllEqual(
            [c for c in components[start:start + 4] for _ in range(c)],
            results.values)
        self.assertAllEqual([min(4,
                                 len(components) - start), 12],
                            results.dense_shape)

      with self.assertRaises(errors.OutOfRangeError):
        sess.run(get_next)

  def testDenseToSparseBatchDatasetWithUnknownShape(self):
    components = np.random.randint(5, size=(40,)).astype(np.int32)
    iterator = (
        dataset_ops.Dataset.from_tensor_slices(components)
        .map(lambda x: array_ops.fill([x, x], x)).apply(
            batching.dense_to_sparse_batch(
                4, [5, None])).make_initializable_iterator())
    init_op = iterator.initializer
    get_next = iterator.get_next()

    with self.test_session() as sess:
      sess.run(init_op)

      for start in range(0, len(components), 4):
        results = sess.run(get_next)
        self.assertAllEqual([[i, j, z]
                             for i, c in enumerate(components[start:start + 4])
                             for j in range(c)
                             for z in range(c)], results.indices)
        self.assertAllEqual([
            c
            for c in components[start:start + 4] for _ in range(c)
            for _ in range(c)
        ], results.values)
        self.assertAllEqual([
            min(4,
                len(components) - start), 5,
            np.max(components[start:start + 4])
        ], results.dense_shape)

      with self.assertRaises(errors.OutOfRangeError):
        sess.run(get_next)

  def testDenseToSparseBatchDatasetWithInvalidShape(self):
    input_tensor = array_ops.constant([[1]])
    with self.assertRaisesRegexp(ValueError, "Dimension -2 must be >= 0"):
      dataset_ops.Dataset.from_tensors(input_tensor).apply(
          batching.dense_to_sparse_batch(4, [-2])).make_initializable_iterator()

  def testDenseToSparseBatchDatasetShapeErrors(self):
    input_tensor = array_ops.placeholder(dtypes.int32)
    iterator = (
        dataset_ops.Dataset.from_tensors(input_tensor).apply(
            batching.dense_to_sparse_batch(4, [12]))
        .make_initializable_iterator())
    init_op = iterator.initializer
    get_next = iterator.get_next()

    with self.test_session() as sess:
      # Initialize with an input tensor of incompatible rank.
      sess.run(init_op, feed_dict={input_tensor: [[1]]})
      with self.assertRaisesRegexp(errors.InvalidArgumentError,
                                   "incompatible with the row shape"):
        sess.run(get_next)

      # Initialize with an input tensor that is larger than `row_shape`.
      sess.run(init_op, feed_dict={input_tensor: range(13)})
      with self.assertRaisesRegexp(errors.DataLossError,
                                   "larger than the row shape"):
        sess.run(get_next)

  def testUnbatchScalarDataset(self):
    data = tuple([math_ops.range(10) for _ in range(3)])
    data = dataset_ops.Dataset.from_tensor_slices(data)
    expected_types = (dtypes.int32,) * 3
    data = data.batch(2)
    self.assertEqual(expected_types, data.output_types)
    data = data.apply(batching.unbatch())
    self.assertEqual(expected_types, data.output_types)

    iterator = data.make_one_shot_iterator()
    op = iterator.get_next()

    with self.test_session() as sess:
      for i in range(10):
        self.assertEqual((i,) * 3, sess.run(op))

      with self.assertRaises(errors.OutOfRangeError):
        sess.run(op)

  def testUnbatchSingleElementTupleDataset(self):
    data = tuple([(math_ops.range(10),) for _ in range(3)])
    data = dataset_ops.Dataset.from_tensor_slices(data)
    expected_types = ((dtypes.int32,),) * 3
    data = data.batch(2)
    self.assertEqual(expected_types, data.output_types)
    data = data.apply(batching.unbatch())
    self.assertEqual(expected_types, data.output_types)

    iterator = data.make_one_shot_iterator()
    op = iterator.get_next()

    with self.test_session() as sess:
      for i in range(10):
        self.assertEqual(((i,),) * 3, sess.run(op))

      with self.assertRaises(errors.OutOfRangeError):
        sess.run(op)

  def testUnbatchMultiElementTupleDataset(self):
    data = tuple([(math_ops.range(10 * i, 10 * i + 10),
                   array_ops.fill([10], "hi")) for i in range(3)])
    data = dataset_ops.Dataset.from_tensor_slices(data)
    expected_types = ((dtypes.int32, dtypes.string),) * 3
    data = data.batch(2)
    self.assertAllEqual(expected_types, data.output_types)
    data = data.apply(batching.unbatch())
    self.assertAllEqual(expected_types, data.output_types)

    iterator = data.make_one_shot_iterator()
    op = iterator.get_next()

    with self.test_session() as sess:
      for i in range(10):
        self.assertEqual(((i, b"hi"), (10 + i, b"hi"), (20 + i, b"hi")),
                         sess.run(op))

      with self.assertRaises(errors.OutOfRangeError):
        sess.run(op)

  def testBatchAndDropRemainder(self):
    components = (np.arange(7),
                  np.array([[1, 2, 3]]) * np.arange(7)[:, np.newaxis],
                  np.array(37.0) * np.arange(7))

    batch_size = array_ops.placeholder(dtypes.int64, shape=[])

    iterator = (
        dataset_ops.Dataset.from_tensor_slices(components).apply(
            batching.batch_and_drop_remainder(batch_size))
        .make_initializable_iterator())

    next_element = iterator.get_next()

    with self.test_session() as sess:
      for test_batch_size in [1, 3, 7, 10]:
        sess.run(iterator.initializer, feed_dict={batch_size: test_batch_size})
        num_batches = 7 // test_batch_size
        for i in range(num_batches):
          result = sess.run(next_element)
          for component, result_component in zip(components, result):
            for j in range(test_batch_size):
              self.assertAllEqual(component[(i * test_batch_size + j)],
                                  result_component[j])
        with self.assertRaises(errors.OutOfRangeError):
          sess.run(next_element)

  def testBatchAndDropRemainderSparse(self):

    def _sparse(i):
      return sparse_tensor.SparseTensorValue(
          indices=[[0]], values=(i * [1]), dense_shape=[1])

    iterator = dataset_ops.Dataset.range(12).map(_sparse).apply(
        batching.batch_and_drop_remainder(5)).make_initializable_iterator()
    init_op = iterator.initializer
    get_next = iterator.get_next()

    with self.test_session() as sess:
      sess.run(init_op)
      for i in range(2):
        actual = sess.run(get_next)
        expected = sparse_tensor.SparseTensorValue(
            indices=[[0, 0], [1, 0], [2, 0], [3, 0], [4, 0]],
            values=[i * 5, i * 5 + 1, i * 5 + 2, i * 5 + 3, i * 5 + 4],
            dense_shape=[5, 1])
        self.assertTrue(sparse_tensor.is_sparse(actual))
        self.assertSparseValuesEqual(actual, expected)
      with self.assertRaises(errors.OutOfRangeError):
        sess.run(get_next)

  def testPaddedBatchAndDropRemainder(self):
    els = []
    for length in [3, 6, 9, 4, 12, 10, 2]:
      els.append((np.array(length), np.arange(length) + 1,
                  np.array(length * 2)))

    dataset = dataset_ops.Dataset.from_tensors(els[0])
    for el in els[1:]:
      dataset = dataset.concatenate(dataset_ops.Dataset.from_tensors(el))

    batch_size = array_ops.placeholder(dtypes.int64, shape=[])
    iterator = (
        dataset.apply(
            batching.padded_batch_and_drop_remainder(
                batch_size, ([], [None], []))).make_initializable_iterator())

    next_element = iterator.get_next()

    with self.test_session() as sess:
      for test_batch_size in [1, 3, 7, 10]:
        sess.run(iterator.initializer, feed_dict={batch_size: test_batch_size})
        num_batches = 7 // test_batch_size
        for i in range(num_batches):
          result = sess.run(next_element)
          for component_idx, result_component in enumerate(result):
            for j in range(test_batch_size):
              data_idx = i * test_batch_size + j
              comp = result_component[j]
              unpadded = comp[comp > 0]
              if np.isscalar(comp):
                # The boolean mask indexing above adds a dim back. Rm it.
                unpadded = unpadded[0]
              self.assertAllEqual(els[data_idx][component_idx], unpadded)
        with self.assertRaises(errors.OutOfRangeError):
          sess.run(next_element)

  def testPaddedBatchAndDropRemainderSparseError(self):

    def _map_fn(i):
      return sparse_tensor.SparseTensorValue(
          indices=[[0, 0]], values=(i * [1]), dense_shape=[1, 1]), i

    with self.assertRaises(TypeError):
      _ = dataset_ops.Dataset.range(10).map(_map_fn).apply(
          batching.padded_batch_and_drop_remainder(5))

  def testBatchAndDropRemainderShapeInference(self):
    components = (array_ops.placeholder(dtypes.int32),
                  (array_ops.placeholder(dtypes.int32, shape=[None]),
                   array_ops.placeholder(dtypes.int32, shape=[20, 30])))

    # Test with a statically known batch size.
    dataset = (
        dataset_ops.Dataset.from_tensor_slices(components).apply(
            batching.batch_and_drop_remainder(128)))

    self.assertIs(None, dataset.output_shapes[0].ndims)
    self.assertEqual([128], dataset.output_shapes[1][0].as_list())
    self.assertEqual([128, 30], dataset.output_shapes[1][1].as_list())

    # Test with a dynamic batch size: the static shape will be unknown, because
    # `batch_size` is a placeholder.
    batch_size = array_ops.placeholder(dtypes.int64)
    dataset = (
        dataset_ops.Dataset.from_tensor_slices(components).apply(
            batching.batch_and_drop_remainder(batch_size)))

    self.assertIs(None, dataset.output_shapes[0].ndims)
    self.assertEqual([None], dataset.output_shapes[1][0].as_list())
    self.assertEqual([None, 30], dataset.output_shapes[1][1].as_list())

  def _testBatchAndMapDatasetHelper(self, num_parallel_batches=1):
    """Test a dataset that maps a TF function across its input elements."""
    # The pipeline is TensorSliceDataset ->
    # RepeatDataset(count) -> BatchAndMapDataset(square_3, batch_size).
    components = (np.arange(7),
                  np.array([[1, 2, 3]]) * np.arange(7)[:, np.newaxis],
                  np.array(37.0) * np.arange(7))

    count = array_ops.placeholder(dtypes.int64, shape=[])
    batch_size = array_ops.placeholder(dtypes.int64, shape=[])

    def _map_fn(x, y, z):
      return math_ops.square(x), math_ops.square(y), math_ops.square(z)

    iterator = (
        dataset_ops.Dataset.from_tensor_slices(components).repeat(count).apply(
            batching.map_and_batch(
                map_func=_map_fn,
                batch_size=batch_size,
                num_parallel_batches=num_parallel_batches))
        .make_initializable_iterator())
    init_op = iterator.initializer
    get_next = iterator.get_next()

    self.assertEqual([[None] + list(c.shape[1:]) for c in components],
                     [t.shape.as_list() for t in get_next])

    with self.test_session() as sess:
      # Batch of a finite input, where the batch_size divides the
      # total number of elements.
      sess.run(init_op, feed_dict={count: 28, batch_size: 14})
      num_batches = (28 * 7) // 14
      for i in range(num_batches):
        result = sess.run(get_next)
        for component, result_component in zip(components, result):
          for j in range(14):
            self.assertAllEqual(component[(i * 14 + j) % 7]**2,
                                result_component[j])
      with self.assertRaises(errors.OutOfRangeError):
        sess.run(get_next)

      # Batch of a finite input, where the batch_size does not
      # divide the total number of elements.
      sess.run(init_op, feed_dict={count: 14, batch_size: 8})

      # We expect (num_batches - 1) full-sized batches.
      num_batches = int(math.ceil((14 * 7) / 8))
      for i in range(num_batches - 1):
        result = sess.run(get_next)
        for component, result_component in zip(components, result):
          for j in range(8):
            self.assertAllEqual(component[(i * 8 + j) % 7]**2,
                                result_component[j])
      result = sess.run(get_next)
      for component, result_component in zip(components, result):
        for j in range((14 * 7) % 8):
          self.assertAllEqual(component[((num_batches - 1) * 8 + j) % 7]**2,
                              result_component[j])
      with self.assertRaises(errors.OutOfRangeError):
        sess.run(get_next)

      # Batch of an empty input should fail straight away.
      sess.run(init_op, feed_dict={count: 0, batch_size: 8})
      with self.assertRaises(errors.OutOfRangeError):
        sess.run(get_next)

      # Empty batch should be an initialization time error.
      with self.assertRaises(errors.InvalidArgumentError):
        sess.run(init_op, feed_dict={count: 14, batch_size: 0})

  def testBatchAndMapDataset(self):
    return self._testBatchAndMapDatasetHelper()

  def testBatchAndMapDatasetWithParallelBatching(self):
    return self._testBatchAndMapDatasetHelper(num_parallel_batches=10)

  def testMapAndBatchSparse(self):

    def _sparse(i):
      return sparse_tensor.SparseTensorValue(
          indices=[[0]], values=(i * [1]), dense_shape=[1])

    iterator = dataset_ops.Dataset.range(10).apply(
        batching.map_and_batch(_sparse, 5)).make_initializable_iterator()
    init_op = iterator.initializer
    get_next = iterator.get_next()

    with self.test_session() as sess:
      sess.run(init_op)
      for i in range(2):
        actual = sess.run(get_next)
        expected = sparse_tensor.SparseTensorValue(
            indices=[[0, 0], [1, 0], [2, 0], [3, 0], [4, 0]],
            values=[i * 5, i * 5 + 1, i * 5 + 2, i * 5 + 3, i * 5 + 4],
            dense_shape=[5, 1])
        self.assertTrue(sparse_tensor.is_sparse(actual))
        self.assertSparseValuesEqual(actual, expected)
      with self.assertRaises(errors.OutOfRangeError):
        sess.run(get_next)

  def testBatchAndMapDatasetFails(self):
    """Test a dataset that maps a TF function across its input elements."""
    dataset = dataset_ops.Dataset.from_tensors(
        array_ops.check_numerics(
            constant_op.constant(1.0) / constant_op.constant(0.0), "oops"))
    batch_size = array_ops.placeholder(dtypes.int64, shape=[])
    iterator = (
        dataset.apply(batching.map_and_batch(lambda x: x, batch_size))
        .make_initializable_iterator())
    init_op = iterator.initializer
    with self.test_session() as sess:
      with self.assertRaisesRegexp(errors.InvalidArgumentError, "oops"):
        sess.run(init_op, feed_dict={batch_size: 14})

  def testBatchAndMapDatasetShapeMismatch(self):
    """Test a dataset that maps a TF function across its input elements."""

    def generator():
      yield [1]
      yield [2]
      yield [3]
      yield [[4, 5, 6]]

    dataset = dataset_ops.Dataset.from_generator(
        generator, output_types=dtypes.int32)
    batch_size = 4
    iterator = (
        dataset.apply(batching.map_and_batch(lambda x: x, batch_size))
        .make_initializable_iterator())
    init_op = iterator.initializer
    get_next = iterator.get_next()
    with self.test_session() as sess:
      sess.run(init_op)
      with self.assertRaisesRegexp(errors.InvalidArgumentError,
                                   "number of elements does not match"):
        sess.run(get_next)

class BatchDatasetSerializationTest(
    dataset_serialization_test_base.DatasetSerializationTestBase):

  def build_dataset(self, multiplier=15.0, tensor_slice_len=2, batch_size=2):
    components = (
        np.arange(tensor_slice_len),
        np.array([[1, 2, 3]]) * np.arange(tensor_slice_len)[:, np.newaxis],
        np.array(multiplier) * np.arange(tensor_slice_len))

    return dataset_ops.Dataset.from_tensor_slices(components).batch(batch_size)

  def testCore(self):
    tensor_slice_len = 8
    batch_size = 2
    num_outputs = tensor_slice_len // batch_size
    self.run_core_tests(
        lambda: self.build_dataset(15.0, tensor_slice_len, batch_size),
        lambda: self.build_dataset(20.0, tensor_slice_len, batch_size),
        num_outputs)

  def _build_dataset_dense_to_sparse(self, components):
    return dataset_ops.Dataset.from_tensor_slices(components).map(
        lambda x: array_ops.fill([x], x)).apply(
            batching.dense_to_sparse_batch(4, [12]))

  # TODO(b/70988345): Re-enable when sparse tensors are properly supported by
  # the DatasetSerializationTestBase.
  def _testDenseToSparseBatchDatasetCore(self):
    components = np.random.randint(5, size=(40,)).astype(np.int32)
    diff_comp = np.random.randint(2, size=(100,)).astype(np.int32)

    num_outputs = len(components) // 4
    self.run_core_tests(lambda: self._build_dataset_dense_to_sparse(components),
                        lambda: self._build_dataset_dense_to_sparse(diff_comp),
                        num_outputs)

  def _sparse(self, i):
    return sparse_tensor.SparseTensorValue(
        indices=[[0]], values=(i * [1]), dense_shape=[1])

  def _build_dataset_sparse(self, batch_size=5):
    return dataset_ops.Dataset.range(10).map(self._sparse).batch(batch_size)

  def testSparseCore(self):
    self.run_core_tests(self._build_dataset_sparse,
                        lambda: self._build_dataset_sparse(2), 2)

  def _build_dataset_nested_sparse(self):
    return dataset_ops.Dataset.range(10).map(self._sparse).batch(5).batch(2)

  def testNestedSparseCore(self):
    self.run_core_tests(self._build_dataset_nested_sparse, None, 1)

class PaddedBatchDatasetSerializationTest(
    dataset_serialization_test_base.DatasetSerializationTestBase):

  def testPaddedBatch(self):

    def build_dataset(seq_lens):
      return dataset_ops.Dataset.from_tensor_slices(seq_lens).map(
          lambda x: array_ops.fill([x], x)).padded_batch(
              4, padded_shapes=[-1])

    seq_lens1 = np.random.randint(1, 20, size=(32,)).astype(np.int32)
    seq_lens2 = np.random.randint(21, 40, size=(32,)).astype(np.int32)
    self.run_core_tests(lambda: build_dataset(seq_lens1),
                        lambda: build_dataset(seq_lens2), 8)

  def testPaddedBatchNonDefaultPadding(self):

    def build_dataset(seq_lens):

      def fill_tuple(x):
        filled = array_ops.fill([x], x)
        return (filled, string_ops.as_string(filled))

      padded_shape = [-1]
      return dataset_ops.Dataset.from_tensor_slices(seq_lens).map(
          fill_tuple).padded_batch(
              4,
              padded_shapes=(padded_shape, padded_shape),
              padding_values=(-1, "<end>"))

    seq_lens1 = np.random.randint(1, 20, size=(32,)).astype(np.int32)
    seq_lens2 = np.random.randint(21, 40, size=(32,)).astype(np.int32)
    self.run_core_tests(lambda: build_dataset(seq_lens1),
                        lambda: build_dataset(seq_lens2), 8)

if __name__ == "__main__":
  test.main()


text:
<?xml version = '1.0' encoding = 'UTF-8'?>
<Table class="oracle.dbtools.crest.model.design.relational.Table" directorySegmentName="seg_0" id="B8180E20-F79B-4277-19D7-2FBDBB8F7DAF" name="commitee_dim">
<sourceDDLFile>ddl.sql</sourceDDLFile>
<createdBy>beng</createdBy>
<createdTime>2016-07-18 13:11:10 UTC</createdTime>
<ownerDesignName>VoterModel</ownerDesignName>
<ordsAlias>ifv14</ordsAlias>
<typeID>2</typeID>
<allowColumnReorder>false</allowColumnReorder>
<existDependencyGenerateInDDl>true</existDependencyGenerateInDDl>
<parsed>true</parsed>
<columns itemClass="oracle.dbtools.crest.model.design.relational.Column">
<Column name="`commitee_id`" id="0777C8EA-6E11-34B4-8C2D-9E3B4CB8528F">
<sourceDDLFile>ddl.sql</sourceDDLFile>
<createdBy>beng</createdBy>
<createdTime>2016-07-18 13:11:10 UTC</createdTime>
<ownerDesignName>VoterModel</ownerDesignName>
<use>0</use>
<logicalDatatype>LOGDT011</logicalDatatype>
<domain>7CF6FDF1-812F-56E4-E492-4B0E518E67D7</domain>
<ownDataTypeParameters>,,</ownDataTypeParameters>
<autoIncrementCycle>false</autoIncrementCycle>
</Column>
<Column name="`name`" id="2C3EDA05-F23F-F464-E500-10ABD48EDAE9">
<sourceDDLFile>ddl.sql</sourceDDLFile>
<createdBy>beng</createdBy>
<createdTime>2016-07-18 13:11:10 UTC</createdTime>
<ownerDesignName>VoterModel</ownerDesignName>
<nullsAllowed>true</nullsAllowed>
<use>0</use>
<logicalDatatype>LOGDT024</logicalDatatype>
<domain>5405A833-1D5D-6035-B6A5-1160C370591E</domain>
<ownDataTypeParameters>,,</ownDataTypeParameters>
<autoIncrementCycle>false</autoIncrementCycle>
</Column>
<Column name="`committee_type`" id="48657111-558A-B109-466E-BE77B958B02F">
<sourceDDLFile>ddl.sql</sourceDDLFile>
<createdBy>beng</createdBy>
<createdTime>2016-07-18 13:11:10 UTC</createdTime>
<ownerDesignName>VoterModel</ownerDesignName>
<nullsAllowed>true</nullsAllowed>
<use>0</use>
<logicalDatatype>LOGDT024</logicalDatatype>
<domain>5405A833-1D5D-6035-B6A5-1160C370591E</domain>
<ownDataTypeParameters>,,</ownDataTypeParameters>
<autoIncrementCycle>false</autoIncrementCycle>
</Column>
<Column name="`party`" id="7C1D8311-6531-9F93-FE64-F9C511C2F0B3">
<sourceDDLFile>ddl.sql</sourceDDLFile>
<createdBy>beng</createdBy>
<createdTime>2016-07-18 13:11:10 UTC</createdTime>
<ownerDesignName>VoterModel</ownerDesignName>
<nullsAllowed>true</nullsAllowed>
<use>0</use>
<logicalDatatype>LOGDT024</logicalDatatype>
<domain>5405A833-1D5D-6035-B6A5-1160C370591E</domain>
<ownDataTypeParameters>,,</ownDataTypeParameters>
<autoIncrementCycle>false</autoIncrementCycle>
</Column>
<Column name="`fec_reference_id`" id="01D871CB-CE1C-B43A-341E-09A962E4974D">
<sourceDDLFile>ddl.sql</sourceDDLFile>
<createdBy>beng</createdBy>
<createdTime>2016-07-18 13:11:10 UTC</createdTime>
<ownerDesignName>VoterModel</ownerDesignName>
<nullsAllowed>true</nullsAllowed>
<use>0</use>
<logicalDatatype>LOGDT024</logicalDatatype>
<domain>5405A833-1D5D-6035-B6A5-1160C370591E</domain>
<ownDataTypeParameters>,,</ownDataTypeParameters>
<autoIncrementCycle>false</autoIncrementCycle>
</Column>
</columns>
<indexes itemClass="oracle.dbtools.crest.model.design.relational.Index">
<ind_PK_UK id="39FC99FE-EA13-2CBE-0D44-CFE306CFD7A6" name="commitee_dim_PK">
<sourceDDLFile>ddl.sql</sourceDDLFile>
<createdBy>beng</createdBy>
<createdTime>2016-07-18 13:11:10 UTC</createdTime>
<ownerDesignName>VoterModel</ownerDesignName>
<pk>true</pk>
<indexState>Primary Constraint</indexState>
<isSurrogateKey>false</isSurrogateKey>
<indexColumnUsage>
<colUsage columnID="0777C8EA-6E11-34B4-8C2D-9E3B4CB8528F"/>
</indexColumnUsage>
</ind_PK_UK>
</indexes>
</Table>


text:
from buffpy.api import API

# check http://bufferapp.com/developers/apps to retrieve a token
# or generate one with the example
token = 'awesome_token'

# instantiate the api object
api = API(client_id='client_id',
          client_secret='client_secret',
          access_token=token)

# get api's info
print api.info