text:
Strukt Framework
================

[![Build Status](https://travis-ci.org/pitsolu/strukt-framework.svg?branch=master)](https://packagist.org/packages/strukt/framework)
[![Latest Stable Version](https://poser.pugx.org/strukt/framework/v/stable)](https://packagist.org/packages/strukt/framework)
[![Total Downloads](https://poser.pugx.org/strukt/framework/downloads)](https://packagist.org/packages/strukt/framework)
[![Latest Unstable Version](https://poser.pugx.org/strukt/framework/v/unstable)](https://packagist.org/packages/strukt/framework)
[![License](https://poser.pugx.org/strukt/framework/license)](https://packagist.org/packages/strukt/framework)

The is the package that unifies all [strukt-strukt](https://github.com/pitsolu/strukt-strukt)
components under the framework.

Rarely should anyone use this on its own.

# Setup, Configuration & Environment

## Configuration

```php
$cfg = new Strukt\Framework\Configuration();
$cfg->getSetup();//Already called in instance above
$cfg->get($type);//Configuration type "providers", "middlewares" or "commands"
```

## Environment Setup

This class is defaultly found in [strukt-commons](github.com/pitsolu/strukt-commons)

```php
Strukt\Env::withFile();//default .env file in your root folder
Strukt\Env::withFile(".env-dev");
Strukt\Env::set("root_dir", getcwd());//custom environment variable
Strukt\Env::get("root_dir");
```

## Setting Application Type

```php
use Strukt\Framework\App as FrameworkApp;

FrameworkApp::create($type); //Can only be "App:Idx" for web or "App:Cli" for command line
FrameworkApp::getType(); //get application type
```

## Setup Registry Packages

```php
$repo = array(

	"core"=>Strukt\Package\Core::class, //Default in built package for default middlewares and providers
	"pkg-do"=>Strukt\Package\PkgDo::class,//https://github.com/pitsolu/pkg-do
	"pkg-roles"=>Strukt\Package\PkgRoles::class,//https://github.com/pitsolu/pkg-roles
	"pkg-audit"=>Strukt\Package\PkgAudit::class,//https://github.com/pitsolu/pkg-audit
	"pkg-books"=>Strukt\Package\PkgBooks::class,//https://github.com/pitsolu/pkg-books
	"pkg-tests"=>Strukt\Package\PkgTests::class,//https://github.com/pitsolu/pkg-tests
	"pkg-asset"=>Strukt\Package\PkgAsset::class//https://github.com/pitsolu/pkg-asset
);

FrameworkApp::mayBeRepo($repo);
FrameworkApp::getRepo();
```

## Some Application Methods

```php
//The line below sets up namespace for with application name
//	the ns will translate into Payroll\AuthModule\Command\PermissionAdd
//	if your app's name is payroll
$cls = FrameworkApp::newCls("{{app}}\AuthModule\Command\PermissionAdd");

//Get app_name from cfg/app.ini file
$app_name = FrameworkApp::getName(); //payroll
```

# Packages

```php
//Get installed and published packages
FrameworkApp::packages("installed");
FrameworkApp::packages("published");
```

## Default Package

```php
$core = new Strukt\Package\Core();//implements Strukt\Package\Pkg

//returns array of middlewares, commands and providers
$core->getSettings($type);//type is "App:Idx" or "App:Cli"

$core->getName();//core
$core->getCmdName();//null
$core->getFiles();//null
$core->getModules();//null
$core->isPublished();//true by default
$core->getRequirements();//null or array
```

The above methods are in abstract class `Strukt\Package\Pkg` you can use them to create your package.

## Building Packages

Your first step in developing your package will require you to install `strukt-framework`
and execute `composer exec strukt-cfg` command that will create your folder structure. You'll need to create `src` and `package` folders.

See structure of package below.

```sh
├── bootstrap.php
├── cfg
├── console
├── index.php
├── lib
├── tpl
├── vendor
├── composer.json
├── LICENSE
├── package #Place all your packages files here
├── README.md
└── src
    └── Strukt
        └── Package
            └── Pkg{{Package Name}}.php #Identify your package resources here

```

Your package class in `src/Strukt/Package/Pkg<Package Name>.php` will have methods
listed in the [Default Package](#default-package) section that is it should implement the
interface `Strukt\Contract\Package`

### Package Autoloading

You may require to autoload libraries both from your root directory and package resources.

```php
$loader = require "vendor/autoload.php";
$loader->addPsr4("App\\", [

	__DIR__."/lib/App",
	__DIR__."/packages/lib/App"
]);
```

### Note

For packages that require installation into your `app/src/{{AppName}}` folder, there
are a few tricks you could use while building your package. The `publish:package` command
takes argument `package` for publishing packages that are currently in development,
since your source will be in the root folder in a subfolder called `package`.

This will require you to enter into your `cfg/repo.php` (See [Setup Registry Packages](#setup-registry-packages)) and indicate your currently in-development package with the key/keyword `package` which will allow the publisher to install files in the your app source folder `app/src`.

# Validator

## Example

```php
$loginFrm = new class($request) extends \Strukt\Contract\Form{

	protected function validation(){

		$service = $this->getValidatorService();

		$this->setMessage("email", $service->getNew($this->get("email"))
										->isNotEmpty()
										->isEmail());

		$this->setMessage("password", $service->getNew($this->get("password"))
										->isNotEmpty()
										->isLen(8));

		$this->setMessage("confirm_password", $service->getNew($this->get("confirm_password"))
												->isNotEmpty()
												->equalTo($this->get("password")));
	}
};

$messages = $loginFrm->validate();
```

The `$request` above is `Strukt\Http\Request`

## Validator Methods

```php
Strukt\Validator::isAlpha()
Strukt\Validator::isAlphaNum()
Strukt\Validator::isNumeric()
Strukt\Validator::isEmail()
Strukt\Validator::isDate(string $format="Y-m-d")
Strukt\Validator::isNotEmpty()
Strukt\Validator::isIn(array $enum)
Strukt\Validator::equalsTo($val)
Strukt\Validator::isLen($len)
```

## Adding Validators

New validators can be added is in your `lib/App/Validator/Extra.php`
There you can find an example `App\Validator\Extra::isLenGt`


text:
#import <SAObjects/SASettingResponse.h>

#import "SASettingSupplier-Protocol.h"

@class SASettingEntity;

@interface SASettingSetValueResponse : SASettingResponse <SASettingSupplier>
{
}

+ (id)setValueResponseWithDictionary:(id)arg1 context:(id)arg2;
+ (id)setValueResponse;
- (id)encodedClassName;
- (id)groupIdentifier;

// Remaining properties
@property(retain, nonatomic) SASettingEntity *setting; // @dynamic setting;

@end

redpajama_set_name: RedPajamaGithub


text:
{% extends 'mainapp/base.html' %}

{% load staticfiles %}

{% block main_block %}

    {% if user.is_authenticated %}

Welcome {{user}}

        {% if current_review %}

You are currently working on the

{{ current_review }} review.

            Change Review
            Create Review
        {% else %}

You aren't working on any reviews

            Select Review
            Create Review
        {% endif %}

    {% else %}

Welcome to SysReviewer - The Systematic Review App

Powered by PubMed, SysReview gives you the tools to create, organise and track your systematic review.

Create and edit your search queries making sure you only have the results that are important to you.

Review and approve papers at each stage of your systematic review.

SysReviewer gives you quick access to each stage of your systematic review.

The SysReviewer Systematic Review Building Blocks

Create/edit Multiple Reviewd

Create multiple queries, test your results, select the best query for your review

Abstract Pool - Review,approve or reject your reviews papers based on their abstracts

Document Pool - Visit the full paper, select the papers that are relevant to your review

Final Pool - View the final papers that are necessary for your review

            Login
            Register

    {% endif %}
{% endblock %}


text:
<!DOCTYPE html>
<html>
<head>
<script>

var value = 0;
var strings = new Array();

function setValue(newValue)
{
    var oldValue = value;
    UndoManager_addAction(setValue,oldValue);

    strings.push("value = "+newValue);
    value = newValue;
}

function performTest()
{
    setValue(1);
    setValue(2);
    UndoManager_newGroup();

    setValue(3);
    setValue(4);
    UndoManager_newGroup();

    setValue(5);
    setValue(6);
    UndoManager_newGroup();

    setValue(7);
    setValue(8);
    UndoManager_newGroup();

    strings.push("\nTwo steps backwards");
    UndoManager_undo();
    UndoManager_undo();

    strings.push("\nMore changes");
    UndoManager_newGroup();
    setValue(20);
    setValue(21);

    strings.push("\nOne step forwards (should do nothing)");
    UndoManager_redo();

    strings.push("\nOne step backwards, one step forwards");
    UndoManager_undo();
    UndoManager_redo();

    strings.push("\nTwo steps backwards, two steps forwards");
    UndoManager_undo();
    UndoManager_undo();
    UndoManager_redo();
    UndoManager_redo();

    strings.push("\nOne step forwards (should do nothing)");
    UndoManager_redo();

    return strings.join("\n");
}
</script>
</head>
<body>
</body>
</html>


text:
namespace testing {
namespace internal {

// Joins a vector of strings as if they are fields of a tuple; returns
// the joined string.
GTEST_API_ std::string JoinAsKeyValueTuple(
    const std::vector<const char*>& names, const Strings& values) {
  GTEST_CHECK_(names.size() == values.size());
  if (values.empty()) {
    return "";
  }
  const auto build_one = [&](const size_t i) {
    return std::string(names[i]) + ": " + values[i];
  };
  std::string result = "(" + build_one(0);
  for (size_t i = 1; i < values.size(); i++) {
    result += ", ";
    result += build_one(i);
  }
  result += ")";
  return result;
}

// Converts an identifier name to a space-separated list of lower-case
// words.  Each maximum substring of the form [A-Za-z][a-z]*|\d+ is
// treated as one word.  For example, both "FooBar123" and
// "foo_bar_123" are converted to "foo bar 123".
GTEST_API_ std::string ConvertIdentifierNameToWords(const char* id_name) {
  std::string result;
  char prev_char = '\0';
  for (const char* p = id_name; *p != '\0'; prev_char = *(p++)) {
    // We don't care about the current locale as the input is
    // guaranteed to be a valid C++ identifier name.
    const bool starts_new_word = IsUpper(*p) ||
                                 (!IsAlpha(prev_char) && IsLower(*p)) ||
                                 (!IsDigit(prev_char) && IsDigit(*p));

    if (IsAlNum(*p)) {
      if (starts_new_word && result != "") result += ' ';
      result += ToLower(*p);
    }
  }
  return result;
}

// This class reports Google Mock failures as Google Test failures.  A
// user can define another class in a similar fashion if they intend to
// use Google Mock with a testing framework other than Google Test.
class GoogleTestFailureReporter : public FailureReporterInterface {
 public:
  void ReportFailure(FailureType type, const char* file, int line,
                     const std::string& message) override {
    AssertHelper(type == kFatal ? TestPartResult::kFatalFailure
                                : TestPartResult::kNonFatalFailure,
                 file, line, message.c_str()) = Message();
    if (type == kFatal) {
      posix::Abort();
    }
  }
};

// Returns the global failure reporter.  Will create a
// GoogleTestFailureReporter and return it the first time called.
GTEST_API_ FailureReporterInterface* GetFailureReporter() {
  // Points to the global failure reporter used by Google Mock.  gcc
  // guarantees that the following use of failure_reporter is
  // thread-safe.  We may need to add additional synchronization to
  // protect failure_reporter if we port Google Mock to other
  // compilers.
  static FailureReporterInterface* const failure_reporter =
      new GoogleTestFailureReporter();
  return failure_reporter;
}

// Protects global resources (stdout in particular) used by Log().
static GTEST_DEFINE_STATIC_MUTEX_(g_log_mutex);

// Returns true if and only if a log with the given severity is visible
// according to the --gmock_verbose flag.
GTEST_API_ bool LogIsVisible(LogSeverity severity) {
  if (GMOCK_FLAG_GET(verbose) == kInfoVerbosity) {
    // Always show the log if --gmock_verbose=info.
    return true;
  } else if (GMOCK_FLAG_GET(verbose) == kErrorVerbosity) {
    // Always hide it if --gmock_verbose=error.
    return false;
  } else {
    // If --gmock_verbose is neither "info" nor "error", we treat it
    // as "warning" (its default value).
    return severity == kWarning;
  }
}

// Prints the given message to stdout if and only if 'severity' >= the level
// specified by the --gmock_verbose flag.  If stack_frames_to_skip >=
// 0, also prints the stack trace excluding the top
// stack_frames_to_skip frames.  In opt mode, any positive
// stack_frames_to_skip is treated as 0, since we don't know which
// function calls will be inlined by the compiler and need to be
// conservative.
GTEST_API_ void Log(LogSeverity severity, const std::string& message,
                    int stack_frames_to_skip) {
  if (!LogIsVisible(severity)) return;

  // Ensures that logs from different threads don't interleave.
  MutexLock l(&g_log_mutex);

  if (severity == kWarning) {
    // Prints a GMOCK WARNING marker to make the warnings easily searchable.
    std::cout << "\nGMOCK WARNING:";
  }
  // Pre-pends a new-line to message if it doesn't start with one.
  if (message.empty() || message[0] != '\n') {
    std::cout << "\n";
  }
  std::cout << message;
  if (stack_frames_to_skip >= 0) {
#ifdef NDEBUG
    // In opt mode, we have to be conservative and skip no stack frame.
    const int actual_to_skip = 0;
#else
    // In dbg mode, we can do what the caller tell us to do (plus one
    // for skipping this function's stack frame).
    const int actual_to_skip = stack_frames_to_skip + 1;
#endif  // NDEBUG

    // Appends a new-line to message if it doesn't end with one.
    if (!message.empty() && *message.rbegin() != '\n') {
      std::cout << "\n";
    }
    std::cout << "Stack trace:\n"
              << ::testing::internal::GetCurrentOsStackTraceExceptTop(
                     ::testing::UnitTest::GetInstance(), actual_to_skip);
  }
  std::cout << ::std::flush;
}

GTEST_API_ WithoutMatchers GetWithoutMatchers() { return WithoutMatchers(); }

GTEST_API_ void IllegalDoDefault(const char* file, int line) {
  internal::Assert(
      false, file, line,
      "You are using DoDefault() inside a composite action like "
      "DoAll() or WithArgs().  This is not supported for technical "
      "reasons.  Please instead spell out the default action, or "
      "assign the default action to an Action variable and use "
      "the variable in various places.");
}

constexpr char UnBase64Impl(char c, const char* const base64, char carry) {
  return *base64 == 0   ? static_cast<char>(65)
         : *base64 == c ? carry
                        : UnBase64Impl(c, base64 + 1, carry + 1);
}

template <size_t... I>
constexpr std::array<char, 256> UnBase64Impl(IndexSequence<I...>,
                                             const char* const base64) {
  return {{UnBase64Impl(static_cast<char>(I), base64, 0)...}};
}

constexpr std::array<char, 256> UnBase64(const char* const base64) {
  return UnBase64Impl(MakeIndexSequence<256>{}, base64);
}

static constexpr char kBase64[] =
    "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
static constexpr std::array<char, 256> kUnBase64 = UnBase64(kBase64);

bool Base64Unescape(const std::string& encoded, std::string* decoded) {
  decoded->clear();
  size_t encoded_len = encoded.size();
  decoded->reserve(3 * (encoded_len / 4) + (encoded_len % 4));
  int bit_pos = 0;
  char dst = 0;
  for (int src : encoded) {
    if (std::isspace(src) || src == '=') {
      continue;
    }
    char src_bin = kUnBase64[static_cast<size_t>(src)];
    if (src_bin >= 64) {
      decoded->clear();
      return false;
    }
    if (bit_pos == 0) {
      dst |= src_bin << 2;
      bit_pos = 6;
    } else {
      dst |= static_cast<char>(src_bin >> (bit_pos - 2));
      decoded->push_back(dst);
      dst = static_cast<char>(src_bin << (10 - bit_pos));
      bit_pos = (bit_pos + 6) % 8;
    }
  }
  return true;
}

}  // namespace internal
}  // namespace testing


text:
import { configure } from 'enzyme';
import Adapter from 'enzyme-adapter-react-16';

configure({ adapter: new Adapter() });

var scopedTests = require.context('../src', true, /\/__tests__\/[a-zA-Z]+Spec\.js$/);

scopedTests.keys().forEach(scopedTests);

var allTests = require.context('.', true, /(spec|integration)[a-zA-Z]+Spec\.js$/);

allTests.keys().forEach(allTests);


text:
```c
#include <gate.h>
```
> The header can be found in the include directory (Gate repository top level).

### Primitives

#### Compile-time definitions

```c
GATE_API_VERSION
```
> The C API version.  The current version is 0.

```c
GATE_PACKET_ALIGNMENT
```
> Granularity of I/O packet buffers.

```c
GATE_MAX_RECV_SIZE
```
> Size limit for received packets.

```c
GATE_PACKET_CODE_SERVICES
```
> Packet code for service discovery.

```c
GATE_PACKET_DOMAIN_CALL
GATE_PACKET_DOMAIN_INFO
GATE_PACKET_DOMAIN_FLOW
GATE_PACKET_DOMAIN_DATA
```
> Packet domains.

```c
GATE_SERVICE_STATE_AVAIL
```
> Service state flag.

#### Macros

```c
GATE_ALIGN_PACKET(size)
```
> Rounds packet size up to a multiple of packet alignment.

#### Types

```c
struct gate_iovec {
	void *iov_base;
	size_t iov_len;
};
```
> Specifies buffers for scatter-gather I/O.

#### Functions

```c
uint64_t gate_clock_realtime(void);
uint64_t gate_clock_monotonic(void);
```
> Get current wall-clock or monotonic time in nanoseconds.  Actual resolution
> is unspecified.

```
gate_debug(arg)     // C macro
gate_debug(arg...)  // C++ function template
```
> Write to the debug log if enabled by runtime and `NDEBUG` wasn't defined
> during compilation.  The arguments can be integers, strings or void-pointers.
> No implicit delimiters (spaces or newlines) are written.
>
> C supports only one argument; C++ supports arbitrary number of arguments.

```c
void gate_debug_int(int64_t n);
void gate_debug_uint(uint64_t n);
```
> Write a decimal number to the debug log if enabled by runtime and `NDEBUG`
> wasn't defined during compilation.

```c
void gate_debug_hex(uint64_t n);
void gate_debug_ptr(const void *ptr);
```
> Write a hexadecimal number to the debug log if enabled by runtime and
> `NDEBUG` wasn't defined during compilation.  The "ptr" variant writes "0x"
> before the number.

```c
void gate_debug_str(const char *s);
void gate_debug_data(const char *data, size_t size);
```
> Write a UTF-8 string to the debug log if enabled by runtime and `NDEBUG`
> wasn't defined during compilation.

```c
void gate_exit(int status);
```
> Terminate the program, indicating that execution must not be resumed later.
> Status 0 indicates success and 1 indicates failure.  Other values are
> interpreted as 1.

```c
void gate_io(const struct gate_iovec *recv,
             int recvveclen,
             size_t *nreceived,
             const struct gate_iovec *send,
             int sendveclen,
             size_t *nsent,
             int64_t timeout);
size_t gate_recv(void *buf, size_t size, int64_t timeout);
size_t gate_send(const void *data, size_t size, int64_t timeout);
```
> Receive and/or send packet data.  The transferred data sizes are returned
> through the *nreceived* and *nsent* pointers, or as return values.
>
> A packet is padded so that its buffer size is rounded up to the next multiple
> of `GATE_PACKET_ALIGNMENT`.  When sending a packet, `0` to
> `GATE_PACKET_ALIGNMENT-1` padding bytes must be sent after the packet to
> ensure alignment.
>
> TODO: document timeout parameter
> The call may still be interrupted without any bytes having been transferred.

#### Packet header

```c
struct gate_packet {
	uint32_t size;
	int16_t code;
	uint8_t domain;
	uint8_t index;
};
```
> The size includes the header and the trailing contents, but not the padding.
>
> Non-negative codes identify discovered services.  Negative codes are for
> built-in functionality.
>
> The index field must be zero in sent packets.  It must be ignored in received
> packets unless it is specified for a domain.
>
> The four least significant bits of the domain field specify the domain.  The
> four most significant bits must be unset in sent packets and ignored in
> received packets.
>
> The `GATE_PACKET_DOMAIN_CALL` domain is used for sending requests to
> services.  Each request is matched with one response.  Pending requests form
> a queue (per service); a response resolves the request in the queue at the
> index specified in the response packet.
>
> The `GATE_PACKET_DOMAIN_INFO` domain is used for sending signals to services,
> and receiving state change notifications from services.  Sent and received
> info packets don't necessarily have any relation.  A service won't start
> sending notifications before at least one packet is sent to that service.
>
> The struct declaration may contain additional reserved fields which must be
> zeroed in sent packets.

### Service discovery

```c
struct gate_service_name_packet {
	struct gate_packet header;
	uint16_t count;
	char names[0]; // Variable length.
};
```
> Service discovery request, sent with the `GATE_PACKET_CODE_SERVICES` code.
> *count* indicates how many service names are concatenated in *names*.  The
> *states* array in the response packet will be in the same order as *names*.
>
> Service names are encoded by prefixing each string with its length as a
> single byte.
>
> Services may be discovered in multiple steps.
>
> The struct declaration may contain additional reserved fields which must be
> zeroed in sent packets.

```c
struct gate_service_state_packet {
	struct gate_packet header;
	uint16_t count;
	uint8_t states[0]; // Variable length.
};
```
> Service discovery response or state change notification, received with
> `GATE_PACKET_CODE_SERVICES` code.   *count* is the total number of discovered
> services; *states* is a concatenation of all previously and newly discovered
> services.  A state item contains service state flags
> (`GATE_SERVICE_STATE_AVAIL`).
>
> If a call reply packet has a *count* which is less than the number of
> requested services, it means that the maximum service count has been reached.
>
> When the `GATE_SERVICE_STATE_AVAIL` flag is unset for a service, sending of
> packets to that service must cease.  If a partial packet has been sent to the
> service, transmission of the packet can be completed.  It is a fatal error to
> send more.  No data can be sent to services which haven't been available.

### Streaming

Services may provide uni- or bi-directional byte streams.  They may be
implicit, or opened explicitly via a call.  Either way, they are subject to
flow control and don't generate packets before permitted by the program.

```c
struct gate_flow_packet {
	struct gate_packet header;
	struct gate_flow flows[0]; // Variable length.
};
```
> Reception capacity notification for streams.  All streams belong to the
> service identified by the code in the packet header.
>
> A packet may contain multiple entries for a single stream: the entries must
> be processed in order.
>
> The struct declaration may contain additional reserved fields which must be
> zeroed in sent packets.

```c
struct gate_flow {
	int32_t id;
	int32_t increment;
};
```
> Information associated with the stream identified by *id*.  If *increment* is
> positive, it indicates that the reception capacity has increased by as many
> bytes.  If *increment* is zero, it indicates that no more data will be
> subscribed to (the stream is being closed).
>
> If *increment* is negative, it conveys service-specific information, such as
> an error code (the stream will stay open).
>
> The reception capacity of a stream must not exceed (2^31)-1 at any given
> time.

```c
struct gate_data_packet {
	struct gate_packet header;
	int32_t id;
	int32_t note;
	char data[0]; // Variable length.
};
```
> Data transfer for the stream identified by *id* which belongs to the service
> identified by the code in the packet header.  The length of *data* is
> implicitly decremented from the reception capacity.  If there is no data, it
> indicates that no more data will be received (the stream is being closed).
>
> The *note* field conveys service-specific information, such as an error code
> (the stream will stay open if the packet contains data).
>
> The struct declaration may contain additional reserved fields which must be
> zeroed in sent packets.


text:
import os
import sys
import string
import argparse
import tempfile
import shutil
import fnmatch

sys.path.append(os.path.join(os.path.dirname(__file__), "..", "..", "..", "scripts"))

from build.common import *
from build.config import *
from build.build import *

class Module:
	def __init__ (self, name, dirName, binName):
		self.name		= name
		self.dirName	= dirName
		self.binName	= binName

VULKAN_MODULE		= Module("dEQP-VK", "../external/vulkancts/modules/vulkan", "deqp-vk")
DEFAULT_BUILD_DIR	= os.path.join(tempfile.gettempdir(), "spirv-binaries", "{targetName}-{buildType}")
DEFAULT_TARGET		= "null"
DEFAULT_DST_DIR		= os.path.join(DEQP_DIR, "external", "vulkancts", "data", "vulkan", "prebuilt")
DEFAULT_VULKAN_VERSION	= "1.1"

def getBuildConfig (buildPathPtrn, targetName, buildType):
	buildPath = buildPathPtrn.format(
		targetName	= targetName,
		buildType	= buildType)

	return BuildConfig(buildPath, buildType, ["-DDEQP_TARGET=%s" % targetName])

def cleanDstDir (dstPath):
	binFiles = [f for f in os.listdir(dstPath) if os.path.isfile(os.path.join(dstPath, f)) and fnmatch.fnmatch(f, "*.spv")]

	for binFile in binFiles:
		print "Removing %s" % os.path.join(dstPath, binFile)
		os.remove(os.path.join(dstPath, binFile))

def execBuildPrograms (buildCfg, generator, module, dstPath, vulkanVersion):
	fullDstPath	= os.path.realpath(dstPath)
	workDir		= os.path.join(buildCfg.getBuildDir(), "modules", module.dirName)

	pushWorkingDir(workDir)

	try:
		binPath = generator.getBinaryPath(buildCfg.getBuildType(), os.path.join(".", "vk-build-programs"))
		execute([binPath, "--validate-spv", "--dst-path", fullDstPath, "--target-vulkan-version", vulkanVersion])
	finally:
		popWorkingDir()

def parseArgs ():
	parser = argparse.ArgumentParser(description = "Build SPIR-V programs",
									 formatter_class=argparse.ArgumentDefaultsHelpFormatter)
	parser.add_argument("-b",
						"--build-dir",
						dest="buildDir",
						default=DEFAULT_BUILD_DIR,
						help="Temporary build directory")
	parser.add_argument("-t",
						"--build-type",
						dest="buildType",
						default="Debug",
						help="Build type")
	parser.add_argument("-c",
						"--deqp-target",
						dest="targetName",
						default=DEFAULT_TARGET,
						help="dEQP build target")
	parser.add_argument("-d",
						"--dst-path",
						dest="dstPath",
						default=DEFAULT_DST_DIR,
						help="Destination path")
	parser.add_argument("-u",
						"--target-vulkan-version",
						dest="vulkanVersion",
						default="1.1",
						choices=["1.0", "1.1"],
						help="Target Vulkan version")
	return parser.parse_args()

if __name__ == "__main__":
	args = parseArgs()

	generator	= ANY_GENERATOR
	buildCfg	= getBuildConfig(args.buildDir, args.targetName, args.buildType)
	module		= VULKAN_MODULE

	build(buildCfg, generator, ["vk-build-programs"])

	if not os.path.exists(args.dstPath):
		os.makedirs(args.dstPath)

	execBuildPrograms(buildCfg, generator, module, args.dstPath, args.vulkanVersion)


text:
SELECT r.macId, macAddress, macAnnotation, o.model AS model, r.createdAt createdAt
FROM MAC_SSID_Relation r
INNER JOIN MACAddress a
ON (r.macId = a.macId)
LEFT OUTER JOIN OUI o
ON (a.ouiId = o.ouiId)
GROUP BY r.macId
ORDER BY a.createdAt DESC
LIMIT :limit;


text:
package org.dspace.app.xmlui.aspect.submission.submit;

import org.apache.cocoon.environment.ObjectModelHelper;
import org.apache.cocoon.environment.Request;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.log4j.Logger;
import org.apache.tools.ant.util.StringUtils;
import org.dspace.app.util.SubmissionInfo;
import org.dspace.app.xmlui.aspect.submission.AbstractStep;
import org.dspace.app.xmlui.cocoon.AbstractDSpaceTransformer;
import org.dspace.app.xmlui.utils.HandleUtil;
import org.dspace.app.xmlui.utils.UIException;
import org.dspace.app.xmlui.wing.Message;
import org.dspace.app.xmlui.wing.WingException;
import org.dspace.app.xmlui.wing.element.*;
import org.dspace.authorize.AuthorizeException;
import org.dspace.content.*;
import org.dspace.content.Item;
import org.dspace.core.ConfigurationManager;
import org.dspace.paymentsystem.*;
import org.dspace.submit.AbstractProcessingStep;
import org.dspace.utils.DSpace;
import org.w3c.dom.Document;
import org.xml.sax.SAXException;

import javax.xml.parsers.DocumentBuilderFactory;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.sql.SQLException;
import java.util.Date;

/**
 * CheckoutStep responsible for supplying interface for Paypal Checkout.
 *
 * @author Mark Diggory, mdiggory at atmire.com
 * @author Fabio Bolognesi, fabio at atmire.com
 * @author Lantian Gai, lantian at atmire.com
 */
public class CheckoutStep extends AbstractStep {
    private static final Message T_HEAD = message("xmlui.Submission.submit.Checkout.head");
    private static final Message T_HELP = message("xmlui.Submission.submit.Checkout.help");
    private static final Message T_TRAIL = message("xmlui.Submission.submit.Checkout.trail");
    private static final Message T_FINALIZE_BUTTON = message("xmlui.Submission.submit.OverviewStep.button.finalize");
    private static final Logger log = Logger.getLogger(AbstractDSpaceTransformer.class);
    @Override
    public void addBody(Body body) throws SAXException, WingException, UIException, SQLException, IOException, AuthorizeException {
        Collection collection = submission.getCollection();
        String actionURL = contextPath + "/handle/" + collection.getHandle() + "/submit/" + knot.getId() + ".continue";

        body.addDivision("step-link","step-link").addPara(T_TRAIL);

        Division helpDivision = body.addDivision("general-help","general-help");
        helpDivision.setHead(T_HEAD);
        helpDivision.addPara(T_HELP);

        Division mainDiv = body.addInteractiveDivision("submit-completed-dataset", actionURL, Division.METHOD_POST, "primary submission");

        Request request = ObjectModelHelper.getRequest(objectModel);
        SubmissionInfo submissionInfo=(SubmissionInfo)request.getAttribute("dspace.submission.info");
        org.dspace.content.Item item = submissionInfo.getSubmissionItem().getItem();
        DSpaceObject dso = HandleUtil.obtainHandle(objectModel);
        PaypalService paypalService = new DSpace().getSingletonService(PaypalService.class);
        paypalService.generateUserForm(context,mainDiv,actionURL,knot.getId(),"A",request,item,dso);

    }

}


text:
'use strict';

var By = require('..').By,
    assert = require('../testing/assert'),
    test = require('../lib/test');

test.suite(function(env) {
  var driver;
  test.after(function() { return driver.quit(); });

  test.it('should return lower case tag name', function*() {
    driver = yield env.builder().build();
    yield driver.get(test.Pages.formPage);

    let el = yield driver.findElement(By.id('cheese'));
    return assert(el.getTagName()).equalTo('input');
  });
});


text:
<?php namespace A11yc; ?>
<form id="a11yc_search" action="<?php echo $base_url ?>" method="GET">
	<input type="hidden" name="a" value="docs">
	<label class="a11yc_skip" for="a11yc_str"><?php echo A11YC_LANG_CTRL_KEYWORD_TITLE ?></label><input type="text" name="s" id="a11yc_str" size="24" value="<?php echo $word ?>">
	<input type="submit" value="<?php echo A11YC_LANG_CTRL_SEARCH ?>">
</form>


text:
import os, sys
import ast

print("\nHighlighting code...")

# Find all files to work with.
#  Replace with your path.
path_to_notebooks = '/srv/projects/intro_programming/intro_programming/notebooks/'

filenames = []
for filename in os.listdir(path_to_notebooks):
    if '.html' in filename and filename != 'index.html':
        filenames.append(filename)

# Uncomment to use just one file for testing:
#filenames = ['visualization_earthquakes.html']

# Process each file.
for filename in filenames:

    # Grab the lines from this html file.
    f = open(path_to_notebooks + filename, 'r')
    lines = f.readlines()
    f.close()

    # Open the file for rewriting.
    f = open(path_to_notebooks + filename, 'wb')

    # Create an empty list for the lines that will need to be highlighted.
    highlight_lines = []
    # True when in a code block that has some highlighting left to be done.
    highlighting_active = False

    # Next line to highlight; starts at 0 in each code block.
    #  This takes into account the ###highlight= line, so you don't have
    #  to do a bunch of subtracting while composing notebooks.
    highlight_line = None

    for line in lines:
        if '###highlight' in line:
            # Get lines to highlight, stored as a list.
            #  Lines are in a list, after the equals sign.
            try:
                highlight_lines = ast.literal_eval(line[line.index('highlight=')+10:line.index(']')+1])
            except:
                print("Problem finding lines to highlight in %s near line %d" % (filename, lines.index(line)))

            # We have some lines to highlight. Start tracking lines,
            #  and highlight appropriate lines.
            line_number = 0
            highlighting_active = True

        if highlighting_active:
            # We are in a code block, with some lines left to highlight.
            if line_number == 0:
                # We are on the ###highlight= line, and we don't need this entire line.
                #  We do need some of the line, which sets up the code block.
                # sample line:

###highlight=[2,3]
                # becomes:

                # keep until start of '<span'
                span_index = line.index('<span')
                f.write(line[:span_index].encode('utf-8'))
                line_number += 1
                # Next line to highlight:
                #  Minus 1 accounts for initial comment line ###highlight=...,
                #  which will be removed.
                highlight_line = highlight_lines.pop(0)-1
            elif line_number == highlight_line:
                # Change style so line is highlighted.
                line = "
%s
" % line
                f.write(line.encode('utf-8'))
                line_number += 1
                try:
                    # Get next line to highlight.
                    highlight_line = highlight_lines.pop(0)-1
                except:
                    # No more lines to highlight in this code block.
                    highlight_line = None
                    highlighting_active = False
            else:
                # Write line of code as is.
                f.write(line.encode('utf-8'))
                line_number += 1
        else:
            # No highlighting left to do in this code block, or
            #  not in a code block with highlighting.
            #  Write line as is.
            f.write(line.encode('utf-8'))

    f.close()

print("  Highlighted code.\n")


text:
package com.etapps.trovenla.utils;

import androidx.annotation.NonNull;
import androidx.annotation.Nullable;

import com.etapps.trovenla.jobs.FetchLibraryJob;
import com.evernote.android.job.Job;
import com.evernote.android.job.JobCreator;

/**
 * Created by emanuele on 06/11/17.
 */

public class TroveJobCreator implements JobCreator {

    @Override
    @Nullable
    public Job create(@NonNull String tag) {
        switch (tag) {
            case FetchLibraryJob.TAG:
                return new FetchLibraryJob();
            default:
                return null;
        }
    }
}


text:
stty erase "ˆH" kill "ˆU" intr "ˆC" tabs
echo stty erase "ˆH" kill "ˆU" intr "ˆC" tabs
/usr/ucb/uptime
echo
trap "" 2
cat /v/adm/motd
echo "Enter help for command list."
while true
do
	trap continue 2
	echo "research gateway> \c"
	if read line
	then
		case "$line" in
			"") continue;;
		esac
		set -- $line
		case "$1" in
			help) echo " ping <destination>"
				echo " traceroute <destination>"
				echo " dig <parameters>"
				echo " telnet <destination> [port]"
				echo " finger <destination>"
				echo;;
			ping) shift
				/bin/ping $*;;
			traceroute)
				shift
				/v/bin/traceroute $*;;
			dig) shift
				/v/bin/dig $*;;
			telnet) shift
				/usr/ucb/telnet $*;;
			finger) shift
				/usr/ucb/finger $*;;
			exit) break;;
			quit) break;;
			*) echo "Unknown command - $line."
				echo "Enter help for command list";;
		esac
	else
		break
	fi
done
exit


text:
using namespace cv;
using namespace venus;

static const std::string TAG("Beauty");

void detectSkin(const cv::Mat& image)
{
	TIME_START;

	Mat mask_rgb = Beauty::calculateSkinRegion_RGB(image);
	TIME_STOP("calculateSkinRegion_RGB");

	Mat mask_ycbcr = Beauty::calculateSkinRegion_YCbCr(image);
	TIME_STOP("calculateSkinRegion_YCbCr");

	Mat mask_hsv = Beauty::calculateSkinRegion_HSV(image);
	TIME_STOP("calculateSkinRegion_HSV");

	// How about combining two methods?
	Mat mask_combined;
	double weight = 0.72;
	cv::addWeighted(mask_rgb, weight, mask_hsv, 1.0 - weight, 0.0, mask_combined);

	cv::imshow("original", image);

	cv::imshow("mask_rgb",      mask_rgb);
	cv::imshow("mask_ycbcr",    mask_ycbcr);
	cv::imshow("mask_hsv",      mask_hsv);
	cv::imshow("mask_combined", mask_combined);
	cv::waitKey();
}

void redEyeRemoval_CLI(const cv::Mat& image, float threshold)
{
	Mat gray = Effect::grayscale(image);
	const std::vector<std::vector<Point2f>> faces = Feature::detectFaces(gray, __FUNCTION__, CLASSIFIER_DIR);
	// TODO detectFace only detect one face, multiple faces support will be added later.

	Mat processed = image.clone();
	if(faces.empty())
	{
		// Here supposing the whole image need to be processed, outline the red eye region is better.
		const float x0 = 0, x1 = static_cast<float>(image.cols - 1);
		const float y0 = 0, y1 = static_cast<float>(image.rows - 1);
		std::vector<Point2f> whole{Point2f(x0, x0), Point2f(x1, x0), Point2f(x1, y1), Point2f(x0, y1)};
		Beauty::removeRedEye(processed, processed, whole, threshold);
	}
	else
	{
		const std::vector<Point2f>& points = faces[0];
		Feature feature(image, points);
		for(int i = 0; i < 2; ++i)
		{
			std::vector<Point2f> polygon = feature.calculateEyePolygon(points, i == 0);
			Beauty::removeRedEye(processed, processed, polygon, threshold);
		}
	}

	cv::imshow("original", image);
	cv::imshow("processed", processed);
	cv::waitKey();
}

void redEyeRemoval_GUI(const cv::Mat& image)
{
	Mat gray = Effect::grayscale(image);
	const std::vector<std::vector<Point2f>> faces = Feature::detectFaces(gray, __FUNCTION__, CLASSIFIER_DIR);
	assert(!faces.empty());  // ensure an face is found.
	const std::vector<Point2f>& points = faces[0];

	const std::string title("Red Eye Removal");
	const int max = 512;

	Mat processed = image.clone();
	UserData user_data(title, max, points, image, processed);

	auto onProgressChanged = [](int progress, void* user_data)
	{
		UserData& data = *reinterpret_cast<UserData*>(user_data);
		float threshold = progress / static_cast<float>(data.max);

		data.original.copyTo(data.processed);
		Feature feature(data.original, data.points);
		for(int i = 0; i < 2; ++i)
		{
			std::vector<Point2f> polygon = feature.calculateEyePolygon(data.points, i == 0);
			Beauty::removeRedEye(data.processed, data.processed, polygon, threshold);
		}

		cv::imshow(data.title, data.processed);
	};

	int progress = max / 2;

	cv::namedWindow(title);
	cv::setMouseCallback(title, UserData::onClick, &user_data);
	cv::createTrackbar("amount", title, &progress, max, onProgressChanged, &user_data);

	onProgressChanged(progress, &user_data);

	cv::waitKey();
}

void skinWhiten(const cv::Mat& image)
{
	const std::string title("Skin Whiten");
	const int max = 100;

	Mat processed = image.clone();
	Mat mask(image.rows, image.cols, CV_8UC1, Scalar(255));  // whole
	UserData user_data(title, max, {}, image, processed);
	user_data.setMask(mask);

	auto onProgressChanged = [](int progress, void* user_data)
	{
		UserData& data = *reinterpret_cast<UserData*>(user_data);
		float amount = progress / static_cast<float>(data.max);
		float level = amount * 8 + 2;
		Beauty::whitenSkinByLogCurve(data.processed, data.original, level);
		cv::imshow(data.title, data.processed);
	};

	int progress = max / 2;
	cv::namedWindow(title);
	cv::setMouseCallback(title, UserData::onClick, &user_data);
	cv::createTrackbar("level", title, &progress, max, onProgressChanged, &user_data);

	onProgressChanged(progress, &user_data);

	cv::waitKey();
}

void skinDermabrasion(const cv::Mat& image)
{
	Mat mask = Beauty::calculateSkinRegion_RGB(image);
	skinDermabrasion(image, mask);
}

void skinDermabrasion(const cv::Mat& image, const cv::Mat& mask)
{
	const std::string title("Skin Dermabrasion");
	const int level_max = 20;

	Mat processed = image.clone();
	cv::imshow("mask", mask);

	UserData user_data(title, level_max, {}, image, processed);
	user_data.setMask(mask);

	auto onProgressChanged = [](int level, void* user_data)
	{
		UserData& data = *reinterpret_cast<UserData*>(user_data);
		float radius = 5.0F;  // can be tuned!
		Beauty::beautifySkin(data.processed, data.original, data.mask, radius, level);

		cv::imshow(data.title, data.processed);
	};

	int level = level_max / 2;
	cv::namedWindow(title);
	cv::setMouseCallback(title, UserData::onClick, &user_data);
	cv::createTrackbar("amount", title, &level, level_max, onProgressChanged, &user_data);

	onProgressChanged(level_max, &user_data);

	cv::waitKey();
}


text:
$(document).ready(function() {
	// ViewModel for password dialog (only)
	function UserPasswordViewModel() {
		var self = this;
		self.password = ko.observable();
		self.password2 = ko.observable();

		self.errors = new ko.errors.ErrorsViewModel(['password', 'password2']);

		self.savePassword = function() {
			// Check both passwords are identical
			if (self.password() !== self.password2()) {
				// Show error
				self.errors.reset({}, ['Both passwords must be identical']);
				self.password('');
				self.password2('');
			} else {
				ko.utils.ajax(uri, 'PUT', {password: self.password()}).fail(self.errors.errorHandler).done(function(user) {
					// Add message
					globalViewModel.flashMessages.clear();
					globalViewModel.flashMessages.success('Your password has been changed');
					// Hide dialog
					$('#password-dialog').modal('hide');
				});
			}
		}

		self.editPassword = function() {
			$('#password-dialog').modal('show');
			$('#password_password').focus();
		}

		self.install = self.editPassword;
	}

	globalViewModel.password(new UserPasswordViewModel());
});


text:
package vcs.citydb.wfs.config.server;

import org.citydb.config.project.global.Cache;

import javax.xml.bind.annotation.XmlSchemaType;
import javax.xml.bind.annotation.XmlTransient;
import javax.xml.bind.annotation.XmlType;
import java.nio.file.Path;

@XmlType(name = "ServerType", propOrder = {})
public class Server {
    private String externalServiceURL;
    @XmlSchemaType(name = "nonNegativeInteger")
    private Integer maxParallelRequests = 30;
    @XmlSchemaType(name = "positiveInteger")
    private Integer waitTimeout = 60;
    @XmlSchemaType(name = "positiveInteger")
    private Integer responseCacheTimeout = 300;
    private Boolean enableCORS = true;
    private String timeZone;
    private String textureServiceURL;
    private TextureCache textureCache;
    private Security security;
    private Cache tempCache;

    @XmlTransient
    private Path tempDir;

    public Server() {
        textureCache = new TextureCache();
        security = new Security();
        tempCache = new Cache();
    }

    public String getExternalServiceURL() {
        return externalServiceURL;
    }

    public boolean isSetExternalServiceURL() {
        return externalServiceURL != null && !externalServiceURL.isEmpty();
    }

    public void setExternalServiceURL(String externalServiceURL) {
        this.externalServiceURL = externalServiceURL;
    }

    public int getMaxParallelRequests() {
        return maxParallelRequests;
    }

    public void setMaxParallelRequests(int maxParallelRequests) {
        this.maxParallelRequests = maxParallelRequests >= 0 ? maxParallelRequests : 30;
    }

    public int getWaitTimeout() {
        return waitTimeout;
    }

    public void setWaitTimeout(int waitTimeout) {
        this.waitTimeout = waitTimeout > 0 ? waitTimeout : 60;
    }

    public int getResponseCacheTimeout() {
        return responseCacheTimeout;
    }

    public void setCacheTimeout(int responseCacheTimeout) {
        this.responseCacheTimeout = responseCacheTimeout > 0 ? responseCacheTimeout : 300;
    }

    public boolean isEnableCORS() {
        return enableCORS;
    }

    public void setEnableCORS(boolean enableCORS) {
        this.enableCORS = enableCORS;
    }

    public String getTimeZone() {
        return timeZone;
    }

    public boolean isSetTimeZone() {
        return timeZone != null && !timeZone.isEmpty();
    }

    public void setTimeZone(String timeZone) {
        this.timeZone = timeZone;
    }

    public String getTextureServiceURL() {
        return textureServiceURL;
    }

    public boolean isSetTextureServiceURL() {
        return textureServiceURL != null && !textureServiceURL.isEmpty();
    }

    public void setTextureServiceURL(String textureServiceURL) {
        this.textureServiceURL = textureServiceURL;
    }

    public TextureCache getTextureCache() {
        return textureCache;
    }

    public void setTextureCache(TextureCache textureCache) {
        this.textureCache = textureCache;
    }

    public Security getSecurity() {
        return security;
    }

    public void setSecurity(Security security) {
        this.security = security;
    }

    public Cache getTempCache() {
        return tempCache;
    }

    public void setTempCache(Cache uidCache) {
        this.tempCache = uidCache;
    }

    public Path getTempDir() {
        return tempDir;
    }

    public void setTempDir(Path tempDir) {
        this.tempDir = tempDir;
    }
}


text:
package org.apache.storm.sql;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.PriorityQueue;

import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.sql.runtime.ISqlStreamsDataSource;
import org.apache.storm.streams.Pair;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.IRichBolt;
import org.apache.storm.topology.IRichSpout;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.base.BaseRichBolt;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.junit.rules.ExternalResource;

public class TestUtils {
  public static final ExternalResource mockInsertBoltValueResource = new ExternalResource() {
    @Override
    protected void before() throws Throwable {
      MockInsertBolt.getCollectedValues().clear();
    }
  };

  public static final ExternalResource mockBoltValueResource = new ExternalResource() {
    @Override
    protected void before() throws Throwable {
      MockBolt.getCollectedValues().clear();
    }
  };

  public static class MyPlus {
    public static Integer evaluate(Integer x, Integer y) {
      return x + y;
    }
  }

  public static class MyConcat {
    public static String init() {
      return "";
    }
    public static String add(String accumulator, String val) {
      return accumulator + val;
    }
    public static String result(String accumulator) {
      return accumulator;
    }
  }

  public static class TopN {
    public static PriorityQueue<Integer> init() {
      return new PriorityQueue<>();
    }
    public static PriorityQueue<Integer> add(PriorityQueue<Integer> accumulator, Integer n, Integer val) {
      if (n <= 0) {
        return accumulator;
      }
      if (accumulator.size() >= n) {
        if (val > accumulator.peek()) {
          accumulator.remove();
          accumulator.add(val);
        }
      } else {
        accumulator.add(val);
      }
      return accumulator;
    }
    public static List<Integer> result(PriorityQueue<Integer> accumulator) {
      List<Integer> res = new ArrayList<>(accumulator);
      Collections.reverse(res);
      return res;
    }
  }

  public static class MockSpout extends BaseRichSpout {

    private final List<Values> records;
    private final Fields outputFields;
    private boolean emitted = false;
    private SpoutOutputCollector collector;

    public MockSpout(List<Values> records, Fields outputFields) {
      this.records = records;
      this.outputFields = outputFields;
    }

    @Override
    public void open(Map<String, Object> conf, TopologyContext context, SpoutOutputCollector collector) {
      this.collector = collector;
    }

    @Override
    public void nextTuple() {
      if (emitted) {
        return;
      }

      for (Values r : records) {
        collector.emit(r);
      }

      emitted = true;
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer declarer) {
      declarer.declare(outputFields);
    }
  }

  public static class MockBolt extends BaseRichBolt {
    /**
     * Collect all values in a static variable as the instance will go through serialization and deserialization.
     * NOTE: This should be cleared before or after running each test.
     */
    private transient static final List<Values> VALUES = new ArrayList<>();

    public static List<Values> getCollectedValues() {
      return VALUES;
    }

    @Override
    public void prepare(Map<String, Object> topoConf, TopologyContext context, OutputCollector collector) {

    }

    @Override
    public void execute(Tuple input) {
      VALUES.add((Values) input.getValue(0));
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer declarer) {

    }
  }

  public static class MockInsertBolt extends BaseRichBolt {
    /**
     * Collect all values in a static variable as the instance will go through serialization and deserialization.
     * NOTE: This should be cleared before or after running each test.
     */
    private transient static final List<Pair<Object, Values>> VALUES = new ArrayList<>();

    public static List<Pair<Object, Values>> getCollectedValues() {
      return VALUES;
    }

    @Override
    public void prepare(Map<String, Object> topoConf, TopologyContext context, OutputCollector collector) {

    }

    @Override
    public void execute(Tuple input) {
      VALUES.add(Pair.of(input.getValue(0), (Values) input.getValue(1)));
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer declarer) {

    }
  }

  public static class MockSqlExprDataSource implements ISqlStreamsDataSource {
    @Override
    public IRichSpout getProducer() {
      throw new UnsupportedOperationException("Not supported.");
    }

    @Override
    public IRichBolt getConsumer() {
      return new MockBolt();
    }
  }

  public static class MockSqlStreamsOutputDataSource implements ISqlStreamsDataSource {

    @Override
    public IRichSpout getProducer() {
      throw new UnsupportedOperationException("Not supported.");
    }

    @Override
    public IRichBolt getConsumer() {
      return new MockInsertBolt();
    }

  }

  public static class MockSqlStreamsDataSource implements ISqlStreamsDataSource {
    @Override
    public IRichSpout getProducer() {
      List<Values> records = new ArrayList<>();
      records.add(new Values(0, "a", "y"));
      records.add(new Values(1, "ab", "y"));
      records.add(new Values(2, "abc", "y"));
      records.add(new Values(3, "abcd", "y"));
      records.add(new Values(4, "abcde", "y"));

      Fields outputFields = new Fields("ID", "NAME", "ADDR");
      return new MockSpout(records, outputFields);
    }

    @Override
    public IRichBolt getConsumer() {
      return new MockBolt();
    }

  }

  public static class MockSqlStreamsInsertDataSource extends MockSqlStreamsNestedDataSource {
    @Override
    public IRichBolt getConsumer() {
      return new MockInsertBolt();
    }
  }

  public static class MockSqlStreamsGroupedDataSource implements ISqlStreamsDataSource {
    @Override
    public IRichSpout getProducer() {
      List<Values> records = new ArrayList<>();
      for (int i = 0; i < 5; ++i) {
        records.add(new Values(i, 0, "x", "y", 5 - i, i * 10));
      }

      Fields outputFields = new Fields("ID", "GRPID", "NAME", "ADDR", "AGE", "SCORE");
      return new MockSpout(records, outputFields);
    }

    @Override
    public IRichBolt getConsumer() {
      return new MockBolt();
    }
  }

  public static class MockSqlStreamsInsertGroupedDataSource extends MockSqlStreamsGroupedDataSource {
    @Override
    public IRichBolt getConsumer() {
      return new MockInsertBolt();
    }
  }

  public static class MockSqlStreamsJoinDataSourceEmp implements ISqlStreamsDataSource {
    @Override
    public IRichSpout getProducer() {
      List<Values> records = new ArrayList<>();
      Fields outputFields = new Fields("EMPID", "EMPNAME", "DEPTID");

      for (int i = 0; i < 5; ++i) {
        records.add(new Values(i, "emp-" + i, i % 2));
      }
      for (int i = 10; i < 15; ++i) {
        records.add(new Values(i, "emp-" + i, i));
      }

      return new MockSpout(records, outputFields);
    }

    @Override
    public IRichBolt getConsumer() {
      return new MockBolt();
    }
  }

  public static class MockSqlStreamsInsertJoinDataSourceEmp extends MockSqlStreamsJoinDataSourceEmp {
    @Override
    public IRichBolt getConsumer() {
      return new MockInsertBolt();
    }
  }

  public static class MockSqlStreamsJoinDataSourceDept implements ISqlStreamsDataSource {
    @Override
    public IRichSpout getProducer() {
      List<Values> records = new ArrayList<>();
      Fields outputFields = new Fields("DEPTID", "DEPTNAME");

      for (int i = 0; i < 5; ++i) {
        records.add(new Values(i, "dept-" + i));
      }

      return new MockSpout(records, outputFields);
    }

    @Override
    public IRichBolt getConsumer() {
      return new MockBolt();
    }
  }

  public static class MockSqlStreamsInsertJoinDataSourceDept extends MockSqlStreamsJoinDataSourceDept {
    @Override
    public IRichBolt getConsumer() {
      return new MockInsertBolt();
    }
  }

  public static class MockSqlStreamsNestedDataSource implements ISqlStreamsDataSource {
    @Override
    public IRichSpout getProducer() {
      List<Values> records = new ArrayList<>();
      Fields outputFields = new Fields("ID", "MAPFIELD", "NESTEDMAPFIELD", "ARRAYFIELD");

      List<Integer> ints = Arrays.asList(100, 200, 300);
      for (int i = 0; i < 5; ++i) {
        Map<String, Integer> map = new HashMap<>();
        map.put("b", i);
        map.put("c", i*i);
        Map<String, Map<String, Integer>> mm = new HashMap<>();
        mm.put("a", map);
        records.add(new Values(i, map, mm, ints));
      }

      return new MockSpout(records, outputFields);
    }

    @Override
    public IRichBolt getConsumer() {
      return new MockBolt();
    }
  }

  public static class MockSqlStreamsInsertNestedDataSource extends MockSqlStreamsNestedDataSource {
    @Override
    public IRichBolt getConsumer() {
      return new MockInsertBolt();
    }
  }

  public static long monotonicNow() {
    final long NANOSECONDS_PER_MILLISECOND = 1000000;
    return System.nanoTime() / NANOSECONDS_PER_MILLISECOND;
  }
}


text:
package libpages

import (
	"context"
	"database/sql"
	"io/ioutil"
	"os"
	"path/filepath"
	"sort"
	"time"

	"github.com/keybase/client/go/kbfs/data"
	"github.com/keybase/client/go/kbfs/tlf"
	"go.uber.org/zap"
)

type activity struct {
	tlfID tlf.ID
	host  string
}

type fileBasedActivityStatsStorer struct {
	root   string
	logger *zap.Logger
	ch     chan activity
}

const (
	dirnameTlfStamps  = "tlf-stamps"
	dirnameHostStamps = "host-stamps"

	fbassChSize        = 1000
	fbassStoreInterval = time.Second * 10
)

func (s *fileBasedActivityStatsStorer) processLoop() {
	// We won't worry about cleaning up these two maps since this storer is
	// meant to be only used for up to ~1000 entries anyway.
	recentProcessedTlfs := make(map[tlf.ID]time.Time)
	recentProcessedHosts := make(map[string]time.Time)
	for a := range s.ch {
		// The end result we want is a file with mtime set to now. os.Create
		// uses the O_TRUNC flag which does that for existing files.

		lastProcessed, ok := recentProcessedTlfs[a.tlfID]
		if !ok || time.Since(lastProcessed) > fbassStoreInterval {
			if f, err := os.Create(filepath.Join(
				s.root, dirnameTlfStamps, a.tlfID.String())); err == nil {
				f.Close()
				recentProcessedTlfs[a.tlfID] = time.Now()
			} else {
				s.logger.Warn("os.Create", zap.Error(err))
			}
		}

		lastProcessed, ok = recentProcessedHosts[a.host]
		if !ok || time.Since(lastProcessed) > fbassStoreInterval {
			if f, err := os.Create(filepath.Join(
				s.root, dirnameHostStamps, a.host)); err == nil {
				f.Close()
				recentProcessedHosts[a.host] = time.Now()
			} else {
				s.logger.Warn("os.Create", zap.Error(err))
			}
		}
	}
}

// NewFileBasedActivityStatsStorer creates an ActivityStatsStorer that stores
// activities on a local filesystem.
//
// NOTE that this is meant to be for development and
// testing only and does not scale well.
func NewFileBasedActivityStatsStorer(
	rootPath string, logger *zap.Logger) (ActivityStatsStorer, error) {
	err := os.MkdirAll(filepath.Join(rootPath, dirnameTlfStamps), os.ModeDir|0700)
	if err != nil {
		return nil, err
	}
	err = os.MkdirAll(filepath.Join(rootPath, dirnameHostStamps), os.ModeDir|0700)
	if err != nil {
		return nil, err
	}
	s := &fileBasedActivityStatsStorer{
		root:   rootPath,
		logger: logger,
		ch:     make(chan activity, fbassChSize),
	}
	go s.processLoop()
	return s, nil
}

// RecordActives implement the ActivityStatsStorer interface.
func (s *fileBasedActivityStatsStorer) RecordActives(tlf tlf.ID, host string) {
	s.ch <- activity{tlfID: tlf, host: host}
}

type fileinfoActivesGetter struct {
	tlfs   []os.FileInfo
	hosts  []os.FileInfo
	sorted bool
}

func (g *fileinfoActivesGetter) GetActives(
	dur time.Duration) (tlfs, hosts int, err error) {
	if !g.sorted {
		// Sort in decreasing order by time.
		sort.Slice(g.tlfs, func(i int, j int) bool {
			return g.tlfs[i].ModTime().After(g.tlfs[j].ModTime())
		})
		sort.Slice(g.hosts, func(i int, j int) bool {
			return g.hosts[i].ModTime().After(g.hosts[j].ModTime())
		})
		g.sorted = true
	}
	cutoff := time.Now().Add(-dur)
	// sort.Search requires a false,false...true,true... sequence.
	tlfs = sort.Search(len(g.tlfs), func(i int) bool {
		return cutoff.After(g.tlfs[i].ModTime())
	})
	hosts = sort.Search(len(g.hosts), func(i int) bool {
		return cutoff.After(g.hosts[i].ModTime())
	})

	return tlfs, hosts, nil
}

// GetActivesGetter implement the ActivityStatsStorer interface.
func (s *fileBasedActivityStatsStorer) GetActivesGetter() (
	getter ActivesGetter, err error) {
	tlfStamps, err := ioutil.ReadDir(filepath.Join(s.root, dirnameTlfStamps))
	if err != nil {
		return nil, err
	}
	hostStamps, err := ioutil.ReadDir(filepath.Join(s.root, dirnameHostStamps))
	if err != nil {
		return nil, err
	}
	return &fileinfoActivesGetter{
		tlfs:  tlfStamps,
		hosts: hostStamps,
	}, nil
}

// MigrateActivityStatsStorerFromFileBasedToMySQL should only be used as part
// of a commandline tool.
func MigrateActivityStatsStorerFromFileBasedToMySQL(
	logger *zap.Logger, fbRootDir string, mysqlDSN string) {
	logger.Info("open mysql", zap.String("dsn", mysqlDSN))
	db, err := sql.Open("mysql", mysqlDSN)
	if err != nil {
		logger.Error("open mysql", zap.Error(err))
		return
	}
	logger.Info("create tables")
	mysqlStorer := newMySQLActivityStatsStorerNoStart(data.WallClock{}, db, logger)
	err = mysqlStorer.createTablesIfNotExists(context.Background())
	if err != nil {
		logger.Error("create tables", zap.Error(err))
		return
	}

	logger.Info("tlf stamps")
	tlfStamps, err := ioutil.ReadDir(filepath.Join(fbRootDir, dirnameTlfStamps))
	if err != nil {
		logger.Error("ReadDir tlf stamps", zap.Error(err))
		return
	}
	for _, fi := range tlfStamps {
		tlfID := tlf.ID{}
		err := tlfID.UnmarshalText([]byte(fi.Name()))
		if err != nil {
			logger.Error("skipping stamp file", zap.String("filename", fi.Name()), zap.Error(err))
			continue
		}
		mysqlStorer.tlfs[tlfID] = fi.ModTime()
	}

	logger.Info("host stamps")
	hostStamps, err := ioutil.ReadDir(filepath.Join(fbRootDir, dirnameHostStamps))
	if err != nil {
		logger.Error("ReadDir host stamps", zap.Error(err))
		return
	}
	for _, fi := range hostStamps {
		mysqlStorer.hosts[fi.Name()] = fi.ModTime()
	}

	logger.Info("flush inserts")
	mysqlStorer.flushInserts()
}


text:
kOps supports two types of addons:

 * Managed addons, which are configurable through the [cluster spec](cluster_spec.md)
 * Static addons, which are manifest files that are applied as-is

## Managed addons

The following addons are managed by kOps and will be upgraded following the kOps and kubernetes lifecycle, and configured based on your cluster spec. kOps will consider both the configuration of the addon itself as well as what other settings you may have configured where applicable.

### Available addons

#### AWS Load Balancer Controller
{{ kops_feature_table(kops_added_default='1.20') }}

AWS Load Balancer Controller offers additional functionality for provisioning ELBs.

```yaml
spec:
  awsLoadBalancerController:
    enabled: true
```

Though the AWS Load Balancer Controller can integrate the AWS WAF and
Shield services with your Application Load Balancers (ALBs), kOps
disables those capabilities by default.

{{ kops_feature_table(kops_added_default='1.24') }}

You can enable use of either or both of the WAF and WAF Classic
services by including the following fields in the cluster spec:

```yaml
spec:
  awsLoadBalancerController:
    enabled: true
    enableWAF: true
    enableWAFv2: true
```

Note that the controller will only succeed in associating one WAF with
a given ALB at a time, despite it accepting both the
"alb.ingress.kubernetes.io/waf-acl-id" and
"alb.ingress.kubernetes.io/wafv2-acl-arn" annotations on the same
_Ingress_ object.

You can enable use of Shield Advanced by including the following fields in the cluster spec:

```yaml
spec:
  awsLoadBalancerController:
    enabled: true
    enableShield: true
```

Support for the WAF and Shield services in kOps is currently **beta**, meaning
that the accepted configuration and the AWS resources involved may
change.

Read more in the [official documentation](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/).

#### Cluster autoscaler
{{ kops_feature_table(kops_added_default='1.19') }}

Cluster autoscaler can be enabled to automatically adjust the size of the kubernetes cluster.

```yaml
spec:
  clusterAutoscaler:
    enabled: true
    expander: least-waste
    balanceSimilarNodeGroups: false
    awsUseStaticInstanceList: false
    scaleDownUtilizationThreshold: 0.5
    skipNodesWithLocalStorage: true
    skipNodesWithSystemPods: true
    newPodScaleUpDelay: 0s
    scaleDownDelayAfterAdd: 10m0s
    scaleDownUnneededTime: 10m0s
    scaleDownUnreadyTime: 20m0s
    image: <the latest supported image for the specified kubernetes version>
    cpuRequest: "100m"
    memoryRequest: "300Mi"
```

Read more about cluster autoscaler in the [official documentation](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler).

##### Expander strategies
Cluster autoscaler supports several different [expander strategies](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-are-expanders).

Note that the `priority` expander requires additional configuration through a ConfigMap as described in [its documentation](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/expander/priority/readme.md) - you will need to create this ConfigMap in your cluster before selecting this expander.

##### Disabling cluster autoscaler for a given instance group
{{ kops_feature_table(kops_added_default='1.20') }}

You can disable the autoscaler for a given instance group by adding the following to the instance group spec.

```yaml
spec:
  autoscale: false
```

#### Cert-manager
{{ kops_feature_table(kops_added_default='1.20', k8s_min='1.16') }}

Cert-manager handles x509 certificates for your cluster.

```yaml
spec:
  certManager:
    enabled: true
    defaultIssuer: yourDefaultIssuer
```

**Warning: cert-manager only supports one installation per cluster. If you are already running cert-manager, you need to
either remove this installation prior to enabling this addon, or mark cert-manger as not being managed by kOps (see below).
As long as you are using v1 versions of the cert-manager resources, it is safe to remove existing installs and replace it with this addon**

##### Self-provisioned cert-manager
{{ kops_feature_table(kops_added_default='1.20.2', k8s_min='1.16') }}

The following cert-manager configuration allows provisioning cert-manager externally and allows all dependent plugins
to be deployed. Please note that addons might run into errors until cert-manager is deployed.

```yaml
spec:
  certManager:
    enabled: true
    managed: false
```

##### DNS nameserver configuration for cert-manager pod
{{ kops_feature_table(kops_added_default='1.23.3', k8s_min='1.16') }}

Optional list of DNS nameserver IP addresses for the cert-manager pod to use.
This is useful if you have a public and private DNS zone for the same domain to ensure that cert-manager can access ingress, or DNS01 challenge TXT records at all times.

You can set pod DNS nameserver configuration for cert-manager like so:
```yaml
spec:
  certManager:
    enabled: true
    nameservers:
      - 1.1.1.1
      - 8.8.8.8
```

##### Enabling dns-01 challenges

{{ kops_feature_table(kops_added_default='1.25.0') }}

Cert Manager may be granted the necessary IAM privileges to solve dns-01 challenges by adding a list of hostedzone IDs.
This requires [external permissions for service accounts](/cluster_spec/#service-account-issuer-discovery-and-aws-iam-roles-for-service-accounts-irsa) to be enabled.

```yaml
spec:
  certManager:
    enabled: true
    hostedZoneIDs:
    - ZONEID
  iam:
    useServiceAccountExternalPermissions: true
```

Read more about cert-manager in the [official documentation](https://cert-manager.io/docs/)

#### Karpenter
{{ kops_feature_table(kops_added_default='1.24') }}

The Karpenter addon enables Karpenter-managed InstanceGroups.

```yaml
spec:
  karpenter:
    enabled: true
```

See more details on how to configure Karpenter in the [kOps Karpenter docs](/operations/karpenter) and the [official documentation](https://karpenter.sh)

#### Metrics server
{{ kops_feature_table(kops_added_default='1.19') }}

Metrics Server is a scalable, efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines.

```yaml
spec:
  metricsServer:
    enabled: true
```

Read more about Metrics Server in the [official documentation](https://github.com/kubernetes-sigs/metrics-server).

##### Secure TLS

{{ kops_feature_table(kops_added_default='1.20') }}

By default, API server will not verify the metrics server TLS certificate. To enable TLS verification, set the following in the cluster spec:

```yaml
spec:
  certManager:
    enabled: true
  metricsServer:
    enabled: true
    insecure: false
```

This requires that cert-manager is installed in the cluster.

#### Node local DNS cache
{{ kops_feature_table(kops_added_default='1.18', k8s_min='1.15') }}

NodeLocal DNSCache can be enabled if you are using CoreDNS. It is used to improve the Cluster DNS performance by running a dns caching agent on cluster nodes as a DaemonSet.

`memoryRequest` and `cpuRequest` for the `node-local-dns` pods can also be configured. If not set, they will be configured by default to `5Mi` and `25m` respectively.

If `forwardToKubeDNS` is enabled, kubedns will be used as a default upstream

```yaml
spec:
  kubeDNS:
    provider: CoreDNS
    nodeLocalDNS:
      enabled: true
      memoryRequest: 5Mi
      cpuRequest: 25m
```

#### Node termination handler

{{ kops_feature_table(kops_added_default='1.19') }}

[Node Termination Handler](https://github.com/aws/aws-node-termination-handler) ensures that the Kubernetes control plane responds appropriately to events that can cause your EC2 instance to become unavailable, such as EC2 maintenance events, EC2 Spot interruptions, and EC2 instance rebalance recommendations. If not handled, your application code may not stop gracefully, take longer to recover full availability, or accidentally schedule work to nodes that are going down.

```yaml
spec:
  nodeTerminationHandler:
    cpuRequest: 200m
    enabled: true
    enableRebalanceMonitoring: true
    enableSQSTerminationDraining: true
    managedASGTag: "aws-node-termination-handler/managed"
    prometheusEnable: true
```

##### Queue Processor Mode

{{ kops_feature_table(kops_added_default='1.21') }}

If `enableSQSTerminationDraining` is true Node Termination Handler will operate in Queue Processor mode. In addition to the events mentioned above, Queue Processor mode allows Node Termination Handler to take care of ASG Scale-In, AZ-Rebalance, Unhealthy Instances, EC2 Instance Termination via the API or Console, and more. kOps will provision the necessary infrastructure: an SQS queue, EventBridge rules, and ASG Lifecycle hooks. `managedASGTag` can be configured with Queue Processor mode to distinguish resource ownership between multiple clusters.

The kOps CLI requires additional IAM permissions to manage the requisite EventBridge rules and SQS queue:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "events:DeleteRule",
        "events:ListRules",
        "events:ListTargetsByRule",
        "events:ListTagsForResource",
        "events:PutEvents",
        "events:PutRule",
        "events:PutTargets",
        "events:RemoveTargets",
        "events:TagResource",
        "sqs:CreateQueue",
        "sqs:DeleteQueue",
        "sqs:GetQueueAttributes",
        "sqs:ListQueues",
        "sqs:ListQueueTags"
      ],
      "Resource": "*"
    }
  ]
}
```

**Warning:** If you switch between the two operating modes on an existing cluster, the old resources have to be manually deleted. For IMDS to Queue Processor, this means deleting the k8s nth daemonset. For Queue Processor to IMDS, this means deleting the Kubernetes NTH deployment and the AWS resources: the SQS queue, EventBridge rules, and ASG Lifecycle hooks.

#### Node Problem Detector

{{ kops_feature_table(kops_added_default='1.22') }}

[Node Problem Detector](https://github.com/kubernetes/node-problem-detector) aims to make various node problems visible to the upstream layers in the cluster management stack. It is a daemon that runs on each node, detects node problems and reports them to apiserver.

```yaml
spec:
  nodeProblemDetector:
    enabled: true
    memoryRequest: 32Mi
    cpuRequest: 10m
```

#### Pod Identity Webhook

{{ kops_feature_table(kops_added_default='1.23') }}

When using [IAM roles for Service Accounts](/cluster_spec/#service-account-issuer-discovery-and-aws-iam-roles-for-service-accounts-irsa) (IRSA), Pods require an additinal token to authenticate with the AWS API. In addition, the SDK requires specific environment variables set to make use of these tokens. This addon will mutate Pods configured to use IRSA so that users do not need to do this themselves.

All ServiceAccounts configured with AWS privileges in the Cluster spec will automatically be mutated to assume the configured role.

```yaml
spec:
  certManager:
    enabled: true
  podIdentityWebhook:
    enabled: true
```

The EKS annotations on ServiceAccounts are typically not necessary as kOps will configure the webhook with all ServiceAccount to role mapping configured in the Cluster spec. But if you need specific configuration, you may annotate the ServiceAccount, overriding the kOps configuration.

Read more about Pod Identity Webhook in the [official documentation](https://github.com/aws/amazon-eks-pod-identity-webhook).

#### Snapshot controller

{{ kops_feature_table(kops_added_default='1.21', k8s_min='1.20') }}

Snapshot controller implements the [volume snapshot features](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) of the Container Storage Interface (CSI).

You can enable the snapshot controller by adding the following to the cluster spec:

```yaml
spec:
  snapshotController:
    enabled: true
```

Note that the in-tree volume drivers do not support this feature. If you are running a cluster on AWS, you can enable the EBS CSI driver by adding the following:

```yaml
spec:
  cloudConfig:
    awsEBSCSIDriver:
      enabled: true
```

##### Self-managed aws-ebs-csi-driver

{{ kops_feature_table(kops_added_default='1.25') }}

The following configuration allows for a self-managed aws-ebs-csi-driver. Please note that if you're using Amazon EBS volumes, you must install the Amazon EBS CSI driver. If the Amazon EBS CSI plugin is not installed, then volume operations will fail.

If IRSA is not enabled, the control plane will have the permissions to provision nodes, and the self-managed controllers should run on the control plane. If IRSA is enabled, kOps will create the respective AWS IAM Role, assign the policy, and establish a trust relationship allowing the ServiceAccount to assume the IAM Role. To configure Pods to assume the given IAM roles, enable the [Pod Identity Webhook](https://kops.sigs.k8s.io/addons/#pod-identity-webhook). Without this webhook, you need to modify your Pod specs yourself for your Pod to assume the defined roles.

```yaml
spec:
  cloudConfig:
    awsEBSCSIDriver:
      enabled: true
      managed: false
```

## Custom addons

The command `kops create cluster` does not support specifying addons to be added to the cluster when it is created. Instead they can be added after cluster creation using kubectl. Alternatively when creating a cluster from a yaml manifest, addons can be specified using `spec.addons`.

```yaml
spec:
  addons:
  - manifest: s3://my-kops-addons/addon.yaml
```

The docs about the [addon management](contributing/addons.md#addon-management) describe in more detail how to define a addon resource with regards to versioning.
Here is a minimal example of an addon manifest that would install two different addons.

```yaml
kind: Addons
metadata:
  name: example
spec:
  addons:
  - name: foo.addons.org.io
    version: 0.0.1
    selector:
      k8s-addon: foo.addons.org.io
    manifest: foo.addons.org.io/v0.0.1.yaml
  - name: bar.addons.org.io
    version: 0.0.1
    selector:
      k8s-addon: bar.addons.org.io
    manifest: bar.addons.org.io/v0.0.1.yaml
```

In this example the folder structure should look like this;

```
addon.yaml
  foo.addons.org.io
    v0.0.1.yaml
  bar.addons.org.io
    v0.0.1.yaml
```

The yaml files in the foo/bar folders can be any kubernetes resource. Typically this file structure would be pushed to S3 or another of the supported backends and then referenced as above in `spec.addons`. In order for master nodes to be able to access the S3 bucket containing the addon manifests, one might have to add additional iam policies to the master nodes using `spec.additionalPolicies`, like so:

```yaml
spec:
  additionalPolicies:
    master: |
      [
        {
          "Effect": "Allow",
          "Action": [
            "s3:GetObject"
          ],
          "Resource": ["arn:aws:s3:::my-kops-addons/*"]
        },
        {
          "Effect": "Allow",
          "Action": [
            "s3:GetBucketLocation",
            "s3:ListBucket"
          ],
          "Resource": ["arn:aws:s3:::my-kops-addons"]
        }
      ]
```
The masters will poll for changes in the bucket and keep the addons up to date.


text:
/**
* @author Adam Œmigielski
* @file test.cpp
**/

#include "PCH.hpp"

#include <Unit_Tests\UnitTests.hpp>

#include "Float.hpp"

#include <cstring>

UNIT_TEST(Float16_constants)
{
    TEST_ASSERT(true, Math::float16::m_Has_sign);

    /* Sign */
    TEST_ASSERT(0x8000, Math::float16::Sign_bit::m_Bit);
    TEST_ASSERT(0x7fff, Math::float16::Sign_bit::m_Bit_inv);
    TEST_ASSERT(0x8000, Math::float16::Sign_bit::m_MSE_bits);
    TEST_ASSERT(0x7fff, Math::float16::Sign_bit::m_MSE_inv_bits);
    TEST_ASSERT(0xffff, Math::float16::Sign_bit::m_LSE_bits);
    TEST_ASSERT(0x0000, Math::float16::Sign_bit::m_LSE_inv_bits);

    /* Exponent */
    TEST_ASSERT(0xffff, Math::float16::Exponent_mask::Masks::m_All);
    TEST_ASSERT(0x0001, Math::float16::Exponent_mask::Masks::m_One);
    TEST_ASSERT(0x0000, Math::float16::Exponent_mask::Masks::m_None);

    TEST_ASSERT(0x4000, Math::float16::Exponent_mask::MS_bit::m_Bit);
    TEST_ASSERT(0xbfff, Math::float16::Exponent_mask::MS_bit::m_Bit_inv);
    TEST_ASSERT(0xc000, Math::float16::Exponent_mask::MS_bit::m_MSE_bits);
    TEST_ASSERT(0x3fff, Math::float16::Exponent_mask::MS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x7fff, Math::float16::Exponent_mask::MS_bit::m_LSE_bits);
    TEST_ASSERT(0x8000, Math::float16::Exponent_mask::MS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x0400, Math::float16::Exponent_mask::LS_bit::m_Bit);
    TEST_ASSERT(0xfbff, Math::float16::Exponent_mask::LS_bit::m_Bit_inv);
    TEST_ASSERT(0xfc00, Math::float16::Exponent_mask::LS_bit::m_MSE_bits);
    TEST_ASSERT(0x03ff, Math::float16::Exponent_mask::LS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x07ff, Math::float16::Exponent_mask::LS_bit::m_LSE_bits);
    TEST_ASSERT(0xf800, Math::float16::Exponent_mask::LS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x7c00, Math::float16::Exponent_mask::m_Mask);
    TEST_ASSERT(0x001f, Math::float16::Exponent_mask::m_Mask_value);

    /* Significand */
    TEST_ASSERT(0xffff, Math::float16::Significand_mask::Masks::m_All);
    TEST_ASSERT(0x0001, Math::float16::Significand_mask::Masks::m_One);
    TEST_ASSERT(0x0000, Math::float16::Significand_mask::Masks::m_None);

    TEST_ASSERT(0x0200, Math::float16::Significand_mask::MS_bit::m_Bit);
    TEST_ASSERT(0xfdff, Math::float16::Significand_mask::MS_bit::m_Bit_inv);
    TEST_ASSERT(0xfe00, Math::float16::Significand_mask::MS_bit::m_MSE_bits);
    TEST_ASSERT(0x01ff, Math::float16::Significand_mask::MS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x03ff, Math::float16::Significand_mask::MS_bit::m_LSE_bits);
    TEST_ASSERT(0xfc00, Math::float16::Significand_mask::MS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x0001, Math::float16::Significand_mask::LS_bit::m_Bit);
    TEST_ASSERT(0xfffe, Math::float16::Significand_mask::LS_bit::m_Bit_inv);
    TEST_ASSERT(0xffff, Math::float16::Significand_mask::LS_bit::m_MSE_bits);
    TEST_ASSERT(0x0000, Math::float16::Significand_mask::LS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x0001, Math::float16::Significand_mask::LS_bit::m_LSE_bits);
    TEST_ASSERT(0xfffe, Math::float16::Significand_mask::LS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x03ff, Math::float16::Significand_mask::m_Mask);
    TEST_ASSERT(0x03ff, Math::float16::Significand_mask::m_Mask_value);

    return Passed;
}

UNIT_TEST(Float32_constants)
{
    TEST_ASSERT(true, Math::float32::m_Has_sign);

    /* Sign */
    TEST_ASSERT(0x80000000, Math::float32::Sign_bit::m_Bit);
    TEST_ASSERT(0x7fffffff, Math::float32::Sign_bit::m_Bit_inv);
    TEST_ASSERT(0x80000000, Math::float32::Sign_bit::m_MSE_bits);
    TEST_ASSERT(0x7fffffff, Math::float32::Sign_bit::m_MSE_inv_bits);
    TEST_ASSERT(0xffffffff, Math::float32::Sign_bit::m_LSE_bits);
    TEST_ASSERT(0x00000000, Math::float32::Sign_bit::m_LSE_inv_bits);

    /* Exponent */
    TEST_ASSERT(0xffffffff, Math::float32::Exponent_mask::Masks::m_All);
    TEST_ASSERT(0x00000001, Math::float32::Exponent_mask::Masks::m_One);
    TEST_ASSERT(0x00000000, Math::float32::Exponent_mask::Masks::m_None);

    TEST_ASSERT(0x40000000, Math::float32::Exponent_mask::MS_bit::m_Bit);
    TEST_ASSERT(0xbfffffff, Math::float32::Exponent_mask::MS_bit::m_Bit_inv);
    TEST_ASSERT(0xc0000000, Math::float32::Exponent_mask::MS_bit::m_MSE_bits);
    TEST_ASSERT(0x3fffffff, Math::float32::Exponent_mask::MS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x7fffffff, Math::float32::Exponent_mask::MS_bit::m_LSE_bits);
    TEST_ASSERT(0x80000000, Math::float32::Exponent_mask::MS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x00800000, Math::float32::Exponent_mask::LS_bit::m_Bit);
    TEST_ASSERT(0xff7fffff, Math::float32::Exponent_mask::LS_bit::m_Bit_inv);
    TEST_ASSERT(0xff800000, Math::float32::Exponent_mask::LS_bit::m_MSE_bits);
    TEST_ASSERT(0x007fffff, Math::float32::Exponent_mask::LS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x00ffffff, Math::float32::Exponent_mask::LS_bit::m_LSE_bits);
    TEST_ASSERT(0xff000000, Math::float32::Exponent_mask::LS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x7f800000, Math::float32::Exponent_mask::m_Mask);
    TEST_ASSERT(0x000000ff, Math::float32::Exponent_mask::m_Mask_value);

    /* Significand */
    TEST_ASSERT(0xffffffff, Math::float32::Significand_mask::Masks::m_All);
    TEST_ASSERT(0x00000001, Math::float32::Significand_mask::Masks::m_One);
    TEST_ASSERT(0x00000000, Math::float32::Significand_mask::Masks::m_None);

    TEST_ASSERT(0x00400000, Math::float32::Significand_mask::MS_bit::m_Bit);
    TEST_ASSERT(0xffbfffff, Math::float32::Significand_mask::MS_bit::m_Bit_inv);
    TEST_ASSERT(0xffc00000, Math::float32::Significand_mask::MS_bit::m_MSE_bits);
    TEST_ASSERT(0x003fffff, Math::float32::Significand_mask::MS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x007fffff, Math::float32::Significand_mask::MS_bit::m_LSE_bits);
    TEST_ASSERT(0xff800000, Math::float32::Significand_mask::MS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x00000001, Math::float32::Significand_mask::LS_bit::m_Bit);
    TEST_ASSERT(0xfffffffe, Math::float32::Significand_mask::LS_bit::m_Bit_inv);
    TEST_ASSERT(0xffffffff, Math::float32::Significand_mask::LS_bit::m_MSE_bits);
    TEST_ASSERT(0x00000000, Math::float32::Significand_mask::LS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x00000001, Math::float32::Significand_mask::LS_bit::m_LSE_bits);
    TEST_ASSERT(0xfffffffe, Math::float32::Significand_mask::LS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x007fffff, Math::float32::Significand_mask::m_Mask);
    TEST_ASSERT(0x007fffff, Math::float32::Significand_mask::m_Mask_value);

    return Passed;
}

UNIT_TEST(Float64_constants)
{
    TEST_ASSERT(true, Math::float64::m_Has_sign);

    /* Sign */
    TEST_ASSERT(0x8000000000000000, Math::float64::Sign_bit::m_Bit);
    TEST_ASSERT(0x7fffffffffffffff, Math::float64::Sign_bit::m_Bit_inv);
    TEST_ASSERT(0x8000000000000000, Math::float64::Sign_bit::m_MSE_bits);
    TEST_ASSERT(0x7fffffffffffffff, Math::float64::Sign_bit::m_MSE_inv_bits);
    TEST_ASSERT(0xffffffffffffffff, Math::float64::Sign_bit::m_LSE_bits);
    TEST_ASSERT(0x0000000000000000, Math::float64::Sign_bit::m_LSE_inv_bits);

    /* Exponent */
    TEST_ASSERT(0xffffffffffffffff, Math::float64::Exponent_mask::Masks::m_All);
    TEST_ASSERT(0x0000000000000001, Math::float64::Exponent_mask::Masks::m_One);
    TEST_ASSERT(0x0000000000000000, Math::float64::Exponent_mask::Masks::m_None);

    TEST_ASSERT(0x4000000000000000, Math::float64::Exponent_mask::MS_bit::m_Bit);
    TEST_ASSERT(0xbfffffffffffffff, Math::float64::Exponent_mask::MS_bit::m_Bit_inv);
    TEST_ASSERT(0xc000000000000000, Math::float64::Exponent_mask::MS_bit::m_MSE_bits);
    TEST_ASSERT(0x3fffffffffffffff, Math::float64::Exponent_mask::MS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x7fffffffffffffff, Math::float64::Exponent_mask::MS_bit::m_LSE_bits);
    TEST_ASSERT(0x8000000000000000, Math::float64::Exponent_mask::MS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x0010000000000000, Math::float64::Exponent_mask::LS_bit::m_Bit);
    TEST_ASSERT(0xffefffffffffffff, Math::float64::Exponent_mask::LS_bit::m_Bit_inv);
    TEST_ASSERT(0xfff0000000000000, Math::float64::Exponent_mask::LS_bit::m_MSE_bits);
    TEST_ASSERT(0x000fffffffffffff, Math::float64::Exponent_mask::LS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x001fffffffffffff, Math::float64::Exponent_mask::LS_bit::m_LSE_bits);
    TEST_ASSERT(0xffe0000000000000, Math::float64::Exponent_mask::LS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x7ff0000000000000, Math::float64::Exponent_mask::m_Mask);
    TEST_ASSERT(0x00000000000007ff, Math::float64::Exponent_mask::m_Mask_value);

    /* Significand */
    TEST_ASSERT(0xffffffffffffffff, Math::float64::Significand_mask::Masks::m_All);
    TEST_ASSERT(0x0000000000000001, Math::float64::Significand_mask::Masks::m_One);
    TEST_ASSERT(0x0000000000000000, Math::float64::Significand_mask::Masks::m_None);

    TEST_ASSERT(0x0008000000000000, Math::float64::Significand_mask::MS_bit::m_Bit);
    TEST_ASSERT(0xfff7ffffffffffff, Math::float64::Significand_mask::MS_bit::m_Bit_inv);
    TEST_ASSERT(0xfff8000000000000, Math::float64::Significand_mask::MS_bit::m_MSE_bits);
    TEST_ASSERT(0x0007ffffffffffff, Math::float64::Significand_mask::MS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x000fffffffffffff, Math::float64::Significand_mask::MS_bit::m_LSE_bits);
    TEST_ASSERT(0xfff0000000000000, Math::float64::Significand_mask::MS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x0000000000000001, Math::float64::Significand_mask::LS_bit::m_Bit);
    TEST_ASSERT(0xfffffffffffffffe, Math::float64::Significand_mask::LS_bit::m_Bit_inv);
    TEST_ASSERT(0xffffffffffffffff, Math::float64::Significand_mask::LS_bit::m_MSE_bits);
    TEST_ASSERT(0x0000000000000000, Math::float64::Significand_mask::LS_bit::m_MSE_inv_bits);
    TEST_ASSERT(0x0000000000000001, Math::float64::Significand_mask::LS_bit::m_LSE_bits);
    TEST_ASSERT(0xfffffffffffffffe, Math::float64::Significand_mask::LS_bit::m_LSE_inv_bits);

    TEST_ASSERT(0x000fffffffffffff, Math::float64::Significand_mask::m_Mask);
    TEST_ASSERT(0x000fffffffffffff, Math::float64::Significand_mask::m_Mask_value);

    return Passed;
}

UNIT_TEST(Float_usage)
{
    static const double values[] =
    {
        123.456,
        0.0,
        1.0,
        -0.0,
        -1.0
    };
    static const size_t n_values = sizeof(values) / sizeof(values[0]);

    for (size_t i = 0; i < n_values; ++i)
    {
        const float val = float(values[i]);
        const Math::float32 f(values[i]);

        TEST_ASSERT(0, memcmp(&val, &(f.m_Raw), sizeof(float)));

        const double d_val(val);
        const double d_f(f.Value());

        TEST_ASSERT(d_val, d_f);
    }

    return Passed;
}


text:
'use strict';

var DataProvider = function() {
	this.dbProxyRoot = 'db';
	this.database = 'materials';
};

DataProvider.prototype._buildUrl = function() {
	return ['/', this.dbProxyRoot, '/', this.database].join('');
};

DataProvider.prototype._makeRequest = function(data, type, url) {
	url = url || this._buildUrl();
	console.log({
		url: url,
		contentType: 'application/json',
		data: data,
		dataType: 'json',
		type: type
	});
	if (['post', 'put'].indexOf(type.toLowerCase()) !== -1) {
		data = JSON.stringify(data);
	}
	return $.ajax({
		url: url,
		contentType: 'application/json',
		data: data,
		dataType: 'json',
		type: type
	});
};

DataProvider.prototype.create = function(data) {
	return this._makeRequest(data, 'POST');
};

DataProvider.prototype.update = function(data) {
	var url = [this._buildUrl(), data._id].join('/');
	return this._makeRequest(data, 'PUT', url);
};

DataProvider.prototype.fetchView = function(viewName, params) {
	var url = [this._buildUrl(), '/_design/materials/_view/', viewName].join('');
	return this._makeRequest(params, 'GET', url);
};