text:
 .. Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at

 ..   http://www.apache.org/licenses/LICENSE-2.0

 .. Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.

FAQ
========

Why isn't my task getting scheduled?
------------------------------------

There are very many reasons why your task might not be getting scheduled.
Here are some of the common causes:

- Does your script "compile", can the Airflow engine parse it and find your
  DAG object? To test this, you can run ``airflow dags list`` and
  confirm that your DAG shows up in the list. You can also run
  ``airflow tasks list foo_dag_id --tree`` and confirm that your task
  shows up in the list as expected. If you use the CeleryExecutor, you
  may want to confirm that this works both where the scheduler runs as well
  as where the worker runs.

- Does the file containing your DAG contain the string "airflow" and "DAG" somewhere
  in the contents? When searching the DAG directory, Airflow ignores files not containing
  "airflow" and "DAG" in order to prevent the DagBag parsing from importing all python
  files collocated with user's DAGs.

- Is your ``start_date`` set properly? The Airflow scheduler triggers the
  task soon after the ``start_date + schedule_interval`` is passed.

- Is your ``schedule_interval`` set properly? The default ``schedule_interval``
  is one day (``datetime.timedelta(1)``). You must specify a different ``schedule_interval``
  directly to the DAG object you instantiate, not as a ``default_param``, as task instances
  do not override their parent DAG's ``schedule_interval``.

- Is your ``start_date`` beyond where you can see it in the UI? If you
  set your ``start_date`` to some time say 3 months ago, you won't be able to see
  it in the main view in the UI, but you should be able to see it in the
  ``Menu -> Browse ->Task Instances``.

- Are the dependencies for the task met? The task instances directly
  upstream from the task need to be in a ``success`` state. Also,
  if you have set ``depends_on_past=True``, the previous task instance
  needs to have succeeded (except if it is the first run for that task).
  Also, if ``wait_for_downstream=True``, make sure you understand
  what it means - all tasks *immediately* downstream of the *previous*
  task instance must have succeeded.
  You can view how these properties are set from the ``Task Instance Details``
  page for your task.

- Are the DagRuns you need created and active? A DagRun represents a specific
  execution of an entire DAG and has a state (running, success, failed, ...).
  The scheduler creates new DagRun as it moves forward, but never goes back
  in time to create new ones. The scheduler only evaluates ``running`` DagRuns
  to see what task instances it can trigger. Note that clearing tasks
  instances (from the UI or CLI) does set the state of a DagRun back to
  running. You can bulk view the list of DagRuns and alter states by clicking
  on the schedule tag for a DAG.

- Is the ``concurrency`` parameter of your DAG reached? ``concurrency`` defines
  how many ``running`` task instances a DAG is allowed to have, beyond which
  point things get queued.

- Is the ``max_active_runs`` parameter of your DAG reached? ``max_active_runs`` defines
  how many ``running`` concurrent instances of a DAG there are allowed to be.

You may also want to read the Scheduler section of the docs and make
sure you fully understand how it proceeds.

How do I trigger tasks based on another task's failure?
-------------------------------------------------------

Check out the :ref:`concepts/trigger_rule`.

What's the deal with ``start_date``?
------------------------------------

``start_date`` is partly legacy from the pre-DagRun era, but it is still
relevant in many ways. When creating a new DAG, you probably want to set
a global ``start_date`` for your tasks using ``default_args``. The first
DagRun to be created will be based on the ``min(start_date)`` for all your
task. From that point on, the scheduler creates new DagRuns based on
your ``schedule_interval`` and the corresponding task instances run as your
dependencies are met. When introducing new tasks to your DAG, you need to
pay special attention to ``start_date``, and may want to reactivate
inactive DagRuns to get the new task onboarded properly.

We recommend against using dynamic values as ``start_date``, especially
``datetime.now()`` as it can be quite confusing. The task is triggered
once the period closes, and in theory an ``@hourly`` DAG would never get to
an hour after now as ``now()`` moves along.

Previously, we also recommended using rounded ``start_date`` in relation to your
``schedule_interval``. This meant an ``@hourly`` would be at ``00:00``
minutes:seconds, a ``@daily`` job at midnight, a ``@monthly`` job on the
first of the month. This is no longer required. Airflow will now auto align
the ``start_date`` and the ``schedule_interval``, by using the ``start_date``
as the moment to start looking.

You can use any sensor or a ``TimeDeltaSensor`` to delay
the execution of tasks within the schedule interval.
While ``schedule_interval`` does allow specifying a ``datetime.timedelta``
object, we recommend using the macros or cron expressions instead, as
it enforces this idea of rounded schedules.

When using ``depends_on_past=True``, it's important to pay special attention
to ``start_date``, as the past dependency is not enforced only on the specific
schedule of the ``start_date`` specified for the task. It's also
important to watch DagRun activity status in time when introducing
new ``depends_on_past=True``, unless you are planning on running a backfill
for the new task(s).

It is also important to note that the task's ``start_date``, in the context of a
backfill CLI command, gets overridden by the backfill's ``start_date`` commands.
This allows for a backfill on tasks that have ``depends_on_past=True`` to
actually start. If this were not the case, the backfill just would not start.

How can I create DAGs dynamically?
----------------------------------

Airflow looks in your ``DAGS_FOLDER`` for modules that contain ``DAG`` objects
in their global namespace and adds the objects it finds in the
``DagBag``. Knowing this all, we need is a way to dynamically assign
variable in the global namespace. This is easily done in python using the
``globals()`` function for the standard library, which behaves like a
simple dictionary.

.. code:: python

    def create_dag(dag_id):
        """
        A function returning a DAG object.
        """

        return DAG(dag_id)

    for i in range(10):
        dag_id = f'foo_{i}'
        globals()[dag_id] = DAG(dag_id)

        # or better, call a function that returns a DAG object!
        other_dag_id = f'bar_{i}'
        globals()[other_dag_id] = create_dag(other_dag_id)

What are all the ``airflow tasks run`` commands in my process list?
-------------------------------------------------------------------

There are many layers of ``airflow tasks run`` commands, meaning it can call itself.

- Basic ``airflow tasks run``: fires up an executor, and tell it to run an
  ``airflow tasks run --local`` command. If using Celery, this means it puts a
  command in the queue for it to run remotely on the worker. If using
  LocalExecutor, that translates into running it in a subprocess pool.
- Local ``airflow tasks run --local``: starts an ``airflow tasks run --raw``
  command (described below) as a subprocess and is in charge of
  emitting heartbeats, listening for external kill signals
  and ensures some cleanup takes place if the subprocess fails.
- Raw ``airflow tasks run --raw`` runs the actual operator's execute method and
  performs the actual work.

How can my airflow dag run faster?
----------------------------------

There are a few variables we can control to improve airflow dag performance:

- ``parallelism``: This variable controls the number of task instances that runs simultaneously across the whole Airflow cluster. User could increase the ``parallelism`` variable in the ``airflow.cfg``.
- ``concurrency``: The Airflow scheduler will run no more than ``concurrency`` task instances for your DAG at any given time. Concurrency is defined in your Airflow DAG. If you do not set the concurrency on your DAG, the scheduler will use the default value from the ``dag_concurrency`` entry in your ``airflow.cfg``.
- ``task_concurrency``: This variable controls the number of concurrent running task instances across ``dag_runs`` per task.
- ``max_active_runs``: the Airflow scheduler will run no more than ``max_active_runs`` DagRuns of your DAG at a given time. If you do not set the ``max_active_runs`` in your DAG, the scheduler will use the default value from the ``max_active_runs_per_dag`` entry in your ``airflow.cfg``.
- ``pool``: This variable controls the number of concurrent running task instances assigned to the pool.

How can we reduce the airflow UI page load time?
------------------------------------------------

If your dag takes long time to load, you could reduce the value of ``default_dag_run_display_number`` configuration in ``airflow.cfg`` to a smaller value. This configurable controls the number of dag run to show in UI with default value 25.

How to fix Exception: Global variable explicit_defaults_for_timestamp needs to be on (1)?
-----------------------------------------------------------------------------------------

This means ``explicit_defaults_for_timestamp`` is disabled in your mysql server and you need to enable it by:

#. Set ``explicit_defaults_for_timestamp = 1`` under the ``mysqld`` section in your ``my.cnf`` file.
#. Restart the Mysql server.

How to reduce airflow dag scheduling latency in production?
-----------------------------------------------------------

- ``max_threads``: Scheduler will spawn multiple threads in parallel to schedule dags. This is controlled by ``max_threads`` with default value of 2. User should increase this value to a larger value (e.g numbers of cpus where scheduler runs - 1) in production.
- ``scheduler_heartbeat_sec``: User should consider to increase ``scheduler_heartbeat_sec`` config to a higher value (e.g 60 secs) which controls how frequent the airflow scheduler gets the heartbeat and updates the job's entry in database.

Why next_ds or prev_ds might not contain expected values?
---------------------------------------------------------

- When scheduling DAG, the ``next_ds`` ``next_ds_nodash`` ``prev_ds`` ``prev_ds_nodash`` are calculated using
  ``execution_date`` and ``schedule_interval``. If you set ``schedule_interval`` as ``None`` or ``@once``,
  the ``next_ds``, ``next_ds_nodash``, ``prev_ds``, ``prev_ds_nodash`` valueS will be set to ``None``.
- When manually triggering DAG, the schedule will be ignored, and ``prev_ds == next_ds == ds``

How do I stop the sync perms happening multiple times per webserver?
--------------------------------------------------------------------

Set the value of ``update_fab_perms`` configuration in ``airflow.cfg`` to ``False``.


text:
require File.expand_path(File.dirname(__FILE__) + '/../test_config.rb')

describe "BlogComments Model" do
  it 'can construct a new instance' do
    @blog_comments = BlogComments.new
    refute_nil @blog_comments
  end
end

redpajama_set_name: RedPajamaGithub


text:
using System;

namespace _02.AddTwoNumbers
{
    class Program
    {
        static void Main(string[] args)
        {
            int a = int.Parse(Console.ReadLine());
            int b = int.Parse(Console.ReadLine());
            int sum = a + b;
            Console.WriteLine($"{a} + {b} = {sum}");
        }
    }
}


text:
/**
 *
 */
package com.rng.admin.web.controllers;

import com.rng.common.services.JCLogger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.MessageSource;
import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.web.bind.annotation.ModelAttribute;

import com.rng.admin.security.AuthenticatedUser;

public abstract class AdminBaseController
{
	protected final JCLogger logger = JCLogger.getLogger(getClass());

	@Autowired protected MessageSource messageSource;

	protected abstract String getHeaderTitle();

	public String getMessage(String code)
	{
		return messageSource.getMessage(code, null, null);
	}

	public String getMessage(String code, String defaultMsg)
	{
		return messageSource.getMessage(code, null, defaultMsg, null);
	}

	@ModelAttribute("authenticatedUser")
    public AuthenticatedUser authenticatedUser(@AuthenticationPrincipal AuthenticatedUser authenticatedUser)
    {
        return authenticatedUser;
    }

	public static AuthenticatedUser getCurrentUser() {

	    Object principal = SecurityContextHolder.getContext().getAuthentication().getPrincipal();
	    if (principal instanceof AuthenticatedUser) {
	    	return ((AuthenticatedUser) principal);
	    }
	    // principal object is either null or represents anonymous user -
	    // neither of which our domain User object can represent - so return null
	    return null;
	}

	public static boolean isLoggedIn() {
	    return getCurrentUser() != null;
	}
}


text:
import logging
import sys

from ncclient import manager

def connect(host, port, user, password):
    conn = manager.connect(host=host,
                           port=port,
                           username=user,
                           password=password,
                           timeout=60,
                           device_params={'name': 'junos'},
                           hostkey_verify=False)

    conn.lock()

    new_host_name = 'foo-bar'

    # configuration as a text string
    location = """
    system {
        location {
            building "Main Campus, E";
            floor 15;
            rack 1117;
        }
    }
    """

    load_config_result = conn.load_configuration(format='text', config=location)
    logging.info(load_config_result)

    # configuration as an argument
    load_config_result = conn.load_configuration(format='text', config="""
    system {
        host-name %s;
    }
    """ % new_host_name)
    logging.info(load_config_result)

    validate_result = conn.validate()
    logging.info(validate_result)

    compare_config_result = conn.compare_configuration()
    logging.info(compare_config_result)

    conn.commit()
    conn.unlock()
    conn.close_session()

if __name__ == '__main__':
    LOG_FORMAT = '%(asctime)s %(levelname)s %(filename)s:%(lineno)d %(message)s'
    logging.basicConfig(stream=sys.stdout, level=logging.INFO, format=LOG_FORMAT)

    connect('router', '22', 'netconf', 'juniper!')


text:
.RadTreeView_OHCHRSkins {
  color: #0066CC!Important;
  font:Verdana, Arial, Helvetica, sans-serif!Important; }
  .RadTreeView_OHCHRSkins .rtHover .rtIn {
    color: #000;
    border-color: #b5b5b5 #b5b5b5 #9e9e9e;
    background-color: #C5C5C5;
    background-image: url('TreeView/ItemHoveredBg.png'); }
  .RadTreeView_OHCHRSkins .rtSelected .rtIn {
    color:#0066CC!important;
	border-color:transparent!important;
	background-color:transparent!important;
	background-image: none!important;
	font: Verdana, Arial, Helvetica, sans-serif!Important; }
  .RadTreeView_OHCHRSkins .rtEdit .rtIn {
    border-color: #6c6c6c;
    background: #ffffff; }
    .RadTreeView_OHCHRSkins .rtEdit .rtIn input {
      color: #333333;
      font:Verdana, Arial, Helvetica, sans-serif;
      color: black;
      border-color: #535353 #2e2e2e #1a1a1a;
      border: 0;
      background-color: #313131;
      background-image: url('TreeView/ItemSelectedBg.png'); }
  .RadTreeView_OHCHRSkins .rtChecked,
  .RadTreeView_OHCHRSkins .rtUnchecked,
  .RadTreeView_OHCHRSkins .rtIndeterminate {
    background-image: url('Common/radFormToggleSprite.png');
    _background-image: url('Common/radFormToggleSpriteIE6.png'); }
  .RadTreeView_OHCHRSkins .rtLoadingIcon {
    background-image: url('Common/loading_small.gif'); }
  .RadTreeView_OHCHRSkins .rtSelected .rtLoadingBefore,
  .RadTreeView_OHCHRSkins .rtSelected .rtLoadingAfter,
  .RadTreeView_OHCHRSkins .rtSelected .rtLoadingBelow {
    color: #333333; }
  .RadTreeView_OHCHRSkins .rtPlus,
  .RadTreeView_OHCHRSkins .rtMinus {
    background-image: url('TreeView/PlusMinus.png'); }

.RadTreeView_OHCHRSkins_disabled .rtIn,
.RadTreeView_OHCHRSkins .rtDisabled .rtIn {
  color: #ccc; }

.RadTreeView a.rtIn {
	color:#0066CC!Important;
	font: Verdana, Arial, Helvetica, sans-serif!Important;

}

.RadTreeView a.rtIn:hover {
    text-decoration:underline;
    color:#FF0000!Important;
	font: Verdana, Arial, Helvetica, sans-serif!Important;
}


text:
namespace ConditionalBuild
{
  partial class frmMain
  {
    /// <summary>
    /// Required designer variable.
    /// </summary>
    private System.ComponentModel.IContainer components = null;

    /// <summary>
    /// Clean up any resources being used.
    /// </summary>
    /// <param name="disposing">true if managed resources should be disposed; otherwise, false.</param>
    protected override void Dispose(bool disposing)
    {
      if (disposing && (components != null))
      {
        components.Dispose();
      }
      base.Dispose(disposing);
    }

    #region Windows Form Designer generated code

    /// <summary>
    /// Required method for Designer support - do not modify
    /// the contents of this method with the code editor.
    /// </summary>
    private void InitializeComponent()
    {
      this.lblMessage = new System.Windows.Forms.Label();
      this.SuspendLayout();
      //
      // lblMessage
      //
      this.lblMessage.AutoSize = true;
      this.lblMessage.Location = new System.Drawing.Point(12, 9);
      this.lblMessage.Name = "lblMessage";
      this.lblMessage.Size = new System.Drawing.Size(89, 13);
      this.lblMessage.TabIndex = 0;
      this.lblMessage.Text = "This is not a build";
      //
      // frmMain
      //
      this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
      this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
      this.ClientSize = new System.Drawing.Size(284, 262);
      this.Controls.Add(this.lblMessage);
      this.Name = "frmMain";
      this.Text = "ConditionalBuild";
      this.Load += new System.EventHandler(this.frmMain_Load);
      this.ResumeLayout(false);
      this.PerformLayout();

    }

    #endregion

    private System.Windows.Forms.Label lblMessage;
  }
}


text:
exports.foo = function() {
  return require("b").good();
};

exports.bar = function() {
  return require("c").bad();
};


text:
SQLiteParser
============

A parser for SQLite files to upload them into a MySQL DB


text:
<?php
namespace Smadevo;

use PDOStatement;

/**
 * Encapsulates a database connection.
 */
interface Database
{
    /**
     * Builds and executes a prepared statement.
     *
     * @param string $statement
     * @param array  $parameters
     *
     * @return PDOStatement
     */
    public function execute(string $statement, array $parameters = []): PDOStatement;
}


text:
'use strict';

Object.defineProperty(exports, "__esModule", {
    value: true
});

var _getPrototypeOf = require('babel-runtime/core-js/object/get-prototype-of');

var _getPrototypeOf2 = _interopRequireDefault(_getPrototypeOf);

var _classCallCheck2 = require('babel-runtime/helpers/classCallCheck');

var _classCallCheck3 = _interopRequireDefault(_classCallCheck2);

var _createClass2 = require('babel-runtime/helpers/createClass');

var _createClass3 = _interopRequireDefault(_createClass2);

var _possibleConstructorReturn2 = require('babel-runtime/helpers/possibleConstructorReturn');

var _possibleConstructorReturn3 = _interopRequireDefault(_possibleConstructorReturn2);

var _inherits2 = require('babel-runtime/helpers/inherits');

var _inherits3 = _interopRequireDefault(_inherits2);

var _Statement2 = require('../Statement');

var _Statement3 = _interopRequireDefault(_Statement2);

function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }

var ForOfStatement = function (_Statement) {
    (0, _inherits3.default)(ForOfStatement, _Statement);

    function ForOfStatement(childNodes) {
        (0, _classCallCheck3.default)(this, ForOfStatement);

        var _this = (0, _possibleConstructorReturn3.default)(this, (ForOfStatement.__proto__ || (0, _getPrototypeOf2.default)(ForOfStatement)).call(this, 'ForOfStatement', childNodes));

        _this.each = false;
        return _this;
    }

    (0, _createClass3.default)(ForOfStatement, [{
        key: '_acceptChildren',
        value: function _acceptChildren(children) {
            children.passToken('Keyword', 'for');
            children.skipNonCode();

            children.passToken('Punctuator', '(');
            children.skipNonCode();

            var left = void 0;
            if (children.isNode('VariableDeclaration')) {
                left = children.currentElement;
                children.moveNext();
            } else {
                left = children.passAssignable();
            }
            children.skipNonCode();

            children.passToken('Identifier', 'of');
            children.skipNonCode();

            var right = children.passExpression();
            children.skipNonCode();

            children.passToken('Punctuator', ')');
            children.skipNonCode();

            var body = children.passStatement();

            children.assertEnd();

            this.left = left;
            this.right = right;
            this.body = body;
        }
    }]);
    return ForOfStatement;
}(_Statement3.default);

exports.default = ForOfStatement;
//# sourceMappingURL=ForOfStatement.js.map


text:
package org.intellij.lang.xpath.xslt.impl;

import com.intellij.ide.FileIconPatcher;
import com.intellij.openapi.project.Project;
import com.intellij.openapi.vfs.VirtualFile;
import com.intellij.psi.PsiFile;
import com.intellij.psi.PsiManager;
import org.intellij.lang.xpath.xslt.XsltSupport;

import javax.swing.*;

public class XsltIconProvider implements FileIconPatcher {

  @Override
  public Icon patchIcon(Icon baseIcon, VirtualFile file, int flags, Project project) {
    if (project == null) return baseIcon;

    final PsiFile psiFile = PsiManager.getInstance(project).findFile(file);
    if (psiFile != null && XsltSupport.isXsltFile(psiFile)) {
      return XsltSupport.createXsltIcon(baseIcon);
    }
    return baseIcon;
  }

}


text:
module SeriesWorkDecorator
  def db_detail_link(options = {})
    name = options.delete(:name).presence || id
    link_to(name, db_edit_series_work_path(self), options)
  end

  def to_values
    self.class::DIFF_FIELDS.each_with_object({}) do |field, hash|
      hash[field] = case field
      when :work_id
        path = work_path(work_id: work.id)
        link_to(work.local_title, path, target: "_blank")
      else
        send(field)
      end

      hash
    end
  end
end


text:
==========================
Ganeti daemons refactoring
==========================

:Created: 2013-Sep-27
:Status: Implemented
:Ganeti-Version: 2.12.0

.. contents:: :depth: 2

This is a design document detailing the plan for refactoring the internal
structure of Ganeti, and particularly the set of daemons it is divided into.

Current state and shortcomings
==============================

Ganeti is comprised of a growing number of daemons, each dealing with part of
the tasks the cluster has to face, and communicating with the other daemons
using a variety of protocols.

Specifically, as of Ganeti 2.8, the situation is as follows:

``Master daemon (MasterD)``
  It is responsible for managing the entire cluster, and it's written in Python.
  It is executed on a single node (the master node). It receives the commands
  given by the cluster administrator (through the remote API daemon or the
  command line tools) over the LUXI protocol.  The master daemon is responsible
  for creating and managing the jobs that will execute such commands, and for
  managing the locks that ensure the cluster will not incur in race conditions.

  Each job is managed by a separate Python thread, that interacts with the node
  daemons via RPC calls.

  The master daemon is also responsible for managing the configuration of the
  cluster, changing it when required by some job. It is also responsible for
  copying the configuration to the other master candidates after updating it.

``RAPI daemon (RapiD)``
  It is written in Python and runs on the master node only. It waits for
  requests issued remotely through the remote API protocol. Then, it forwards
  them, using the LUXI protocol, to the master daemon (if they are commands) or
  to the query daemon if they are queries about the configuration (including
  live status) of the cluster.

``Node daemon (NodeD)``
  It is written in Python. It runs on all the nodes. It is responsible for
  receiving the master requests over RPC and execute them, using the appropriate
  backend (hypervisors, DRBD, LVM, etc.). It also receives requests over RPC for
  the execution of queries gathering live data on behalf of the query daemon.

``Configuration daemon (ConfD)``
  It is written in Haskell. It runs on all the master candidates. Since the
  configuration is replicated only on the master node, this daemon exists in
  order to provide information about the configuration to nodes needing them.
  The requests are done through ConfD's own protocol, HMAC signed,
  implemented over UDP, and meant to be used by parallely querying all the
  master candidates (or a subset thereof) and getting the most up to date
  answer. This is meant as a way to provide a robust service even in case master
  is temporarily unavailable.

``Query daemon (QueryD)``
  It is written in Haskell. It runs on all the master candidates. It replies
  to Luxi queries about the current status of the system, including live data it
  obtains by querying the node daemons through RPCs.

``Monitoring daemon (MonD)``
  It is written in Haskell. It runs on all nodes, including the ones that are
  not vm-capable. It is meant to provide information on the status of the
  system. Such information is related only to the specific node the daemon is
  running on, and it is provided as JSON encoded data over HTTP, to be easily
  readable by external tools.
  The monitoring daemon communicates with ConfD to get information about the
  configuration of the cluster. The choice of communicating with ConfD instead
  of MasterD allows it to obtain configuration information even when the cluster
  is heavily degraded (e.g.: when master and some, but not all, of the master
  candidates are unreachable).

The current structure of the Ganeti daemons is inefficient because there are
many different protocols involved, and each daemon needs to be able to use
multiple ones, and has to deal with doing different things, thus making
sometimes unclear which daemon is responsible for performing a specific task.

Also, with the current configuration, jobs are managed by the master daemon
using python threads. This makes terminating a job after it has started a
difficult operation, and it is the main reason why this is not possible yet.

The master daemon currently has too many different tasks, that could be handled
better if split among different daemons.

Proposed changes
================

In order to improve on the current situation, a new daemon subdivision is
proposed, and presented hereafter.

.. digraph:: "new-daemons-structure"

  {rank=same; RConfD LuxiD;}
  {rank=same; Jobs rconfigdata;}
  node [shape=box]
  RapiD [label="RapiD [M]"]
  LuxiD [label="LuxiD [M]"]
  WConfD [label="WConfD [M]"]
  Jobs [label="Jobs [M]"]
  RConfD [label="RConfD [MC]"]
  MonD [label="MonD [All]"]
  NodeD [label="NodeD [All]"]
  Clients [label="gnt-*\nclients [M]"]
  p1 [shape=none, label=""]
  p2 [shape=none, label=""]
  p3 [shape=none, label=""]
  p4 [shape=none, label=""]
  configdata [shape=none, label="config.data"]
  rconfigdata [shape=none, label="config.data\n[MC copy]"]
  locksdata [shape=none, label="locks.data"]

  RapiD -> LuxiD [label="LUXI"]
  LuxiD -> WConfD [label="WConfD\nproto"]
  LuxiD -> Jobs [label="fork/exec"]
  Jobs -> WConfD [label="WConfD\nproto"]
  Jobs -> NodeD [label="RPC"]
  LuxiD -> NodeD [label="RPC"]
  rconfigdata -> RConfD
  configdata -> rconfigdata [label="sync via\nNodeD RPC"]
  WConfD -> NodeD [label="RPC"]
  WConfD -> configdata
  WConfD -> locksdata
  MonD -> RConfD [label="RConfD\nproto"]
  Clients -> LuxiD [label="LUXI"]
  p1 -> MonD [label="MonD proto"]
  p2 -> RapiD [label="RAPI"]
  p3 -> RConfD [label="RConfD\nproto"]
  p4 -> Clients [label="CLI"]

``LUXI daemon (LuxiD)``
  It will be written in Haskell. It will run on the master node and it will be
  the only LUXI server, replying to all the LUXI queries. These includes both
  the queries about the live configuration of the cluster, previously served by
  QueryD, and the commands actually changing the status of the cluster by
  submitting jobs. Therefore, this daemon will also be the one responsible with
  managing the job queue. When a job needs to be executed, the LuxiD will spawn
  a separate process tasked with the execution of that specific job, thus making
  it easier to terminate the job itself, if needed.  When a job requires locks,
  LuxiD will request them from WConfD.
  In order to keep availability of the cluster in case of failure of the master
  node, LuxiD will replicate the job queue to the other master candidates, by
  RPCs to the NodeD running there (the choice of RPCs for this task might be
  reviewed at a second time, after implementing this design).

``Configuration management daemon (WConfD)``
  It will run on the master node and it will be responsible for the management
  of the authoritative copy of the cluster configuration (that is, it will be
  the daemon actually modifying the ``config.data`` file). All the requests of
  configuration changes will have to pass through this daemon, and will be
  performed using a LUXI-like protocol ("WConfD proto" in the graph. The exact
  protocol will be defined in the separate design document that will detail the
  WConfD separation).  Having a single point of configuration management will
  also allow Ganeti to get rid of possible race conditions due to concurrent
  modifications of the configuration.  When the configuration is updated, it
  will have to push the received changes to the other master candidates, via
  RPCs, so that RConfD daemons and (in case of a failure on the master node)
  the WConfD daemon on the new master can access an up-to-date version of it
  (the choice of RPCs for this task might be reviewed at a second time). This
  daemon will also be the one responsible for managing the locks, granting them
  to the jobs requesting them, and taking care of freeing them up if the jobs
  holding them crash or are terminated before releasing them.  In order to do
  this, each job, after being spawned by LuxiD, will open a local unix socket
  that will be used to communicate with it, and will be destroyed when the job
  terminates.  LuxiD will be able to check, after a timeout, whether the job is
  still running by connecting here, and to ask WConfD to forcefully remove the
  locks if the socket is closed.
  Also, WConfD should hold a serialized list of the locks and their owners in a
  file (``locks.data``), so that it can keep track of their status in case it
  crashes and needs to be restarted (by asking LuxiD which of them are still
  running).
  Interaction with this daemon will be performed using Unix sockets.

``Configuration query daemon (RConfD)``
  It is written in Haskell, and it corresponds to the old ConfD. It will run on
  all the master candidates and it will serve information about the static
  configuration of the cluster (the one contained in ``config.data``). The
  provided information will be highly available (as in: a response will be
  available as long as a stable-enough connection between the client and at
  least one working master candidate is available) and its freshness will be
  best effort (the most recent reply from any of the master candidates will be
  returned, but it might still be older than the one available through WConfD).
  The information will be served through the ConfD protocol.

``Rapi daemon (RapiD)``
  It remains basically unchanged, with the only difference that all of its LUXI
  query are directed towards LuxiD instead of being split between MasterD and
  QueryD.

``Monitoring daemon (MonD)``
  It remains unaffected by the changes in this design document. It will just get
  some of the data it needs from RConfD instead of the old ConfD, but the
  interfaces of the two are identical.

``Node daemon (NodeD)``
  It remains unaffected by the changes proposed in the design document. The only
  difference being that it will receive its RPCs from LuxiD (for job queue
  replication), from WConfD (for configuration replication) and for the
  processes executing single jobs (for all the operations to be performed by
  nodes) instead of receiving them just from MasterD.

This restructuring will allow us to reorganize and improve the codebase,
introducing cleaner interfaces and giving well defined and more restricted tasks
to each daemon.

Furthermore, having more well-defined interfaces will allow us to have easier
upgrade procedures, and to work towards the possibility of upgrading single
components of a cluster one at a time, without the need for immediately
upgrading the entire cluster in a single step.

Implementation
==============

While performing this refactoring, we aim to increase the amount of
Haskell code, thus benefiting from the additional type safety provided by its
wide compile-time checks. In particular, all the job queue management and the
configuration management daemon will be written in Haskell, taking over the role
currently fulfilled by Python code executed as part of MasterD.

The changes describe by this design document are quite extensive, therefore they
will not be implemented all at the same time, but through a sequence of steps,
leaving the codebase in a consistent and usable state.

#. Rename QueryD to LuxiD.
   A part of LuxiD, the one replying to configuration
   queries including live information about the system, already exists in the
   form of QueryD. This is being renamed to LuxiD, and will form the first part
   of the new daemon. NB: this is happening starting from Ganeti 2.8. At the
   beginning, only the already existing queries will be replied to by LuxiD.
   More queries will be implemented in the next versions.

#. Let LuxiD be the interface for the queries and MasterD be their executor.
   Currently, MasterD is the only responsible for receiving and executing LUXI
   queries, and for managing the jobs they create.
   Receiving the queries and managing the job queue will be extracted from
   MasterD into LuxiD.
   Actually executing jobs will still be done by MasterD, that contains all the
   logic for doing that and for properly managing locks and the configuration.
   At this stage, scheduling will simply consist in starting jobs until a fixed
   maximum number of simultaneously running jobs is reached.

#. Extract WConfD from MasterD.
   The logic for managing the configuration file is factored out to the
   dedicated WConfD daemon. All configuration changes, currently executed
   directly by MasterD, will be changed to be IPC requests sent to the new
   daemon.

#. Extract locking management from MasterD.
   The logic for managing and granting locks is extracted to WConfD as well.
   Locks will not be taken directly anymore, but asked via IPC to WConfD.
   This step can be executed on its own or at the same time as the previous one.

#. Jobs are executed as processes.
   The logic for running jobs is rewritten so that each job can be managed by an
   independent process. LuxiD will spawn a new (Python) process for every single
   job. The RPCs will remain unchanged, and the LU code will stay as is as much
   as possible.
   MasterD will cease to exist as a daemon on its own at this point, but not
   before.

#. Improve job scheduling algorithm.
   The simple algorithm for scheduling jobs will be replaced by a more
   intelligent one. Also, the implementation of :doc:`design-optables` can be
   started.

Job death detection
-------------------

**Requirements:**

- It must be possible to reliably detect a death of a process even under
  uncommon conditions such as very heavy system load.
- A daemon must be able to detect a death of a process even if the
  daemon is restarted while the process is running.
- The solution must not rely on being able to communicate with
  a process.
- The solution must work for the current situation where multiple jobs
  run in a single process.
- It must be POSIX compliant.

These conditions rule out simple solutions like checking a process ID
(because the process might be eventually replaced by another process
with the same ID) or keeping an open connection to a process.

**Solution:** As a job process is spawned, before attempting to
communicate with any other process, it will create a designated empty
lock file, open it, acquire an *exclusive* lock on it, and keep it open.
When connecting to a daemon, the job process will provide it with the
path of the file. If the process dies unexpectedly, the operating system
kernel automatically cleans up the lock.

Therefore, daemons can check if a process is dead by trying to acquire
a *shared* lock on the lock file in a non-blocking mode:

- If the locking operation succeeds, it means that the exclusive lock is
  missing, therefore the process has died, but the lock
  file hasn't been cleaned up yet. The daemon should release the lock
  immediately. Optionally, the daemon may delete the lock file.
- If the file is missing, the process has died and the lock file has
  been cleaned up.
- If the locking operation fails due to a lock conflict, it means
  the process is alive.

Using shared locks for querying lock files ensures that the detection
works correctly even if multiple daemons query a file at the same time.

A job should close and remove its lock file when completely finishes.
The WConfD daemon will be responsible for removing stale lock files of
jobs that didn't remove its lock files themselves.

**Statelessness of the protocol:** To keep our protocols stateless,
the job id and the path the to lock file are sent as part of every
request that deals with resources, in particular the Ganeti Locks.
All resources are owned by the pair (job id, lock file). In this way,
several jobs can live in the same process (as it will be in the
transition period), but owner death detection still only depends on the
owner of the resource. In particular, no additional lookup table is
needed to obtain the lock file for a given owner.

**Considered alternatives:** An alternative to creating a separate lock
file would be to lock the job status file. However, file locks are kept
only as long as the file is open. Therefore any operation followed by
closing the file would cause the process to release the lock. In
particular, with jobs as threads, the master daemon wouldn't be able to
keep locks and operate on job files at the same time.

Job execution
-------------

As the Luxi daemon will be responsible for executing jobs, it needs to
start jobs in such a way that it can properly detect if the job dies
under any circumstances (such as Luxi daemon being restarted in the
process).

The name of the lock file will be stored in the corresponding job file
so that anybody is able to check the status of the process corresponding
to a job.

The proposed procedure:

#. The Luxi daemon saves the name of its own lock file into the job file.
#. The Luxi daemon forks, creating a bi-directional pipe with the child
   process.
#. The child process creates and locks its own, proper lock file and
   handles its name to the Luxi daemon through the pipe.
#. The Luxi daemon saves the name of the lock file into the job file and
   confirms it to the child process.
#. Only then the child process can replace itself by the actual job
   process.

If the child process detects that the pipe is broken before receiving the
confirmation, it must terminate, not starting the actual job.
This way, the actual job is only started if it is ensured that its lock
file name is written to the job file.

If the Luxi daemon detects that the pipe is broken before successfully
sending the confirmation in step 4., it assumes that the job has failed.
If the pipe gets broken after sending the confirmation, no further
action is necessary. If the child doesn't receive the confirmation,
it will die and its death will be detected by Luxid eventually.

If the Luxi daemon dies before completing the procedure, the job will
not be started. If the job file contained the daemon's lock file name,
it will be detected as dead (because the daemon process died). If the
job file already contained its proper lock file, it will also be
detected as dead (because the child process won't start the actual job
and die).

WConfD details
--------------

WConfD will communicate with its clients through a Unix domain socket for both
configuration management and locking. Clients can issue multiple RPC calls
through one socket. For each such a call the client sends a JSON request
document with a remote function name and data for its arguments. The server
replies with a JSON response document containing either the result of
signalling a failure.

Any state associated with client processes will be mirrored on persistent
storage and linked to the identity of processes so that the WConfD daemon will
be able to resume its operation at any point after a restart or a crash. WConfD
will track each client's process start time along with its process ID to be
able detect if a process dies and it's process ID is reused.  WConfD will clear
all locks and other state associated with a client if it detects it's process
no longer exists.

Configuration management
++++++++++++++++++++++++

The new configuration management protocol will be implemented in the following
steps:

Step 1:
  #. Implement the following functions in WConfD and export them through
     RPC:

     - Obtain a single internal lock, either in shared or
       exclusive mode. This lock will substitute the current lock
       ``_config_lock`` in config.py.
     - Release the lock.
     - Return the whole configuration data to a client.
     - Receive the whole configuration data from a client and replace the
       current configuration with it. Distribute it to master candidates
       and distribute the corresponding *ssconf*.

     WConfD must detect deaths of its clients (see `Job death
     detection`_) and release locks automatically.

  #. In config.py modify public methods that access configuration:

     - Instead of acquiring a local lock, obtain a lock from WConfD
       using the above functions
     - Fetch the current configuration from WConfD.
     - Use it to perform the method's task.
     - If the configuration was modified, send it to WConfD at the end.
     - Release the lock to WConfD.

  This will decouple the configuration management from the master daemon,
  even though the specific configuration tasks will still performed by
  individual jobs.

  After this step it'll be possible access the configuration from separate
  processes.

Step 2:
  #. Reimplement all current methods of ``ConfigWriter`` for reading and
     writing the configuration of a cluster in WConfD.
  #. Expose each of those functions in WConfD as a separate RPC function.
     This will allow easy future extensions or modifications.
  #. Replace ``ConfigWriter`` with a stub (preferably automatically
     generated from the Haskell code) that will contain the same methods
     as the current ``ConfigWriter`` and delegate all calls to its
     methods to WConfD.

Step 3:
  In a later step, the impact of the config lock will be reduced by moving
  it more and more into an internal detail of WConfD. This forthcoming process
  of :doc:`design-configlock` is described separately.

Locking
+++++++

The new locking protocol will be implemented as follows:

Re-implement the current locking mechanism in WConfD and expose it for RPC
calls. All current locks will be mapped into a data structure that will
uniquely identify them (storing lock's level together with it's name).

WConfD will impose a linear order on locks. The order will be compatible
with the current ordering of lock levels so that existing code will work
without changes.

WConfD will keep the set of currently held locks for each client. The
protocol will allow the following operations on the set:

*Update:*
  Update the current set of locks according to a given list. The list contains
  locks and their desired level (release / shared / exclusive). To prevent
  deadlocks, WConfD will check that all newly requested locks (or already held
  locks requested to be upgraded to *exclusive*) are greater in the sense of
  the linear order than all currently held locks, and fail the operation if
  not. Only the locks in the list will be updated, other locks already held
  will be left intact. If the operation fails, the client's lock set will be
  left intact.
*Opportunistic union:*
  Add as much as possible locks from a given set to the current set within a
  given timeout. WConfD will again check the proper order of locks and
  acquire only the ones that are allowed wrt. the current set.  Returns the
  set of acquired locks, possibly empty. Immediate. Never fails. (It would also
  be possible to extend the operation to try to wait until a given number of
  locks is available, or a given timeout elapses.)
*List:*
  List the current set of held locks. Immediate, never fails.
*Intersection:*
  Retain only a given set of locks in the current one. This function is
  provided for convenience, it's redundant wrt. *list* and *update*. Immediate,
  never fails.

Additional restrictions due to lock implications:
  Ganeti supports locks that act as if a lock on a whole group (like all nodes)
  were held. To avoid dead locks caused by the additional blockage of those
  group locks, we impose certain restrictions. Whenever `A` is a group lock and
  `B` belongs to `A`, then the following holds.

  - `A` is in lock order before `B`.
  - All locks that are in the lock order between `A` and `B` also belong to `A`.
  - It is considered a lock-order violation to ask for an exclusive lock on `B`
    while holding a shared lock on `A`.

After this step it'll be possible to use locks from jobs as separate processes.

The above set of operations allows the clients to use various work-flows. In particular:

Pessimistic strategy:
  Lock all potentially relevant resources (for example all nodes), determine
  which will be needed, and release all the others.
Optimistic strategy:
  Determine what locks need to be acquired without holding any. Lock the
  required set of locks. Determine the set of required locks again and check if
  they are all held. If not, release everything and restart.

.. COMMENTED OUT:
  Start with the smallest set of locks and when determining what more
  relevant resources will be needed, expand the set. If an *union* operation
  fails, release all locks, acquire the desired union and restart the
  operation so that all preconditions and possible concurrent changes are
  checked again.

Future aims:

-  Add more fine-grained locks to prevent unnecessary blocking of jobs. This
   could include locks on parameters of entities or locks on their states (so that
   a node remains online, but otherwise can change, etc.). In particular,
   adding, moving and removing instances currently blocks the whole node.
-  Add checks that all modified configuration parameters belong to entities
   the client has locked and log violations.
-  Make the above checks mandatory.
-  Automate optimistic locking and checking the locks in logical units.
   For example, this could be accomplished by allowing some of the initial
   phases of `LogicalUnit` (such as `ExpandNames` and `DeclareLocks`) to be run
   repeatedly, checking if the set of locks requested the second time is
   contained in the set acquired after the first pass.
-  Add the possibility for a job to reserve hardware resources such as disk
   space or memory on nodes. Most likely as a new, special kind of instances
   that would only block its resources and allow to be converted to a regular
   instance. This would allow long-running jobs such as instance creation or
   move to lock the corresponding nodes, acquire the resources and turn the
   locks into shared ones, keeping an exclusive lock only on the instance.
-  Use more sophisticated algorithm for preventing deadlocks such as a
   `wait-for graph`_. This would allow less *union* failures and allow more
   optimistic, scalable acquisition of locks.

.. _`wait-for graph`: http://en.wikipedia.org/wiki/Wait-for_graph

Further considerations
======================

There is a possibility that a job will finish performing its task while LuxiD
and/or WConfD will not be available.
In order to deal with this situation, each job will update its job file
in the queue. This is race free, as LuxiD will no longer touch the job file,
once the job is started; a corollary of this is that the job also has to
take care of replicating updates to the job file. LuxiD will watch job files for
changes to determine when a job was cleanly finished. To determine jobs
that died without having the chance of updating the job file, the `Job death
detection`_ mechanism will be used.

.. vim: set textwidth=72 :
.. Local Variables:
.. mode: rst
.. fill-column: 72
.. End:


text:
/* eslint-disable */
/* Remove the eslint-disable when you remove the .skip and refactor this to fit with jest 20 */
import { updateFieldValue, trimField } from '../form-field-updaters';

describe.skip('updateFieldValue', () => {
  beforeEach(() => {
    // Simple implementation of our setState
    const safeSetState = function(updater) {
      if (typeof updater !== 'function') {
        const newState = Object.assign({}, updater);
        updater = () => newState;
      }
      this.state = Object.assign({}, this.state, updater(this.state));
    };

    this.setupTestEnvironment = initialState => {
      if (!initialState) {
        throw new Error('setupTestEnvironment needs an initialState');
      }

      // Setup the event object for use but expect test to fill in value itself
      this.e = {
        target: {},
      };

      // Create mock component
      const _this = {};
      _this.safeSetState = safeSetState.bind(_this);
      _this.state = initialState;
      return _this;
    };
  });

  // Test the safeSetState we just wrote above
  describe('.test.safeSetState', () => {
    it('handles function argument', () => {
      const _this = this.setupTestEnvironment({ a: 1 });

      _this.safeSetState(prevState => ({ a: prevState.a + 1 }));
      expect(_this.state).toEqual({ a: 2 });

      _this.safeSetState(prevState => ({ a: 5 }));
      expect(_this.state).toEqual({ a: 5 });

      _this.safeSetState(prevState => ({ a: 17, b: 3 }));
      expect(_this.state).toEqual({ a: 17, b: 3 });

      _this.safeSetState(prevState => ({ b: 10 }));
      expect(_this.state).toEqual({ a: 17, b: 10 });
    });

    it('handles object argument', () => {
      const _this = this.setupTestEnvironment({ a: 1 });

      _this.safeSetState({ a: 5 });
      expect(_this.state).toEqual({ a: 5 });

      _this.safeSetState({ a: 17, b: 3 });
      expect(_this.state).toEqual({ a: 17, b: 3 });

      _this.safeSetState({ b: 10 });
      expect(_this.state).toEqual({ a: 17, b: 10 });
    });
  });

  it('updates state with simple keys', () => {
    const _this = this.setupTestEnvironment({ a: 'initial' });
    const e = this.e;
    const key = 'a';
    const boundUpdate = updateFieldValue.bind(_this, key, undefined);

    e.target.value = 'test';
    boundUpdate(e);
    expect(_this.state).toEqual({ a: 'test' });
  });

  it('updates state with complex keys', () => {
    const initialState = {
      form: {
        subset: {
          keyA: 'initial',
        },
      },
    };
    const targetState = {
      form: {
        subset: {
          keyA: 'test',
        },
      },
    };

    const _this = this.setupTestEnvironment(initialState);
    const e = this.e;
    const key = 'form.subset.keyA';
    const boundUpdate = updateFieldValue.bind(_this, key, undefined);

    e.target.value = 'test';
    boundUpdate(e);
    expect(_this.state).toEqual(targetState);
  });

  it("doesn't delete other keys in nested objects", () => {
    const initialState = {
      form: {
        subset: {
          keyA: 'initial',
          keyB: 'dummyB',
          keyC: 'dummyC',
        },
      },
    };
    const targetState = {
      form: {
        subset: {
          keyA: 'test',
          keyB: 'dummyB',
          keyC: 'dummyC',
        },
      },
    };

    const _this = this.setupTestEnvironment(initialState);
    const e = this.e;
    const key = 'form.subset.keyA';
    const boundUpdate = updateFieldValue.bind(_this, key, undefined);

    e.target.value = 'test';
    boundUpdate(e);
    expect(_this.state).toEqual(targetState);
  });

  it('handles sequential updates', () => {
    const _this = this.setupTestEnvironment({ a: 'initial' });
    const e = this.e;
    const key = 'a';
    const boundUpdate = updateFieldValue.bind(_this, key, undefined);

    e.target.value = 'first';
    boundUpdate(e);
    expect(_this.state).toEqual({ a: 'first' });

    e.target.value = 'second';
    boundUpdate(e);
    expect(_this.state).toEqual({ a: 'second' });
  });

  it('throws error correctly relating to keypath', () => {
    const _this = this.setupTestEnvironment({ a: 'initial' });
    const e = this.e;
    e.target.value = 'test';

    const key1 = 'b';
    const boundUpdate1 = updateFieldValue.bind(_this, key1, undefined);
    expect(boundUpdate1.bind(null, e)).not.toThrow();

    _this.state = { a: 'initial' };
    const key2 = 'a.b';
    const boundUpdate2 = updateFieldValue.bind(_this, key2, undefined);
    expect(boundUpdate2.bind(null, e)).toThrow();

    _this.state = { a: 'initial' };
    const key3 = 'b.c';
    const boundUpdate3 = updateFieldValue.bind(_this, key3, undefined);
    expect(boundUpdate3.bind(null, e)).toThrow();
  });

  it('throws error on invalid option', () => {
    const _this = this.setupTestEnvironment({ a: 'initial' });
    const e = this.e;
    const key = 'a';
    const boundUpdate = updateFieldValue.bind(_this, key);

    const options = {
      invalidOption: 'test',
    };
    expect(boundUpdate.bind(null, options, e)).toThrow();

    // Check when there is also a valid one
    options.isMaterialSelect = true;
    expect(boundUpdate.bind(null, options, e)).toThrow();
  });

  it('handles Material Select element correctly', () => {
    const _this = this.setupTestEnvironment({ a: 'initial' });
    const e = this.e;
    const key = 'a';
    const options = {
      isMaterialSelect: true,
    };
    const boundUpdate = updateFieldValue.bind(_this, key, options);

    e.target.value = 'wrong';
    const value = 'right';
    boundUpdate(e, null, value);
    expect(_this.state).toEqual({ a: 'right' });
  });

  it('handles trim option', () => {
    const _this = this.setupTestEnvironment({ a: 'initial' });
    const e = this.e;
    const key = 'a';
    const options = {
      trim: 5,
    };
    const boundUpdate = updateFieldValue.bind(_this, key, options);

    e.target.value = 'hello this is a test';
    boundUpdate(e);
    expect(_this.state).toEqual({ a: 'hello' });
  });
});

describe('trimField', () => {
  // 20 characters long value
  const value = 'hello this is a test';

  it('trims field', () => {
    expect(trimField(5, value)).toBe('hello');

    expect(trimField(25, value)).toBe(value);
  });

  it('handles empty/null values', () => {
    expect(trimField(5, '')).toBe('');

    expect(trimField(5, null)).toBe('');

    expect(trimField(5, undefined)).toBe('');
  });
});


text:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE locale SYSTEM "../../../../../lib/pkp/dtd/locale.dtd">

<!--
  * plugins/reports/subscriptions/locale/nl_NL/locale.xml
  *
  * Copyright (c) 2003-2013 John Willinsky
  * Distributed under the GNU GPL v2. For full terms see the file docs/COPYING.
  *
  * Localization strings.
  *
  -->

<locale name="nl_NL" full_name="Nederlands">
	<message key="plugins.reports.subscriptions.displayName">Abonnementen rapport</message>
	<message key="plugins.reports.subscriptions.description">Deze plugin levert een CSV rapport met een lijst abonnementen en bijbehorende informatie</message>
	<message key="plugins.reports.subscriptions.institutionMailingAddress">Postadres instituut</message>
	<message key="plugins.reports.subscriptions.ipRanges">IP reeksen</message>
</locale>


text:
'use strict';

const common = require('../common');
const events = require('events');
const assert = require('assert');

const symbol = Symbol('symbol');

const e = new events.EventEmitter();
e.setMaxListeners(1);

process.on('warning', common.mustCall((warning) => {
  assert.ok(warning instanceof Error);
  assert.strictEqual(warning.name, 'MaxListenersExceededWarning');
  assert.strictEqual(warning.emitter, e);
  assert.strictEqual(warning.count, 2);
  assert.strictEqual(warning.type, symbol);
  assert.ok(warning.message.includes(
    '2 Symbol(symbol) listeners added to [EventEmitter].'));
}));

e.on(symbol, () => {});
e.on(symbol, () => {});


text:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<title>CompressionStatistics (yui-compressor-ant-task)</title>
<meta name="date" content="2017-03-03">
<link rel="stylesheet" type="text/css" href="../../../../../../stylesheet.css" title="Style">

JavaScript is disabled on your browser.

Skip navigation links

Package

Class

Use

Tree

Deprecated

Index

Help

Prev Class

Next Class

Frames

No Frames

All Classes

Summary:

Nested |

Field |

Constr |

Method

Detail:

Field |

Constr |

Method

net.noha.tools.ant.yuicompressor.tasks

Class CompressionStatistics

java.lang.Object

net.noha.tools.ant.yuicompressor.tasks.CompressionStatistics

public class CompressionStatistics
extends java.lang.Object

Author:

Viktor Lieskovsky

Field Summary

Fields

Modifier and Type
Field and Description

private long
cssFileCount

private long
cssFromBytes

private long
cssToBytes

private long
fileCount

private long
fromBytes

private long
jsFileCount

private long
jsFromBytes

private long
jsToBytes

private long
toBytes

Constructor Summary

Constructors

Constructor and Description

CompressionStatistics()

Method Summary

All Methods Instance Methods Concrete Methods

Modifier and Type
Method and Description

java.lang.String
getCssStats()

java.lang.String
getFileStats(java.io.File inFile,
            java.io.File outFile,
            java.lang.String fileType)

java.lang.String
getJsStats()

java.lang.String
getTotalStats()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fromBytes

private long fromBytes

toBytes

private long toBytes

fileCount

private long fileCount

jsFromBytes

private long jsFromBytes

jsToBytes

private long jsToBytes

jsFileCount

private long jsFileCount

cssFromBytes

private long cssFromBytes

cssToBytes

private long cssToBytes

cssFileCount

private long cssFileCount

Constructor Detail

CompressionStatistics

public CompressionStatistics()

Method Detail

getFileStats

public java.lang.String getFileStats(java.io.File inFile,
                                     java.io.File outFile,
                                     java.lang.String fileType)

getTotalStats

public java.lang.String getTotalStats()

getJsStats

public java.lang.String getJsStats()

getCssStats

public java.lang.String getCssStats()

Skip navigation links

Package

Class

Use

Tree

Deprecated

Index

Help

Prev Class

Next Class

Frames

No Frames

All Classes

Summary:

Nested |

Field |

Constr |

Method

Detail:

Field |

Constr |

Method


text:
#include"mem/cache/ldsp/ldsprefetcher.hh"

std::string LDSPrefetcher::name()
{
     return cpu->name() + ".LDSPrefetcher";
}
void  LDSPrefetcher::setPattern_Table(Pattern_Table *pattern_table)
{
     _pattern_table=pattern_table;
}
void  LDSPrefetcher::setTRB(TRB *trb)
{
     _trb=trb;
}
void  LDSPrefetcher::setBQ(BQ * bq)
{
     _bq=bq;
}
void  LDSPrefetcher::setDirection_Predictor(Dire_Pred * dire_pred)
{
     _dire_pred=dire_pred;
}
void  LDSPrefetcher::Regstats()
{
     LDSpref_senttoL2
        .name(name() + ".pref_senttoL2")
        .desc("Number of prefs sent to L2 cache")
        .prereq(ldspref_senttoL2);

     LDSpref_useful
        .name(name() + ".pref_useful")
        .desc("Number of prefs useful")
        .prereq(LDSpref_useful);

     LDSpref_LateToUse
        .name(name() + ".pref_LateToUse")
        .desc("Number of prefs late To Use")
        .prereq(LDSpref_LateToUse);

     LDSDemandMissCausedByPref
        .name(name() + ".DemandMissCausedByPref")
        .desc("Number of DemandMiss Caused By Prefetch")
        .prereq(LDSDemandMissCausedByPref);

     LDSTotalDemandMiss
        .name(name() + ".TotalDemandMiss")
        .desc("Number of Total DemandMiss")
        .prereq(LDSTotalDemandMiss);

     LDSPref_BQFullcycles
        .name(name() + ".Pref_BQFullcycles")
        .desc("Number of cycles that BQ is Full!")
        .prereq(LDSPref_BQFullcycles);

     LDSPref_Squashcycles
        .name(name() + ".pref_Squashcycles")
        .desc("Number  of cycles that lds_pref is Squashed!")
        .prereq(LDSPref_Squashcycles);

}

void  LDSPrefetcher::drainSanityCheck() const
{
    /*
      assert(_pattern_table->pt_line.empty());
      assert(_bq->_BQ.empty());
    */

}

void   LDSPrefetcher::isdrained() const
{

}

void  LDSPrefetcher::takeoverfrom()
{

}


text:
require "megaphone/version"
require "megaphone/engine"
require "megaphone/notifiable"
require "megaphone/table_prefix"


text:
<!DOCTYPE html>

    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
    <title>Mesh face material</title>
    <meta name="description" content="">
    <meta name="keywords" content="">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0, maximum-scale=1.0">


text:
'use strict';

var windowHeight = window.innerHeight,
    windowHeightExtra = 0;
var safari = /^((?!chrome|android).)*safari/i.test(navigator.userAgent),
    mobile = /Mobi/.test(navigator.userAgent);

// check if safari - extend height
if (safari && !mobile) {
	windowHeightExtra = window.outerHeight - window.innerHeight;
}

if (mobile) {
	windowHeight = window.screen.availHeight; // stops from jumping
	windowHeightExtra = (window.screen.availHeight - window.innerHeight) / 2; // prevents white spaces
}

// position parallax
var positionParallax = function positionParallax(container, speed, parallax, elem) {
	var bgScroll = container.top / speed - windowHeightExtra;
	parallax[elem].dongwon.top = bgScroll + 'px';
};

// animate parallax
var animateParallax = function animateParallax(parallax, speed) {
	for (var i = 0; parallax.length > i; i++) {
		var container = parallax[i].parentElement.parentElement.getBoundingClientRect();

		// only animate if on screen
		if (container.top + container.height >= 0 && container.top <= windowHeight) {
			positionParallax(container, speed, parallax, i);
		}
	}
};

// determine height
var calculateHeight = function calculateHeight(parallax, speed) {
	for (var i = 0; parallax.length > i; i++) {
		var container = parallax[i].parentElement.parentElement.getBoundingClientRect();
		var containerTop = parallax[i].parentElement.parentElement.offsetTop;
		var elemOffsetTop = (windowHeight - container.height) / 2;

		// set bgHeight & check if it needs to stretch beyond container bottom
		var bgHeight = windowHeight > container.height + containerTop ? container.height + containerTop - containerTop / speed : container.height + (elemOffsetTop - elemOffsetTop / speed) * 2;

		parallax[i].dongwon.height = bgHeight + windowHeightExtra * 2 + 'px';
		positionParallax(container, speed, parallax, i);
	}
};

var universalParallax = function universalParallax() {
	var up = function up(parallax, speed) {

		// check that speed is not a negative number
		if (speed < 1) {
			speed = 1;
		}

		// set height on elements
		calculateHeight(parallax, speed);

		// recalculate height on resize
		if (!mobile) {
			window.addEventListener('resize', function () {
				windowHeight = window.innerHeight;
				calculateHeight(parallax, speed);
			});
		}

		// Add scroll event listener
		window.addEventListener('scroll', function () {
			animateParallax(parallax, speed);
		});
	};

	// Ready all elements
	this.init = function (param) {
		if (typeof param === 'undefined') {
			param = {};
		}

		param = {
			speed: typeof param.speed !== 'undefined' ? param.speed : 1.5,
			className: typeof param.className !== 'undefined' ? param.className : 'parallax'
		};
		var parallax = document.getElementsByClassName(param.className);

		for (var i = 0; parallax.length > i; i++) {
			// make container div
			var wrapper = document.createElement('div');
			parallax[i].parentNode.insertBefore(wrapper, parallax[i]);
			wrapper.appendChild(parallax[i]);
			var parallaxContainer = parallax[i].parentElement;
			parallaxContainer.className += 'parallax__container';

			// parent elem need position: relative for effect to work - if not already defined, add it
			if (window.getComputedStyle(parallaxContainer.parentElement, null).getPropertyValue('position') !== 'relative') {
				parallaxContainer.parentElement.style.position = 'relative';
			}

			var imgData = parallax[i].dataset.parallaxImage;
			// add image to div if none is specified
			if (typeof imgData !== 'undefined') {
				parallax[i].dongwon.backgroundImage = 'url(' + imgData + ')';
				// if no other class than .parallax is specified, add CSS
				if (parallax[i].classList.length === 1 && parallax[i].classList[0] === 'parallax') {
					parallax[i].dongwon.backgroundRepeat = 'no-repeat';
					parallax[i].dongwon.backgroundPosition = 'center';
					parallax[i].dongwon.backgroundSize = 'cover';
				}
			}
		};

		// when page is loaded && init completed -> run function
		document.addEventListener('readystatechange', function (event) {
			if (event.target.readyState === 'complete') {
				up(parallax, param.speed);
			}
		});
	};
};


text:
package dagger.internal.codegen.xprocessing;

import static com.google.common.base.Preconditions.checkNotNull;
import static dagger.internal.codegen.extension.DaggerStreams.toImmutableList;
import static kotlin.streams.jdk8.StreamsKt.asStream;

import androidx.room.compiler.processing.XHasModifiers;
import androidx.room.compiler.processing.XMethodElement;
import androidx.room.compiler.processing.XTypeElement;
import androidx.room.compiler.processing.XTypeParameterElement;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.squareup.javapoet.TypeVariableName;

// TODO(bcorso): Consider moving these methods into XProcessing library.
/** A utility class for {@link XTypeElement} helper methods. */
public final class XTypeElements {
  private enum Visibility {
    PUBLIC,
    PRIVATE,
    OTHER;

    /** Returns the visibility of the given {@link XTypeElement}. */
    private static Visibility of(XTypeElement element) {
      checkNotNull(element);
      if (element.isPrivate()) {
        return Visibility.PRIVATE;
      } else if (element.isPublic()) {
        return Visibility.PUBLIC;
      } else {
        return Visibility.OTHER;
      }
    }
  }

  // TODO(bcorso): Consider XParameterizable interface to handle both methods and types.
  /** Returns the type arguments for the given type as a list of {@link TypeVariableName}. */
  public static ImmutableList<TypeVariableName> typeVariableNames(XTypeElement typeElement) {
    return typeElement.getTypeParameters().stream()
        .map(XTypeParameterElement::getTypeVariableName)
        .collect(toImmutableList());
  }

  /** Returns {@code true} if the given element is nested. */
  public static boolean isNested(XTypeElement typeElement) {
    return typeElement.getEnclosingTypeElement() != null;
  }

  /** Returns {@code true} if the given {@code type} has type parameters. */
  public static boolean hasTypeParameters(XTypeElement typeElement) {
    return !typeElement.getTypeParameters().isEmpty();
  }

  /** Returns all non-private, non-static, abstract methods in {@code type}. */
  public static ImmutableList<XMethodElement> getAllUnimplementedMethods(XTypeElement type) {
    return getAllNonPrivateInstanceMethods(type).stream()
        .filter(XHasModifiers::isAbstract)
        .collect(toImmutableList());
  }

  /** Returns all non-private, non-static methods in {@code type}. */
  public static ImmutableList<XMethodElement> getAllNonPrivateInstanceMethods(XTypeElement type) {
    return getAllMethods(type).stream()
        .filter(method -> !method.isPrivate() && !method.isStatic())
        .collect(toImmutableList());
  }

  // TODO(b/229784604): This is needed until the XProcessing getAllMethods fix is upstreamed. Due
  // to the existing bug, XTypeElement#getAllMethods() will currently contain some inaccessible
  // package-private methods from base classes, so we filter them manually here.
  public static ImmutableList<XMethodElement> getAllMethods(XTypeElement type) {
    return asStream(type.getAllMethods())
        .filter(method -> isAccessibleFrom(method, type))
        .collect(toImmutableList());
  }

  private static boolean isAccessibleFrom(XMethodElement method, XTypeElement type) {
    if (method.isPublic() || method.isProtected()) {
      return true;
    }
    if (method.isPrivate()) {
      return false;
    }
    return method
        .getClosestMemberContainer()
        .getClassName()
        .packageName()
        .equals(type.getClassName().packageName());
  }

  public static boolean isEffectivelyPublic(XTypeElement element) {
    return allVisibilities(element).stream()
        .allMatch(visibility -> visibility.equals(Visibility.PUBLIC));
  }

  public static boolean isEffectivelyPrivate(XTypeElement element) {
    return allVisibilities(element).contains(Visibility.PRIVATE);
  }

  /**
   * Returns a list of visibilities containing visibility of the given element and the visibility of
   * its enclosing elements.
   */
  private static ImmutableSet<Visibility> allVisibilities(XTypeElement element) {
    checkNotNull(element);
    ImmutableSet.Builder<Visibility> visibilities = ImmutableSet.builder();
    XTypeElement currentElement = element;
    while (currentElement != null) {
      visibilities.add(Visibility.of(currentElement));
      currentElement = currentElement.getEnclosingTypeElement();
    }
    return visibilities.build();
  }

  private XTypeElements() {}
}


text:
#pragma once
#include <aws/machinelearning/MachineLearning_EXPORTS.h>
#include <aws/core/utils/memory/stl/AWSString.h>
#include <aws/core/utils/memory/stl/AWSMap.h>
#include <aws/machinelearning/model/DetailsAttributes.h>
#include <utility>

namespace Aws
{
namespace Utils
{
namespace Json
{
  class JsonValue;
} // namespace Json
} // namespace Utils
namespace MachineLearning
{
namespace Model
{

  /**
   *
The output from a Predict operation:

   * Details - Contains the following attributes:
   * DetailsAttributes.PREDICTIVE_MODEL_TYPE - REGRESSION | BINARY |
   * MULTICLASS DetailsAttributes.ALGORITHM - SGD

   *
 PredictedLabel - Present for either a BINARY or
   * MULTICLASS MLModel request.

   * PredictedScores - Contains the raw classification score
   * corresponding to each label.

 PredictedValue -
   * Present for a REGRESSION MLModel request.

   *

See Also:
   AWS
   * API Reference

   */
  class AWS_MACHINELEARNING_API Prediction
  {
  public:
    Prediction();
    Prediction(const Aws::Utils::Json::JsonValue& jsonValue);
    Prediction& operator=(const Aws::Utils::Json::JsonValue& jsonValue);
    Aws::Utils::Json::JsonValue Jsonize() const;

    /**
     *
The prediction label for either a BINARY or
     * MULTICLASS MLModel.

     */
    inline const Aws::String& GetPredictedLabel() const{ return m_predictedLabel; }

    /**
     *
The prediction label for either a BINARY or
     * MULTICLASS MLModel.

     */
    inline void SetPredictedLabel(const Aws::String& value) { m_predictedLabelHasBeenSet = true; m_predictedLabel = value; }

    /**
     *
The prediction label for either a BINARY or
     * MULTICLASS MLModel.

     */
    inline void SetPredictedLabel(Aws::String&& value) { m_predictedLabelHasBeenSet = true; m_predictedLabel = std::move(value); }

    /**
     *
The prediction label for either a BINARY or
     * MULTICLASS MLModel.

     */
    inline void SetPredictedLabel(const char* value) { m_predictedLabelHasBeenSet = true; m_predictedLabel.assign(value); }

    /**
     *
The prediction label for either a BINARY or
     * MULTICLASS MLModel.

     */
    inline Prediction& WithPredictedLabel(const Aws::String& value) { SetPredictedLabel(value); return *this;}

    /**
     *
The prediction label for either a BINARY or
     * MULTICLASS MLModel.

     */
    inline Prediction& WithPredictedLabel(Aws::String&& value) { SetPredictedLabel(std::move(value)); return *this;}

    /**
     *
The prediction label for either a BINARY or
     * MULTICLASS MLModel.

     */
    inline Prediction& WithPredictedLabel(const char* value) { SetPredictedLabel(value); return *this;}

    /**
     * The prediction value for REGRESSION MLModel.
     */
    inline double GetPredictedValue() const{ return m_predictedValue; }

    /**
     * The prediction value for REGRESSION MLModel.
     */
    inline void SetPredictedValue(double value) { m_predictedValueHasBeenSet = true; m_predictedValue = value; }

    /**
     * The prediction value for REGRESSION MLModel.
     */
    inline Prediction& WithPredictedValue(double value) { SetPredictedValue(value); return *this;}

    inline const Aws::Map<Aws::String, double>& GetPredictedScores() const{ return m_predictedScores; }

    inline void SetPredictedScores(const Aws::Map<Aws::String, double>& value) { m_predictedScoresHasBeenSet = true; m_predictedScores = value; }

    inline void SetPredictedScores(Aws::Map<Aws::String, double>&& value) { m_predictedScoresHasBeenSet = true; m_predictedScores = std::move(value); }

    inline Prediction& WithPredictedScores(const Aws::Map<Aws::String, double>& value) { SetPredictedScores(value); return *this;}

    inline Prediction& WithPredictedScores(Aws::Map<Aws::String, double>&& value) { SetPredictedScores(std::move(value)); return *this;}

    inline Prediction& AddPredictedScores(const Aws::String& key, double value) { m_predictedScoresHasBeenSet = true; m_predictedScores.emplace(key, value); return *this; }

    inline Prediction& AddPredictedScores(Aws::String&& key, double value) { m_predictedScoresHasBeenSet = true; m_predictedScores.emplace(std::move(key), value); return *this; }

    inline Prediction& AddPredictedScores(const char* key, double value) { m_predictedScoresHasBeenSet = true; m_predictedScores.emplace(key, value); return *this; }

    inline const Aws::Map<DetailsAttributes, Aws::String>& GetDetails() const{ return m_details; }

    inline void SetDetails(const Aws::Map<DetailsAttributes, Aws::String>& value) { m_detailsHasBeenSet = true; m_details = value; }

    inline void SetDetails(Aws::Map<DetailsAttributes, Aws::String>&& value) { m_detailsHasBeenSet = true; m_details = std::move(value); }

    inline Prediction& WithDetails(const Aws::Map<DetailsAttributes, Aws::String>& value) { SetDetails(value); return *this;}

    inline Prediction& WithDetails(Aws::Map<DetailsAttributes, Aws::String>&& value) { SetDetails(std::move(value)); return *this;}

    inline Prediction& AddDetails(const DetailsAttributes& key, const Aws::String& value) { m_detailsHasBeenSet = true; m_details.emplace(key, value); return *this; }

    inline Prediction& AddDetails(DetailsAttributes&& key, const Aws::String& value) { m_detailsHasBeenSet = true; m_details.emplace(std::move(key), value); return *this; }

    inline Prediction& AddDetails(const DetailsAttributes& key, Aws::String&& value) { m_detailsHasBeenSet = true; m_details.emplace(key, std::move(value)); return *this; }

    inline Prediction& AddDetails(DetailsAttributes&& key, Aws::String&& value) { m_detailsHasBeenSet = true; m_details.emplace(std::move(key), std::move(value)); return *this; }

    inline Prediction& AddDetails(DetailsAttributes&& key, const char* value) { m_detailsHasBeenSet = true; m_details.emplace(std::move(key), value); return *this; }

    inline Prediction& AddDetails(const DetailsAttributes& key, const char* value) { m_detailsHasBeenSet = true; m_details.emplace(key, value); return *this; }

  private:
    Aws::String m_predictedLabel;
    bool m_predictedLabelHasBeenSet;
    double m_predictedValue;
    bool m_predictedValueHasBeenSet;
    Aws::Map<Aws::String, double> m_predictedScores;
    bool m_predictedScoresHasBeenSet;
    Aws::Map<DetailsAttributes, Aws::String> m_details;
    bool m_detailsHasBeenSet;
  };

} // namespace Model
} // namespace MachineLearning
} // namespace Aws


text:
const $ = require('jquery');
const bannersRotator = require('../com/banners-rotator');

const COUNTRY_CODE = null;
const LINK = null;

$(function () {
  function initBanners() {
    $.ajax({ url: "https://data.services.jetbrains.com/geo" }).done(function (data) {
      if (COUNTRY_CODE && data.code === COUNTRY_CODE) {
        let template = `<a class="link-banner_pl" href="${LINK}"></a>`;
        bannersRotator.createBanner({
          id: "banner-default-country",
          type: bannersRotator.Banner.TYPE.slideup,
          width: 'auto',
          height: 'auto',
          template: template,
          className: 'link-banner'
        });
      }
    });
  }

  initBanners();
});


text:
class AddPurposeToBapps < ActiveRecord::Migration[4.2]
  def change
    add_column :bapps, :purpose, :text
  end
end


text:
class CreateFuelingsAgain < ActiveRecord::Migration[4.2]
  def up
    return if table_exists? 'FTK_MAIN'

    # NOTE: This is a partial table compared to what's in the EAM DB,
    # but it contains all of the columns we actually use in this application.
    create_table 'FTK_MAIN', primary_key: 'row_id' do |t|
      t.decimal  'qty_fuel'
      t.integer  'meter_1'
      t.datetime 'ftk_date'
      t.datetime 'X_datetime_insert'
      t.string   'EQ_equip_no'
    end
  end

  def down
    return if ActiveRecord.version < Gem::Version.new('5.0')

    drop_table 'FTK_MAIN'
  end
end


text: instance_eval File.read(File.join(File.dirname(__FILE__), 'test.rb'))


text:
function disp(X,name)
%DISP Command window display of a tensor.
%
%   DISP(X) displays a tensor with no name.
%
%   DISP(X,NAME) displays a tensor with the given name.
%
%   See also TENSOR, TENSOR/DISPLAY.
%
%MATLAB Tensor Toolbox.
%Copyright 2012, Sandia Corporation.

% This is the MATLAB Tensor Toolbox by T. Kolda, B. Bader, and others.
% http://www.sandia.gov/~tgkolda/TensorToolbox.
% Copyright (2012) Sandia Corporation. Under the terms of Contract
% DE-AC04-94AL85000, there is a non-exclusive license for use of this
% work by or on behalf of the U.S. Government. Export of this data may
% require a license from the United States Government.
% The full license terms can be found in the file LICENSE.txt

if ~exist('name','var')
    name = 'ans';
end

fprintf(1,'%s is a tensor of size %s\n',name,tt_size2str(X.size));

if isempty(X.data)
    fprintf(1,'\t%s = []\n',name);
end

s = shiftdim(num2cell(X.data,1:2),2);

for i = 1:numel(s)
    fprintf('\t%s',name);
    if ndims(X) == 1
        fprintf('(:)');
    elseif ndims(X) == 2
        fprintf('(:,:)');
    elseif ndims(X) > 2
        fprintf('(:,:');
        fprintf(',%d',tt_ind2sub(X.size(3:end),i));
        fprintf(')');
    end
    fprintf(' = \n');
    output = tt_matrix2cellstr(s{i});
    fprintf('\t%s\n',output{:});
end

end


text:
#ifndef QSCRIPTDEBUGGERCOMMANDSCHEDULERFRONTEND_P_H
#define QSCRIPTDEBUGGERCOMMANDSCHEDULERFRONTEND_P_H

//
//  W A R N I N G
//  -------------
//
// This file is not part of the Qt API.  It exists purely as an
// implementation detail.  This header file may change from version to
// version without notice, or even be removed.
//
// We mean it.
//

#include <QtCore/qobjectdefs.h>

#include <QtCore/qstring.h>

QT_BEGIN_NAMESPACE

class QScriptDebuggerCommandSchedulerInterface;
class QScriptDebuggerResponseHandlerInterface;
class QScriptDebuggerCommand;
class QScriptDebuggerValue;
class QScriptBreakpointData;

class Q_AUTOTEST_EXPORT QScriptDebuggerCommandSchedulerFrontend
{
public:
    QScriptDebuggerCommandSchedulerFrontend(
        QScriptDebuggerCommandSchedulerInterface *scheduler,
        QScriptDebuggerResponseHandlerInterface *responseHandler);
    ~QScriptDebuggerCommandSchedulerFrontend();

     // execution control
     int scheduleInterrupt();
     int scheduleContinue();
     int scheduleStepInto(int count = 1);
     int scheduleStepOver(int count = 1);
     int scheduleStepOut();
     int scheduleRunToLocation(const QString &fileName, int lineNumber);
     int scheduleRunToLocation(qint64 scriptId, int lineNumber);
     int scheduleForceReturn(int contextIndex, const QScriptDebuggerValue &value);

     // breakpoints
     int scheduleSetBreakpoint(const QString &fileName, int lineNumber);
     int scheduleSetBreakpoint(const QScriptBreakpointData &data);
     int scheduleDeleteBreakpoint(int id);
     int scheduleDeleteAllBreakpoints();
     int scheduleGetBreakpoints();
     int scheduleGetBreakpointData(int id);
     int scheduleSetBreakpointData(int id, const QScriptBreakpointData &data);

     // scripts
     int scheduleGetScripts();
     int scheduleGetScriptData(qint64 id);
     int scheduleScriptsCheckpoint();
     int scheduleGetScriptsDelta();
     int scheduleResolveScript(const QString &fileName);

     // stack
     int scheduleGetBacktrace();
     int scheduleGetContextCount();
     int scheduleGetContextState(int contextIndex);
     int scheduleGetContextInfo(int contextIndex);
     int scheduleGetContextId(int contextIndex);
     int scheduleGetThisObject(int contextIndex);
     int scheduleGetActivationObject(int contextIndex);
     int scheduleGetScopeChain(int contextIndex);
     int scheduleContextsCheckpoint();
     int scheduleGetPropertyExpressionValue(int contextIndex, int lineNumber,
                                            const QStringList &path);
     int scheduleGetCompletions(int contextIndex, const QStringList &path);

     // iteration
     int scheduleNewScriptValueIterator(const QScriptDebuggerValue &object);
     int scheduleGetPropertiesByIterator(int id, int count);
     int scheduleDeleteScriptValueIterator(int id);

     // evaluation
     int scheduleEvaluate(int contextIndex, const QString &program,
                          const QString &fileName = QString(),
                          int lineNumber = 1);

     int scheduleScriptValueToString(const QScriptDebuggerValue &value);
     int scheduleSetScriptValueProperty(const QScriptDebuggerValue &object,
                                        const QString &name,
                                        const QScriptDebuggerValue &value);

     int scheduleClearExceptions();

     int scheduleNewScriptObjectSnapshot();
     int scheduleScriptObjectSnapshotCapture(int id, const QScriptDebuggerValue &object);
     int scheduleDeleteScriptObjectSnapshot(int id);

private:
    int scheduleCommand(const QScriptDebuggerCommand &command);

    QScriptDebuggerCommandSchedulerInterface *m_scheduler;
    QScriptDebuggerResponseHandlerInterface *m_responseHandler;

    Q_DISABLE_COPY(QScriptDebuggerCommandSchedulerFrontend)
};

QT_END_NAMESPACE

#endif


text:
package fixtures.custombaseurimoreoptions.implementation;

import fixtures.custombaseurimoreoptions.AutoRestParameterizedCustomHostTestClient;
import fixtures.custombaseurimoreoptions.Paths;
import com.microsoft.rest.ServiceClient;
import com.microsoft.rest.RestClient;
import okhttp3.OkHttpClient;
import retrofit2.Retrofit;

/**
 * Initializes a new instance of the AutoRestParameterizedCustomHostTestClient class.
 */
public class AutoRestParameterizedCustomHostTestClientImpl extends ServiceClient implements AutoRestParameterizedCustomHostTestClient {

    /** The subscription id with value 'test12'. */
    private String subscriptionId;

    /**
     * Gets The subscription id with value 'test12'.
     *
     * @return the subscriptionId value.
     */
    public String subscriptionId() {
        return this.subscriptionId;
    }

    /**
     * Sets The subscription id with value 'test12'.
     *
     * @param subscriptionId the subscriptionId value.
     * @return the service client itself
     */
    public AutoRestParameterizedCustomHostTestClientImpl withSubscriptionId(String subscriptionId) {
        this.subscriptionId = subscriptionId;
        return this;
    }

    /** A string value that is used as a global part of the parameterized host. Default value 'host'. */
    private String dnsSuffix;

    /**
     * Gets A string value that is used as a global part of the parameterized host. Default value 'host'.
     *
     * @return the dnsSuffix value.
     */
    public String dnsSuffix() {
        return this.dnsSuffix;
    }

    /**
     * Sets A string value that is used as a global part of the parameterized host. Default value 'host'.
     *
     * @param dnsSuffix the dnsSuffix value.
     * @return the service client itself
     */
    public AutoRestParameterizedCustomHostTestClientImpl withDnsSuffix(String dnsSuffix) {
        this.dnsSuffix = dnsSuffix;
        return this;
    }

    /**
     * The Paths object to access its operations.
     */
    private Paths paths;

    /**
     * Gets the Paths object to access its operations.
     * @return the Paths object.
     */
    public Paths paths() {
        return this.paths;
    }

    /**
     * Initializes an instance of AutoRestParameterizedCustomHostTestClient client.
     */
    public AutoRestParameterizedCustomHostTestClientImpl() {
        this("https://{vault}{secret}{dnsSuffix}");
    }

    /**
     * Initializes an instance of AutoRestParameterizedCustomHostTestClient client.
     *
     * @param baseUrl the base URL of the host
     */
    private AutoRestParameterizedCustomHostTestClientImpl(String baseUrl) {
        super(baseUrl);
        initialize();
    }

    /**
     * Initializes an instance of AutoRestParameterizedCustomHostTestClient client.
     *
     * @param clientBuilder the builder for building an OkHttp client, bundled with user configurations
     * @param restBuilder the builder for building an Retrofit client, bundled with user configurations
     */
    public AutoRestParameterizedCustomHostTestClientImpl(OkHttpClient.Builder clientBuilder, Retrofit.Builder restBuilder) {
        this("https://{vault}{secret}{dnsSuffix}", clientBuilder, restBuilder);
        initialize();
    }

    /**
     * Initializes an instance of AutoRestParameterizedCustomHostTestClient client.
     *
     * @param baseUrl the base URL of the host
     * @param clientBuilder the builder for building an OkHttp client, bundled with user configurations
     * @param restBuilder the builder for building an Retrofit client, bundled with user configurations
     */
    private AutoRestParameterizedCustomHostTestClientImpl(String baseUrl, OkHttpClient.Builder clientBuilder, Retrofit.Builder restBuilder) {
        super(baseUrl, clientBuilder, restBuilder);
        initialize();
    }

    /**
     * Initializes an instance of AutoRestParameterizedCustomHostTestClient client.
     *
     * @param restClient the REST client containing pre-configured settings
     */
    public AutoRestParameterizedCustomHostTestClientImpl(RestClient restClient) {
        super(restClient);
        initialize();
    }

    private void initialize() {
        this.dnsSuffix = "host";
        this.paths = new PathsImpl(retrofit(), this);
    }
}


text:
import sys, os
sys.path.append(os.path.dirname(os.path.realpath(__file__))+'/lib')

from cloudagents import CloudAgent
from keystoneclient.v2_0 import client
import swiftclient
import novaclient.v1_1
import json
import urllib2
from collections import defaultdict
import StringIO
import csv

ca = CloudAgent()

ca.required_config = {
	"name": "Usage Change",
	"version": "0.1.0",
	"author": "Jeff Kramer",
	"url": "http://www.hpcloud.com/",
	"help": """This script audits what resources you're using across the cloud, and sends you a notification with details and changes.""",
	"config":
		[
		]
	}

def agent():

	ca.log("Starting!")

	keystone = client.Client(token=ca.creds['token'], tenant_id=ca.creds['tenantId'],
							auth_url=ca.creds['identity_url'])

	catalog = keystone.service_catalog.catalog
	text = ""
	if ca.datastore:
		old_usage = json.loads(ca.datastore)
	else:
		old_usage = AutoVivification()
	usage = AutoVivification()
	changes = AutoVivification()
	object_csv_data = []
	compute_csv_data = []
	files = []

	for entry in catalog['serviceCatalog']:
		if entry['type'] == 'object-store':
			text += "\nObject Storage\n--------------\n\n"
			text += "%-15s %-25s %10s %10s %10s\n" % ("Region", "Container",
													"Objects",
													"Size",
													"$/month")
			for endpoint in entry['endpoints']:
				headers, containers = swiftclient.get_account(endpoint['publicURL'],
										ca.creds['token'],full_listing=True)
				for container in containers:
					try:
						old_container = old_usage['object-store'][endpoint['region']][container['name']]
					except ValueError:
						old_container = dict
					if old_container.get('count',0) != container['count'] or old_container.get('bytes',0) != container['bytes']:
						if old_container.get('bytes',0) < container['bytes'] or old_container.get('count',0) < container['count']:
							changes['object-store'][endpoint['region']][container['name']]['bytes'] = container['bytes'] - old_container.get('bytes',0)
							changes['object-store'][endpoint['region']][container['name']]['count'] = container['count'] - old_container.get('count',0)

					usage['object-store'][endpoint['region']][container['name']]['count'] = container['count']
					usage['object-store'][endpoint['region']][container['name']]['bytes'] = container['bytes']
					try:
						old_usage['object-store'][endpoint['region']].pop(container['name'])
					except ValueError:
						pass

					object_csv_data.append([endpoint['region'],container['name'],container['count'],container['bytes']])

					text += "%-15s %-25s %10s %10s %10s\n" % (endpoint['region'], container['name'],
													container['count'],
													sizeof_fmt(container['bytes']),
													"$"+storage_price(container['bytes']))

		if entry['type'] == 'compute':

			text += "\nCompute\n--------------\n\n"
			text += "%-20s %-40s %7s %7s\n" % ("Region","Name", "Size", "$/month")
			for endpoint in entry['endpoints']:
				nova = novaclient.v1_1.client.Client(None,None,None,auth_url="")
				nova.set_management_url(endpoint['publicURL'])
				nova.client.auth_token = ca.creds['token']
				flavors = dict()
				for flavor in nova.flavors.list():
					flavors[unicode(flavor.id)] = dict()
					flavors[unicode(flavor.id)]["display"] = sizeof_fmt(flavor.ram*1048576)
					flavors[unicode(flavor.id)]["price"] = get_compute_price(flavor.ram)
					flavors[unicode(flavor.id)]["ram"] = flavor.ram
					flavors[unicode(flavor.id)]["name"] = flavor.name

				for server in nova.servers.list():
					try:
						oldserver = old_usage['compute'][endpoint['region']][str(server.id)]
					except KeyError:
						oldserver = None
					if not oldserver:
						changes['compute'][endpoint['region']][server.id]['size'] = flavors[server.flavor['id']]["ram"]
						changes['compute'][endpoint['region']][server.id]['name'] = server.name
						changes['compute'][endpoint['region']][server.id]['added'] = True
					else:
						try:
							old_usage['compute'][endpoint['region']].pop(str(server.id))
						except ValueError:
							pass

					compute_csv_data.append([endpoint['region'],str(server.id),str(server.name),flavors[server.flavor['id']]["name"],flavors[server.flavor['id']]["ram"]])

					usage['compute'][endpoint['region']][server.id]['size'] = flavors[server.flavor['id']]["ram"]
					usage['compute'][endpoint['region']][server.id]['name'] = server.name
					text += "%-20s %-40s %7s %7s\n" % (endpoint['region'], server.name,
																flavors[server.flavor['id']]["display"],
																"$"+compute_price(flavors[server.flavor['id']]["price"]))
	if object_csv_data:
		object_csv = StringIO.StringIO()
		object_csv_writer = csv.writer(object_csv)
		object_csv_writer.writerows([["region","container","objects","bytes"]])
		object_csv_writer.writerows(object_csv_data)
		csv_data = object_csv.getvalue()
		files.append({"file-name":"object-storage.csv","contents":csv_data})

	if compute_csv_data:
		compute_csv = StringIO.StringIO()
		compute_csv_writer = csv.writer(compute_csv)
		compute_csv_writer.writerows([["region","id","name","flavor","ram"]])
		compute_csv_writer.writerows(compute_csv_data)
		csv_data = compute_csv.getvalue()
		files.append({"file-name":"compute.csv","contents":csv_data})

	#print text
	ca.store(json.dumps(usage))

	changes_html = ""

	for dataset in [changes, old_usage]:
		for service, service_data in dataset.items():
			for zone, zone_data in service_data.items():
				for change, change_data in zone_data.items():
					if service == 'object-store':
						changes_html += "
"+str(service)+"	"
						changes_html += ""+str(zone)+"	"
						changes_html += ""+str(change)+"	"
						if change_data.get("count"):
							if change_data['count'] > 0:
								changes_html += "+"+str(change_data['count'])+" obj"
							elif change_data['count'] < 0:
								changes_html += "-"+str(change_data['count'])+" obj"
						changes_html += "	"
						if change_data.get("bytes"):
							if change_data['bytes'] > 0:
								changes_html += "+"+sizeof_fmt(change_data['count'])+""
							elif change_data['bytes'] < 0:
								changes_html += "-"+sizeof_fmt(change_data['count'])+""
						changes_html += "
\n"
					elif service == 'compute':
						if change_data.get('name'):
							changes_html += "
"+str(service)+"	"
							changes_html += ""+str(zone)+"	"
							changes_html += ""+str(change_data.get('name'))+"	"
							if change_data.get('added'):
								changes_html += "Created "+sizeof_fmt(change_data['size']*1048576)+" server"
							else:
								changes_html += "Deleted "+sizeof_fmt(change_data['size']*1048576)+" server"
							changes_html += "
\n"

	if changes_html != '':
		changes_html = """

		Service
		Zone
		Item
		Change

		""" + changes_html + """

		"""

		ca.email("Usage Change",text, changes_html, files)

def sizeof_fmt(num):
    for x in [' B','KB','MB','GB']:
        if num < 1024.0 and num > -1024.0:
            return "%3.1f %s" % (num, x)
        num /= 1024.0
    return "%3.1f %s" % (num, 'TB')

def storage_price(num):
	return "%0.2f" % ((float(num)/1073741824.0)*0.1,)

def compute_price(hourly):
	return "%0.2f" % (float(hourly)*24.0*30.0,)

def get_compute_price(ram):
	ram = int(ram)
	if ram == 1024:
		return 0.035
	elif ram == 2048:
		return 0.07
	elif ram == 4096:
		return 0.14
	elif ram == 8192:
		return 0.28
	elif ram == 16384:
		return 0.56
	elif ram == 16384:
		return 1.12
	else:
		return 0.0

class AutoVivification(dict):
    """Implementation of perl's autovivification feature."""
    def __getitem__(self, item):
        try:
            return dict.__getitem__(self, item)
        except KeyError:
            value = self[item] = type(self)()
            return value

#		ca.email("File missing: "+ca.conf['container']+"/"+path,'''
#	The container '%s' appears to be missing the file '%s'.
#	''' % (ca.conf['container'], path))

ca.run(agent)


text:
Change anything you want to customize inside morpheus-master folder or even duplicate morpheus-master into another folder to keep source of different skins.
Then type:

`mvn clean install`

If you want to change any configuration in _defaults.scss to easily customize your own skin you just need to point to the file with those changes. It will be copied by maven in the proper place:

`mvn clean install -Dsakai.skin.customization.file=/folder/to/your/file.scss`

For example:

 - To generate without icons:
 `mvn clean install -Dsakai.skin.customization.file=./src/morpheus-master/sass/examples/_customization_example_withouticons.scss`
 - To generate with another google font:
 `mvn clean install -Dsakai.skin.customization.file=./src/morpheus-master/sass/examples/_customization_example_typography.scss`

By default source is morpheus-master and target is morpheus-default, but you can change these folders typing:

`mvn clean install -Dsakai.skin.source=<morpheus-source> -Dsakai.skin.target=<morpheus-target>`

We have uploaded a non-icons version compiled by:

`mvn clean install -Dsakai.skin.target=morpheus-default-noicons -Dsakai.skin.customization.file=./src/morpheus-master/sass/examples/_customization_example_withouticons.scss`

Feel free to repeat this commands to generate as many skins as you want.

## I want to design my skin from scratch using morpheus

You can use and test your own compass instalation to generate skins by typing inside morpheus-master folder:

`compass compile --css-dir=../webapp/skin/morpheus-custom --http-path=/library/skin/morpheus-custom`