text:
title: Calico Networking with rkt
no-canonical: true
---

This tutorial describes how to set up a Calico cluster in a pure rkt environment.
rkt is used for running both the Calico components and the workloads.

## 1. Environment setup

This tutorial walks through getting a cluster set up with Vagrant.

### 1.1 Install dependencies

* [VirtualBox][virtualbox] 5.0.0 or greater.
* [Vagrant][vagrant] 1.7.4 or greater.
* [Git][git]

### 1.2 Clone this project

    git clone https://github.com/projectcalico/calico.git

### 1.3 Startup and SSH
Change into the directory for this guide:

    cd calico/{{page.version}}/getting-started/rkt/vagrant/

Run

    vagrant up

To connect to your servers

* Linux/Mac OS X
    * run `vagrant ssh <hostname>`
* Windows
    * Follow instructions from https://github.com/nickryand/vagrant-multi-putty
    * run `vagrant putty <hostname>`

### 1.4 Verify environment

You should now have two CoreOS servers. The servers are named calico-01 and calico-02
and have IP addresses 172.18.18.101 and 172.18.18.102.

At this point, it's worth checking that your servers can ping each other.

From calico-01

    ping 172.18.18.102

From calico-02

    ping 172.18.18.101

If you see ping failures, the likely culprit is a problem with the VirtualBox network between the VMs.  You should
check that each host is connected to the same virtual network adapter in VirtualBox and rebooting the host may also
help.  Remember to shut down the VMs with `vagrant halt` before you reboot.

You should also verify each host can access etcd.  The following will return an error if etcd is not available.

    curl -L http://localhost:2379/version

## 2. Starting Calico services

Once you have your cluster up and running, start Calico on both hosts

    sudo calicoctl node  --runtime=rkt

This will create a systemd unit called `calico-node` which runs the Calico components under `rkt`

You can check that it's running

	$ sudo rkt list
	UUID            APP     IMAGE NAME                               STATE   NETWORKS
	bc13af40        node    registry-1.docker.io/calico/node:v0.22.0 running

You can check the status and logs using normal systemd commands e.g. `systemctl status calico-node` and `journalctl -u calico-node`

## 3. Create the networks

You can configure multiple networks when using rkt. Each network is represented by a configuration file in
`/etc/rkt/net.d/`. By default, connections to a given container are only allowed from containers on the same network.
This can be changed by applying additional Calico policy.

Containers on multiple networks can be accessed by containers on each network that it is connected to.
- The container only gets a single Calico IP address and single ethernet interface.
- The container is associated with the Calico profiles for each of the networks.

To define a rkt network for Calico, create a configuration file in `/etc/rkt/net.d/`.
- Each network should be given a unique "name". This corresponds to a "profile" in Calico.
- To use Calico networking, specify "type": "calico"
- To use Calico IPAM, specify "type": "calico-ipam" in the "ipam" section.

This worked example creates two rkt networks. Run these commands on both `calico-01` and `calico-02`

```shell
cat >/etc/rkt/net.d/10-calico-backend.conf <<EOF
{
    "name": "backend",
    "type": "calico",
    "ipam": {
        "type": "calico-ipam"
    }
}
EOF

cat >/etc/rkt/net.d/10-calico-frontend.conf <<EOF
{
    "name": "frontend",
    "type": "calico",
    "ipam": {
        "type": "calico-ipam"
    }
}
EOF
```

## 4. Create services

With the networks created, let's start some services. We'll create a "frontend" and a "backend" service, one on `calico-01` and the other on `calico-02`
Both "services" will just be a `httpd` running in a `busybox` container. The service just serves up the filesystem in the container over HTTP.

### On calico-01

Create the "frontend" service using `systemd-run`.

    sudo systemd-run --unit=frontend sudo rkt run --net=frontend registry-1.docker.io/library/busybox --exec httpd -- -f -h /

Normal [systemd commands][systemd-run] can then be used to get the status of the container and view its logs (e.g. `sudo journalctl -u frontend`). Use `rkt list` to see the IP.

	$ sudo rkt list
	UUID		APP	IMAGE NAME					STATE	CREATED		STARTEDNETWORKS
	05f8779a	node	registry-1.docker.io/calico/node:v0.22.0		running	1 hour ago	1 hour ago
	c89f2f35	busybox	registry-1.docker.io/library/busybox:latest	running	2 seconds ago	1 second ago	frontend:ip4=192.168.0.0, default-restricted:ip4=172.16.28.2

We now have a `busybox` container running on the network `frontend` with an IP address of `192.168.0.0`
rkt also creates a second network called `default-restricted`. This is used for communication with the rkt metadata service running on the host and is covered in the [rkt documentation](https://github.com/coreos/rkt/blob/master/Documentation/networking/overview.md#the-default-restricted-network)

### On calico-02

Repeat for a "backend" service on `calico-02`

    sudo systemd-run --unit=backend sudo rkt run --net=backend registry-1.docker.io/library/busybox --exec httpd -- -f -h /

	$ sudo rkt list
	UUID		APP	IMAGE NAME					STATE	CREATED		STARTEDNETWORKS
	2cc27ce1	node	registry-1.docker.io/calico/node:v0.22.0		running	1 hour ago	1 hour ago
	407208a5	busybox	registry-1.docker.io/library/busybox:latest	running	11 seconds ago	10 seconds ago	backend:ip4=192.168.0.64, default-restricted:ip4=172.16.28.2

We now have a `busybox` container running on the network `backend` with an IP address of `192.168.0.1`

## 5. Validate access to services

Now that we've created the services and we know their IP addresses, we can access them using `wget` from containers running on
either host, as long as they are created on the same network.

e.g. On `calico-02` use wget to access the frontend service which is running on `calico-01`

	sudo rkt run --net=frontend registry-1.docker.io/library/busybox --exec=/bin/wget -- -T 10 192.168.0.0/etc/passwd 2>/dev/null

Expected output

	[62032.807862] wget[4]: Connecting to 192.168.0.0 (192.168.0.0:80)
	[62032.813662] wget[4]: passwd               100% |*******************************|   334   0:00:00 ETA

This command runs the `wget` command in a busybox container to fetch the `passwd` file from our host. '-T 1' tells wget to only wait a second for a response.
Stderr is redirected to `/dev/null` as we're not interested in the logs from `rkt` for this command.

You can repeat this command on calico-01 and check that access works the same from any server in your cluster.

### 5.1 Checking network isolation
Repeat the above command but try to access the backend from the frontend. Because we've not allowed access between these networks, the command will fail.

	sudo rkt run --net=backend registry-1.docker.io/library/busybox --exec=/bin/wget -- -T 2 192.168.0.0/etc/passwd 2>/dev/null

Expected output

	[62128.109283] wget[4]: Connecting to 192.168.0.0 (192.168.0.0:80)
	[62129.109472] wget[4]: wget: download timed out

Calico always allows access to the containers running on a host _from that host only_. The rules are bypassed in this case only.
This means that the backend service can be accessed directly from `calico-02` (the host it's running on), but not from `calico-01`

## 6. Add network policy
To view the existing network policy, use the `calicoctl` command.

On either host, run

	calicoctl profile backend rule show

Expected output

	Inbound rules:
		 1 allow from tag backend
	Outbound rules:
		 1 allow

The "frontend" profile produces a similar result.

### Open access to backends

We want the backends to allow inbound traffic from the frontends, but only on port 80.

Run

	calicoctl profile backend rule add inbound allow tcp from tag frontend to ports 80
	calicoctl profile backend rule show

To produce the following output

	Inbound rules:
		 1 allow from tag backend
		 2 allow tcp from tag frontend to ports 80
	Outbound rules:
		 1 allow

And we can now access our backend service from the frontend containers.

On either host, run

	sudo rkt run --net=frontend registry-1.docker.io/library/busybox --exec=/bin/wget -- -T 10 192.168.0.64/etc/passwd 2>/dev/null

### Open access to frontends
We want to allow everyone to access our frontends, but only on port 80.

	calicoctl profile frontend rule add inbound allow tcp to ports 80
	calicoctl profile frontend rule show

To produce the following output

	calicoctl profile frontend rule show
	Inbound rules:
		 1 allow from tag frontend
		 2 allow tcp to ports 80
	Outbound rules:
		 1 allow

Now on either host, we can access the container directly

	wget -T 10 192.168.0.0/etc/passwd

## 7. Resetting/Cleanup up
If you want to start again from the beginning, then run the following commands on both hosts to ensure that all the rkt containers and systemd jobs are removed.

	# Stop the frontend (in case the job failed, manually remove the service too)
	sudo systemctl stop frontend; sudo rm -rf /run/systemd/system/frontend.service /run/systemd/system/frontend.service.d; sudo systemctl daemon-reload

	# Stop the backend (in case the job failed, manually remove the service too)
	sudo systemctl stop backend; sudo rm -rf /run/systemd/system/backend.service /run/systemd/system/backend.service.d; sudo systemctl daemon-reload

	# Stop the calico-node (in case the job failed, manually remove the service too)
	sudo systemctl stop calico-node; sudo rm -rf /run/systemd/system/calico-node.service /run/systemd/system/calico-node.service.d; sudo systemctl daemon-reload

	# Remove any stopped busybox containers
	sudo rkt list --no-legend | cut -f1 |sudo xargs rkt rm

	# Wipe Calico data from etcd
	etcdctl rm --recursive /calico

[systemd-run]: https://github.com/coreos/rkt/blob/master/Documentation/using-rkt-with-systemd.md#systemd-run
[virtualbox]: https://www.virtualbox.org/
[vagrant]: https://www.vagrantup.com/downloads.html
[git]: http://git-scm.com/


text:
var socketPhoenix = new Socket("/socket", {});
socketPhoenix.connect();
var channelScore = socketPhoenix.channel("score:join", {});
var language;

setupSocket();

function setupSocket(){
    channelScore.join()
        .receive("ok", function (data) {
        console.log(data);
        })
        .receive("error", function (resp) {
        console.log(resp);
        });

}

function loadConfig(lang) {
    language = lang;
}

function setup() {
	language = $("select").val();
    var json = JSON.stringify({"language": language});
    channelScore.push('score:setupInstructions' , json);
	$("#startButton").prop("disabled", false);

}

function start() {
	var seconds = $("#countdown").val();
	if (seconds==="") {
		seconds="default";
	}
	language = $("select").val();
	var json = JSON.stringify({"language": language, "countdownSeconds": seconds});
    channelScore.push('score:start', json);
	$("#startButton").prop("disabled",true);
}

function reset() {
    channelScore.push('score:resetGame');
}

redpajama_set_name: RedPajamaGithub


text:
package wire

import (
	"io"

	"github.com/btcboost/copernicus/util"
)

// MsgPing implements the Message interface and represents a bitcoin ping
// message.
//
// For versions BIP0031Version and earlier, it is used primarily to confirm
// that a connection is still valid.  A transmission error is typically
// interpreted as a closed connection and that the peer should be removed.
// For versions AFTER BIP0031Version it contains an identifier which can be
// returned in the pong message to determine network timing.
//
// The payload for this message just consists of a nonce used for identifying
// it later.
type MsgPing struct {
	// Unique value associated with message that is used to identify
	// specific ping message.
	Nonce uint64
}

// Decode decodes r using the bitcoin protocol encoding into the receiver.
// This is part of the Message interface implementation.
func (msg *MsgPing) Decode(r io.Reader, pver uint32, enc MessageEncoding) error {
	// There was no nonce for BIP0031Version and earlier.
	// NOTE: > is not a mistake here.  The BIP0031 was defined as AFTER
	// the version unlike most others.
	if pver > BIP0031Version {
		err := util.ReadElements(r, &msg.Nonce)
		if err != nil {
			return err
		}
	}

	return nil
}

// Encode encodes the receiver to w using the bitcoin protocol encoding.
// This is part of the Message interface implementation.
func (msg *MsgPing) Encode(w io.Writer, pver uint32, enc MessageEncoding) error {
	// There was no nonce for BIP0031Version and earlier.
	// NOTE: > is not a mistake here.  The BIP0031 was defined as AFTER
	// the version unlike most others.
	if pver > BIP0031Version {
		err := util.WriteElements(w, msg.Nonce)
		if err != nil {
			return err
		}
	}

	return nil
}

// Command returns the protocol command string for the message.  This is part
// of the Message interface implementation.
func (msg *MsgPing) Command() string {
	return CmdPing
}

// MaxPayloadLength returns the maximum length the payload can be for the
// receiver.  This is part of the Message interface implementation.
func (msg *MsgPing) MaxPayloadLength(pver uint32) uint32 {
	plen := uint32(0)
	// There was no nonce for BIP0031Version and earlier.
	// NOTE: > is not a mistake here.  The BIP0031 was defined as AFTER
	// the version unlike most others.
	if pver > BIP0031Version {
		// Nonce 8 bytes.
		plen += 8
	}

	return plen
}

// NewMsgPing returns a new bitcoin ping message that conforms to the Message
// interface.  See MsgPing for details.
func NewMsgPing(nonce uint64) *MsgPing {
	return &MsgPing{
		Nonce: nonce,
	}
}


text:
document.querySelector("#printButton").onclick = () => {
  window.print();
};

(() => {
  const jobs = [
    {
      company: "Atlassian • Contract",
      position: "Full-Stack Developer",
      responsibilities: [
        "Launched retiring old content on Atlassian support docs (support.atlassian.com).",
        "Led project efforts on TypeScript conversion and helped lead CI / CD pipeline improvements."
      ],
      technologies: [
        "TypeScript",
        "JavaScript",
        "React",
        "Express",
        "GraphQL",
        "Docker",
        "Contentful CMS"
      ],
      startDate: "February 2020",
      endDate: "Present"
      // reference: {
      //   name: "",
      //   position: "Engineering Team Lead",
      //   contact: "",
      //   contactType: "tel:"
      // }
    },
    {
      company: "Smartly.io",
      position: "Full-Stack Developer",
      responsibilities: [
        "Developed the text engine for the company's web-based video editor and renderer, which has produced millions of video ads for Fortune 500 customers.",
        "Implemented the video and image management system (Rails, TypeScript) integrated across the company's core offerings, enhancing customers' experience in creating ad campaigns (100% of company revenue).",
        "Owned the company UI component library (React, TypeScript) in close collaboration with product designers.",
        "Interviewed engineering candidates on a survey of front-end, back-end and general technical software skills."
      ],
      technologies: [
        "TypeScript",
        "JavaScript",
        "React",
        "Redux",
        "Ruby",
        "Rails",
        "PHP",
        "Python",
        "Docker"
      ],
      startDate: "September 2018",
      endDate: "November 2019"
      // reference: {
      //   name: "",
      //   position: "Engineering Team Lead",
      //   contact: "",
      //   contactType: "tel:"
      // }
    },
    {
      company: "Gofore",
      position: "Software Developer",
      responsibilities: [
        "Founded and led a project to automate the company's internal cloud billing processes, reducing management costs by €1000s per month and improving reliability of billing processes.",
        "Utilized natural language chat bot solutions (DialogFlow) on internal sales data to help developers and designers request relevant projects using technologies of their interests."
      ],
      technologies: [
        "Python",
        "Flask",
        "JavaScript",
        "TypeScript",
        "Node.js",
        "React",
        "Redux",
        "Angular 2",
        "Express",
        "AWS S3",
        "AWS Lambda"
      ],
      startDate: "February 2018",
      endDate: "September 2018",
      // reference: {
      //   name: "Juho Salmi",
      //   position: "Product Owner",
      //   contact: "+358405377012",
      //   contactType: "tel:"
      // }
    },
    {
      company: "Pearson Education",
      position: "Full-Stack Intern (AngularJS, Express)",
      responsibilities: [
        "Developed a data visualization tool for over 1000 employees, replacing an expensive OracleSQL tool, while improving the efficiency of the development-testing lifecycle."
      ],
      technologies: [
        // "HTML",
        // "CSS",
        // "JavaScript",
        // "Node.js",
        // "MongoDB",
        // "SQLite",
        // "AngularJS",
        // "ExpressJS",
        // "Bootstrap",
        // "Perl"
      ],
      startDate: "May 2015",
      endDate: "August 2015",
      // reference: {
      //   name: "Ankur Kapadia",
      //   position: "Manager",
      //   contact: "anku13@gmail.com",
      //   contactType: "mailto:"
      // }
    },
    {
      company: "SolarWinds",
      position: "UI Intern (Angular)",
      responsibilities: [],
      technologies: [
        // "HTML",
        // "CSS",
        // "JavaScript",
        // "TypeScript",
        // "Node.js",
        // "AngularJS",
        // "Angular 2",
        // "Sass",
        // "ExpressJS",
        // "Grunt",
        // "Webpack"
      ],
      startDate: "June 2016",
      endDate: "August 2016"
    },
    {
      company: "Educational Design Studios",
      position: "Full-Stack Intern (Ruby)",
      responsibilities: [],
      technologies: [
        // "HTML",
        // "CSS",
        // "JavaScript",
        // "Ruby",
        // "Ruby on Rails",
      ],
      startDate: "June 2013",
      endDate: "June 2014"
    },
    {
      company: "Rackspace Hosting",
      position: "Full-Stack Intern (Ruby)",
      responsibilities: [],
      technologies: [
        // "HTML",
        // "CSS",
        // "JavaScript",
        // "Ruby",
        // "Sinatra",
      ],
      startDate: "June 2013",
      endDate: "August 2013"
    }
  ];

  const projects = [
    {
      title: "MoFi",
      description:
        "Movie Recommendations. Hackathon project that helps groups decide on a movie to watch based on the moods of the group. I was in charge of the Back-End.",
      technologies: ["Koa", "Mongoose", "MongoDB", "React"]
    },
    {
      title: "NinjaPup OS",
      description:
        "Basic Operating System written with Rust nightly with paged x86 64-bit memory management.",
      technologies: ["Rust", "Kernel Development"]
    },
    {
      title: "todolist.social",
      description:
        "Todo list application that allows users to keep track of and share their goals with friends.",
      technologies: [
        "HTML",
        "CSS",
        "JavaScript",
        "AngularJS",
        "Node.js",
        "ExpressJS",
        "SQLite"
      ]
    },
    {
      title: "PintOS",
      description:
        "Simple OS I wrote on top of, implementing virtual memory management, system calls, child processes, process scheduling, and a simple ext2 file system.",
      technologies: [
        "C",
        "Kernel Development",
        "File Systems",
        "Process Scheduling",
        "Concurrency"
      ],
      hiddenPrint: true
    }
  ];

  const skillsets = [
    {
      short: "Languages",
      title: "Programming Languages",
      skills: [
        "JavaScript",
        "TypeScript",
        "Ruby",
        "Python",
        "Java",
        "C#",
        "PHP",
        "C",
        "C++",
        "Rust"
      ]
    },
    {
      short: "Front-End",
      title: "Web Front-End",
      skills: [
        "HTML",
        "CSS / Sass",
        "JavaScript",
        "TypeScript",
        "React",
        "Redux",
        "AngularJS",
        "Angular 2+",
        "RxJS (Exposure)",
        "jQuery",
        "Unit Testing"
      ]
    },
    {
      short: "Back-End",
      title: "Web Back-End",
      skills: [
        "ExpressJS",
        "Ruby on Rails",
        "Python Flask",
        "MongoDB",
        "REST-ful APIs",
        "GraphQL",
        "PostgreSQL",
        "Docker",
        "CI Tools",
        "Concurrency",
        "Unit Testing"
      ]
    },
    {
      short: "Low-Level",
      title: "Low-Level",
      skills: [
        "C",
        "C++",
        "Rust",
        "Kernel Development",
        "File Systems",
        "System Calls",
        "Concurrency"
      ],
      hiddenPrint: true
    }
  ];

  angular
    .module("ResumeApp", ["ngMaterial", "ngSanitize"])
    .controller("ExperienceCtrl", function($scope) {
      $scope.jobs = jobs;
    })
    .controller("ProjectsCtrl", function($scope) {
      $scope.projects = projects;
    })
    .controller("SkillsCtrl", function($scope) {
      $scope.skillsets = skillsets;
    })
    .controller("ReferencesCtrl", function($scope) {
      $scope.jobs = jobs;
    })
    .directive("resumeSection", function() {
      return {
        restrict: "E",
        templateUrl: "components/resume-section.html",
        scope: {
          title: "@sectionTitle"
        },
        transclude: true
      };
    })
    .directive("resumeChips", function() {
      return {
        restrict: "E",
        templateUrl: "components/resume-chips.html",
        scope: {
          source: "="
        }
      };
    });
})();


text:
@interface OCSSingletonScopeTests : XCTestCase

@end


text:
using System;
using System.Collections.Generic;
using System.Text;
using System.Collections;
using System.Drawing.Drawing2D;
using System.Drawing;

namespace fxpa
{
    public static class MathHelper
    {
        public enum DirectionEnum
        {
            Equal = 0,
            Up,
            Down
        }

        /// <summary>
        /// Will round the value to a given count of resulting symbols.
        /// </summary>
        /// <returns></returns>
        public static double RoundToSymbolsCount(double inputValue, int requiredSymbolsCount)
        {
            int symbols = (int)Math.Ceiling(Math.Log10(inputValue));
            // Values below 0 mean spaces already existing after the zero point - discard them.
            symbols = Math.Max(0, symbols);
            return Math.Round(inputValue, requiredSymbolsCount - symbols);

            //double symbolsd = Math.Log10(inputDataUnits[0].Close);
            //double symbols2d = Math.Log10(1.2);
            //int symbols2 = (int)Math.Ceiling(Math.Log10(1.2));
            //double symbols3d = Math.Log10(10.2);
            //int symbols3 = (int)Math.Ceiling(Math.Log10(10.2));
        }

        /// <summary>
        /// Perform a complex calculation to establish the distance between these points in absolute coordinates,
        /// considering both X and Y scaling.
        /// </summary>
        public static float GetAbsoluteDistance(Matrix transformationMatrix, PointF point1, PointF point2)
        {
            float xDifference = Math.Abs(point1.X - point2.X);
            float yDifference = Math.Abs(point1.Y - point2.Y);

            // Apply scale.
            xDifference = xDifference * transformationMatrix.Elements[0];
            yDifference = yDifference * transformationMatrix.Elements[3];

            return (float)Math.Sqrt(xDifference * xDifference + yDifference * yDifference);
        }

        /// <summary>
        /// Will show where line 1 crosses line 2; 1 for cross up, 2 for cross down, 0 for no crossing.
        /// </summary>
        static public double[] CreateLineCrossings(double[] line1, double[] line2)
        {
            System.Diagnostics.Debug.Assert(line1.Length == line2.Length);

            double[] results = new double[line2.Length];

            for (int k = 0; k < line1.Length; k++)
            {
                if (k == 0)
                {
                    //if (line1[k] == line2[k])
                    //{
                    //    results[k] = 1;
                    //}
                }
                else
                {
                    if ((line1[k - 1] >= line2[k - 1] && line1[k] <= line2[k]))
                    {
                        results[k] = 2;
                    }
                    else if (line1[k - 1] <= line2[k - 1] && line1[k] >= line2[k])
                    {
                        results[k] = 1;
                    }
                }
            }

            return results;
        }

        /// <summary>
        ///
        /// </summary>
        static public double[] CreateFixedLineResultLength(double value, int count)
        {
            double[] line = new double[count];
            for (int i = 0; i < count; i++)
            {
                line[i] = value;
            }
            return line;
        }

        /// <summary>
        /// Calculates a "connection value" line between the 2 values (like a vector connecting 2 points).
        /// </summary>
        static public double[] CreateConnectionValues(double initialValue, double secondaryValue, int connectionValuesCount)
        {
            double[] results = new double[connectionValuesCount];
            //System.Diagnostics.Debug.Assert(connectionValuesCount >= 2);

            results[0] = initialValue;
            results[connectionValuesCount - 1] = secondaryValue;

            double stepValue = (secondaryValue - initialValue) / (connectionValuesCount - 1);
            for (int i = 1; i < connectionValuesCount - 1; i++)
            {
                results[i] = results[i - 1] + stepValue;
            }

            return results;
        }

        static public DirectionEnum[] EstablishDirection(double[] values)
        {
            DirectionEnum[] results = new DirectionEnum[values.Length];

            for (int i = 1; i < values.Length; i++)
            {
                if (values[i - 1] > values[i])
                {
                    results[i] = DirectionEnum.Down;
                }
                else if (values[i - 1] < values[i])
                {
                    results[i] = DirectionEnum.Up;
                }
            }

            return results;
        }

        /// <summary>
        /// This is a quick approximation of the Moving average.
        /// The formula used here is as follows :
        /// An estimate of the moving average if the bin size for moving average is n may be obtained by:
        /// NewAverage = (((n-1) * OldAverage) + newValue)/n
        /// This works once the bin is full (sample number >= n). The bin partially full is often dealt with by using a seed value for the initial moving average (OldAverage) and then using this calculation.
        /// This assumes normal distribution of values etc.
        /// </summary>
        static public double[] CalculateQuickMA(double[] values, int period)
        {
            double[] results = new double[values.Length];
            double average = 0;
            for (int i = 0; i < values.Length; i++)
            {
                average = (((period - 1) * average) + values[i]) / period;
                results[i] = average;
            }
            return results;
        }

        /// <summary>
        /// Provides a way to see the distribution of the values in periodsCount separate periods.
        /// </summary>
        /// <param name="values"></param>
        /// <param name="periodsCount"></param>
        /// <returns></returns>
        static public Dictionary<double, double> CalculateValueDistribution(IEnumerable<double> values, int periodsCount)
        {
            List<double> list = new List<double>(values);
            list.Sort();

            double min = list[0];
            double max = list[list.Count - 1];

            double periodSize = (max - min) / periodsCount;

            Dictionary<double, double> results = new Dictionary<double,double>();

            int j=0;
            for (double i=min; i<max; i+=periodSize)
            {
                int count = 0;
                for (; j < list.Count; j++)
                {
                    if (list[j] <= i + periodSize)
                    {
                        count++;
                    }
                    else
                    {
                        break;
                    }
                }

                results.Add(i, count);
            }

            return results;
        }

    }
}


text:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<TITLE>
net.floodlightcontroller.core.module
</TITLE>

<META NAME="date" CONTENT="2013-07-13">

<LINK REL ="stylesheet" TYPE="text/css" HREF="../../../../stylesheet.css" TITLE="Style">

      Overview
    Package
      Class
      Use
      Tree
      Deprecated
      Index
      Help

 PREV PACKAGE
 NEXT PACKAGE

  FRAMES
 NO FRAMES

  All Classes

Package net.floodlightcontroller.core.module

Interface Summary

IFloodlightModule
Defines an interface for loadable Floodlight modules.

IFloodlightModuleContext

IFloodlightService
This is the base interface for any IFloodlightModule package that provides
 a service.

Class Summary

FloodlightModuleContext
The service registry for an IFloodlightProvider.

FloodlightModuleLoader
Finds all Floodlight modules in the class path and loads/starts them.

ModuleLoaderResource
Returns list of modules loaded by Floodlight.

Exception Summary

FloodlightModuleException

      Overview
    Package
      Class
      Use
      Tree
      Deprecated
      Index
      Help

 PREV PACKAGE
 NEXT PACKAGE

  FRAMES
 NO FRAMES

  All Classes


text:
/**
 * Creates a jQuery object with a form element. The formTransform function
 * argument can be used to fill the form with input fields.
 *
 * Example: create$form(function($form) {
 *   $hidden = create$hidden('fruit', 'apple');
 *   $form.append($hidden);
 * });
 *
 * @param formTransform
 * @returns {*|jQuery|HTMLElement}
 */
function create$form(formTransform) {
  var $form = $(document.createElement('form'));
  $form.attr('action', 'dump.php');
  $form.attr('method', 'POST');

  if(typeof formTransform == 'function') {
    formTransform($form);
  }

  return $form;
}

/**
 * Creates a jQuery object with a hidden form field.
 *
 * @param name
 * @param value
 * @returns {*|jQuery|HTMLElement}
 */
function create$hidden(name, value) {
  var $input =  $(document.createElement('input'));
  $input.attr('name', name);
  $input.val(value);
  return $input;
}

/**
 * Performs an AJAX request as if the given form was submitted.
 * Options:
 *   callback: Function run on success. Goes into success option of
 *             the performed $.ajax call.
 *   jsonVar: If given, all form data is packed into a single JSON-
 *            -encoded variable.
 *
 * @param form
 * @param options
 */
function ajaxSendForm(form, options) {
  var $form = form instanceof jQuery ? form : $(form);
  var options = $.extend({}, ajaxSendForm.defaults, options);

  $.ajax({
    url: $form.attr('action'),
    type: $form.attr('method'),
    success: options.callback,
    data: options.json ? Form2Json._getFormDataJson($form) : Form2Json._getFormData($form)
  });
}
ajaxSendForm.defaults = {
  json: false,
  callback: undefined
};

/**
 * Performs an asyncTest sending a form via AJAX to the dump
 * script that returns a JSON object with all request variables.
 * It compares the send data with the received data, which
 * must be equal.
 * If the json switch is on, all the form data is sent packed
 * into a single json variable named __JSON.
 *
 * @param title
 * @param expected
 * @param formTransform
 * @param json
 */
function formAsyncTest(title, expected, formTransform, json) {
  var suffix = 'AJAX';
  if(json) {
    suffix += ', JSON';
  }
  suffix = ' (' + suffix + ')';

  asyncTest(title + suffix, function() {
    expect(1);

    var $form = create$form(formTransform);
    var callback = function(data) {
      deepEqual(data, expected);
      start();
    };

    ajaxSendForm($form, {
      callback: callback,
      json: json
    });
  });
}

/**
 * Runs a series of tests to check whether the form data
 * is collected, send and unpacked correctly.
 *
 * Firstly, the expected argument is compared to a result
 * of a _getFormData call on the formTransform form. This
 * tests the _getFormData function.
 *
 * Secondly, the form is sent via AJAX to a dump script
 * that returns a JSON object just with all the request
 * parameters it gets. This tests that the form is sent
 * correctly as well as dumped correctly.
 *
 * Thirdly, the form is sent to the dump script again,
 * but this time with all its data packed into a single
 * JSON variable. This tests the transparent JSON
 * unpacking on the server side.
 *
 * @param title
 * @param expected
 * @param formTransform
 */
function formAsyncTests(title, expected, formTransform) {
  test(title + ' (_getFormData)', function() {
    var $form = create$form(formTransform);

    deepEqual(Form2Json._getFormData($form), expected);
  });

  formAsyncTest(title, expected, formTransform, false);
  formAsyncTest(title, expected, formTransform, true);
}

/* *** *** HELPER FUNCTIONS END HERE *** *** */

/* *** *** TESTS START HERE *** *** */
module('_getPushKey');

test('Empty object push key is found', function() {
  var obj = {};
  equal(Form2Json._getPushKey(obj), 0);
});

test('Sequential numeric push key is found', function() {
  var obj = { 0: 'fruit', 1: 'vegetable' };
  equal(Form2Json._getPushKey(obj), 2);
});

test('Non-sequential numeric push key is found', function() {
  var obj = { 0: 'fruit', 5: 'vegetable', 2: 'nut' };
  equal(Form2Json._getPushKey(obj), 6);
});

test('First push key in named object is found', function() {
  var obj = { fruit: 'apple' };
  equal(Form2Json._getPushKey(obj), 0);
});

test('Another push key in named object is found', function() {
  var obj = { fruit: 'apple', 5: 'vegetable', nut: 'chestnut', 2: 'unknown' };
  equal(Form2Json._getPushKey(obj), 6);
});

module('_getFormData');

formAsyncTests('Empty form is submitted.', {});

formAsyncTests(
  'Form with single value is submitted.',
  { fruit: 'apple' },
  function($form) {
    $form.append(create$hidden('fruit', 'apple'));
  }
);

formAsyncTests(
  'Form with multiple values is submitted.',
  { fruit: 'apple', vegetable: 'carrot', nut: 'cashew' },
  function($form) {
    $form.append(create$hidden('fruit', 'apple'));
    $form.append(create$hidden('vegetable', 'carrot'));
    $form.append(create$hidden('nut', 'cashew'));
  }
);

formAsyncTests(
  'Form with named arrays is submitted.',
  {
    fruit: { red: 'apple', orange: 'orange' },
    vegetables: { orange: 'carrot', green: 'broccoli' }
  },
  function($form) {
    $form.append(create$hidden('fruit[red]', 'apple'));
    $form.append(create$hidden('fruit[orange]', 'orange'));
    $form.append(create$hidden('vegetables[orange]', 'carrot'));
    $form.append(create$hidden('vegetables[green]', 'broccoli'));
  }
);

formAsyncTests(
  'Form with numbered arrays is submitted.',
  {
    fruit: { 0: 'apple', 2: 'orange' },
    vegetables: { 1: 'broccoli', 3: 'carrot' }
  },
  function($form) {
    $form.append(create$hidden('fruit[0]', 'apple'));
    $form.append(create$hidden('fruit[2]', 'orange'));
    $form.append(create$hidden('vegetables[3]', 'carrot'));
    $form.append(create$hidden('vegetables[1]', 'broccoli'));
  }
);

formAsyncTests(
  'Form with decimal numbered arrays is submitted.',
  {
    fruit: { 0.1: 'apple', 2.23: 'orange' },
    vegetables: { 1.333: 'broccoli', 3.6: 'carrot' }
  },
  function($form) {
    $form.append(create$hidden('fruit[0.1]', 'apple'));
    $form.append(create$hidden('fruit[2.23]', 'orange'));
    $form.append(create$hidden('vegetables[3.6]', 'carrot'));
    $form.append(create$hidden('vegetables[1.333]', 'broccoli'));
  }
);

formAsyncTests(
  'Form with one implicitly numbered array is submitted.',
  {
    fruit: { 0: 'apple', 1: 'orange', 2: 'pear' }
  },
  function($form) {
    $form.append(create$hidden('fruit[]', 'apple'));
    $form.append(create$hidden('fruit[]', 'orange'));
    $form.append(create$hidden('fruit[]', 'pear'));
  }
);

formAsyncTests(
  'Form with more implicitly numbered arrays is submitted.',
  {
    fruit: { 0: 'apple', 1: 'orange' },
    vegetables: { 0: 'carrot', 1: 'broccoli' },
    nuts: { 0: 'chestnut', 1: 'walnut' }
  },
  function($form) {
    $form.append(create$hidden('fruit[]', 'apple'));
    $form.append(create$hidden('fruit[]', 'orange'));
    $form.append(create$hidden('vegetables[]', 'carrot'));
    $form.append(create$hidden('vegetables[]', 'broccoli'));
    $form.append(create$hidden('nuts[]', 'chestnut'));
    $form.append(create$hidden('nuts[]', 'walnut'));
  }
);

formAsyncTests(
  'Form with multi-dimensional named arrays is submitted.',
  {
    fruit: {
      apple: { color: 'red', taste: 'sweet' },
      lemon: { color: 'yellow', taste: 'sour' }
    },
    vegetables: {
      carrot: { color: 'orange', taste: 'sweet' },
      broccoli: { color: 'green', taste: 'bitter' }
    }
  },
  function($form) {
    $form.append(create$hidden('fruit[apple][color]', 'red'));
    $form.append(create$hidden('fruit[apple][taste]', 'sweet'));
    $form.append(create$hidden('fruit[lemon][color]', 'yellow'));
    $form.append(create$hidden('fruit[lemon][taste]', 'sour'));
    $form.append(create$hidden('vegetables[carrot][color]', 'orange'));
    $form.append(create$hidden('vegetables[carrot][taste]', 'sweet'));
    $form.append(create$hidden('vegetables[broccoli][color]', 'green'));
    $form.append(create$hidden('vegetables[broccoli][taste]', 'bitter'));
  }
);

formAsyncTests(
  'Form with multi-dimensional mixed arrays is submitted.',
  {
    fruit: {
      apple: {
        colors: {
          0: 'red',
          1: 'green',
          2: 'yellow'
        },
        taste: 'sweet'
      },
      lemon: { color: 'yellow', taste: 'sour' }
    },
    vegetables: {
      carrot: { color: 'orange', taste: 'sweet' },
      broccoli: {
        color: 'green',
        tastes: {
          0: 'bitter',
          1: 'sweet',
          common: 'sweet'
        }
      }
    },
    nuts: {
      0: 'chestnut',
      5: 'walnut',
      6: 'cashew',
      big: 'coconut',
      again: {
        0: 'chestnut',
        5: 'walnut',
        6: 'cashew',
        big: 'coconut'
      }
    }
  },
  function($form) {
    $form.append(create$hidden('fruit[apple][colors][]', 'red'));
    $form.append(create$hidden('fruit[apple][colors][]', 'green'));
    $form.append(create$hidden('fruit[apple][colors][]', 'yellow'));
    $form.append(create$hidden('fruit[apple][taste]', 'sweet'));
    $form.append(create$hidden('fruit[lemon][color]', 'yellow'));
    $form.append(create$hidden('fruit[lemon][taste]', 'sour'));
    $form.append(create$hidden('vegetables[carrot][color]', 'orange'));
    $form.append(create$hidden('vegetables[carrot][taste]', 'sweet'));
    $form.append(create$hidden('vegetables[broccoli][color]', 'green'));
    $form.append(create$hidden('vegetables[broccoli][tastes][]', 'bitter'));
    $form.append(create$hidden('vegetables[broccoli][tastes][]', 'sweet'));
    $form.append(create$hidden('vegetables[broccoli][tastes][common]', 'sweet'));
    $form.append(create$hidden('nuts[]', 'chestnut'));
    $form.append(create$hidden('nuts[5]', 'walnut'));
    $form.append(create$hidden('nuts[]', 'cashew'));
    $form.append(create$hidden('nuts[big]', 'coconut'));
    $form.append(create$hidden('nuts[again][]', 'chestnut'));
    $form.append(create$hidden('nuts[again][5]', 'walnut'));
    $form.append(create$hidden('nuts[again][]', 'cashew'));
    $form.append(create$hidden('nuts[again][big]', 'coconut'));
  }
);

formAsyncTests(
  'Form with multi-dimensional implicit arrays is submitted.',
  {
    fruit: {
      0: {
        0: 'red apple'
      },
      1: {
        0: 'yellow lemon'
      }
    },
    vegetables: {
      carrot: {
        0: {
          0: 'orange and sweet'
        },
        1: {
          0: 'brown and rotten'
        }
      }
    }
  },
  function($form) {
    $form.append(create$hidden('fruit[][]', 'red apple'));
    $form.append(create$hidden('fruit[][]', 'yellow lemon'));
    $form.append(create$hidden('vegetables[carrot][][]', 'orange and sweet'));
    $form.append(create$hidden('vegetables[carrot][][]', 'brown and rotten'));
  }
);

module('_getFormFromOptions');

test('Form got when directly supplied.', function() {
  var form = document.createElement('form');

  var options = { form: form };
  strictEqual(form, Form2Json._getFormFromOptions(options));
});

asyncTest('Form got from the directly supplied key even if event with another form is supplied too.', function() {
  expect(3);

  var form = document.createElement('form');
  var anotherForm = document.createElement('form');
  notEqual(form, anotherForm);

  $(anotherForm).on('submit', function(event) {
    event.preventDefault();

    var options = {
      form: form,
      event: event
    };

    var gotForm = Form2Json._getFormFromOptions(options);

    strictEqual(form, gotForm);
    notEqual(anotherForm, gotForm); // notStrictEqual would do too
    start();
  }).trigger('submit');
});

asyncTest('Form got from submit event.', function() {
  expect(1);

  var form = document.createElement('form');

  $(form).on('submit', function(event) {
    event.preventDefault();

    var options = { event: event };
    strictEqual(form, Form2Json._getFormFromOptions(options));
    start();
  }).trigger('submit');
});

asyncTest('Form got from button click event.', function() {
  expect(1);

  var form = document.createElement('form');
  var button = document.createElement('input');
  button.type = 'submit';
  $(form).append(button);

  $(button).on('click', function(event) {
    event.preventDefault();

    var options = { event: event };
    strictEqual(form, Form2Json._getFormFromOptions(options));
    start();
  }).trigger('click');
});

module('_cloneForm');

test('Plain empty form is cloned.', function() {
  var form = document.createElement('form');
  var clonedForm = Form2Json._cloneForm(form);

  notEqual(form, clonedForm);
  deepEqual(form, clonedForm);
});

test('Simple empty form is cloned.', function() {
  var form = document.createElement('form');
  form.action = 'http://www.example.com/form?action=submmit#hash';
  form.method = 'POST';
  form.enctype = 'multipart/form-data';
  var clonedForm = Form2Json._cloneForm(form);

  notEqual(form, clonedForm);
  deepEqual(form, clonedForm);
});

test('Non-empty form is cloned as empty.', function() {
  var form = document.createElement('form');
  form.action = 'http://www.example.com/form?action=submmit#hash';
  form.method = 'POST';
  form.enctype = 'multipart/form-data';
  var $form = $(form);

  var input = document.createElement('input');
  var $input = $(input);
  $(form).append(input);

  var clonedForm = Form2Json._cloneForm(form);
  var $clonedForm = $(clonedForm);

  // There really is the input element.
  equal($form.find('input').length, 1);

  // The new form is empty, the old one isn't.
  notEqual(form, clonedForm);
  notDeepEqual(form, clonedForm);

  equal($clonedForm.find('input').length, 0);

  // Remove the input element from the old form to check
  // deep equality.
  $input.remove();
  // It is really removed.
  equal($form.find('input').length, 0);

  // Both forms are now empty – they are not identical,
  // but they have same attributes.
  notEqual(form, clonedForm);
  deepEqual(form, clonedForm);
});

module('_afterInvisible');

test('Element not in DOM raises an error.', function() {
  var before = document.createElement('form');
  var after = document.createElement('form');

  throws(function() {
    Form2Json._afterInvisible(before, after);
  }, 'Element is not in the document's <body>.');
});

test('Element not in body raises an error.', function() {
  var before = document.createElement('form');
  $('head').append(before);
  var after = document.createElement('form');

  throws(function() {
    Form2Json._afterInvisible(before, after);
  }, 'Element is not in the document's <body>.');
});

test('The new element is inserted after the original one.', function() {
  var $body = $('body');
  var before = document.createElement('form');
  $body.append(before);
  var after = document.createElement('form');

  Form2Json._afterInvisible(before, after);

  strictEqual($(before).next()[0], after);

  // Teardown.
  $(before).add(after).remove();
});

test('The new element is invisible.', function() {
  var $body = $('body');
  var before = document.createElement('form');
  $body.append(before);
  var after = document.createElement('form');

  Form2Json._afterInvisible(before, after);

  equal($(before).filter(':visible').length, 1);
  equal($(after).filter(':visible').length, 0);

  // Teardown.
  $(before).add(after).remove();
});

module('submitFormAsJson');

function submitFormAsJsonTest(testOptions) {
  asyncTest(testOptions.testName, function() {
    expect(testOptions.expectCount);

    // Attaching to a body for the cloned form to has interceptable submit event.
    var $body = $('body');
    var $form = create$form(function($form) {
      $.each(submitFormAsJsonTest.formData, function(k, v) {
        $form.append(create$hidden(k, v));
      });
    });
    $body.append($form);

    var callback = function(formData) {
      $(this).on('submit', function(event) {
        event.preventDefault();
        if(typeof testOptions.callback == 'function') {
          testOptions.callback.call(this, event, formData);
        }
        start();
      });
    };

    var options = {
      form: $form[0],
      callback: callback
    };
    if(typeof testOptions.prepare == 'function') {
      testOptions.prepare.call($form[0], options);
    }

    Form2Json.submitFormAsJson(options);

    // Teardown.
    $form.remove();
  });
}
submitFormAsJsonTest.formData = {
  fruit: 'apple',
  vegetable: 'carrot'
};

submitFormAsJsonTest({
  testName: 'Form is submitted with right values.',
  expectCount: 2,
  callback: function(event) {
    var val = $(this).find('input').val();
    var data = {
      fruit: 'apple',
      vegetable: 'carrot'
    };

    equal(val, JSON.stringify(data));
    deepEqual(JSON.parse(val), data);
  }
});

submitFormAsJsonTest({
  testName: 'Form is packed into a field with a default name.',
  expectCount: 2,
  callback: function(event) {
    var $input = $(this).find('input');

    equal($input.length, 1);
    equal($input.attr('name'), Form2Json.submitFormAsJson.defaults.varName);
  }
});

submitFormAsJsonTest({
  testName: 'Form can be packed into a field with a custom name.',
  expectCount: 2,
  prepare: function(options) {
    options.varName = '_JSON_';
  },
  callback: function(event) {
    var $input = $(this).find('input');

    equal($input.length, 1);
    equal($input.attr('name'), '_JSON_');
  }
});

submitFormAsJsonTest({
  testName: 'Callback is called right before submit in the context of the form.',
  expectCount: 2,
  callback: function() {
    var $input = $(this).find('input');

    // The hidden fields is there with the right name.
    // The form is ready to be submitted.
    equal(1, $input.length);
    equal(Form2Json.submitFormAsJson.defaults.varName, $input.attr('name'));
  }
});

submitFormAsJsonTest({
  testName: 'Button clicking sends its value along the form data.',
  expectCount: 1,
  prepare: function(options) {
    options.form = undefined;

    var $submit = $(document.createElement('input'));
    $submit
      .attr('type', 'submit')
      .attr('name', 'button')
      .val('clicked')
      .on('click', function(event) {
        event.preventDefault();
        options.event = event;
      });

    $(this).append($submit);
    $submit.trigger('click');
  },
  callback: function(event, formData) {
    var expected = $.extend({}, submitFormAsJsonTest.formData, {
      button: 'clicked'
    });
    deepEqual(expected, formData);
  }
});

module('jQuery');

test('jQuery object has form2json method callable on collection.', function() {
  var $form = create$form();
  equal('function', typeof $form.form2json);
});

test('jQuery form2json method returns the collection (allows chaining).', function() {
  var $form = create$form();
  strictEqual($form, $form.form2json());
});

// If the submit isn't intercepted, the page would be redirected.
// Is it sufficent to test nothing assuming that if we are still
// here seeing the test results, it means that the tess passed?
// Is there a way to test whether a form has been submitted and
// its submitment intercepted?
asyncTest('Form has its submit method intercepted.', function() {
  expect(1);

  var options = {
    callback: function() {
      $(this).on('submit', function(event) {
        // Intercepting the original form isn't enough. Even though
        // this callback call isn't covered by this test, it has to
        // be used to prevent redirect.
        event.preventDefault();
      });
    }
  };

  var $form = create$form().form2json(options);
  $form.on('submit', function(event) {
    // Don't prevent default here! It's form2json's job.
    equal(true, event.isDefaultPrevented());
    start();
  });
  $form.trigger('submit');
});

test('The OnSubmit method is attached to the form submit event.', function() {
  expect(1);
  var $form = create$form().form2json();
  var events = $._data($form[0], 'events').submit;
  $.each(events, function() {
    if(typeof this.handler.origin != 'undefined' && this.handler.origin === Form2Json.eventHandler) {
      ok(true);
    }
  })
});

asyncTest('The submitFormAsJson method is called, callback being handled.', function() {
  expect(0);

  var options = {
    callback: function() {
      $(this).on('submit', function(event) {
        event.preventDefault();
        // Test just the call itself.
        // Almost identical to the last test, but this time this callback
        // right before submitting the JSON form is tested. Thanks to the
        // last test it can be already assumed that the submitment of the
        // original form is intercepted.
        start();
      });
    }
  };
  var $form = create$form().form2json(options);
  $form.trigger('submit');
});

asyncTest('Button clicking adds its value to the form data.', function() {
  expect(1);

  function formTransform($form) {
    $form.append(create$hidden('fruit', 'apple'));
    $form.append(create$hidden('vegetable', 'carrot'));

    var submit = document.createElement('input');
    submit.type = 'submit';
    submit.name = 'button';
    submit.value = 'clicked';
    $form.append(submit);
  }

  var options = {
    callback: function(formData) {
      var $form = $(this);
      $form.on('submit', function(event) {
        event.preventDefault();

        var expected = {
          fruit: 'apple',
          vegetable: 'carrot',
          button: 'clicked'
        };
        deepEqual(expected, formData);
        start();
      });
    }
  };

  var $form = create$form(formTransform).form2json(options);
  $form.find('input[type=submit]').trigger('click');
});


text:
package com.ritesh.idea.plugin.exception;

/**
 * @author ritesh
 */
public class InvalidCredentialException extends RuntimeException {
    public InvalidCredentialException(String message) {
        super(message);
    }
}


text:
"""
The MIT License (MIT)

Copyright (c) Serenity Software, LLC

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
# pylint: disable=invalid-name,too-many-public-methods,missing-docstring
from cahoots.parsers.programming.lexer import \
    ProgrammingLexerThread, \
    ProgrammingLexer
from pygments.lexers.web import PhpLexer
import unittest

class ProgrammingLexerThreadTests(unittest.TestCase):

    def test_phpLexerDetectsExpectedTokens(self):
        data_string = "echo 'Hello World';"
        lexer = ProgrammingLexerThread(
            'php',
            PhpLexer(startinline=True),
            data_string
        )

        lexer.start()
        lexer.join()

        self.assertEqual(3, lexer.result)

    def test_phpLexerDetectsExpectedNoTokens(self):
        data_string = " "
        lexer = ProgrammingLexerThread(
            'php',
            PhpLexer(startinline=True),
            data_string
        )

        lexer.start()
        lexer.join()

        self.assertFalse(lexer.result)

class ProgrammingLexerTests(unittest.TestCase):

    def test_phpIsDetectedUsingProgrammingLexer(self):
        data_string = "echo 'Hello World';"
        lexer = ProgrammingLexer(['php'], data_string)
        result = lexer.lex()

        self.assertEqual({'php': 3}, result)


text:
<?php

/**
 * Compose HTTP RFC compliant date
 *
 * @phpstub
 *
 * @param int $timestamp
 *
 * @return string Returns the HTTP date as string.
 */
function http_date($timestamp = NULL)
{
}


text:
<!DOCTYPE html>

    <meta charset="utf-8">
    <title>Flattened Pack Example</title>


text:
//  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
//  This source code is licensed under both the GPLv2 (found in the
//  COPYING file in the root directory) and Apache 2.0 License
//  (found in the LICENSE.Apache file in the root directory).

#ifndef ROCKSDB_LITE

#include "rocksdb/utilities/write_batch_with_index.h"

#include <memory>

#include "db/column_family.h"
#include "db/db_impl/db_impl.h"
#include "db/merge_context.h"
#include "db/merge_helper.h"
#include "memory/arena.h"
#include "memtable/skiplist.h"
#include "options/db_options.h"
#include "rocksdb/comparator.h"
#include "rocksdb/iterator.h"
#include "util/cast_util.h"
#include "util/string_util.h"
#include "utilities/write_batch_with_index/write_batch_with_index_internal.h"

namespace ROCKSDB_NAMESPACE {

// when direction == forward
// * current_at_base_ <=> base_iterator > delta_iterator
// when direction == backwards
// * current_at_base_ <=> base_iterator < delta_iterator
// always:
// * equal_keys_ <=> base_iterator == delta_iterator
class BaseDeltaIterator : public Iterator {
 public:
  BaseDeltaIterator(Iterator* base_iterator, WBWIIterator* delta_iterator,
                    const Comparator* comparator,
                    const ReadOptions* read_options = nullptr)
      : forward_(true),
        current_at_base_(true),
        equal_keys_(false),
        status_(Status::OK()),
        base_iterator_(base_iterator),
        delta_iterator_(delta_iterator),
        comparator_(comparator),
        iterate_upper_bound_(read_options ? read_options->iterate_upper_bound
                                          : nullptr) {}

  ~BaseDeltaIterator() override {}

  bool Valid() const override {
    return current_at_base_ ? BaseValid() : DeltaValid();
  }

  void SeekToFirst() override {
    forward_ = true;
    base_iterator_->SeekToFirst();
    delta_iterator_->SeekToFirst();
    UpdateCurrent();
  }

  void SeekToLast() override {
    forward_ = false;
    base_iterator_->SeekToLast();
    delta_iterator_->SeekToLast();
    UpdateCurrent();
  }

  void Seek(const Slice& k) override {
    forward_ = true;
    base_iterator_->Seek(k);
    delta_iterator_->Seek(k);
    UpdateCurrent();
  }

  void SeekForPrev(const Slice& k) override {
    forward_ = false;
    base_iterator_->SeekForPrev(k);
    delta_iterator_->SeekForPrev(k);
    UpdateCurrent();
  }

  void Next() override {
    if (!Valid()) {
      status_ = Status::NotSupported("Next() on invalid iterator");
      return;
    }

    if (!forward_) {
      // Need to change direction
      // if our direction was backward and we're not equal, we have two states:
      // * both iterators are valid: we're already in a good state (current
      // shows to smaller)
      // * only one iterator is valid: we need to advance that iterator
      forward_ = true;
      equal_keys_ = false;
      if (!BaseValid()) {
        assert(DeltaValid());
        base_iterator_->SeekToFirst();
      } else if (!DeltaValid()) {
        delta_iterator_->SeekToFirst();
      } else if (current_at_base_) {
        // Change delta from larger than base to smaller
        AdvanceDelta();
      } else {
        // Change base from larger than delta to smaller
        AdvanceBase();
      }
      if (DeltaValid() && BaseValid()) {
        if (comparator_->Equal(delta_iterator_->Entry().key,
                               base_iterator_->key())) {
          equal_keys_ = true;
        }
      }
    }
    Advance();
  }

  void Prev() override {
    if (!Valid()) {
      status_ = Status::NotSupported("Prev() on invalid iterator");
      return;
    }

    if (forward_) {
      // Need to change direction
      // if our direction was backward and we're not equal, we have two states:
      // * both iterators are valid: we're already in a good state (current
      // shows to smaller)
      // * only one iterator is valid: we need to advance that iterator
      forward_ = false;
      equal_keys_ = false;
      if (!BaseValid()) {
        assert(DeltaValid());
        base_iterator_->SeekToLast();
      } else if (!DeltaValid()) {
        delta_iterator_->SeekToLast();
      } else if (current_at_base_) {
        // Change delta from less advanced than base to more advanced
        AdvanceDelta();
      } else {
        // Change base from less advanced than delta to more advanced
        AdvanceBase();
      }
      if (DeltaValid() && BaseValid()) {
        if (comparator_->Equal(delta_iterator_->Entry().key,
                               base_iterator_->key())) {
          equal_keys_ = true;
        }
      }
    }

    Advance();
  }

  Slice key() const override {
    return current_at_base_ ? base_iterator_->key()
                            : delta_iterator_->Entry().key;
  }

  Slice value() const override {
    return current_at_base_ ? base_iterator_->value()
                            : delta_iterator_->Entry().value;
  }

  Status status() const override {
    if (!status_.ok()) {
      return status_;
    }
    if (!base_iterator_->status().ok()) {
      return base_iterator_->status();
    }
    return delta_iterator_->status();
  }

 private:
  void AssertInvariants() {
#ifndef NDEBUG
    bool not_ok = false;
    if (!base_iterator_->status().ok()) {
      assert(!base_iterator_->Valid());
      not_ok = true;
    }
    if (!delta_iterator_->status().ok()) {
      assert(!delta_iterator_->Valid());
      not_ok = true;
    }
    if (not_ok) {
      assert(!Valid());
      assert(!status().ok());
      return;
    }

    if (!Valid()) {
      return;
    }
    if (!BaseValid()) {
      assert(!current_at_base_ && delta_iterator_->Valid());
      return;
    }
    if (!DeltaValid()) {
      assert(current_at_base_ && base_iterator_->Valid());
      return;
    }
    // we don't support those yet
    assert(delta_iterator_->Entry().type != kMergeRecord &&
           delta_iterator_->Entry().type != kLogDataRecord);
    int compare = comparator_->Compare(delta_iterator_->Entry().key,
                                       base_iterator_->key());
    if (forward_) {
      // current_at_base -> compare < 0
      assert(!current_at_base_ || compare < 0);
      // !current_at_base -> compare <= 0
      assert(current_at_base_ && compare >= 0);
    } else {
      // current_at_base -> compare > 0
      assert(!current_at_base_ || compare > 0);
      // !current_at_base -> compare <= 0
      assert(current_at_base_ && compare <= 0);
    }
    // equal_keys_ <=> compare == 0
    assert((equal_keys_ || compare != 0) && (!equal_keys_ || compare == 0));
#endif
  }

  void Advance() {
    if (equal_keys_) {
      assert(BaseValid() && DeltaValid());
      AdvanceBase();
      AdvanceDelta();
    } else {
      if (current_at_base_) {
        assert(BaseValid());
        AdvanceBase();
      } else {
        assert(DeltaValid());
        AdvanceDelta();
      }
    }
    UpdateCurrent();
  }

  void AdvanceDelta() {
    if (forward_) {
      delta_iterator_->Next();
    } else {
      delta_iterator_->Prev();
    }
  }
  void AdvanceBase() {
    if (forward_) {
      base_iterator_->Next();
    } else {
      base_iterator_->Prev();
    }
  }
  bool BaseValid() const { return base_iterator_->Valid(); }
  bool DeltaValid() const { return delta_iterator_->Valid(); }
  void UpdateCurrent() {
// Suppress false positive clang analyzer warnings.
#ifndef __clang_analyzer__
    status_ = Status::OK();
    while (true) {
      WriteEntry delta_entry;
      if (DeltaValid()) {
        assert(delta_iterator_->status().ok());
        delta_entry = delta_iterator_->Entry();
      } else if (!delta_iterator_->status().ok()) {
        // Expose the error status and stop.
        current_at_base_ = false;
        return;
      }
      equal_keys_ = false;
      if (!BaseValid()) {
        if (!base_iterator_->status().ok()) {
          // Expose the error status and stop.
          current_at_base_ = true;
          return;
        }

        // Base has finished.
        if (!DeltaValid()) {
          // Finished
          return;
        }
        if (iterate_upper_bound_) {
          if (comparator_->Compare(delta_entry.key, *iterate_upper_bound_) >=
              0) {
            // out of upper bound -> finished.
            return;
          }
        }
        if (delta_entry.type == kDeleteRecord ||
            delta_entry.type == kSingleDeleteRecord) {
          AdvanceDelta();
        } else {
          current_at_base_ = false;
          return;
        }
      } else if (!DeltaValid()) {
        // Delta has finished.
        current_at_base_ = true;
        return;
      } else {
        int compare =
            (forward_ ? 1 : -1) *
            comparator_->Compare(delta_entry.key, base_iterator_->key());
        if (compare <= 0) {  // delta bigger or equal
          if (compare == 0) {
            equal_keys_ = true;
          }
          if (delta_entry.type != kDeleteRecord &&
              delta_entry.type != kSingleDeleteRecord) {
            current_at_base_ = false;
            return;
          }
          // Delta is less advanced and is delete.
          AdvanceDelta();
          if (equal_keys_) {
            AdvanceBase();
          }
        } else {
          current_at_base_ = true;
          return;
        }
      }
    }

    AssertInvariants();
#endif  // __clang_analyzer__
  }

  bool forward_;
  bool current_at_base_;
  bool equal_keys_;
  Status status_;
  std::unique_ptr<Iterator> base_iterator_;
  std::unique_ptr<WBWIIterator> delta_iterator_;
  const Comparator* comparator_;  // not owned
  const Slice* iterate_upper_bound_;
};

typedef SkipList<WriteBatchIndexEntry*, const WriteBatchEntryComparator&>
    WriteBatchEntrySkipList;

class WBWIIteratorImpl : public WBWIIterator {
 public:
  WBWIIteratorImpl(uint32_t column_family_id,
                   WriteBatchEntrySkipList* skip_list,
                   const ReadableWriteBatch* write_batch)
      : column_family_id_(column_family_id),
        skip_list_iter_(skip_list),
        write_batch_(write_batch) {}

  ~WBWIIteratorImpl() override {}

  bool Valid() const override {
    if (!skip_list_iter_.Valid()) {
      return false;
    }
    const WriteBatchIndexEntry* iter_entry = skip_list_iter_.key();
    return (iter_entry != nullptr &&
            iter_entry->column_family == column_family_id_);
  }

  void SeekToFirst() override {
    WriteBatchIndexEntry search_entry(
        nullptr /* search_key */, column_family_id_,
        true /* is_forward_direction */, true /* is_seek_to_first */);
    skip_list_iter_.Seek(&search_entry);
  }

  void SeekToLast() override {
    WriteBatchIndexEntry search_entry(
        nullptr /* search_key */, column_family_id_ + 1,
        true /* is_forward_direction */, true /* is_seek_to_first */);
    skip_list_iter_.Seek(&search_entry);
    if (!skip_list_iter_.Valid()) {
      skip_list_iter_.SeekToLast();
    } else {
      skip_list_iter_.Prev();
    }
  }

  void Seek(const Slice& key) override {
    WriteBatchIndexEntry search_entry(&key, column_family_id_,
                                      true /* is_forward_direction */,
                                      false /* is_seek_to_first */);
    skip_list_iter_.Seek(&search_entry);
  }

  void SeekForPrev(const Slice& key) override {
    WriteBatchIndexEntry search_entry(&key, column_family_id_,
                                      false /* is_forward_direction */,
                                      false /* is_seek_to_first */);
    skip_list_iter_.SeekForPrev(&search_entry);
  }

  void Next() override { skip_list_iter_.Next(); }

  void Prev() override { skip_list_iter_.Prev(); }

  WriteEntry Entry() const override {
    WriteEntry ret;
    Slice blob, xid;
    const WriteBatchIndexEntry* iter_entry = skip_list_iter_.key();
    // this is guaranteed with Valid()
    assert(iter_entry != nullptr &&
           iter_entry->column_family == column_family_id_);
    auto s = write_batch_->GetEntryFromDataOffset(
        iter_entry->offset, &ret.type, &ret.key, &ret.value, &blob, &xid);
    assert(s.ok());
    assert(ret.type == kPutRecord || ret.type == kDeleteRecord ||
           ret.type == kSingleDeleteRecord || ret.type == kDeleteRangeRecord ||
           ret.type == kMergeRecord);
    return ret;
  }

  Status status() const override {
    // this is in-memory data structure, so the only way status can be non-ok is
    // through memory corruption
    return Status::OK();
  }

  const WriteBatchIndexEntry* GetRawEntry() const {
    return skip_list_iter_.key();
  }

 private:
  uint32_t column_family_id_;
  WriteBatchEntrySkipList::Iterator skip_list_iter_;
  const ReadableWriteBatch* write_batch_;
};

struct WriteBatchWithIndex::Rep {
  explicit Rep(const Comparator* index_comparator, size_t reserved_bytes = 0,
               size_t max_bytes = 0, bool _overwrite_key = false)
      : write_batch(reserved_bytes, max_bytes),
        comparator(index_comparator, &write_batch),
        skip_list(comparator, &arena),
        overwrite_key(_overwrite_key),
        last_entry_offset(0),
        last_sub_batch_offset(0),
        sub_batch_cnt(1) {}
  ReadableWriteBatch write_batch;
  WriteBatchEntryComparator comparator;
  Arena arena;
  WriteBatchEntrySkipList skip_list;
  bool overwrite_key;
  size_t last_entry_offset;
  // The starting offset of the last sub-batch. A sub-batch starts right before
  // inserting a key that is a duplicate of a key in the last sub-batch. Zero,
  // the default, means that no duplicate key is detected so far.
  size_t last_sub_batch_offset;
  // Total number of sub-batches in the write batch. Default is 1.
  size_t sub_batch_cnt;

  // Remember current offset of internal write batch, which is used as
  // the starting offset of the next record.
  void SetLastEntryOffset() { last_entry_offset = write_batch.GetDataSize(); }

  // In overwrite mode, find the existing entry for the same key and update it
  // to point to the current entry.
  // Return true if the key is found and updated.
  bool UpdateExistingEntry(ColumnFamilyHandle* column_family, const Slice& key);
  bool UpdateExistingEntryWithCfId(uint32_t column_family_id, const Slice& key);

  // Add the recent entry to the update.
  // In overwrite mode, if key already exists in the index, update it.
  void AddOrUpdateIndex(ColumnFamilyHandle* column_family, const Slice& key);
  void AddOrUpdateIndex(const Slice& key);

  // Allocate an index entry pointing to the last entry in the write batch and
  // put it to skip list.
  void AddNewEntry(uint32_t column_family_id);

  // Clear all updates buffered in this batch.
  void Clear();
  void ClearIndex();

  // Rebuild index by reading all records from the batch.
  // Returns non-ok status on corruption.
  Status ReBuildIndex();
};

bool WriteBatchWithIndex::Rep::UpdateExistingEntry(
    ColumnFamilyHandle* column_family, const Slice& key) {
  uint32_t cf_id = GetColumnFamilyID(column_family);
  return UpdateExistingEntryWithCfId(cf_id, key);
}

bool WriteBatchWithIndex::Rep::UpdateExistingEntryWithCfId(
    uint32_t column_family_id, const Slice& key) {
  if (!overwrite_key) {
    return false;
  }

  WBWIIteratorImpl iter(column_family_id, &skip_list, &write_batch);
  iter.Seek(key);
  if (!iter.Valid()) {
    return false;
  }
  if (comparator.CompareKey(column_family_id, key, iter.Entry().key) != 0) {
    return false;
  }
  WriteBatchIndexEntry* non_const_entry =
      const_cast<WriteBatchIndexEntry*>(iter.GetRawEntry());
  if (LIKELY(last_sub_batch_offset <= non_const_entry->offset)) {
    last_sub_batch_offset = last_entry_offset;
    sub_batch_cnt++;
  }
  non_const_entry->offset = last_entry_offset;
  return true;
}

void WriteBatchWithIndex::Rep::AddOrUpdateIndex(
    ColumnFamilyHandle* column_family, const Slice& key) {
  if (!UpdateExistingEntry(column_family, key)) {
    uint32_t cf_id = GetColumnFamilyID(column_family);
    const auto* cf_cmp = GetColumnFamilyUserComparator(column_family);
    if (cf_cmp != nullptr) {
      comparator.SetComparatorForCF(cf_id, cf_cmp);
    }
    AddNewEntry(cf_id);
  }
}

void WriteBatchWithIndex::Rep::AddOrUpdateIndex(const Slice& key) {
  if (!UpdateExistingEntryWithCfId(0, key)) {
    AddNewEntry(0);
  }
}

void WriteBatchWithIndex::Rep::AddNewEntry(uint32_t column_family_id) {
  const std::string& wb_data = write_batch.Data();
  Slice entry_ptr = Slice(wb_data.data() + last_entry_offset,
                          wb_data.size() - last_entry_offset);
  // Extract key
  Slice key;
  bool success __attribute__((__unused__));
  success =
      ReadKeyFromWriteBatchEntry(&entry_ptr, &key, column_family_id != 0);
  assert(success);

  auto* mem = arena.Allocate(sizeof(WriteBatchIndexEntry));
  auto* index_entry =
      new (mem) WriteBatchIndexEntry(last_entry_offset, column_family_id,
                                      key.data() - wb_data.data(), key.size());
  skip_list.Insert(index_entry);
}

void WriteBatchWithIndex::Rep::Clear() {
  write_batch.Clear();
  ClearIndex();
}

void WriteBatchWithIndex::Rep::ClearIndex() {
  skip_list.~WriteBatchEntrySkipList();
  arena.~Arena();
  new (&arena) Arena();
  new (&skip_list) WriteBatchEntrySkipList(comparator, &arena);
  last_entry_offset = 0;
  last_sub_batch_offset = 0;
  sub_batch_cnt = 1;
}

Status WriteBatchWithIndex::Rep::ReBuildIndex() {
  Status s;

  ClearIndex();

  if (write_batch.Count() == 0) {
    // Nothing to re-index
    return s;
  }

  size_t offset = WriteBatchInternal::GetFirstOffset(&write_batch);

  Slice input(write_batch.Data());
  input.remove_prefix(offset);

  // Loop through all entries in Rep and add each one to the index
  uint32_t found = 0;
  while (s.ok() && !input.empty()) {
    Slice key, value, blob, xid;
    uint32_t column_family_id = 0;  // default
    char tag = 0;

    // set offset of current entry for call to AddNewEntry()
    last_entry_offset = input.data() - write_batch.Data().data();

    s = ReadRecordFromWriteBatch(&input, &tag, &column_family_id, &key,
                                  &value, &blob, &xid);
    if (!s.ok()) {
      break;
    }

    switch (tag) {
      case kTypeColumnFamilyValue:
      case kTypeValue:
      case kTypeColumnFamilyDeletion:
      case kTypeDeletion:
      case kTypeColumnFamilySingleDeletion:
      case kTypeSingleDeletion:
      case kTypeColumnFamilyMerge:
      case kTypeMerge:
        found++;
        if (!UpdateExistingEntryWithCfId(column_family_id, key)) {
          AddNewEntry(column_family_id);
        }
        break;
      case kTypeLogData:
      case kTypeBeginPrepareXID:
      case kTypeBeginPersistedPrepareXID:
      case kTypeBeginUnprepareXID:
      case kTypeEndPrepareXID:
      case kTypeCommitXID:
      case kTypeRollbackXID:
      case kTypeNoop:
        break;
      default:
        return Status::Corruption("unknown WriteBatch tag in ReBuildIndex",
                                  ToString(static_cast<unsigned int>(tag)));
    }
  }

  if (s.ok() && found != write_batch.Count()) {
    s = Status::Corruption("WriteBatch has wrong count");
  }

  return s;
}

WriteBatchWithIndex::WriteBatchWithIndex(
    const Comparator* default_index_comparator, size_t reserved_bytes,
    bool overwrite_key, size_t max_bytes)
    : rep(new Rep(default_index_comparator, reserved_bytes, max_bytes,
                  overwrite_key)) {}

WriteBatchWithIndex::~WriteBatchWithIndex() {}

WriteBatchWithIndex::WriteBatchWithIndex(WriteBatchWithIndex&&) = default;

WriteBatchWithIndex& WriteBatchWithIndex::operator=(WriteBatchWithIndex&&) =
    default;

WriteBatch* WriteBatchWithIndex::GetWriteBatch() { return &rep->write_batch; }

size_t WriteBatchWithIndex::SubBatchCnt() { return rep->sub_batch_cnt; }

WBWIIterator* WriteBatchWithIndex::NewIterator() {
  return new WBWIIteratorImpl(0, &(rep->skip_list), &rep->write_batch);
}

WBWIIterator* WriteBatchWithIndex::NewIterator(
    ColumnFamilyHandle* column_family) {
  return new WBWIIteratorImpl(GetColumnFamilyID(column_family),
                              &(rep->skip_list), &rep->write_batch);
}

Iterator* WriteBatchWithIndex::NewIteratorWithBase(
    ColumnFamilyHandle* column_family, Iterator* base_iterator,
    const ReadOptions* read_options) {
  if (rep->overwrite_key == false) {
    assert(false);
    return nullptr;
  }
  return new BaseDeltaIterator(base_iterator, NewIterator(column_family),
                               GetColumnFamilyUserComparator(column_family),
                               read_options);
}

Iterator* WriteBatchWithIndex::NewIteratorWithBase(Iterator* base_iterator) {
  if (rep->overwrite_key == false) {
    assert(false);
    return nullptr;
  }
  // default column family's comparator
  return new BaseDeltaIterator(base_iterator, NewIterator(),
                               rep->comparator.default_comparator());
}

Status WriteBatchWithIndex::Put(ColumnFamilyHandle* column_family,
                                const Slice& key, const Slice& value) {
  rep->SetLastEntryOffset();
  auto s = rep->write_batch.Put(column_family, key, value);
  if (s.ok()) {
    rep->AddOrUpdateIndex(column_family, key);
  }
  return s;
}

Status WriteBatchWithIndex::Put(const Slice& key, const Slice& value) {
  rep->SetLastEntryOffset();
  auto s = rep->write_batch.Put(key, value);
  if (s.ok()) {
    rep->AddOrUpdateIndex(key);
  }
  return s;
}

Status WriteBatchWithIndex::Delete(ColumnFamilyHandle* column_family,
                                   const Slice& key) {
  rep->SetLastEntryOffset();
  auto s = rep->write_batch.Delete(column_family, key);
  if (s.ok()) {
    rep->AddOrUpdateIndex(column_family, key);
  }
  return s;
}

Status WriteBatchWithIndex::Delete(const Slice& key) {
  rep->SetLastEntryOffset();
  auto s = rep->write_batch.Delete(key);
  if (s.ok()) {
    rep->AddOrUpdateIndex(key);
  }
  return s;
}

Status WriteBatchWithIndex::SingleDelete(ColumnFamilyHandle* column_family,
                                         const Slice& key) {
  rep->SetLastEntryOffset();
  auto s = rep->write_batch.SingleDelete(column_family, key);
  if (s.ok()) {
    rep->AddOrUpdateIndex(column_family, key);
  }
  return s;
}

Status WriteBatchWithIndex::SingleDelete(const Slice& key) {
  rep->SetLastEntryOffset();
  auto s = rep->write_batch.SingleDelete(key);
  if (s.ok()) {
    rep->AddOrUpdateIndex(key);
  }
  return s;
}

Status WriteBatchWithIndex::Merge(ColumnFamilyHandle* column_family,
                                  const Slice& key, const Slice& value) {
  rep->SetLastEntryOffset();
  auto s = rep->write_batch.Merge(column_family, key, value);
  if (s.ok()) {
    rep->AddOrUpdateIndex(column_family, key);
  }
  return s;
}

Status WriteBatchWithIndex::Merge(const Slice& key, const Slice& value) {
  rep->SetLastEntryOffset();
  auto s = rep->write_batch.Merge(key, value);
  if (s.ok()) {
    rep->AddOrUpdateIndex(key);
  }
  return s;
}

Status WriteBatchWithIndex::PutLogData(const Slice& blob) {
  return rep->write_batch.PutLogData(blob);
}

void WriteBatchWithIndex::Clear() { rep->Clear(); }

Status WriteBatchWithIndex::GetFromBatch(ColumnFamilyHandle* column_family,
                                         const DBOptions& options,
                                         const Slice& key, std::string* value) {
  Status s;
  MergeContext merge_context;
  const ImmutableDBOptions immuable_db_options(options);

  WriteBatchWithIndexInternal::Result result =
      WriteBatchWithIndexInternal::GetFromBatch(
          immuable_db_options, this, column_family, key, &merge_context,
          &rep->comparator, value, rep->overwrite_key, &s);

  switch (result) {
    case WriteBatchWithIndexInternal::Result::kFound:
    case WriteBatchWithIndexInternal::Result::kError:
      // use returned status
      break;
    case WriteBatchWithIndexInternal::Result::kDeleted:
    case WriteBatchWithIndexInternal::Result::kNotFound:
      s = Status::NotFound();
      break;
    case WriteBatchWithIndexInternal::Result::kMergeInProgress:
      s = Status::MergeInProgress();
      break;
    default:
      assert(false);
  }

  return s;
}

Status WriteBatchWithIndex::GetFromBatchAndDB(DB* db,
                                              const ReadOptions& read_options,
                                              const Slice& key,
                                              std::string* value) {
  assert(value != nullptr);
  PinnableSlice pinnable_val(value);
  assert(!pinnable_val.IsPinned());
  auto s = GetFromBatchAndDB(db, read_options, db->DefaultColumnFamily(), key,
                             &pinnable_val);
  if (s.ok() && pinnable_val.IsPinned()) {
    value->assign(pinnable_val.data(), pinnable_val.size());
  }  // else value is already assigned
  return s;
}

Status WriteBatchWithIndex::GetFromBatchAndDB(DB* db,
                                              const ReadOptions& read_options,
                                              const Slice& key,
                                              PinnableSlice* pinnable_val) {
  return GetFromBatchAndDB(db, read_options, db->DefaultColumnFamily(), key,
                           pinnable_val);
}

Status WriteBatchWithIndex::GetFromBatchAndDB(DB* db,
                                              const ReadOptions& read_options,
                                              ColumnFamilyHandle* column_family,
                                              const Slice& key,
                                              std::string* value) {
  assert(value != nullptr);
  PinnableSlice pinnable_val(value);
  assert(!pinnable_val.IsPinned());
  auto s =
      GetFromBatchAndDB(db, read_options, column_family, key, &pinnable_val);
  if (s.ok() && pinnable_val.IsPinned()) {
    value->assign(pinnable_val.data(), pinnable_val.size());
  }  // else value is already assigned
  return s;
}

Status WriteBatchWithIndex::GetFromBatchAndDB(DB* db,
                                              const ReadOptions& read_options,
                                              ColumnFamilyHandle* column_family,
                                              const Slice& key,
                                              PinnableSlice* pinnable_val) {
  return GetFromBatchAndDB(db, read_options, column_family, key, pinnable_val,
                           nullptr);
}

Status WriteBatchWithIndex::GetFromBatchAndDB(
    DB* db, const ReadOptions& read_options, ColumnFamilyHandle* column_family,
    const Slice& key, PinnableSlice* pinnable_val, ReadCallback* callback) {
  Status s;
  MergeContext merge_context;
  const ImmutableDBOptions& immuable_db_options =
      static_cast_with_check<DBImpl>(db->GetRootDB())->immutable_db_options();

  // Since the lifetime of the WriteBatch is the same as that of the transaction
  // we cannot pin it as otherwise the returned value will not be available
  // after the transaction finishes.
  std::string& batch_value = *pinnable_val->GetSelf();
  WriteBatchWithIndexInternal::Result result =
      WriteBatchWithIndexInternal::GetFromBatch(
          immuable_db_options, this, column_family, key, &merge_context,
          &rep->comparator, &batch_value, rep->overwrite_key, &s);

  if (result == WriteBatchWithIndexInternal::Result::kFound) {
    pinnable_val->PinSelf();
    return s;
  }
  if (result == WriteBatchWithIndexInternal::Result::kDeleted) {
    return Status::NotFound();
  }
  if (result == WriteBatchWithIndexInternal::Result::kError) {
    return s;
  }
  if (result == WriteBatchWithIndexInternal::Result::kMergeInProgress &&
      rep->overwrite_key == true) {
    // Since we've overwritten keys, we do not know what other operations are
    // in this batch for this key, so we cannot do a Merge to compute the
    // result.  Instead, we will simply return MergeInProgress.
    return Status::MergeInProgress();
  }

  assert(result == WriteBatchWithIndexInternal::Result::kMergeInProgress ||
         result == WriteBatchWithIndexInternal::Result::kNotFound);

  // Did not find key in batch OR could not resolve Merges.  Try DB.
  if (!callback) {
    s = db->Get(read_options, column_family, key, pinnable_val);
  } else {
    DBImpl::GetImplOptions get_impl_options;
    get_impl_options.column_family = column_family;
    get_impl_options.value = pinnable_val;
    get_impl_options.callback = callback;
    s = static_cast_with_check<DBImpl>(db->GetRootDB())
            ->GetImpl(read_options, key, get_impl_options);
  }

  if (s.ok() || s.IsNotFound()) {  // DB Get Succeeded
    if (result == WriteBatchWithIndexInternal::Result::kMergeInProgress) {
      // Merge result from DB with merges in Batch
      auto cfh = reinterpret_cast<ColumnFamilyHandleImpl*>(column_family);
      const MergeOperator* merge_operator =
          cfh->cfd()->ioptions()->merge_operator;
      Statistics* statistics = immuable_db_options.statistics.get();
      Env* env = immuable_db_options.env;
      Logger* logger = immuable_db_options.info_log.get();

      Slice* merge_data;
      if (s.ok()) {
        merge_data = pinnable_val;
      } else {  // Key not present in db (s.IsNotFound())
        merge_data = nullptr;
      }

      if (merge_operator) {
        std::string merge_result;
        s = MergeHelper::TimedFullMerge(merge_operator, key, merge_data,
                                        merge_context.GetOperands(),
                                        &merge_result, logger, statistics, env);
        pinnable_val->Reset();
        *pinnable_val->GetSelf() = std::move(merge_result);
        pinnable_val->PinSelf();
      } else {
        s = Status::InvalidArgument("Options::merge_operator must be set");
      }
    }
  }

  return s;
}

void WriteBatchWithIndex::MultiGetFromBatchAndDB(
    DB* db, const ReadOptions& read_options, ColumnFamilyHandle* column_family,
    const size_t num_keys, const Slice* keys, PinnableSlice* values,
    Status* statuses, bool sorted_input) {
  MultiGetFromBatchAndDB(db, read_options, column_family, num_keys, keys,
                         values, statuses, sorted_input, nullptr);
}

void WriteBatchWithIndex::MultiGetFromBatchAndDB(
    DB* db, const ReadOptions& read_options, ColumnFamilyHandle* column_family,
    const size_t num_keys, const Slice* keys, PinnableSlice* values,
    Status* statuses, bool sorted_input, ReadCallback* callback) {
  const ImmutableDBOptions& immuable_db_options =
      static_cast_with_check<DBImpl>(db->GetRootDB())->immutable_db_options();

  autovector<KeyContext, MultiGetContext::MAX_BATCH_SIZE> key_context;
  autovector<KeyContext*, MultiGetContext::MAX_BATCH_SIZE> sorted_keys;
  // To hold merges from the write batch
  autovector<std::pair<WriteBatchWithIndexInternal::Result, MergeContext>,
             MultiGetContext::MAX_BATCH_SIZE>
      merges;
  // Since the lifetime of the WriteBatch is the same as that of the transaction
  // we cannot pin it as otherwise the returned value will not be available
  // after the transaction finishes.
  for (size_t i = 0; i < num_keys; ++i) {
    MergeContext merge_context;
    PinnableSlice* pinnable_val = &values[i];
    std::string& batch_value = *pinnable_val->GetSelf();
    Status* s = &statuses[i];
    WriteBatchWithIndexInternal::Result result =
        WriteBatchWithIndexInternal::GetFromBatch(
            immuable_db_options, this, column_family, keys[i], &merge_context,
            &rep->comparator, &batch_value, rep->overwrite_key, s);

    if (result == WriteBatchWithIndexInternal::Result::kFound) {
      pinnable_val->PinSelf();
      continue;
    }
    if (result == WriteBatchWithIndexInternal::Result::kDeleted) {
      *s = Status::NotFound();
      continue;
    }
    if (result == WriteBatchWithIndexInternal::Result::kError) {
      continue;
    }
    if (result == WriteBatchWithIndexInternal::Result::kMergeInProgress &&
        rep->overwrite_key == true) {
      // Since we've overwritten keys, we do not know what other operations are
      // in this batch for this key, so we cannot do a Merge to compute the
      // result.  Instead, we will simply return MergeInProgress.
      *s = Status::MergeInProgress();
      continue;
    }

    assert(result == WriteBatchWithIndexInternal::Result::kMergeInProgress ||
           result == WriteBatchWithIndexInternal::Result::kNotFound);
    key_context.emplace_back(column_family, keys[i], &values[i],
                             /*timestamp*/ nullptr, &statuses[i]);
    merges.emplace_back(result, std::move(merge_context));
  }

  for (KeyContext& key : key_context) {
    sorted_keys.emplace_back(&key);
  }

  // Did not find key in batch OR could not resolve Merges.  Try DB.
  static_cast_with_check<DBImpl>(db->GetRootDB())
      ->PrepareMultiGetKeys(key_context.size(), sorted_input, &sorted_keys);
  static_cast_with_check<DBImpl>(db->GetRootDB())
      ->MultiGetWithCallback(read_options, column_family, callback,
                             &sorted_keys);

  ColumnFamilyHandleImpl* cfh =
      reinterpret_cast<ColumnFamilyHandleImpl*>(column_family);
  const MergeOperator* merge_operator = cfh->cfd()->ioptions()->merge_operator;
  for (auto iter = key_context.begin(); iter != key_context.end(); ++iter) {
    KeyContext& key = *iter;
    if (key.s->ok() || key.s->IsNotFound()) {  // DB Get Succeeded
      size_t index = iter - key_context.begin();
      std::pair<WriteBatchWithIndexInternal::Result, MergeContext>&
          merge_result = merges[index];
      if (merge_result.first ==
          WriteBatchWithIndexInternal::Result::kMergeInProgress) {
        // Merge result from DB with merges in Batch
        Statistics* statistics = immuable_db_options.statistics.get();
        Env* env = immuable_db_options.env;
        Logger* logger = immuable_db_options.info_log.get();

        Slice* merge_data;
        if (key.s->ok()) {
          merge_data = iter->value;
        } else {  // Key not present in db (s.IsNotFound())
          merge_data = nullptr;
        }

        if (merge_operator) {
          *key.s = MergeHelper::TimedFullMerge(
              merge_operator, *key.key, merge_data,
              merge_result.second.GetOperands(), key.value->GetSelf(), logger,
              statistics, env);
          key.value->PinSelf();
        } else {
          *key.s =
              Status::InvalidArgument("Options::merge_operator must be set");
        }
      }
    }
  }
}

void WriteBatchWithIndex::SetSavePoint() { rep->write_batch.SetSavePoint(); }

Status WriteBatchWithIndex::RollbackToSavePoint() {
  Status s = rep->write_batch.RollbackToSavePoint();

  if (s.ok()) {
    rep->sub_batch_cnt = 1;
    rep->last_sub_batch_offset = 0;
    s = rep->ReBuildIndex();
  }

  return s;
}

Status WriteBatchWithIndex::PopSavePoint() {
  return rep->write_batch.PopSavePoint();
}

void WriteBatchWithIndex::SetMaxBytes(size_t max_bytes) {
  rep->write_batch.SetMaxBytes(max_bytes);
}

size_t WriteBatchWithIndex::GetDataSize() const {
  return rep->write_batch.GetDataSize();
}

}  // namespace ROCKSDB_NAMESPACE
#endif  // !ROCKSDB_LITE


text:
import pika
connection=pika.BlockingConnection(pika.ConnectionParameters(host="10.0.0.25"))  #创建链接
channel=connection.channel()  #创建rabbitMQ频道
channel.queue_declare(queue="hello")  #创建hello名字的队列
channel.basic_publish(exchange="",routing_key="hello",body="hello dkjsadj", properties=pika.BasicProperties(delivery_mode=2,))  #做持久化
connection.close()


text:
GitBucket is a mobile app for GitHub. You can use it anywhere, free of charge.

It is developed using the MVVM architectural pattern and ReactiveCocoa framework.

The code of GitBucket is open source now, you can find it at MVVMReactiveCocoa.


text:
"""Reorganize Pokemon, PokemonForm, etc. to Species, Pokemon, etc.

This is an unmaintained one-shot script, only included in the repo for
reference.

"""

import csv
import os

from pokedex import defaults

number_of_species = 649
high_id_start = 10000

csv_dir = defaults.get_default_csv_dir()

def to_dict(filename):
    fullname = os.path.join(csv_dir, filename)
    reader = csv.reader(open(fullname))
    column_names = reader.next()
    entries = dict()
    for row in reader:
        row_dict = dict(zip(column_names, row))
        entries[row_dict.get('id', row_dict.get('pokemon_id'))] = row_dict
    return entries, column_names

pokemon, pokemon_columns = to_dict('pokemon.csv')
forms, form_columns = to_dict('pokemon_forms.csv')
form_groups, form_group_columns = to_dict('pokemon_form_groups.csv')
evolution_chains, evolution_chain_columns = to_dict('evolution_chains.csv')

result_columns = dict(
    species='''id identifier generation_id evolves_from_species_id
        evolution_chain_id color_id shape_id habitat_id
        growth_rate_id gender_rate capture_rate base_happiness is_baby
        hatch_counter has_gender_differences forms_switchable'''.split(),
    pokemon='''id species_id height weight base_experience order'''.split(),
    form='''id form_identifier pokemon_id introduced_in_version_group_id
        is_default is_battle_only order'''.split(),
    chain='''id baby_trigger_item_id'''.split(),
    )

def normalize_id(id):
    id = int(id)
    if id > number_of_species:
        id = id - high_id_start + number_of_species
    return id

def put(dct, entry):
    """Put entry in dct. If already there, check it's the same.
    """
    id = int(entry['id'])
    if id in dct:
        if entry == dct[id]:
            pass
        else:
            print entry
            print dct[id]
            assert False
    else:
        dct[id] = entry

forms_switchable = dict(
        castform=True,
        unown=False,
        darmanitan=True,
        basculin=False,
        rotom=True,
        shaymin=True,
        deerling=True,
        sawsbuck=True,
        arceus=True,
        pichu=False,
        giratina=True,
        burmy=True,
        wormadam=False,
        deoxys=True,
        genesect=True,
        meloetta=True,
        gastrodon=False,
        cherrim=True,
        shellos=False,
    )

result_species = dict()
result_pokemon = dict()
result_forms = dict()
result_chains = dict()

for form_id, source_form in forms.items():
    pokemon_id = source_form['unique_pokemon_id'] or source_form['form_base_pokemon_id']
    species_id = source_form['form_base_pokemon_id']
    source_pokemon = pokemon[pokemon_id]
    source_evolution_chain = evolution_chains[source_pokemon['evolution_chain_id']]
    try:
        source_group = form_groups[species_id]
    except KeyError:
        source_group = dict(is_battle_only=0)
    all_fields = dict(source_form)
    all_fields.update(source_group)
    all_fields.update(source_pokemon)
    all_fields.update(source_evolution_chain)
    del all_fields['id']
    new_species = dict()
    for column_name in result_columns['species']:
        if column_name == 'id':
            new_species[column_name] = normalize_id(species_id)
        elif column_name == 'evolves_from_species_id':
            new_species[column_name] = pokemon[species_id]['evolves_from_pokemon_id']
        elif column_name == 'shape_id':
            new_species[column_name] = all_fields['pokemon_shape_id']
        elif column_name == 'forms_switchable':
            if species_id in form_groups:
                new_species[column_name] = forms_switchable[source_pokemon['identifier']]
            else:
                new_species[column_name] = 0
        else:
            new_species[column_name] = all_fields[column_name]
    put(result_species, new_species)
    new_pokemon = dict()
    for column_name in result_columns['pokemon']:
        if column_name == 'id':
            new_pokemon[column_name] = normalize_id(pokemon_id)
        elif column_name == 'species_id':
            new_pokemon[column_name] = species_id
        else:
            new_pokemon[column_name] = all_fields[column_name]
    put(result_pokemon, new_pokemon)
    new_form = dict()
    for column_name in result_columns['form']:
        if column_name == 'id':
            new_form[column_name] = normalize_id(form_id)
        elif column_name == 'pokemon_id':
            new_form[column_name] = normalize_id(pokemon_id)
        elif column_name == 'form_identifier':
            new_form[column_name] = source_form['identifier']
        elif column_name == 'is_battle_only':
            if source_form['unique_pokemon_id'] == source_form['form_base_pokemon_id']:
                # Default form, herefore not battle-only
                new_form[column_name] = '0'
            else:
                # Keep
                new_form[column_name] = all_fields[column_name]
        else:
            new_form[column_name] = all_fields[column_name]
    put(result_forms, new_form)
    new_chain = dict(source_evolution_chain)
    del new_chain['growth_rate_id']
    put(result_chains, new_chain)

def write_csv(dct, fieldnames, filename):
    fullname = os.path.join(csv_dir, filename)
    reader = csv.DictWriter(open(fullname, 'w'), fieldnames)
    reader.writerow(dict((n,n) for n in fieldnames))
    for id, row in sorted(dct.items()):
        reader.writerow(row)

write_csv(result_species, result_columns['species'], 'pokemon_species.csv')
write_csv(result_pokemon, result_columns['pokemon'], 'pokemon.csv')
write_csv(result_forms, result_columns['form'], 'pokemon_forms.csv')
write_csv(result_chains, result_columns['chain'], 'evolution_chains.csv')