text:
title: Getting started with Azure Red Hat OpenShift
date: 2019-11-18 11:45:00 +0100
type: post
classes: wide
published: true
status: publish
categories:
- Containers
- Microsoft
- Azure
- Cloud
- Cloud-Native
- Red Hat
tags:
- Microsoft
- Containers
- ARO
- Red Hat
- OpenShift
- Cloud-Native
- Azure
author: juan_manuel_rey
comments: true
---

Azure Red Hat OpenShift was [announced](https://azure.microsoft.com/en-us/blog/openshift-on-azure-the-easiest-fully-managed-openshift-in-the-cloud/) during last year Red Hat Summit, since then the service has been first in private preview and then during this year Red Hat Summit the service [was declared GA](https://azure.microsoft.com/en-us/blog/generally-available-azure-red-hat-openshift/).

After the GA however not every Microsoft and Red Hat customer was able to freely try it in the classic pay as yo go model, instead there was a requirement to reserve four application nodes up-front for the first cluster. However this requirement has been finally lifted during Microsoft Ignite and from now on you can deploy Azure Red Hat OpenShift, or ARO for short, in a PAYG fashion like it should be with any cloud service.

With this in mind I am going to describe the process to deploy your first ARO cluster.

# Prerequisites

Before deploying a cluster we will need to meet several requirements in order to integrate the authentication with Azure Active Directory. All the processes for the prerequisites using Azure Portal are very well described in ARO documentation [here](https://docs.microsoft.com/en-us/azure/openshift/howto-aad-app-configuration#create-an-azure-ad-app-registration), so instead of repeating that part I will instead demonstrate the process using Azure CLI.

We will need to perform two distinct operations:

- Create an Azure AD security group
- Create an Azure AD app registration

## Create an Azure AD security group

First create the Azure AD security group

```
$ az ad group create --display-name aro-group --mail-nickname aro-group -o json
{
  "deletionTimestamp": null,
  "description": null,
  "dirSyncEnabled": null,
  "displayName": "aro-group",
  "lastDirSyncTime": null,
  "mail": null,
  "mailEnabled": false,
  "mailNickname": "aro-group",
  "objectId": "4f2aefef-a111-4f84-b980-99c989a4e0cc",
  "objectType": "Group",
  "odata.metadata": "https://graph.windows.net/b40a365a-9f29-4480-b1f9-28f9179421de/$metadata#directoryObjects/@Element",
  "odata.type": "Microsoft.DirectoryServices.Group",
  "onPremisesDomainName": null,
  "onPremisesNetBiosName": null,
  "onPremisesSamAccountName": null,
  "onPremisesSecurityIdentifier": null,
  "provisioningErrors": [],
  "proxyAddresses": [],
  "securityEnabled": true
}
```

With the group created take note of the `objectId` property, we will need later to create the cluster. Next step is to add the accounts fo the future ARO administrators to this group, in my case I  will add a custom account I created in my tenant.

```
$ az ad group member add --group aro-group --member-id 804fb062-a879-4e72-aaca-8be50e6dc49a
$ az ad group member list --group aro-group --query '[].{Name:displayName, NickName:mailNickname, Id:objectId}' -o json
[
  {
    "Id": "804fb062-a879-4e72-aaca-8be50e6dc49a",
    "Name": "ARO Admin",
    "NickName": "aroadmin"
  }
]
```

## Create an Azure AD app

To be able to integrate our cluster with Azure Active Directory we will need to create an Azure AD application.

```
$ appId=$(az ad app create --display-name aro-aad --query appId -o tsv)
$ echo $appId
57887cd0-868b-4e0d-88a7-519a0ad590cf
```

### Confgure the application

With our application created we need to configure it, first set the owner which in my case is the `aroadmin` user.

```
$ az ad app owner add --id $appId --owner-object-id 804fb062-a879-4e72-aaca-8be50e6dc49a
$ az ad app owner list --id $appId --query '[].{Name:displayName}' -o json
[
  {
    "Name": "ARO Admin"
  }
]
```

Next step is add the permissions to the specific Azure Active Directory Graph APIs. We need to add permissions for `Directory.Read.All` and `User.Read`. The first one is defined as an *Application* type and will allow the app to *Read directory data*. the second will be granted as *Delegated* and will enable the app to *Sing in and read user profile*

```
$ az ad app permission add --id $appId --api 00000002-0000-0000-c000-000000000000 --api-permissions 311a71cc-e848-46a1-bdf8-97ff7156d8e6=Scope 5778995a-e1bf-45b8-affa-663a9f3f4d04=Role
```

Lets dig a bit in the previous command options before we continue.

- `00000002-0000-0000-c000-000000000000` represents Azure Active Directory Graph API
- `311a71cc-e848-46a1-bdf8-97ff7156d8e6` corresponds to `User.Read` API permission and the `Scope` option indicates this would be a delegated permission.
- `5778995a-e1bf-45b8-affa-663a9f3f4d04` corresponds to `Directory.Read.All` permission and the `Role` option tells Azure AD API this would be an application permission.

To finish the API permissions we need to grant admin consent for this application, in order to grant the consent you need to be the Azure AD admin or request an admin to do it.

```
$ az ad app permission admin-consent --id $appId
$ az ad app permission list --id $appId -o json
[
  {
    "additionalProperties": null,
    "expiryTime": "2020-05-15T16:17:56.6007636",
    "resourceAccess": [
      {
        "additionalProperties": null,
        "id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6",
        "type": "Scope"
      },
      {
        "additionalProperties": null,
        "id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
        "type": "Role"
      }
    ],
    "resourceAppId": "00000002-0000-0000-c000-000000000000"
  }
]
```

Generate a secure password by any method and add as secret to the application.

```
$ az ad app update --id $appId --password <SUPER_SECRET_PASSWORD>
```

Finally update the application to disable [implicit grant flow](https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-implicit-grant-flow) using ID Tokens.

```
az ad app update --id $appId --set oauth2AllowIdTokenImplicitFlow=false
```

There is one more setting to configure, the Redirect URI, but we cannot set it until the cluster is created.

# Creating the cluster

Once all prerequisites are finished we can proceed with the cluster creation. Create a new resource group, although this resource group will not contain the resources of the cluster but I will explain thsi later.

```
$ az group create --name arorg --location westeurope
```

The paremeter needed for the create operation are:

- Cluster name.
- Resource group.
- Azure AD Application ID - The ID of the we created in the previous section.
- Azure AD Application secret - The application password we created.
- Azure AD Tenant ID - The Azure AD tenant ID the clsuter will be integrated with.
- Customer Admin Group ID - The ID of the Azure AD security group created before that includes all the admin accounts.
- Application Node count - By default is 4 but in my case I am deploying just two application nodes.

```
$ az openshift create -n aro-cl1 -g arorg -c 2 --aad-client-app-id $apId --aad-client-app-secret <SUPER_SECRET_PASSWORD> --aad-tenant-id b40a365a-9f29-4480-b1f9-2gf9179421de --customer-admin-group-id 4f2aefef-a111-4f84-b980-99c989a4e0cc -l westeurope
$ az openshift list
Name     Location    ResourceGroup    OpenShiftVersion    ProvisioningState    PublicHostname
-------  ----------  ---------------  ------------------  -------------------  ---------------------------------------------------
aro-cl1  westeurope  arorg            v3.11               Succeeded            openshift.3a298ccfc966456481fe.westeurope.azmosa.io
```

## Update the application redirect URI

Update the Azure AD app registration with the URL `https://<aro_public_hostname>/oauth2callback/Azure%20AD`.

```
$ az ad app update --id $appId --reply-urls https://openshift.3a298ccfc966456481fe.westeurope.azmosa.io/oauth2callback/Azure%20AD
$ az ad app show --id $appId --query replyUrls -o json
[
  "https://openshift.3a298ccfc966456481fe.westeurope.azmosa.io/oauth2callback/Azure%20AD"
]
```

# Access the cluster

To access our newly deployed ARO cluster and start working on it we have two main methods.

- OpenShift Console
- OpenShift CLI

## OpenShift Console

From your favorite browser open the URL `https://<aro_public_hostname>` you will be prompted to login with your Azure AD user, use one of the ARO administrator accounts for this first login.

[![](/assets/images/aro_console.png "Azure Red Hat OpenShift Console")]({{site.url}}/assets/images/aro_console.png)

From here we can navigate to either *Cluster Console*, *Application Console* or *Service Catalog* which is the main landing page. I will not go in detail throughout all those sections since they are [perfectly detailed in O[penShift documentation](https://docs.openshift.com/).

## OpenShift CLI

The OpenShift command line is `oc`, which is basically `kuebctl` with additional functionality for OpenShift. To get the `oc` tool access the *Command Line Tools*, the link can be found in the upper right part of the *Service Catalog* are as shown in the following screen capture.

[![](/assets/images/command_line_tools.png)]({{site.url}}/assets/images/command_line_tools.png)

To login into our ARO cluster wiht `oc` we will use a token. Get your token again from the *Service Catalog* area in the upper right corner by clicking in your user name and then in *Copy Login Command*.

[![](/assets/images/login_command.png)]({{site.url}}/assets/images/login_command.png)

Paste the login command in your shell and execute it to log into the cluster.

```
$ oc login https://openshift.3a298ccfc966456481fe.westeurope.azmosa.io --token=bK2dcoYJuWgEGcwOJwuKOtdGMQf_zuUxYBwb_fsu394
Logged into "https://openshift.3a298ccfc966456481fe.westeurope.azmosa.io:443" as "aroadmin@jreypo.onmicrosoft.com" using the token provided.

You have access to the following projects and can switch between them with 'oc project <projectname>':

  * openshift

Using project "openshift".
$ oc get nodes
NAME                        STATUS   ROLES     AGE   VERSION
compute-1573691909-000000   Ready    compute   3d    v1.11.0+d4cacc0
compute-1573691909-000001   Ready    compute   3d    v1.11.0+d4cacc0
infra-1573691909-000000     Ready    infra     3d    v1.11.0+d4cacc0
infra-1573691909-000001     Ready    infra     3d    v1.11.0+d4cacc0
infra-1573691909-000002     Ready    infra     3d    v1.11.0+d4cacc0
master-000000               Ready    master    3d    v1.11.0+d4cacc0
master-000001               Ready    master    3d    v1.11.0+d4cacc0
master-000002               Ready    master    3d    v1.11.0+d4cacc0
```

And that's it we can now start developing and deploying applications in out new shiny ARO cluster. In future posts posts I will show what kind of operations can be done in the cluster from the administrator perspective and will go deeper in ARO internals.

Comments are welcome. Take care!

--Juanma


text:
package com.Voxce.model;

import java.sql.Date;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.Table;

@Entity
@Table(name="inventory")
public class Inventory {

	private int inventory_id;
	private String part_no;
	private String name;
	private String objectType;
	private String mass;
	private String BatchNumber;
	private String mass_unit;
	private int volume;
	private String volume_unit;
	private int concentration;
	private String concentration_unit;
	private int storage_temp;
	private String inv_condition;
	private String status;
	private String category;
	private String origin;
	private String cat_nbr;
	private String lot_nbr;
	private String serial_nbr;
	private Date issuedate;
	private Date expirydate;
	private Date gooduntildate;
	private String hazard;
	private long created_by;
	private Date date_created;
	private long modified_by;
	private Date date_modified;
	private Integer subscriber_id;

	public Inventory() {

	}

	public Inventory(Inventory copyInventory){

		this.inventory_id = copyInventory.inventory_id;
		this.part_no = copyInventory.part_no;
		this.name  = copyInventory.name;
		this.objectType = copyInventory.objectType;
		this.mass = copyInventory.mass;
		this.BatchNumber = copyInventory.BatchNumber;
		this.mass_unit = copyInventory.mass_unit;
		this.volume = copyInventory.volume;
		this.volume_unit = copyInventory.volume_unit;
		this.concentration = copyInventory.concentration;
		this.concentration_unit = copyInventory.concentration_unit;
		this.storage_temp = copyInventory.storage_temp;
		this.inv_condition = copyInventory.inv_condition;
		this.status = copyInventory.status;
		this.category = copyInventory.category;
		this.origin = copyInventory.origin;
		this.cat_nbr = copyInventory.cat_nbr;
		this.lot_nbr = copyInventory.lot_nbr;
		this.serial_nbr = copyInventory.serial_nbr;
		this.issuedate = copyInventory.issuedate;
		this.expirydate = copyInventory.expirydate;
		this.gooduntildate = copyInventory.gooduntildate;
		this.hazard = copyInventory.hazard;
		this.created_by = copyInventory.created_by;
		this.date_created = copyInventory.date_created;
		this.modified_by = copyInventory.modified_by;
		this.date_modified = copyInventory.date_modified;
	}

	@Id
	@GeneratedValue
	@Column(name="inventory_id")
	public int getInventory_id() {
		return inventory_id;
	}
	public void setInventory_id(int inventory_id) {
		this.inventory_id = inventory_id;
	}

	@Column(name="part_no")
	public String getPartNo() {
		return part_no;
	}
	public void setPartNo(String part_no) {
		this.part_no = part_no;
	}

	@Column(name="name")
	public String getName() {
		return name;
	}
	public void setName(String name) {
		this.name = name;
	}

	@Column(name="objectType")
	public String getObjectType() {
		return objectType;
	}

	public void setObjectType(String objectType) {
		this.objectType = objectType;
	}

	@Column(name="mass")
	public String getMass() {
		return mass;
	}
	public void setMass(String mass) {
		this.mass = mass;
	}

	@Column(name="mass_unit")
	public String getMassUnit() {
		return mass_unit;
	}
	public void setMassUnit(String mass_unit) {
		this.mass_unit = mass_unit;
	}

	@Column(name="batchnumber")
	public String getBatchNumber() {
		return BatchNumber;
	}
	public void setBatchNumber(String batch_number) {
		this.BatchNumber = batch_number;
	}

	@Column(name="volume")
	public int getVolume() {
		return volume;
	}
	public void setVolume(int volume) {
		this.volume = volume;
	}

	@Column(name="volume_unit")
	public String getVolumeUnit() {
		return volume_unit;
	}
	public void setVolumeUnit(String volume_unit) {
		this.volume_unit = volume_unit;
	}

	@Column(name="concentration")
	public int getConcentration() {
		return concentration;
	}
	public void setConcentration(int concentration) {
		this.concentration = concentration;
	}

	@Column(name="concentration_unit")
	public String getConcentrationUnit() {
		return concentration_unit;
	}
	public void setConcentrationUnit(String concentration_unit) {
		this.concentration_unit = concentration_unit;
	}

	@Column(name="storage_temp")
	public int getStorageTemp() {
		return storage_temp;
	}
	public void setStorageTemp(int storage_temp) {
		this.storage_temp = storage_temp;
	}

	@Column(name="inv_condition")
	public String getInvCondition() {
		return inv_condition;
	}
	public void setInvCondition(String inv_condition) {
		this.inv_condition = inv_condition;
	}

	@Column(name="status")
	public String getStatus() {
		return status;
	}
	public void setStatus(String status) {
		this.status = status;
	}

	@Column(name="category")
	public String getCategory() {
		return category;
	}
	public void setCategory(String category) {
		this.category = category;
	}

	@Column(name="origin")
	public String getOrigin() {
		return origin;
	}
	public void setOrigin(String origin) {
		this.origin = origin;
	}

	@Column(name="cat_nbr")
	public String getCatNbr() {
		return cat_nbr;
	}
	public void setCatNbr(String cat_nbr) {
		this.cat_nbr = cat_nbr;
	}

	@Column(name="lot_nbr")
	public String getLotNbr() {
		return lot_nbr;
	}
	public void setLotNbr(String lot_nbr) {
		this.lot_nbr = lot_nbr;
	}

	@Column(name="serial_nbr")
	public String getSerialNbr() {
		return serial_nbr;
	}
	public void setSerialNbr(String serial_nbr) {
		this.serial_nbr = serial_nbr;
	}

	@Column(name="hazard")
	public String getHazard() {
		return hazard;
	}
	public void setHazard(String hazard) {
		this.hazard = hazard;
	}

	@Column(name="issuedate")
	public Date getIssueDate() {
		return issuedate;
	}
	public void setIssueDate(Date issuedate) {
		this.issuedate = issuedate;
	}

	@Column(name="expirydate")
	public Date getExpiryDate() {
		return expirydate;
	}
	public void setExpiryDate(Date expirydate) {
		this.expirydate = expirydate;
	}

	@Column(name="gooduntildate")
	public Date getGoodUntilDate() {
		return gooduntildate;
	}
	public void setGoodUntilDate(Date gooduntildate) {
		this.gooduntildate = gooduntildate;
	}

	@Column(name="created_by")
	public long getCreated_by() {
		return created_by;
	}
	public void setCreated_by(long created_by) {
		this.created_by = created_by;
	}

	@Column(name="date_created")
	public Date getDate_created() {
		return date_created;
	}
	public void setDate_created(Date date_created) {
		this.date_created = date_created;
	}

	@Column(name="modified_by")
	public long getModified_by() {
		return modified_by;
	}
	public void setModified_by(long modified_by) {
		this.modified_by = modified_by;
	}

	@Column(name="date_modified")
	public Date getDate_modified() {
		return date_modified;
	}
	public void setDate_modified(Date date_modified) {
		this.date_modified = date_modified;
	}
	@Column(name="subscriber_id")
	public Integer getSubscriber_id() {
		return subscriber_id;
	}

	public void setSubscriber_id(Integer subscriber_id) {
		this.subscriber_id = subscriber_id;
	}

}

redpajama_set_name: RedPajamaGithub


text:
package org.apache.beam.sdk.io.gcp.datastore;

import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Verify.verify;
import static com.google.datastore.v1beta3.PropertyFilter.Operator.EQUAL;
import static com.google.datastore.v1beta3.PropertyOrder.Direction.DESCENDING;
import static com.google.datastore.v1beta3.QueryResultBatch.MoreResultsType.NOT_FINISHED;
import static com.google.datastore.v1beta3.client.DatastoreHelper.makeFilter;
import static com.google.datastore.v1beta3.client.DatastoreHelper.makeOrder;
import static com.google.datastore.v1beta3.client.DatastoreHelper.makeUpsert;
import static com.google.datastore.v1beta3.client.DatastoreHelper.makeValue;

import org.apache.beam.sdk.annotations.Experimental;
import org.apache.beam.sdk.options.GcpOptions;
import org.apache.beam.sdk.options.PipelineOptions;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.Flatten;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.Values;
import org.apache.beam.sdk.transforms.display.DisplayData;
import org.apache.beam.sdk.transforms.display.DisplayData.Builder;
import org.apache.beam.sdk.util.AttemptBoundedExponentialBackOff;
import org.apache.beam.sdk.util.RetryHttpRequestInitializer;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PBegin;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PDone;

import com.google.api.client.auth.oauth2.Credential;
import com.google.api.client.util.BackOff;
import com.google.api.client.util.BackOffUtils;
import com.google.api.client.util.Sleeper;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.MoreObjects;
import com.google.common.collect.ImmutableList;
import com.google.datastore.v1beta3.CommitRequest;
import com.google.datastore.v1beta3.Entity;
import com.google.datastore.v1beta3.EntityResult;
import com.google.datastore.v1beta3.Key;
import com.google.datastore.v1beta3.Key.PathElement;
import com.google.datastore.v1beta3.PartitionId;
import com.google.datastore.v1beta3.Query;
import com.google.datastore.v1beta3.QueryResultBatch;
import com.google.datastore.v1beta3.RunQueryRequest;
import com.google.datastore.v1beta3.RunQueryResponse;
import com.google.datastore.v1beta3.client.Datastore;
import com.google.datastore.v1beta3.client.DatastoreException;
import com.google.datastore.v1beta3.client.DatastoreFactory;
import com.google.datastore.v1beta3.client.DatastoreHelper;
import com.google.datastore.v1beta3.client.DatastoreOptions;
import com.google.datastore.v1beta3.client.QuerySplitter;
import com.google.protobuf.Int32Value;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.NoSuchElementException;
import javax.annotation.Nullable;

/**
 *
{@link V1Beta3} provides an API to Read and Write {@link PCollection PCollections} of
 * Google Cloud Datastore version v1beta3
 * {@link Entity} objects.
 *
 *
This API currently requires an authentication workaround. To use {@link V1Beta3}, users
 * must use the {@code gcloud} command line tool to get credentials for Datastore:
 *

 * $ gcloud auth login
 *

 *
 *
To read a {@link PCollection} from a query to Datastore, use {@link V1Beta3#read} and
 * its methods {@link V1Beta3.Read#withProjectId} and {@link V1Beta3.Read#withQuery} to
 * specify the project to query and the query to read from. You can optionally provide a namespace
 * to query within using {@link V1Beta3.Read#withNamespace}. You could also optionally specify
 * how many splits you want for the query using {@link V1Beta3.Read#withNumQuerySplits}.
 *
 *
For example:
 *
 *
 {@code
 * // Read a query from Datastore
 * PipelineOptions options = PipelineOptionsFactory.fromArgs(args).create();
 * Query query = ...;
 * String projectId = "...";
 *
 * Pipeline p = Pipeline.create(options);
 * PCollection<Entity> entities = p.apply(
 *     DatastoreIO.v1beta3().read()
 *         .withProjectId(projectId)
 *         .withQuery(query));
 * }

 *
 *
Note: Normally, a Cloud Dataflow job will read from Cloud Datastore in parallel across
 * many workers. However, when the {@link Query} is configured with a limit using
 * {@link com.google.datastore.v1beta3.Query.Builder#setLimit(Int32Value)}, then
 * all returned results will be read by a single Dataflow worker in order to ensure correct data.
 *
 *
To write a {@link PCollection} to a Datastore, use {@link V1Beta3#write},
 * specifying the Cloud Datastore project to write to:
 *
 *
 {@code
 * PCollection<Entity> entities = ...;
 * entities.apply(DatastoreIO.v1beta3().write().withProjectId(projectId));
 * p.run();
 * }

 *
 *
{@link Entity Entities} in the {@code PCollection} to be written must have complete
 * {@link Key Keys}. Complete {@code Keys} specify the {@code name} and {@code id} of the
 * {@code Entity}, where incomplete {@code Keys} do not. A {@code namespace} other than
 * {@code projectId} default may be used by specifying it in the {@code Entity} {@code Keys}.
 *
 *
{@code
 * Key.Builder keyBuilder = DatastoreHelper.makeKey(...);
 * keyBuilder.getPartitionIdBuilder().setNamespace(namespace);
 * }

 *
 *
{@code Entities} will be committed as upsert (update or insert) mutations. Please read
 * Entities, Properties, and
 * Keys for more information about {@code Entity} keys.
 *
 *

Permissions

 * Permission requirements depend on the {@code PipelineRunner} that is used to execute the
 * Dataflow job. Please refer to the documentation of corresponding {@code PipelineRunner}s for
 * more details.
 *
 *
Please see Cloud Datastore Sign Up
 * for security and permission related information specific to Datastore.
 *
 * @see org.apache.beam.sdk.runners.PipelineRunner
 */
@Experimental(Experimental.Kind.SOURCE_SINK)
public class V1Beta3 {

  // A package-private constructor to prevent direct instantiation from outside of this package
  V1Beta3() {}

  /**
   * Datastore has a limit of 500 mutations per batch operation, so we flush
   * changes to Datastore every 500 entities.
   */
  @VisibleForTesting
  static final int DATASTORE_BATCH_UPDATE_LIMIT = 500;

  /**
   * Returns an empty {@link V1Beta3.Read} builder. Configure the source {@code projectId},
   * {@code query}, and optionally {@code namespace} and {@code numQuerySplits} using
   * {@link V1Beta3.Read#withProjectId}, {@link V1Beta3.Read#withQuery},
   * {@link V1Beta3.Read#withNamespace}, {@link V1Beta3.Read#withNumQuerySplits}.
   */
  public V1Beta3.Read read() {
    return new V1Beta3.Read(null, null, null, 0);
  }

  /**
   * A {@link PTransform} that reads the result rows of a Datastore query as {@code Entity}
   * objects.
   *
   * @see DatastoreIO
   */
  public static class Read extends PTransform<PBegin, PCollection<Entity>> {
    private static final Logger LOG = LoggerFactory.getLogger(Read.class);

    /** An upper bound on the number of splits for a query. */
    public static final int NUM_QUERY_SPLITS_MAX = 50000;

    /** A lower bound on the number of splits for a query. */
    static final int NUM_QUERY_SPLITS_MIN = 12;

    /** Default bundle size of 64MB. */
    static final long DEFAULT_BUNDLE_SIZE_BYTES = 64 * 1024 * 1024;

    /**
     * Maximum number of results to request per query.
     *
     *
Must be set, or it may result in an I/O error when querying Cloud Datastore.
     */
    static final int QUERY_BATCH_LIMIT = 500;

    @Nullable
    private final String projectId;

    @Nullable
    private final Query query;

    @Nullable
    private final String namespace;

    private final int numQuerySplits;

    /**
     * Computes the number of splits to be performed on the given query by querying the estimated
     * size from Datastore.
     */
    static int getEstimatedNumSplits(Datastore datastore, Query query, @Nullable String namespace) {
      int numSplits;
      try {
        long estimatedSizeBytes = getEstimatedSizeBytes(datastore, query, namespace);
        numSplits = (int) Math.min(NUM_QUERY_SPLITS_MAX,
            Math.round(((double) estimatedSizeBytes) / DEFAULT_BUNDLE_SIZE_BYTES));
      } catch (Exception e) {
        LOG.warn("Failed the fetch estimatedSizeBytes for query: {}", query, e);
        // Fallback in case estimated size is unavailable.
        numSplits = NUM_QUERY_SPLITS_MIN;
      }
      return Math.max(numSplits, NUM_QUERY_SPLITS_MIN);
    }

    /**
     * Get the estimated size of the data returned by the given query.
     *
     *
Datastore provides no way to get a good estimate of how large the result of a query
     * entity kind being queried, using the __Stat_Kind__ system table, assuming exactly 1 kind
     * is specified in the query.
     *
     *
See https://cloud.google.com/datastore/docs/concepts/stats.
     */
    static long getEstimatedSizeBytes(Datastore datastore, Query query, @Nullable String namespace)
        throws DatastoreException {
      String ourKind = query.getKind(0).getName();
      Query.Builder queryBuilder = Query.newBuilder();
      if (namespace == null) {
        queryBuilder.addKindBuilder().setName("__Stat_Kind__");
      } else {
        queryBuilder.addKindBuilder().setName("__Ns_Stat_Kind__");
      }
      queryBuilder.setFilter(makeFilter("kind_name", EQUAL, makeValue(ourKind).build()));

      // Get the latest statistics
      queryBuilder.addOrder(makeOrder("timestamp", DESCENDING));
      queryBuilder.setLimit(Int32Value.newBuilder().setValue(1));

      RunQueryRequest request = makeRequest(queryBuilder.build(), namespace);

      long now = System.currentTimeMillis();
      RunQueryResponse response = datastore.runQuery(request);
      LOG.debug("Query for per-kind statistics took {}ms", System.currentTimeMillis() - now);

      QueryResultBatch batch = response.getBatch();
      if (batch.getEntityResultsCount() == 0) {
        throw new NoSuchElementException(
            "Datastore statistics for kind " + ourKind + " unavailable");
      }
      Entity entity = batch.getEntityResults(0).getEntity();
      return entity.getProperties().get("entity_bytes").getIntegerValue();
    }

    /** Builds a {@link RunQueryRequest} from the {@code query} and {@code namespace}. */
    static RunQueryRequest makeRequest(Query query, @Nullable String namespace) {
      RunQueryRequest.Builder requestBuilder = RunQueryRequest.newBuilder().setQuery(query);
      if (namespace != null) {
        requestBuilder.getPartitionIdBuilder().setNamespaceId(namespace);
      }
      return requestBuilder.build();
    }

    /**
     * A helper function to get the split queries, taking into account the optional
     * {@code namespace}.
     */
    private static List<Query> splitQuery(Query query, @Nullable String namespace,
        Datastore datastore, QuerySplitter querySplitter, int numSplits) throws DatastoreException {
      // If namespace is set, include it in the split request so splits are calculated accordingly.
      PartitionId.Builder partitionBuilder = PartitionId.newBuilder();
      if (namespace != null) {
        partitionBuilder.setNamespaceId(namespace);
      }

      return querySplitter.getSplits(query, partitionBuilder.build(), numSplits, datastore);
    }

    /**
     * Note that only {@code namespace} is really {@code @Nullable}. The other parameters may be
     * {@code null} as a matter of build order, but if they are {@code null} at instantiation time,
     * an error will be thrown.
     */
    private Read(@Nullable String projectId, @Nullable Query query, @Nullable String namespace,
        int numQuerySplits) {
      this.projectId = projectId;
      this.query = query;
      this.namespace = namespace;
      this.numQuerySplits = numQuerySplits;
    }

    /**
     * Returns a new {@link V1Beta3.Read} that reads from the Datastore for the specified project.
     */
    public V1Beta3.Read withProjectId(String projectId) {
      checkNotNull(projectId, "projectId");
      return new V1Beta3.Read(projectId, query, namespace, numQuerySplits);
    }

    /**
     * Returns a new {@link V1Beta3.Read} that reads the results of the specified query.
     *
     *
Note: Normally, {@code DatastoreIO} will read from Cloud Datastore in parallel
     * across many workers. However, when the {@link Query} is configured with a limit using
     * {@link Query.Builder#setLimit}, then all results will be read by a single worker in order
     * to ensure correct results.
     */
    public V1Beta3.Read withQuery(Query query) {
      checkNotNull(query, "query");
      checkArgument(!query.hasLimit() || query.getLimit().getValue() > 0,
          "Invalid query limit %s: must be positive", query.getLimit().getValue());
      return new V1Beta3.Read(projectId, query, namespace, numQuerySplits);
    }

    /**
     * Returns a new {@link V1Beta3.Read} that reads from the given namespace.
     */
    public V1Beta3.Read withNamespace(String namespace) {
      return new V1Beta3.Read(projectId, query, namespace, numQuerySplits);
    }

    /**
     * Returns a new {@link V1Beta3.Read} that reads by splitting the given {@code query} into
     * {@code numQuerySplits}.
     *
     *
The semantics for the query splitting is defined below:
     *

     *
Any value less than or equal to 0 will be ignored, and the number of splits will be
     *   chosen dynamically at runtime based on the query data size.
     *
Any value greater than {@link Read#NUM_QUERY_SPLITS_MAX} will be capped at
     *   {@code NUM_QUERY_SPLITS_MAX}.
     *
If the {@code query} has a user limit set, then {@code numQuerySplits} will be
     *   ignored and no split will be performed.
     *
Under certain cases Cloud Datastore is unable to split query to the requested number of
     *   splits. In such cases we just use whatever the Datastore returns.
     *

     */
    public V1Beta3.Read withNumQuerySplits(int numQuerySplits) {
      return new V1Beta3.Read(projectId, query, namespace,
          Math.min(Math.max(numQuerySplits, 0), NUM_QUERY_SPLITS_MAX));
    }

    @Nullable
    public Query getQuery() {
      return query;
    }

    @Nullable
    public String getProjectId() {
      return projectId;
    }

    @Nullable
    public String getNamespace() {
      return namespace;
    }

    /**
     * {@inheritDoc}
     */
    @Override
    public PCollection<Entity> apply(PBegin input) {
      V1Beta3Options v1Beta3Options = V1Beta3Options.from(getProjectId(), getQuery(),
          getNamespace());

      /*
       * This composite transform involves the following steps:
       *   1. Create a singleton of the user provided {@code query} and apply a {@link ParDo} that
       *   splits the query into {@code numQuerySplits} and assign each split query a unique
       *   {@code Integer} as the key. The resulting output is of the type
       *   {@code PCollection<KV<Integer, Query>>}.
       *
       *   If the value of {@code numQuerySplits} is less than or equal to 0, then the number of
       *   splits will be computed dynamically based on the size of the data for the {@code query}.
       *
       *   2. The resulting {@code PCollection} is sharded using a {@link GroupByKey} operation. The
       *   queries are extracted from they {@code KV<Integer, Iterable<Query>>} and flattened to
       *   output a {@code PCollection<Query>}.
       *
       *   3. In the third step, a {@code ParDo} reads entities for each query and outputs
       *   a {@code PCollection<Entity>}.
       */
      PCollection<KV<Integer, Query>> queries = input
          .apply(Create.of(query))
          .apply(ParDo.of(new SplitQueryFn(v1Beta3Options, numQuerySplits)));

      PCollection<Query> shardedQueries = queries
          .apply(GroupByKey.<Integer, Query>create())
          .apply(Values.<Iterable<Query>>create())
          .apply(Flatten.<Query>iterables());

      PCollection<Entity> entities = shardedQueries
          .apply(ParDo.of(new ReadFn(v1Beta3Options)));

      return entities;
    }

    @Override
    public void validate(PBegin input) {
      checkNotNull(projectId, "projectId");
      checkNotNull(query, "query");
    }

    @Override
    public void populateDisplayData(DisplayData.Builder builder) {
      super.populateDisplayData(builder);
      builder
          .addIfNotNull(DisplayData.item("projectId", projectId)
              .withLabel("ProjectId"))
          .addIfNotNull(DisplayData.item("namespace", namespace)
              .withLabel("Namespace"))
          .addIfNotNull(DisplayData.item("query", query.toString())
              .withLabel("Query"));
    }

    @Override
    public String toString() {
      return MoreObjects.toStringHelper(getClass())
          .add("projectId", projectId)
          .add("query", query)
          .add("namespace", namespace)
          .toString();
    }

    /**
     * A class for v1beta3 Datastore related options.
     */
    @VisibleForTesting
    static class V1Beta3Options implements Serializable {
      private final Query query;
      private final String projectId;
      @Nullable
      private final String namespace;

      private V1Beta3Options(String projectId, Query query, @Nullable String namespace) {
        this.projectId = checkNotNull(projectId, "projectId");
        this.query = checkNotNull(query, "query");
        this.namespace = namespace;
      }

      public static V1Beta3Options from(String projectId, Query query, @Nullable String namespace) {
        return new V1Beta3Options(projectId, query, namespace);
      }

      public Query getQuery() {
        return query;
      }

      public String getProjectId() {
        return projectId;
      }

      @Nullable
      public String getNamespace() {
        return namespace;
      }
    }

    /**
     * A {@link DoFn} that splits a given query into multiple sub-queries, assigns them unique
     * keys and outputs them as {@link KV}.
     */
    @VisibleForTesting
    static class SplitQueryFn extends DoFn<Query, KV<Integer, Query>> {
      private final V1Beta3Options options;
      // number of splits to make for a given query
      private final int numSplits;

      private final V1Beta3DatastoreFactory datastoreFactory;
      // Datastore client
      private transient Datastore datastore;
      // Query splitter
      private transient QuerySplitter querySplitter;

      public SplitQueryFn(V1Beta3Options options, int numSplits) {
        this(options, numSplits, new V1Beta3DatastoreFactory());
      }

      @VisibleForTesting
      SplitQueryFn(V1Beta3Options options, int numSplits,
          V1Beta3DatastoreFactory datastoreFactory) {
        this.options = options;
        this.numSplits = numSplits;
        this.datastoreFactory = datastoreFactory;
      }

      @StartBundle
      public void startBundle(Context c) throws Exception {
        datastore = datastoreFactory.getDatastore(c.getPipelineOptions(), options.projectId);
        querySplitter = datastoreFactory.getQuerySplitter();
      }

      @ProcessElement
      public void processElement(ProcessContext c) throws Exception {
        int key = 1;
        Query query = c.element();

        // If query has a user set limit, then do not split.
        if (query.hasLimit()) {
          c.output(KV.of(key, query));
          return;
        }

        int estimatedNumSplits;
        // Compute the estimated numSplits if numSplits is not specified by the user.
        if (numSplits <= 0) {
          estimatedNumSplits = getEstimatedNumSplits(datastore, query, options.getNamespace());
        } else {
          estimatedNumSplits = numSplits;
        }

        List<Query> querySplits;
        try {
          querySplits = splitQuery(query, options.getNamespace(), datastore, querySplitter,
              estimatedNumSplits);
        } catch (Exception e) {
          LOG.warn("Unable to parallelize the given query: {}", query, e);
          querySplits = ImmutableList.of(query);
        }

        // assign unique keys to query splits.
        for (Query subquery : querySplits) {
          c.output(KV.of(key++, subquery));
        }
      }

      @Override
      public void populateDisplayData(Builder builder) {
        super.populateDisplayData(builder);
        builder
            .addIfNotNull(DisplayData.item("projectId", options.getProjectId())
                .withLabel("ProjectId"))
            .addIfNotNull(DisplayData.item("namespace", options.getNamespace())
                .withLabel("Namespace"))
            .addIfNotNull(DisplayData.item("query", options.getQuery().toString())
                .withLabel("Query"));
      }
    }

    /**
     * A {@link DoFn} that reads entities from Datastore for each query.
     */
    @VisibleForTesting
    static class ReadFn extends DoFn<Query, Entity> {
      private final V1Beta3Options options;
      private final V1Beta3DatastoreFactory datastoreFactory;
      // Datastore client
      private transient Datastore datastore;

      public ReadFn(V1Beta3Options options) {
        this(options, new V1Beta3DatastoreFactory());
      }

      @VisibleForTesting
      ReadFn(V1Beta3Options options, V1Beta3DatastoreFactory datastoreFactory) {
        this.options = options;
        this.datastoreFactory = datastoreFactory;
      }

      @StartBundle
      public void startBundle(Context c) throws Exception {
        datastore = datastoreFactory.getDatastore(c.getPipelineOptions(), options.getProjectId());
      }

      /** Read and output entities for the given query. */
      @ProcessElement
      public void processElement(ProcessContext context) throws Exception {
        Query query = context.element();
        String namespace = options.getNamespace();
        int userLimit = query.hasLimit()
            ? query.getLimit().getValue() : Integer.MAX_VALUE;

        boolean moreResults = true;
        QueryResultBatch currentBatch = null;

        while (moreResults) {
          Query.Builder queryBuilder = query.toBuilder().clone();
          queryBuilder.setLimit(Int32Value.newBuilder().setValue(
              Math.min(userLimit, QUERY_BATCH_LIMIT)));

          if (currentBatch != null && !currentBatch.getEndCursor().isEmpty()) {
            queryBuilder.setStartCursor(currentBatch.getEndCursor());
          }

          RunQueryRequest request = makeRequest(queryBuilder.build(), namespace);
          RunQueryResponse response = datastore.runQuery(request);

          currentBatch = response.getBatch();

          // MORE_RESULTS_AFTER_LIMIT is not implemented yet:
          // https://groups.google.com/forum/#!topic/gcd-discuss/iNs6M1jA2Vw, so
          // use result count to determine if more results might exist.
          int numFetch = currentBatch.getEntityResultsCount();
          if (query.hasLimit()) {
            verify(userLimit >= numFetch,
                "Expected userLimit %s >= numFetch %s, because query limit %s must be <= userLimit",
                userLimit, numFetch, query.getLimit());
            userLimit -= numFetch;
          }

          // output all the entities from the current batch.
          for (EntityResult entityResult : currentBatch.getEntityResultsList()) {
            context.output(entityResult.getEntity());
          }

          // Check if we have more entities to be read.
          moreResults =
              // User-limit does not exist (so userLimit == MAX_VALUE) and/or has not been satisfied
              (userLimit > 0)
                  // All indications from the API are that there are/may be more results.
                  && ((numFetch == QUERY_BATCH_LIMIT)
                  || (currentBatch.getMoreResults() == NOT_FINISHED));
        }
      }
    }
  }

  /**
   * Returns an empty {@link V1Beta3.Write} builder. Configure the destination
   * {@code projectId} using {@link V1Beta3.Write#withProjectId}.
   */
  public Write write() {
    return new Write(null);
  }

  /**
   * A {@link PTransform} that writes {@link Entity} objects to Cloud Datastore.
   *
   * @see DatastoreIO
   */
  public static class Write extends PTransform<PCollection<Entity>, PDone> {
    @Nullable
    private final String projectId;

    /**
     * Note that {@code projectId} is only {@code @Nullable} as a matter of build order, but if
     * it is {@code null} at instantiation time, an error will be thrown.
     */
    public Write(@Nullable String projectId) {
      this.projectId = projectId;
    }

    /**
     * Returns a new {@link Write} that writes to the Cloud Datastore for the specified project.
     */
    public Write withProjectId(String projectId) {
      checkNotNull(projectId, "projectId");
      return new Write(projectId);
    }

    @Override
    public PDone apply(PCollection<Entity> input) {
      input.apply(ParDo.of(new DatastoreWriterFn(projectId)));
      return PDone.in(input.getPipeline());
    }

    @Override
    public void validate(PCollection<Entity> input) {
      checkNotNull(projectId, "projectId");
    }

    @Nullable
    public String getProjectId() {
      return projectId;
    }

    @Override
    public String toString() {
      return MoreObjects.toStringHelper(getClass())
          .add("projectId", projectId)
          .toString();
    }

    @Override
    public void populateDisplayData(DisplayData.Builder builder) {
      super.populateDisplayData(builder);
      builder
          .addIfNotNull(DisplayData.item("projectId", projectId)
              .withLabel("Output Project"));
    }

    /**
     * A {@link DoFn} that writes {@link Entity} objects to Cloud Datastore. Entities are written in
     * batches, where the maximum batch size is {@link V1Beta3#DATASTORE_BATCH_UPDATE_LIMIT}.
     * Entities are committed as upsert mutations (either update if the key already exists, or
     * insert if it is a new key). If an entity does not have a complete key (i.e., it has no name
     * or id), the bundle will fail.
     *
     *
See
     * Datastore: Entities, Properties, and Keys for information about entity keys and entities.
     *
     *
Commits are non-transactional.  If a commit fails because of a conflict over an entity
     * group, the commit will be retried (up to {@link V1Beta3#DATASTORE_BATCH_UPDATE_LIMIT}
     * times).
     */
    @VisibleForTesting
    static class DatastoreWriterFn extends DoFn<Entity, Void> {
      private static final Logger LOG = LoggerFactory.getLogger(DatastoreWriterFn.class);
      private final String projectId;
      private transient Datastore datastore;
      private final V1Beta3DatastoreFactory datastoreFactory;
      // Current batch of entities to be written.
      private final List<Entity> entities = new ArrayList<>();
      /**
       * Since a bundle is written in batches, we should retry the commit of a batch in order to
       * prevent transient errors from causing the bundle to fail.
       */
      private static final int MAX_RETRIES = 5;

      /**
       * Initial backoff time for exponential backoff for retry attempts.
       */
      private static final int INITIAL_BACKOFF_MILLIS = 5000;

      public DatastoreWriterFn(String projectId) {
        this(projectId, new V1Beta3DatastoreFactory());
      }

      @VisibleForTesting
      DatastoreWriterFn(String projectId, V1Beta3DatastoreFactory datastoreFactory) {
        this.projectId = checkNotNull(projectId, "projectId");
        this.datastoreFactory = datastoreFactory;
      }

      @StartBundle
      public void startBundle(Context c) {
        datastore = datastoreFactory.getDatastore(c.getPipelineOptions(), projectId);
      }

      @ProcessElement
      public void processElement(ProcessContext c) throws Exception {
        // Verify that the entity to write has a complete key.
        if (!isValidKey(c.element().getKey())) {
          throw new IllegalArgumentException(
              "Entities to be written to the Datastore must have complete keys");
        }

        entities.add(c.element());
        if (entities.size() >= V1Beta3.DATASTORE_BATCH_UPDATE_LIMIT) {
          flushBatch();
        }
      }

      @FinishBundle
      public void finishBundle(Context c) throws Exception {
        if (entities.size() > 0) {
          flushBatch();
        }
      }

      /**
       * Writes a batch of entities to Cloud Datastore.
       *
       *
If a commit fails, it will be retried (up to {@link DatastoreWriterFn#MAX_RETRIES}
       * times). All entities in the batch will be committed again, even if the commit was partially
       * successful. If the retry limit is exceeded, the last exception from the Datastore will be
       * thrown.
       *
       * @throws DatastoreException if the commit fails or IOException or InterruptedException if
       * backing off between retries fails.
       */
      private void flushBatch() throws DatastoreException, IOException, InterruptedException {
        LOG.debug("Writing batch of {} entities", entities.size());
        Sleeper sleeper = Sleeper.DEFAULT;
        BackOff backoff = new AttemptBoundedExponentialBackOff(MAX_RETRIES, INITIAL_BACKOFF_MILLIS);

        while (true) {
          // Batch upsert entities.
          try {
            CommitRequest.Builder commitRequest = CommitRequest.newBuilder();
            for (Entity entity: entities) {
              commitRequest.addMutations(makeUpsert(entity));
            }
            commitRequest.setMode(CommitRequest.Mode.NON_TRANSACTIONAL);
            datastore.commit(commitRequest.build());
            // Break if the commit threw no exception.
            break;
          } catch (DatastoreException exception) {
            // Only log the code and message for potentially-transient errors. The entire exception
            // will be propagated upon the last retry.
            LOG.error("Error writing to the Datastore ({}): {}", exception.getCode(),
                exception.getMessage());
            if (!BackOffUtils.next(sleeper, backoff)) {
              LOG.error("Aborting after {} retries.", MAX_RETRIES);
              throw exception;
            }
          }
        }
        LOG.debug("Successfully wrote {} entities", entities.size());
        entities.clear();
      }

      @Override
      public void populateDisplayData(Builder builder) {
        super.populateDisplayData(builder);
        builder
            .addIfNotNull(DisplayData.item("projectId", projectId)
                .withLabel("Output Project"));
      }
    }

    /**
     * Returns true if a Datastore key is complete.  A key is complete if its last element
     * has either an id or a name.
     */
    static boolean isValidKey(Key key) {
      List<PathElement> elementList = key.getPathList();
      if (elementList.isEmpty()) {
        return false;
      }
      PathElement lastElement = elementList.get(elementList.size() - 1);
      return (lastElement.getId() != 0 || !lastElement.getName().isEmpty());
    }
  }

  /**
   * A wrapper factory class for Datastore singleton classes {@link DatastoreFactory} and
   * {@link QuerySplitter}
   *
   *
{@link DatastoreFactory} and {@link QuerySplitter} are not java serializable, hence
   * wrapping them under this class, which implements {@link Serializable}.
   */
  @VisibleForTesting
  static class V1Beta3DatastoreFactory implements Serializable {

    /** Builds a Datastore client for the given pipeline options and project. */
    public Datastore getDatastore(PipelineOptions pipelineOptions, String projectId) {
      DatastoreOptions.Builder builder =
          new DatastoreOptions.Builder()
              .projectId(projectId)
              .initializer(
                  new RetryHttpRequestInitializer()
              );

      Credential credential = pipelineOptions.as(GcpOptions.class).getGcpCredential();
      if (credential != null) {
        builder.credential(credential);
      }

      return DatastoreFactory.get().create(builder.build());
    }

    /** Builds a Datastore {@link QuerySplitter}. */
    public QuerySplitter getQuerySplitter() {
      return DatastoreHelper.getQuerySplitter();
    }
  }
}


text:
package io.warp10.standalone;

import io.warp10.continuum.gts.GTSEncoder;
import io.warp10.continuum.store.GTSDecoderIterator;
import io.warp10.continuum.store.StoreClient;
import io.warp10.continuum.store.thrift.data.Metadata;
import io.warp10.quasar.token.thrift.data.ReadToken;
import io.warp10.quasar.token.thrift.data.WriteToken;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class PlasmaStoreClient implements StoreClient {
  private List<StandalonePlasmaHandlerInterface> plasmaHandlers = new ArrayList<StandalonePlasmaHandlerInterface>();
  @Override
  public GTSDecoderIterator fetch(ReadToken token, List<Metadata> metadatas, long now, long timespan, boolean fromArchive, boolean writeTimestamp) {
    return null;
  }
  @Override
  public void addPlasmaHandler(StandalonePlasmaHandlerInterface handler) {
    this.plasmaHandlers.add(handler);
  }
  public void store(GTSEncoder encoder) throws java.io.IOException {
    for (StandalonePlasmaHandlerInterface plasmaHandler: this.plasmaHandlers) {
      if (plasmaHandler.hasSubscriptions() && null != encoder) {
        plasmaHandler.publish(encoder);
      }
    }
  }

  @Override
  public void archive(int chunk, GTSEncoder encoder) throws IOException {
    throw new IOException("Archive not implemented.");
  }

  @Override
  public long delete(WriteToken token, Metadata metadata, long start, long end) throws IOException { return 0L; }
}


text:
#include <QApplication>
#include <QDesktopWidget>
#include <QPainter>
#include <QSvgRenderer>
#include <QtConcurrentRun>

#include "clipartentrieslist.h"
#include "debug.h"

namespace MoodBox
{

int ClipartImageLoader::maxIconSize = 0;

ClipartImageLoader::ClipartImageLoader(QObject *parent, bool useRequestOrder)
	: QObject(parent), useRequestOrder(useRequestOrder)
{
	connect(&watcher, SIGNAL(finished()), this, SLOT(onJobFinished()));
	connect(qApp, SIGNAL(aboutToQuit()), this, SLOT(clear()));

	if (maxIconSize == 0)
	{
		QDesktopWidget *desktop = qApp->desktop();
		for (int i = 0; i < desktop->numScreens(); i++)
		{
			maxIconSize = qMax(desktop->screenGeometry(i).width(), maxIconSize);
		}
		Q_ASSERT_X(maxIconSize > 0, "ClipartImageLoader::ClipartImageLoader", "Unable to detect max screen width");
		if (maxIconSize <= 0) maxIconSize = 400;
		maxIconSize /= CLIPART_COLUMNS;
	}
}

int ClipartImageLoader::loadPicture(const QString &imageFile, int id)
{
	QImage original;

	bool isLoaded = false;

	if (imageFile.endsWith(CLIPART_NOT_SCALED_PREVIEW_EXT)) // Handle SVG separately
	{
		QSvgRenderer svg(imageFile);
		isLoaded = svg.isValid();
		if (isLoaded)
		{
			QSize svgSize = svg.defaultSize();
			svgSize.scale(maxIconSize, maxIconSize, Qt::KeepAspectRatio);
			original = QImage(svgSize, QImage::Format_ARGB32);
			original.fill(0);
			QPainter painter(&original);
			svg.render(&painter);
		}
	}
	else
	{
		isLoaded = original.load(imageFile);
	}

	if (isLoaded)
	{
		lastResult = ClipartImageLoader::generateItemImages(imageFile, original);
	}
	else
	{
		lastResult.icon = QImage();
		lastResult.preview = QImage();
	}

	return id;
}

void ClipartImageLoader::loadImages(const QString &fileName, int id)
{
	if (useRequestOrder)
		requestOrder.push(id);
	if (jobs.contains(id)) return;

	jobs.insert(id, fileName);
	runJobs();
}

ItemImages ClipartImageLoader::generateItemImages(const QString &fileName, const QImage &originalImage)
{
	ItemImages images;

	images.file = fileName;
	images.icon = originalImage;
	images.preview = originalImage;

	if (images.icon.width() > maxIconSize || images.icon.height() > maxIconSize)
	{
		images.icon = images.icon.scaled(maxIconSize, maxIconSize, Qt::KeepAspectRatio);
	}

	// Make scaled preview
	if (images.icon.width() > PICTURE_PREVIEW_WIDTH || images.icon.height() > PICTURE_PREVIEW_HEIGHT)
		images.preview = images.icon.scaled(PICTURE_PREVIEW_WIDTH, PICTURE_PREVIEW_HEIGHT, Qt::KeepAspectRatio);

	if (fileName.endsWith(CLIPART_NOT_SCALED_PREVIEW_EXT))
	{
		// Upsample too small icons
		if (images.icon.width() < maxIconSize && images.icon.height() < maxIconSize)
		{
			images.icon = images.icon.scaled(maxIconSize, maxIconSize, Qt::KeepAspectRatio);
		}
	}

	return images;
}

int ClipartImageLoader::getMaxIconSize()
{
	return maxIconSize;
}

void ClipartImageLoader::clear()
{
	jobs.clear();
	requestOrder.clear();
	watcher.waitForFinished();
}

void ClipartImageLoader::runJobs()
{
	if (jobs.isEmpty())
	{
		QDEBUG("ClipartImageLoader::runJobs: job list is empty");
		return;
	}

	if (!watcher.isRunning())
	{
		// Get next item to process
		int id = -1;
		if (useRequestOrder)
		{
			do {
				id = requestOrder.pop();
			} while (!requestOrder.isEmpty() && !jobs.contains(id));
		}
		else
		{
			id = jobs.begin().key();
		}

		if (!jobs.contains(id))
		{
			QDEBUG("ClipartImageLoader::runJobs: no more jobs to process");
			return;
		}

		QFuture<int> future = QtConcurrent::run(this, &MoodBox::ClipartImageLoader::loadPicture, jobs.value(id), id);
		watcher.setFuture(future);
	}
}

void ClipartImageLoader::onJobFinished()
{
	int id = watcher.result();

	if (!jobs.isEmpty() && jobs.contains(id))
	{
		jobs.remove(id);
		emit imagesLoaded(lastResult, id);
	}
	else
	{
		QDEBUG("ClipartImageLoader::onJobFinished: job result discarded, id" << id);
	}
	runJobs();
}

}


text:
#include "ElementsCore.h"

class CElementsContainer : public IElementsContainer
{
public:
	CElementsContainer(void);
	~CElementsContainer(void);

// Initialization functions
	void					SetTitle(CString& Title);
	void					AddElement(sElement* pElement);

// IElementsContainer declarations section
public:
	virtual void			ReleaseElements();

	virtual CString			GetTitle();

	ENUMERATOR(mElementsList, sElement*, CreateEnumerator, GetNextElement, posElementsEnum)

// IObject declarations section
public:
	virtual CString			GetClassString();
	virtual void			QueryInterface(int iIDD, void** ppvObject);

private:
	CList<sElement*, sElement*>	mElementsList;

	CString					csTitle;
};


text:
"use strict";
/*jslint eqeqeq: false, onevar: false*/
/*global sinon, module, require, ActiveXObject, XMLHttpRequest, DOMParser*/
/**
 * Minimal Event interface implementation
 *
 * Original implementation by Sven Fuchs: https://gist.github.com/995028
 * Modifications and tests by Christian Johansen.
 *
 * @author Sven Fuchs (svenfuchs@artweb-design.de)
 * @author Christian Johansen (christian@cjohansen.no)
 * @license BSD
 *
 * Copyright (c) 2011 Sven Fuchs, Christian Johansen
 */

if (typeof sinon == "undefined") {
    this.sinon = {};
}

(function () {
    var push = [].push;

    sinon.Event = function Event(type, bubbles, cancelable) {
        this.initEvent(type, bubbles, cancelable);
    };

    sinon.Event.prototype = {
        initEvent: function(type, bubbles, cancelable) {
            this.type = type;
            this.bubbles = bubbles;
            this.cancelable = cancelable;
        },

        stopPropagation: function () {},

        preventDefault: function () {
            this.defaultPrevented = true;
        }
    };

    sinon.EventTarget = {
        addEventListener: function addEventListener(event, listener, useCapture) {
            this.eventListeners = this.eventListeners || {};
            this.eventListeners[event] = this.eventListeners[event] || [];
            push.call(this.eventListeners[event], listener);
        },

        removeEventListener: function removeEventListener(event, listener, useCapture) {
            var listeners = this.eventListeners && this.eventListeners[event] || [];

            for (var i = 0, l = listeners.length; i < l; ++i) {
                if (listeners[i] == listener) {
                    return listeners.splice(i, 1);
                }
            }
        },

        dispatchEvent: function dispatchEvent(event) {
            var type = event.type;
            var listeners = this.eventListeners && this.eventListeners[type] || [];

            for (var i = 0; i < listeners.length; i++) {
                if (typeof listeners[i] == "function") {
                    listeners[i].call(this, event);
                } else {
                    listeners[i].handleEvent(event);
                }
            }

            return !!event.defaultPrevented;
        }
    };
}());

/**
 * @depend event.js
 */
/*jslint eqeqeq: false, onevar: false*/
/*global sinon, module, require, ActiveXObject, XMLHttpRequest, DOMParser*/
/**
 * Fake XMLHttpRequest object
 *
 * @author Christian Johansen (christian@cjohansen.no)
 * @license BSD
 *
 * Copyright (c) 2010-2011 Christian Johansen
 */

if (typeof sinon == "undefined") {
    this.sinon = {};
}
sinon.xhr = { XMLHttpRequest: this.XMLHttpRequest };

// wrapper for global
(function(global) {
    var xhr = sinon.xhr;
    xhr.GlobalXMLHttpRequest = global.XMLHttpRequest;
    xhr.GlobalActiveXObject = global.ActiveXObject;
    xhr.supportsActiveX = typeof xhr.GlobalActiveXObject != "undefined";
    xhr.supportsXHR = typeof xhr.GlobalXMLHttpRequest != "undefined";
    xhr.workingXHR = xhr.supportsXHR ? xhr.GlobalXMLHttpRequest : xhr.supportsActiveX
                                     ? function() { return new xhr.GlobalActiveXObject("MSXML2.XMLHTTP.3.0") } : false;

    /*jsl:ignore*/
    var unsafeHeaders = {
        "Accept-Charset": true,
        "Accept-Encoding": true,
        "Connection": true,
        "Content-Length": true,
        "Cookie": true,
        "Cookie2": true,
        "Content-Transfer-Encoding": true,
        "Date": true,
        "Expect": true,
        "Host": true,
        "Keep-Alive": true,
        "Referer": true,
        "TE": true,
        "Trailer": true,
        "Transfer-Encoding": true,
        "Upgrade": true,
        "User-Agent": true,
        "Via": true
    };
    /*jsl:end*/

    function FakeXMLHttpRequest() {
        this.readyState = FakeXMLHttpRequest.UNSENT;
        this.requestHeaders = {};
        this.requestBody = null;
        this.status = 0;
        this.statusText = "";

        if (typeof FakeXMLHttpRequest.onCreate == "function") {
            FakeXMLHttpRequest.onCreate(this);
        }
    }

    function verifyState(xhr) {
        if (xhr.readyState !== FakeXMLHttpRequest.OPENED) {
            throw new Error("INVALID_STATE_ERR");
        }

        if (xhr.sendFlag) {
            throw new Error("INVALID_STATE_ERR");
        }
    }

    // filtering to enable a white-list version of Sinon FakeXhr,
    // where whitelisted requests are passed through to real XHR
    function each(collection, callback) {
        if (!collection) return;
        for (var i = 0, l = collection.length; i < l; i += 1) {
            callback(collection[i]);
        }
    }
    function some(collection, callback) {
        for (var index = 0; index < collection.length; index++) {
            if(callback(collection[index]) === true) return true;
        };
        return false;
    }
    // largest arity in XHR is 5 - XHR#open
    var apply = function(obj,method,args) {
        switch(args.length) {
        case 0: return obj[method]();
        case 1: return obj[method](args[0]);
        case 2: return obj[method](args[0],args[1]);
        case 3: return obj[method](args[0],args[1],args[2]);
        case 4: return obj[method](args[0],args[1],args[2],args[3]);
        case 5: return obj[method](args[0],args[1],args[2],args[3],args[4]);
        };
    };

    FakeXMLHttpRequest.filters = [];
    FakeXMLHttpRequest.addFilter = function(fn) {
        this.filters.push(fn)
    };
    var IE6Re = /MSIE 6/;
    FakeXMLHttpRequest.defake = function(fakeXhr,xhrArgs) {
        var xhr = new sinon.xhr.workingXHR();
        each(["open","setRequestHeader","send","abort","getResponseHeader",
              "getAllResponseHeaders","addEventListener","overrideMimeType","removeEventListener"],
             function(method) {
                 fakeXhr[method] = function() {
                   return apply(xhr,method,arguments);
                 };
             });

        var copyAttrs = function(args) {
            each(args, function(attr) {
              try {
                fakeXhr[attr] = xhr[attr]
              } catch(e) {
                if(!IE6Re.test(navigator.userAgent)) throw e;
              }
            });
        };

        var stateChange = function() {
            fakeXhr.readyState = xhr.readyState;
            if(xhr.readyState >= FakeXMLHttpRequest.HEADERS_RECEIVED) {
                copyAttrs(["status","statusText"]);
            }
            if(xhr.readyState >= FakeXMLHttpRequest.LOADING) {
                copyAttrs(["responseText"]);
            }
            if(xhr.readyState === FakeXMLHttpRequest.DONE) {
                copyAttrs(["responseXML"]);
            }
            if(fakeXhr.onreadystatechange) fakeXhr.onreadystatechange.call(fakeXhr);
        };
        if(xhr.addEventListener) {
          for(var event in fakeXhr.eventListeners) {
              if(fakeXhr.eventListeners.hasOwnProperty(event)) {
                  each(fakeXhr.eventListeners[event],function(handler) {
                      xhr.addEventListener(event, handler);
                  });
              }
          }
          xhr.addEventListener("readystatechange",stateChange);
        } else {
          xhr.onreadystatechange = stateChange;
        }
        apply(xhr,"open",xhrArgs);
    };
    FakeXMLHttpRequest.useFilters = false;

    function verifyRequestSent(xhr) {
        if (xhr.readyState == FakeXMLHttpRequest.DONE) {
            throw new Error("Request done");
        }
    }

    function verifyHeadersReceived(xhr) {
        if (xhr.async && xhr.readyState != FakeXMLHttpRequest.HEADERS_RECEIVED) {
            throw new Error("No headers received");
        }
    }

    function verifyResponseBodyType(body) {
        if (typeof body != "string") {
            var error = new Error("Attempted to respond to fake XMLHttpRequest with " +
                                 body + ", which is not a string.");
            error.name = "InvalidBodyException";
            throw error;
        }
    }

    sinon.extend(FakeXMLHttpRequest.prototype, sinon.EventTarget, {
        async: true,

        open: function open(method, url, async, username, password) {
            this.method = method;
            this.url = url;
            this.async = typeof async == "boolean" ? async : true;
            this.username = username;
            this.password = password;
            this.responseText = null;
            this.responseXML = null;
            this.requestHeaders = {};
            this.sendFlag = false;
            if(sinon.FakeXMLHttpRequest.useFilters === true) {
                var xhrArgs = arguments;
                var defake = some(FakeXMLHttpRequest.filters,function(filter) {
                    return filter.apply(this,xhrArgs)
                });
                if (defake) {
                  return sinon.FakeXMLHttpRequest.defake(this,arguments);
                }
            }
            this.readyStateChange(FakeXMLHttpRequest.OPENED);
        },

        readyStateChange: function readyStateChange(state) {
            this.readyState = state;

            if (typeof this.onreadystatechange == "function") {
                try {
                    this.onreadystatechange();
                } catch (e) {
                    sinon.logError("Fake XHR onreadystatechange handler", e);
                }
            }

            this.dispatchEvent(new sinon.Event("readystatechange"));
        },

        setRequestHeader: function setRequestHeader(header, value) {
            verifyState(this);

            if (unsafeHeaders[header] || /^(Sec-|Proxy-)/.test(header)) {
                throw new Error("Refused to set unsafe header \"" + header + "\"");
            }

            if (this.requestHeaders[header]) {
                this.requestHeaders[header] += "," + value;
            } else {
                this.requestHeaders[header] = value;
            }
        },

        // Helps testing
        setResponseHeaders: function setResponseHeaders(headers) {
            this.responseHeaders = {};

            for (var header in headers) {
                if (headers.hasOwnProperty(header)) {
                    this.responseHeaders[header] = headers[header];
                }
            }

            if (this.async) {
                this.readyStateChange(FakeXMLHttpRequest.HEADERS_RECEIVED);
            }
        },

        // Currently treats ALL data as a DOMString (i.e. no Document)
        send: function send(data) {
            verifyState(this);

            if (!/^(get|head)$/i.test(this.method)) {
                if (this.requestHeaders["Content-Type"]) {
                    var value = this.requestHeaders["Content-Type"].split(";");
                    this.requestHeaders["Content-Type"] = value[0] + ";charset=utf-8";
                } else {
                    this.requestHeaders["Content-Type"] = "text/plain;charset=utf-8";
                }

                this.requestBody = data;
            }

            this.errorFlag = false;
            this.sendFlag = this.async;
            this.readyStateChange(FakeXMLHttpRequest.OPENED);

            if (typeof this.onSend == "function") {
                this.onSend(this);
            }
        },

        abort: function abort() {
            this.aborted = true;
            this.responseText = null;
            this.errorFlag = true;
            this.requestHeaders = {};

            if (this.readyState > sinon.FakeXMLHttpRequest.UNSENT && this.sendFlag) {
                this.readyStateChange(sinon.FakeXMLHttpRequest.DONE);
                this.sendFlag = false;
            }

            this.readyState = sinon.FakeXMLHttpRequest.UNSENT;
        },

        getResponseHeader: function getResponseHeader(header) {
            if (this.readyState < FakeXMLHttpRequest.HEADERS_RECEIVED) {
                return null;
            }

            if (/^Set-Cookie2?$/i.test(header)) {
                return null;
            }

            header = header.toLowerCase();

            for (var h in this.responseHeaders) {
                if (h.toLowerCase() == header) {
                    return this.responseHeaders[h];
                }
            }

            return null;
        },

        getAllResponseHeaders: function getAllResponseHeaders() {
            if (this.readyState < FakeXMLHttpRequest.HEADERS_RECEIVED) {
                return "";
            }

            var headers = "";

            for (var header in this.responseHeaders) {
                if (this.responseHeaders.hasOwnProperty(header) &&
                    !/^Set-Cookie2?$/i.test(header)) {
                    headers += header + ": " + this.responseHeaders[header] + "\r\n";
                }
            }

            return headers;
        },

        setResponseBody: function setResponseBody(body) {
            verifyRequestSent(this);
            verifyHeadersReceived(this);
            verifyResponseBodyType(body);

            var chunkSize = this.chunkSize || 10;
            var index = 0;
            this.responseText = "";

            do {
                if (this.async) {
                    this.readyStateChange(FakeXMLHttpRequest.LOADING);
                }

                this.responseText += body.substring(index, index + chunkSize);
                index += chunkSize;
            } while (index < body.length);

            var type = this.getResponseHeader("Content-Type");

            if (this.responseText &&
                (!type || /(text\/xml)|(application\/xml)|(\+xml)/.test(type))) {
                try {
                    this.responseXML = FakeXMLHttpRequest.parseXML(this.responseText);
                } catch (e) {
                    // Unable to parse XML - no biggie
                }
            }

            if (this.async) {
                this.readyStateChange(FakeXMLHttpRequest.DONE);
            } else {
                this.readyState = FakeXMLHttpRequest.DONE;
            }
        },

        respond: function respond(status, headers, body) {
            this.setResponseHeaders(headers || {});
            this.status = typeof status == "number" ? status : 200;
            this.statusText = FakeXMLHttpRequest.statusCodes[this.status];
            this.setResponseBody(body || "");
        }
    });

    sinon.extend(FakeXMLHttpRequest, {
        UNSENT: 0,
        OPENED: 1,
        HEADERS_RECEIVED: 2,
        LOADING: 3,
        DONE: 4
    });

    // Borrowed from JSpec
    FakeXMLHttpRequest.parseXML = function parseXML(text) {
        var xmlDoc;

        if (typeof DOMParser != "undefined") {
            var parser = new DOMParser();
            xmlDoc = parser.parseFromString(text, "text/xml");
        } else {
            xmlDoc = new ActiveXObject("Microsoft.XMLDOM");
            xmlDoc.async = "false";
            xmlDoc.loadXML(text);
        }

        return xmlDoc;
    };

    FakeXMLHttpRequest.statusCodes = {
        100: "Continue",
        101: "Switching Protocols",
        200: "OK",
        201: "Created",
        202: "Accepted",
        203: "Non-Authoritative Information",
        204: "No Content",
        205: "Reset Content",
        206: "Partial Content",
        300: "Multiple Choice",
        301: "Moved Permanently",
        302: "Found",
        303: "See Other",
        304: "Not Modified",
        305: "Use Proxy",
        307: "Temporary Redirect",
        400: "Bad Request",
        401: "Unauthorized",
        402: "Payment Required",
        403: "Forbidden",
        404: "Not Found",
        405: "Method Not Allowed",
        406: "Not Acceptable",
        407: "Proxy Authentication Required",
        408: "Request Timeout",
        409: "Conflict",
        410: "Gone",
        411: "Length Required",
        412: "Precondition Failed",
        413: "Request Entity Too Large",
        414: "Request-URI Too Long",
        415: "Unsupported Media Type",
        416: "Requested Range Not Satisfiable",
        417: "Expectation Failed",
        422: "Unprocessable Entity",
        500: "Internal Server Error",
        501: "Not Implemented",
        502: "Bad Gateway",
        503: "Service Unavailable",
        504: "Gateway Timeout",
        505: "HTTP Version Not Supported"
    };

    sinon.useFakeXMLHttpRequest = function () {
        sinon.FakeXMLHttpRequest.restore = function restore(keepOnCreate) {
            if (xhr.supportsXHR) {
                global.XMLHttpRequest = xhr.GlobalXMLHttpRequest;
            }

            if (xhr.supportsActiveX) {
                global.ActiveXObject = xhr.GlobalActiveXObject;
            }

            delete sinon.FakeXMLHttpRequest.restore;

            if (keepOnCreate !== true) {
                delete sinon.FakeXMLHttpRequest.onCreate;
            }
        };
        if (xhr.supportsXHR) {
            global.XMLHttpRequest = sinon.FakeXMLHttpRequest;
        }

        if (xhr.supportsActiveX) {
            global.ActiveXObject = function ActiveXObject(objId) {
                if (objId == "Microsoft.XMLHTTP" || /^Msxml2\.XMLHTTP/i.test(objId)) {

                    return new sinon.FakeXMLHttpRequest();
                }

                return new xhr.GlobalActiveXObject(objId);
            };
        }

        return sinon.FakeXMLHttpRequest;
    };

    sinon.FakeXMLHttpRequest = FakeXMLHttpRequest;
})(this);

if (typeof module == "object" && typeof require == "function") {
    module.exports = sinon;
}

/**
 * @depend fake_xml_http_request.js
 */
/*jslint eqeqeq: false, onevar: false, regexp: false, plusplus: false*/
/*global module, require, window*/
/**
 * The Sinon "server" mimics a web server that receives requests from
 * sinon.FakeXMLHttpRequest and provides an API to respond to those requests,
 * both synchronously and asynchronously. To respond synchronuously, canned
 * answers have to be provided upfront.
 *
 * @author Christian Johansen (christian@cjohansen.no)
 * @license BSD
 *
 * Copyright (c) 2010-2011 Christian Johansen
 */

if (typeof sinon == "undefined") {
    var sinon = {};
}

sinon.fakeServer = (function () {
    var push = [].push;
    function F() {}

    function create(proto) {
        F.prototype = proto;
        return new F();
    }

    function responseArray(handler) {
        var response = handler;

        if (Object.prototype.toString.call(handler) != "[object Array]") {
            response = [200, {}, handler];
        }

        if (typeof response[2] != "string") {
            throw new TypeError("Fake server response body should be string, but was " +
                                typeof response[2]);
        }

        return response;
    }

    var wloc = typeof window !== "undefined" ? window.location : {};
    var rCurrLoc = new RegExp("^" + wloc.protocol + "//" + wloc.host);

    function matchOne(response, reqMethod, reqUrl) {
        var rmeth = response.method;
        var matchMethod = !rmeth || rmeth.toLowerCase() == reqMethod.toLowerCase();
        var url = response.url;
        var matchUrl = !url || url == reqUrl || (typeof url.test == "function" && url.test(reqUrl));

        return matchMethod && matchUrl;
    }

    function match(response, request) {
        var requestMethod = this.getHTTPMethod(request);
        var requestUrl = request.url;

        if (!/^https?:\/\//.test(requestUrl) || rCurrLoc.test(requestUrl)) {
            requestUrl = requestUrl.replace(rCurrLoc, "");
        }

        if (matchOne(response, this.getHTTPMethod(request), requestUrl)) {
            if (typeof response.response == "function") {
                var ru = response.url;
                var args = [request].concat(!ru ? [] : requestUrl.match(ru).slice(1));
                return response.response.apply(response, args);
            }

            return true;
        }

        return false;
    }

    return {
        create: function () {
            var server = create(this);
            this.xhr = sinon.useFakeXMLHttpRequest();
            server.requests = [];

            this.xhr.onCreate = function (xhrObj) {
                server.addRequest(xhrObj);
            };

            return server;
        },

        addRequest: function addRequest(xhrObj) {
            var server = this;
            push.call(this.requests, xhrObj);

            xhrObj.onSend = function () {
                server.handleRequest(this);
            };

            if (this.autoRespond && !this.responding) {
                setTimeout(function () {
                    server.responding = false;
                    server.respond();
                }, this.autoRespondAfter || 10);

                this.responding = true;
            }
        },

        getHTTPMethod: function getHTTPMethod(request) {
            if (this.fakeHTTPMethods && /post/i.test(request.method)) {
                var matches = (request.requestBody || "").match(/_method=([^\b;]+)/);
                return !!matches ? matches[1] : request.method;
            }

            return request.method;
        },

        handleRequest: function handleRequest(xhr) {
            if (xhr.async) {
                if (!this.queue) {
                    this.queue = [];
                }

                push.call(this.queue, xhr);
            } else {
                this.processRequest(xhr);
            }
        },

        respondWith: function respondWith(method, url, body) {
            if (arguments.length == 1 && typeof method != "function") {
                this.response = responseArray(method);
                return;
            }

            if (!this.responses) { this.responses = []; }

            if (arguments.length == 1) {
                body = method;
                url = method = null;
            }

            if (arguments.length == 2) {
                body = url;
                url = method;
                method = null;
            }

            push.call(this.responses, {
                method: method,
                url: url,
                response: typeof body == "function" ? body : responseArray(body)
            });
        },

        respond: function respond() {
            if (arguments.length > 0) this.respondWith.apply(this, arguments);
            var queue = this.queue || [];
            var request;

            while(request = queue.shift()) {
                this.processRequest(request);
            }
        },

        processRequest: function processRequest(request) {
            try {
                if (request.aborted) {
                    return;
                }

                var response = this.response || [404, {}, ""];

                if (this.responses) {
                    for (var i = 0, l = this.responses.length; i < l; i++) {
                        if (match.call(this, this.responses[i], request)) {
                            response = this.responses[i].response;
                            break;
                        }
                    }
                }

                if (request.readyState != 4) {
                    request.respond(response[0], response[1], response[2]);
                }
            } catch (e) {
                sinon.logError("Fake server request processing", e);
            }
        },

        restore: function restore() {
            return this.xhr.restore && this.xhr.restore.apply(this.xhr, arguments);
        }
    };
}());

if (typeof module == "object" && typeof require == "function") {
    module.exports = sinon;
}

/*jslint eqeqeq: false, plusplus: false, evil: true, onevar: false, browser: true, forin: false*/
/*global module, require, window*/
/**
 * Fake timer API
 * setTimeout
 * setInterval
 * clearTimeout
 * clearInterval
 * tick
 * reset
 * Date
 *
 * Inspired by jsUnitMockTimeOut from JsUnit
 *
 * @author Christian Johansen (christian@cjohansen.no)
 * @license BSD
 *
 * Copyright (c) 2010-2011 Christian Johansen
 */

if (typeof sinon == "undefined") {
    var sinon = {};
}

(function (global) {
    var id = 1;

    function addTimer(args, recurring) {
        if (args.length === 0) {
            throw new Error("Function requires at least 1 parameter");
        }

        var toId = id++;
        var delay = args[1] || 0;

        if (!this.timeouts) {
            this.timeouts = {};
        }

        this.timeouts[toId] = {
            id: toId,
            func: args[0],
            callAt: this.now + delay,
            invokeArgs: Array.prototype.slice.call(args, 2)
        };

        if (recurring === true) {
            this.timeouts[toId].interval = delay;
        }

        return toId;
    }

    function parseTime(str) {
        if (!str) {
            return 0;
        }

        var strings = str.split(":");
        var l = strings.length, i = l;
        var ms = 0, parsed;

        if (l > 3 || !/^(\d\d:){0,2}\d\d?$/.test(str)) {
            throw new Error("tick only understands numbers and 'h:m:s'");
        }

        while (i--) {
            parsed = parseInt(strings[i], 10);

            if (parsed >= 60) {
                throw new Error("Invalid time " + str);
            }

            ms += parsed * Math.pow(60, (l - i - 1));
        }

        return ms * 1000;
    }

    function createObject(object) {
        var newObject;

        if (Object.create) {
            newObject = Object.create(object);
        } else {
            var F = function () {};
            F.prototype = object;
            newObject = new F();
        }

        newObject.Date.clock = newObject;
        return newObject;
    }

    sinon.clock = {
        now: 0,

        create: function create(now) {
            var clock = createObject(this);

            if (typeof now == "number") {
                clock.now = now;
            }

            if (!!now && typeof now == "object") {
                throw new TypeError("now should be milliseconds since UNIX epoch");
            }

            return clock;
        },

        setTimeout: function setTimeout(callback, timeout) {
            return addTimer.call(this, arguments, false);
        },

        clearTimeout: function clearTimeout(timerId) {
            if (!this.timeouts) {
                this.timeouts = [];
            }

            if (timerId in this.timeouts) {
                delete this.timeouts[timerId];
            }
        },

        setInterval: function setInterval(callback, timeout) {
            return addTimer.call(this, arguments, true);
        },

        clearInterval: function clearInterval(timerId) {
            this.clearTimeout(timerId);
        },

        tick: function tick(ms) {
            ms = typeof ms == "number" ? ms : parseTime(ms);
            var tickFrom = this.now, tickTo = this.now + ms, previous = this.now;
            var timer = this.firstTimerInRange(tickFrom, tickTo);

            var firstException;
            while (timer && tickFrom <= tickTo) {
                if (this.timeouts[timer.id]) {
                    tickFrom = this.now = timer.callAt;
                    try {
                      this.callTimer(timer);
                    } catch (e) {
                      firstException = firstException || e;
                    }
                }

                timer = this.firstTimerInRange(previous, tickTo);
                previous = tickFrom;
            }

            this.now = tickTo;

            if (firstException) {
              throw firstException;
            }
        },

        firstTimerInRange: function (from, to) {
            var timer, smallest, originalTimer;

            for (var id in this.timeouts) {
                if (this.timeouts.hasOwnProperty(id)) {
                    if (this.timeouts[id].callAt < from || this.timeouts[id].callAt > to) {
                        continue;
                    }

                    if (!smallest || this.timeouts[id].callAt < smallest) {
                        originalTimer = this.timeouts[id];
                        smallest = this.timeouts[id].callAt;

                        timer = {
                            func: this.timeouts[id].func,
                            callAt: this.timeouts[id].callAt,
                            interval: this.timeouts[id].interval,
                            id: this.timeouts[id].id,
                            invokeArgs: this.timeouts[id].invokeArgs
                        };
                    }
                }
            }

            return timer || null;
        },

        callTimer: function (timer) {
            if (typeof timer.interval == "number") {
                this.timeouts[timer.id].callAt += timer.interval;
            } else {
                delete this.timeouts[timer.id];
            }

            try {
                if (typeof timer.func == "function") {
                    timer.func.apply(null, timer.invokeArgs);
                } else {
                    eval(timer.func);
                }
            } catch (e) {
              var exception = e;
            }

            if (!this.timeouts[timer.id]) {
                if (exception) {
                  throw exception;
                }
                return;
            }

            if (exception) {
              throw exception;
            }
        },

        reset: function reset() {
            this.timeouts = {};
        },

        Date: (function () {
            var NativeDate = Date;

            function ClockDate(year, month, date, hour, minute, second, ms) {
                // Defensive and verbose to avoid potential harm in passing
                // explicit undefined when user does not pass argument
                switch (arguments.length) {
                case 0:
                    return new NativeDate(ClockDate.clock.now);
                case 1:
                    return new NativeDate(year);
                case 2:
                    return new NativeDate(year, month);
                case 3:
                    return new NativeDate(year, month, date);
                case 4:
                    return new NativeDate(year, month, date, hour);
                case 5:
                    return new NativeDate(year, month, date, hour, minute);
                case 6:
                    return new NativeDate(year, month, date, hour, minute, second);
                default:
                    return new NativeDate(year, month, date, hour, minute, second, ms);
                }
            }

            return mirrorDateProperties(ClockDate, NativeDate);
        }())
    };

    function mirrorDateProperties(target, source) {
        if (source.now) {
            target.now = function now() {
                return target.clock.now;
            };
        } else {
            delete target.now;
        }

        if (source.toSource) {
            target.toSource = function toSource() {
                return source.toSource();
            };
        } else {
            delete target.toSource;
        }

        target.toString = function toString() {
            return source.toString();
        };

        target.prototype = source.prototype;
        target.parse = source.parse;
        target.UTC = source.UTC;
        target.prototype.toUTCString = source.prototype.toUTCString;
        return target;
    }

    var methods = ["Date", "setTimeout", "setInterval",
                   "clearTimeout", "clearInterval"];

    function restore() {
        var method;

        for (var i = 0, l = this.methods.length; i < l; i++) {
            method = this.methods[i];
            if (global[method].hadOwnProperty) {
                global[method] = this["_" + method];
            } else {
                delete global[method];
            }
        }

        // Prevent multiple executions which will completely remove these props
        this.methods = [];
    }

    function stubGlobal(method, clock) {
        clock[method].hadOwnProperty = Object.prototype.hasOwnProperty.call(global, method);
        clock["_" + method] = global[method];

        if (method == "Date") {
            var date = mirrorDateProperties(clock[method], global[method]);
            global[method] = date;
        } else {
            global[method] = function () {
                return clock[method].apply(clock, arguments);
            };

            for (var prop in clock[method]) {
                if (clock[method].hasOwnProperty(prop)) {
                    global[method][prop] = clock[method][prop];
                }
            }
        }

        global[method].clock = clock;
    }

    sinon.useFakeTimers = function useFakeTimers(now) {
        var clock = sinon.clock.create(now);
        clock.restore = restore;
        clock.methods = Array.prototype.slice.call(arguments,
                                                   typeof now == "number" ? 1 : 0);

        if (clock.methods.length === 0) {
            clock.methods = methods;
        }

        for (var i = 0, l = clock.methods.length; i < l; i++) {
            stubGlobal(clock.methods[i], clock);
        }

        return clock;
    };
}(typeof global != "undefined" && typeof global !== "function" ? global : this));

sinon.timers = {
    setTimeout: setTimeout,
    clearTimeout: clearTimeout,
    setInterval: setInterval,
    clearInterval: clearInterval,
    Date: Date
};

if (typeof module == "object" && typeof require == "function") {
    module.exports = sinon;
}

/**
 * @depend fake_server.js
 * @depend fake_timers.js
 */
/*jslint browser: true, eqeqeq: false, onevar: false*/
/*global sinon*/
/**
 * Add-on for sinon.fakeServer that automatically handles a fake timer along with
 * the FakeXMLHttpRequest. The direct inspiration for this add-on is jQuery
 * 1.3.x, which does not use xhr object's onreadystatehandler at all - instead,
 * it polls the object for completion with setInterval. Dispite the direct
 * motivation, there is nothing jQuery-specific in this file, so it can be used
 * in any environment where the ajax implementation depends on setInterval or
 * setTimeout.
 *
 * @author Christian Johansen (christian@cjohansen.no)
 * @license BSD
 *
 * Copyright (c) 2010-2011 Christian Johansen
 */

(function () {
    function Server() {}
    Server.prototype = sinon.fakeServer;

    sinon.fakeServerWithClock = new Server();

    sinon.fakeServerWithClock.addRequest = function addRequest(xhr) {
        if (xhr.async) {
            if (typeof setTimeout.clock == "object") {
                this.clock = setTimeout.clock;
            } else {
                this.clock = sinon.useFakeTimers();
                this.resetClock = true;
            }

            if (!this.longestTimeout) {
                var clockSetTimeout = this.clock.setTimeout;
                var clockSetInterval = this.clock.setInterval;
                var server = this;

                this.clock.setTimeout = function (fn, timeout) {
                    server.longestTimeout = Math.max(timeout, server.longestTimeout || 0);

                    return clockSetTimeout.apply(this, arguments);
                };

                this.clock.setInterval = function (fn, timeout) {
                    server.longestTimeout = Math.max(timeout, server.longestTimeout || 0);

                    return clockSetInterval.apply(this, arguments);
                };
            }
        }

        return sinon.fakeServer.addRequest.call(this, xhr);
    };

    sinon.fakeServerWithClock.respond = function respond() {
        var returnVal = sinon.fakeServer.respond.apply(this, arguments);

        if (this.clock) {
            this.clock.tick(this.longestTimeout || 0);
            this.longestTimeout = 0;

            if (this.resetClock) {
                this.clock.restore();
                this.resetClock = false;
            }
        }

        return returnVal;
    };

    sinon.fakeServerWithClock.restore = function restore() {
        if (this.clock) {
            this.clock.restore();
        }

        return sinon.fakeServer.restore.apply(this, arguments);
    };
}());


text:
@interface SocketManager ()

@property (strong, nonatomic) NSMutableArray *socketHistory;

@property (copy, nonatomic) NSString *willSenddata;
@property (strong, nonatomic) NSDateFormatter *dateFormatter;
@end

@implementation SocketManager

+ (instancetype)sharedManager
{
    static SocketManager *mgr = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{

        mgr = [[super allocWithZone:NULL] init];
    });
    return mgr;
}

+ (instancetype)allocWithZone:(struct _NSZone *)zone
{
    return [self sharedManager];
}

- (instancetype)init
{
    self = [super init];
    if (self) {
        self.dateFormatter = [[NSDateFormatter alloc] init];
        [self.dateFormatter setDateFormat:@"HH:mm:ss"];

//        self.socketHistory = [NSMutableArray array];
    }
    return self;
}

- (void)socketConnectHost:(NSString *)host
                     port:(UInt16)port
                 delegate:(id<SocketManagerDelegate>)delegate
{
    if (self.socket == nil) {
        self.socket = [[AsyncSocket alloc] initWithDelegate:self];
    }

    NSError *error = nil;

    self.socketHost = host;
    self.socketPort = port;
//    self.socketHost = @"10.211.55.6";
//    self.socketPort = 60000;
    self.delegate = delegate;
    NSLog(@"will connect : %@:%d", self.socketHost, self.socketPort);
    [[NSString stringWithFormat:@"will connect : %@:%d", self.socketHost, self.socketPort] toast];
    BOOL connected = [self.socket connectToHost:self.socketHost
                        onPort:self.socketPort
                         error:&error];

    if (!connected) {
        NSLog(@"error: %@", error);
        abort();
    }else{
        [self.socket setRunLoopModes:@[NSRunLoopCommonModes]];
    }
}

- (void)cutOfSocket
{
    if ([self.socket isConnected]) {
        [self.socket disconnect];
    }
}

- (void)writeData:(NSString *)data
{
    if (self.socket.isConnected == YES) {
        self.willSenddata = data;
        NSData *_data = [data dataUsingEncoding:NSUTF8StringEncoding];
        [self.socket writeData:_data withTimeout:-1 tag:0];
    }
}

- (NSString *)dateFormatText:(NSString *)text send:(BOOL)send
{
    NSString *fmt = @"%@ %@:%@";
    NSString *fmtText = [NSString stringWithFormat:fmt,
                         [self.dateFormatter stringFromDate:[NSDate date]],
                         send?@"发送数据":@"收到数据",
                         text];
    return fmtText;
}

- (NSString *)historyLog
{
    return [self.socketHistory componentsJoinedByString:@"\n"];
}

- (void)addToHistory:(NSString *)text send:(BOOL)send
{
    NSString *fmtText = [self dateFormatText:text send:send];
    if (self.socketHistory == nil) {
        self.socketHistory = [NSMutableArray array];
    }
    [self.socketHistory addObject:fmtText];
    if ([self.delegate respondsToSelector:@selector(socketManager:didUpdateHistory:)]) {
        [self.delegate socketManager:self didUpdateHistory:self.historyLog];
    }
}

#pragma mark - Socket delegate
- (BOOL)onSocketWillConnect:(AsyncSocket *)sock
{
    NSLog(@"%s", __PRETTY_FUNCTION__);
    return YES;
}

- (void)onSocket:(AsyncSocket *)sock didConnectToHost:(NSString *)host port:(UInt16)port
{
    [@"socket 已连接" toast];
    [sock readDataWithTimeout:-1 tag:0];

    if ([self.delegate respondsToSelector:@selector(socketManager:socketDidConnect:)]) {
        [self.delegate socketManager:self socketDidConnect:sock];
    }
}

- (void)onSocket:(AsyncSocket *)sock willDisconnectWithError:(NSError *)err
{
    NSLog(@"%s: %@", __PRETTY_FUNCTION__, err);

    if ([self.delegate respondsToSelector:@selector(socketManager:socketDidDisconnect:)]) {
        [self.delegate socketManager:self socketDidDisconnect:sock];
    }
}

- (void)onSocketDidDisconnect:(AsyncSocket *)sock
{
    [@"socket 已断开连接" toast];

    if ([self.delegate respondsToSelector:@selector(socketManager:socketDidDisconnect:)]) {
        [self.delegate socketManager:self socketDidDisconnect:sock];
    }
}

- (void)onSocket:(AsyncSocket *)sock didWriteDataWithTag:(long)tag
{
    [@"socket 发送数据" toast];
    [self addToHistory:self.willSenddata send:YES];
    [sock readDataWithTimeout:-1 tag:0];
}

- (void)onSocket:(AsyncSocket *)sock didReadData:(NSData *)data withTag:(long)tag
{
    NSString *response = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];

    [self addToHistory:response send:NO];
    [[NSString stringWithFormat:@"socket 收到数据: %@", response] toast];
    [sock readDataWithTimeout:-1 tag:0];
}

@end


text:
<?php
// # GetPaymentSample
// This sample code demonstrate how you can
// retrieve a list of all Payment resources
// you've created using the Payments API.
// Note various query parameters that you can
// use to filter, and paginate through the
// payments list.
// API used: GET /v1/payments/payments

require __DIR__ . '/../bootstrap.php';
use PayPal\Api\Payment;

$paymentId = "PAY-0XL713371A312273YKE2GCNI";

// ### Retrieve payment
// Retrieve the payment object by calling the
// static `get` method
// on the Payment class by passing a valid
// Payment ID
// (See bootstrap.php for more on `ApiContext`)
try {
	$payment = Payment::get($paymentId, $apiContext);
} catch (\PPConnectionException $ex) {
	echo "Exception:" . $ex->getMessage() . PHP_EOL;
	var_dump($ex->getData());
	exit(1);
}
?>

Retrieving Payment ID: <?php echo $paymentId;?>

<?php var_dump($payment->toArray());?>

	Back