text:
N.B MongoSyphon version 2 has had a considerable overhaul of it's configuration file format and functionality.

## Introduction

MongoSyphon is an Extract/Transform/Load (ETL) Engine designed specifically to Merge and Transform data into Document forms. It can read from RDBMS tables or MongoDB and output to JSON, XML or directly into MongoDB. It performs time and memory efficient joins of data internally where the underlying database does no support it or the merge of data is between multiple sources.

When sending data to MongoDB it can make use of the powerful native document update facilities. This differs from many other ETL tools which target relational structures and have a minimal MongoDB add-on. MongoSyphon can be used both for an initial bulk transfer and for ongoing updates.

MongoSyphon does not contain an explicit Change Data Capture (CDC) capability but is able to perform basic CDC tasks through SQL and MongoDB querying or make use of change tables generated by an external CDC.

It required that users are competent in SQL and know the structure of the source data. It is also assumed they can make a judgement and/or measure the impact on the source system when running MongoSyphon.

It is also assumed that the user is able to understand and define the MongoDB target collection.

A core tenet of MongoSyphon is to push work to either the source or to MongoDB wherever possible. The engine itself is intended to me small, lightweight and fast without huge memory requirements.

There is no GUI for MongoSyphon, job scheduling should be via cron or similar for ongoing processes.

## Short Demo
### Build
1. Checkout from Github
2. mvn package

### Prerequisites

1. MongoDB on port 27017 with auth to write (root/password)
2. MySQl on port 3306 with auth to write (root/password)
3. mongo client and mysql in your path

### Demo

First make a sample database in your RDBMS (MySQL)

```
> cd example
> cat mkpets.sql

drop database if exists sdemo;
create database sdemo;
use sdemo;

create table owner ( ownerid int primary key, name VARCHAR(20), address VARCHAR(60));
CREATE TABLE species ( speciesid int primary key ,  species VARCHAR(20));
CREATE TABLE pet ( petid int primary key, name VARCHAR(20), owner int, species int);

insert into species values(1,"Dog");
insert into species values(2,"Cat");

insert into owner values (1,"John","22 Accacia Avenue");
insert into owner values (2,"Sarah","19 Main Street");

insert into pet values (1,"Brea",1,1);
insert into pet values (2,"Harvest",1,1);
insert into pet values (3,"Mittens",2,2);

> mysql -uroot -ppassword < mkpets.sql
> mysql -u root -ppassword
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 595
Server version: 5.7.17 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> describe sdemo;
ERROR 1046 (3D000): No database selected
mysql> use sdemo;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+-----------------+
| Tables_in_sdemo |
+-----------------+
| owner           |
| pet             |
| species         |
+-----------------+
3 rows in set (0.00 sec)

mysql> exit
```

Now we have a three table schema each _owner_ has 0 or more pets, each _pet_ has exactly one _species_ from a small list.

```
>cat owners.js
{
	start: {
		source: {
			uri:  "jdbc:mysql://localhost:3306/sdemo?useSSL=false",
			user: "root",
			password: "password"
		},
		target: {
			mode: "insert",
			uri: "mongodb://localhost:27017/",
			namespace: "sdemo.owners"
		},
		template: {
			_id: "$ownerid",
			name: "$name",
			address : "$address",
			pets : [ "@petsection" ]
		},
		query:{
		   sql: 'SELECT * FROM owner'
		}
	},

	petsection: {
		template: {
			petid: "$petid",
			name: "$name",
			species : "@speciessection"
		},
		query:{
			sql: 'SELECT * FROM pet where owner = ?'
		},
		params: [ "ownerid" ]
	},

	speciessection: {
		template: {
			_value : "$species"
		},
		query: {
			sql: 'SELECT * from species where speciesid = ?'
		},
		params : [ "species" ]
	}
}
>java -jar ../bin/MongoSyphon.jar -c owners.js
2 records converted in 0 seconds at an average of 7 records/s
>mongo -uroot -ppassword
> use sdemo
switched to db sdemo
> db.owners.find().pretty()
{
	"_id" : 1,
	"name" : "John",
	"address" : "22 Accacia Avenue",
	"pets" : [
		{
			"petid" : 1,
			"name" : "Brea",
			"species" : "Dog"
		},
		{
			"petid" : 2,
			"name" : "Harvest",
			"species" : "Dog"
		}
	]
}
{
	"_id" : 2,
	"name" : "Sarah",
	"address" : "19 Main Street",
	"pets" : [
		{
			"petid" : 3,
			"name" : "Mittens",
			"species" : "Cat"
		}
	]
}
>exit
```

## WARNING Version 2.0.1

**MongoSyphon is an earluy stage, open-source tool and has the following characteristics**

* Not well enough tested
* Limited Error Handling
* At-will support from the Author

These will improve over time but usage is **at your own risk** and there are no implied warranties or promises it won't eat your data or lie to you.

Please report any bugs of issues you find.

This is released with an Apache 2.0 license.

## Command line
```
Usage: java -jar MongoSyphon.jar [args]

Args:
   -c <config file>
   -h <help>
   -n <new output config>
```

## Config Files

### Nature

MongoSyphon is driven from JSON format configuration files. Early prototypes used YAML configuration files however this resulted in embedding JSON in YAML and was ultimately less readable. This also lends itself to storing the ETL configurations inside MongoDB itself if desired.

Each configuraton file defines a complete ETL process. At this time that is either an insert, update or upsert into a single MongoDB collection with documents generated from one or more relational tables or from a single MongoDB instance/cluster.

In future parallel generation of codepentant records into multiple collections to generate more 'relational' data in MongoDB may also be supported.

### Config Format

####  Sections and the start point

The config file is divided into named _sections_. Every config file must have a section called _start_  which is the first section processes. A section defines a SQL Query, Mongo Query or Aggregation, a Template, a Parameters section if required and any cacheing/execution options.

A section can have a source, target, query and template.

A simple section might look like this

```

	start: {
		source : {
			uri: "jdbc:mysql://localhost:3306/jmdb?useSSL=false",
			user: "toor",
			password: "p4ssword"
		},
		target : {
			mode: 'insert',
			uri: 'mongodb://localhost:27017',
			namespace: "test.example"
		},
		template: {
			_id: "$ownerid",
			name: "$name",
			address : "$address",
		},
		query: {
			sql: 'SELECT * FROM owner'
		}
	}
```

The simple description of the logic here is.

* Run the  Query
* For Each Row/Document in Results:
	- Apply the Row values to the template to make a Document.
	- Sent that result to the targer.
	- `$name` means the contents of a column called name.
	- without the $ the explicit value is included.
	- Null or empty columns are not include in the output.
	-
#### Source Details

Each section must have a source, a place from which to pull data  - is a source is not explicitly defined for a section then it is inherited from the parent section calling it. Connection to an RDBMS is via JDBC and a suitable JDBC Driver must be in your classpath. JDBC drivers for MySQL and Prostgres are included in the JAR and the Maven build file.

The source part of a section has the following members.

Option|Description|Mandatory|
|------|-----------|---------|
| uri| A JDBC or MongoDB Connection string to the database of your choice, you must have an appropriate driver jar in your classpath for an RDBMS, mysql and postgres build with MongoSyphon and are in it's JAR|Yes|
|user| For JDBC only - user credentials, in MongoDB they are in the URI| Yes |
|password|For JDBC only - user credentials, in MongoDB they are in the URI| Yes |

The full range of Connection string options are available for MongoDB and the RDBMS. To use username/password authentication for MongoDB it should be included in the URI [Mongo URI Format](https://docs.mongodb.com/manual/reference/connection-string/). In future additional password options may be included to avoid passwords in configuration files when using passworded login.

#### Target Details

The target specifies either a MongoDB URI and namespace to write to or an output file format. It also specified, in the MOngoDB cae whther to insert, update or upsert data

|Option|Description|Mandatory|
|------|-----------|---------|
|mode| one of insert,upsert,update,save,JSON, XML or subsection|No|
| uri| A MongoDB Connection string to the database of your choice, you must have an appropriate driver jar in your classpath for an RDBMS, mysql and postgres build with MongoSyphon and are in it's JAR|Yes|
user| A user who can log in and read from the RDBMS| Yes |
|password|A password for the user above , for MongoDB put it in the connection string instead| Yes |

#### Nested Sections

The power of MongSyphon is in it's ability to build nested document structures - to do this us uses the concept of nested sections. When evaluating a section, MongoSyphon can evaluate the contents of another section which may return one or more documented and embed them in the higher level section. This is specified using the `@section` notation.

Taking our Pet Owners example.

```

	start: {
		source : { ... },
		target: { ... },
		template: {
			_id: "$ownerid",
			name: "$name",
			address : "$address",
			pets : [ "@petsection" ]
		},
		query : {
		  sql: 'SELECT * FROM owner'
		}
	},

	petsection: {
		template: {
			petid: "$petid",
			name: "$name",
		},
		query : { sql: 'SELECT * FROM pet where owner = ?' } ,
		params: [ "ownerid" ]

	}
```

In `ownerssection` we specify that for each owner we want an Array (denoted by the square brackets) of results of `petsection`. If the array is empty the field will be omitteed rather than embed an empty array.

`petsection` is similar to `ownersection` except the SQL uses a WHERE clause with SQL placehoders _?_ in the query to retrieve a subset of the data. In this case oly pets for a given owner. The values used to parameterise the query are defined in the `params:` value which is always an array of column names from the parent or calling section. In this case ownerid from ownersection.

***It is Critical that these queries are correctly indexed on the RDBMS***

#### Single Embedded Objects

Calling a sub section with the parameter surrounded by square brackets embeds an array in the parent. There are also two ways to create a nested object rather than array.

The simplest is used to take values from the current query and present them nested. Imagine we had an RDBMS Row of the form

|Name|StreetNumber|StreetName|Town|Country|PostalCode|
|----|------------|----------|----|-------|----------|
|Charlotte|22|Accacia Avenue|Staines|UK|ST1 8BP|

We could have a template like this.

```
 peoplesection: {
        template: {
            name: "$name",
            streetnumber: "$streetnumber",
            streetname: "$streetname",
            town: "$town",
            country: "$country",
            postalcode: "$postalcode"
        },
        query: { sql: 'SELECT * FROM people' }
    }
```

Or we may want a more nested schema from that single row - like this

```
 peoplesection: {
        template: {
	            name: "$name",
	            address : {
		            streetnumber: "$streetnumber",
		            streetname: "$streetname",
		            town: "$town",
		            country: "$country",
		            postalcode: "$postalcode"
            }
        },
        query: { sql: 'SELECT * FROM people' }
    }
```

Which is also valid.

We can also embed an object from a sub-section by calling the subsection but _not including the square brackets_ this causes the subsection to be evaluated _only for the first matching value_. In our pets example we may do this for a lookup table like the pet's species like so. This should only be used normally where you *know* there is only a single matching value.

```

	petsection: {
		template: {
			petid: "$petid",
			name: "$name",
			type : "@speciessection"
		},
		query : { sql: 'SELECT * FROM pet where owner = ?' },
		params: [ "ownerid" ]

	},

	speciessection: {
		template: {
			species : "$species",
			breed : "$breed"
		},
		query : { sql: 'SELECT * from species where speciesid = ?' },
		params : [ "species" ]

	}
```

This would result in a document like:

```
{
	"_id" : 2,
	"name" : "Sarah",
	"address" : "19 Main Street",
	"pets" : [
		{
			"petid" : 3,
			"name" : "Mittens",
			"type" : { species:  "Cat",
			           breed : "Siamese"
			         }
		}
	]
}
```

#### Scalar Values

Sometimes you have only a single value you are interested in being returned from a sub-section. This can be the case regardless of whther it is being returned to an array or a scalar at a higher level - you don't want to return an object as it will have a single member.

Mongosyphon lets us use the special value `_value` to denote that the output of this section should not be an object but in fact a scalar.

This is often the case when the section relates to a lookup table. If, in our example above, we did not have _species_ *and* _breed_ for each pet but just the species then we may choose to define it differently. By putting the species values into `_value` we can get a different output.

```

	petsection: {
		template: {
			petid: "$petid",
			name: "$name",
			type : "@speciessection"
		},
		query: { sql: 'SELECT * FROM pet where owner = ?'},
		params: [ "ownerid" ]

	},

	speciessection: {
		template: {
			_value : "$species",
		},
		query: { sql: 'SELECT * from species where speciesid = ?' },
		params : [ "species" ]

	}
```

We get the output

```
{
	"_id" : 2,
	"name" : "Sarah",
	"address" : "19 Main Street",
	"pets" : [
		{
			"petid" : 3,
			"name" : "Mittens",
			"type" : "Cat"
		}
	]
}
```

Note that _type_ is not a String value not an object.

### Caching Sections

The default mode in mongosyphon is to perform standalone queries (using pre-prepared statements and pooled connections for efficiency) to retrieve each lower section. In some cases this can result in the same query being performed many times - for example retrieving the "species = cat" data in the above example for each cat in the database.

Most RDBMS will cache frequent queries like this but there is still considerable overhead in round trips to the server, preparing the query etc. Mongosyphon allows you to cache a section at the client side if it is expected to return a relatively small number of discrete results. For example we know there are only a few pet species so we can add the parameter `cached: true` in the section. Using this will cause MongoSyphon to cache the output for each set of input params and greatly reduce calls to the server.

Be aware that you can cache any size of sub or nested object but the cache will require RAM and will grow with each new entry there is no cache eviction.

### Merging Sections

In many mappings from RDBMS to MongoDB there is a one to many relationship between one table an another - in our example each owner has many pets. Each pet has a single owner. In this case we can use a more efficient method to generate the data by sorting and merging the source tables. this is the option `mergeon`.

To do this we need to ensure both sections are sorted by the field we are using to join them. 'ownerid' in the owners tables and 'owner' in the pets table.

We can then use 'mergeon' to walk both tables merging the results.

_behind the scenes, MongoSyphon simply keeps the cursor open for the sub document and assumes any results it is looking for will be ordered at the point it previously left off_

Our pet's config, using this efficient merging mechanism, which avoids a query on the pets table per owner looks like this.

```
	ownerssection: {
		template: {
			_id: "$ownerid",
			name: "$name",
			address : "$address",
			pets : [ "@petsection" ]
		},
		query: { sql: 'SELECT *,ownerid as owner FROM owner order by ownerid'}

	},

	petsection: {
		template: {
			petid: "$petid",
			name: "$name",
		},
		query: { sql: 'SELECT * FROM pet order by owner'},
		mergeon:  "owner"

	},

```

Currently MongoSyphon only merges on a field with the same name, therefore we need to alias the field ownerid to owner using SQL. In future this will allow merging on multiple columns with different names.

This typically requires you have an index on the column being sorted however that would also be required for the query based method above and woudl typically exist for retrieval anyway.

### Pushing transformation work  to SQL

All the previous examples have used simple SQL statements against a single database. The reason the JOIN operations have not been pushed ot the underlying database is that there is rarely a good and efficient way to do that for a 1 to Many relationship. You either have to retrieve repeated columns for many rows or use something like MySQL's **GROUP_CONCAT** function and then parse a text string.

For Many to Many and One to One relationships, sometimes it is better to push some logic into the SQL query and MongoSyphon imploses no limits on this other than your own SQL abilities.

We have seen an example of renaming a field above using *AS* where we wanted a column with a different name,

`sql: 'SELECT *,ownerid as owner FROM owner order by ownerid'`

We might also want to perform an operation like a sum or concatenation at the RDBMS side.

`sql: 'SELECT *,LEFT(telno,3) AS AREACODE,MID(telno,3,7) as DIGITS'`

Finally we can use the RDBMS to perform a server side **JOIN** where it is appropriate using any SQL syntax. Again correct indexing is vital to the performance of this but it *may* still be faster than using MongoSyphon to perform the join.

```
sql: 'SELECT ACTOR.NAME as NAME ,ACTORS2MOVIES.ROLE AS ROLE
		 FROM ACTORS,ACTORS2MOVIES WHERE MOVIEID=? AND
		 ACTORS.ACTORID=ACTORS2MOVIES.ACTORID'
```

### Java transformations

Mongosyphon now supports document transformations which can be performed in Java code via a series of transformers which can be applied
prior to the document being inserted or updated. The transformer can be any Java class which implements the `com.johnlpage.mongosyphon.IDocumentTransformer` interface and has a default / no-args constructor. The final source document before insertion will be
passed to the `transform()` method, which should perform any modifications desired on that source document. Below is a very basic example of
a transformer which will add a new field to the document called `fullNameUpper` consisting of the upper-cased contentation of the document
fields `firstName` and `lastName`. The transformer will also remove the document field `somethingExtra`.

```
package com.johnlpage.mongosyphon.transformer.custom;
import org.bson.Document;
import com.johnlpage.mongosyphon.IDocumentTransformer;

public class TestTransform implements IDocumentTransformer {

    public void transform(Document source) {
        String firstName = source.getString("firstName");
        String lastName = source.getString("lastName");
        String fullNameUpper = (firstName + " " + lastName).toUpperCase();
        source.put("fullNameUpper", fullNameUpper);
        source.remove("somethingExtra");
    }
}
```

Configure the transformer(s) via the `documentTransformers` element, for example:

```
{
    start: {
        source: {...},
        target: {...},
        template: {...},
        query:{...},
        documentTransformers: [
            {className: "com.johnlpage.mongosyphon.transformer.custom.TestTransform"}
        ]
    },
    ...
}
```

**NOTE** that transformers are currently only applied to top level document and will not be applied/processed for any subsection. This allows the entire document to be modified in it's final state (after subsections have been applied) before being output.

### Updates and Inserts

All previous examples have used the target `mode` of insert.

It is possible to use MongoSyphon to update an existing MongoDB database as well. To do so this use `mode: "update"`, `mode: "upsert"`,
or `mode: "save"` as target level parameters depending on the behaviour you desire.

The `mode: "save"` option follows the same semantics that `save()` uses in the mongo shell or driver methods. If the document
provided does not contain an `_id` field, then an insert will be performed. Otherwise, if the document contains an `_id` field
an upsert will be performed which will either insert a new document (if one does not exist) or overwrite an existing document.

When using `mode: "update"` or `mode: "upsert"` the format of the top level document is different, it must be of the form:

```
someupdatesection: {
	template: {
	  "$find" : { ... },
	  "$set" : { ... },
	  "$inc" : { ... },
	  "$push" : {...}
	},
query: { sql: 'select X from Y' }
}
```
The template should correspond to a MongoDB update document specification using one of more update operators. The exception is the "$find" operator which specifies which *single* document to find an update. Currently multi-update is not supported.

By using the $ operator you can create update specifications which query an RDBMS table and then push updates into nested documents and arrays in MongoDB. For example if we now had a collection mapping pet_id's to microchip numbers we could push those through to update the pet records, or push them unto an array under the owner.

This is a very powerful feature but will take some thinking about to gain maximum benefit.

### Reading from MongoDB

Any section can also query MongoDB, so do this, in place of the *sql* option we use a *mongoquery* option like this

```
query: {
            database: "test",
            collection: "atest" ,
            find: { },
            project : { },
            limit : 1,
            sort: { }
        }
```

or

```
mongoquery: {
            database: "test",
            collection: "atest" ,
            aggregate: [{$match:{}},
                        {$group:{_id:null,x:{$sum:"$x"}}}]
        }
```

This can be used to convert or move data within  or between MongoDB systems  - including all the Join and Merge operators. It can also be used with an  RDBMS or Multiple MongoDB systems to merge data from multiple sources.

MongoDB queries are parameterised using $1 to $9 in the find() part or the $match part for queries and aggregation respectively. This is equivalent to using ? in the SQL query.

### Performing an initial query

Mongosyphon and run a qury agains either the RDBMS or MongoDB without outputting the records to act as parameters for a section that does output data. As a very simple example. Assume you are converting 10 Million records - they will be streamed from the RDBMS to MongoDB by MongoSyphon.

This stops part way through for some reason and you have N million converted.

As long as when you converted them you were sorting by some useful key, perhaps the primary key. With the Mongo Query facility you can query mongodb for the highest primary key it has loaded and continue from there. This makes it restartable but also allows for ongoing migration where you have an incrementing serial number of datestamped changes.

The MongoDB Query is run against the top level collection you are loading, it returns a single Document which is treated like an RDBMS row. It is configured using three top level paramaeters.

The examples below simply mean, find the highest _id value in the Mongo Database

Example, assuming in the source *movieid* is a sequence, find any new movies. So query all movies, order by _id in reverse adn take the first one. If nothing found then return an _id value of -1

```

start : {
    source: {
        uri: "mongodb://MacPro.local:27017/"
    },
    query: {
        database: "imdb",
        collection: "movies" ,
        find: {} ,
        limit: 1,
        project: { _id: 1},
        sort: { _id : -1},
        default : { _id: 0 }
    },
    target : {
        mode: "subsection",
        uri: "moviesection",
      }
},

"moviessection": {
    source: {
    	uri:  "jdbc:mysql://localhost:3306/sdemo?useSSL=false",
	user: "root",
	password: "password"
	},
   "template": {
      "_id": "$movieid",
      "title": "$title",
      "year": "$year",
       },
    query :{"sql": "SELECT * FROM movies where movieid > ? order by movieid"}
    "params" : [ "_id" ]
  },
```

Using this currently may require some MongoDB schema changes to support it - for example:

Given our owners and pets scenario - new pets are added to the RDBMS - to determine the last petid we saw we need to query MongoDB for the max petid which is in a subdocument.

To do this we can use an aggregation pipeline or a view built on one but that will not be fast.

In each document (owner) we can add a top level field (maxpetid) calculated using it's own section on initial load and using $max on subsequent updates. We can index and sort on this as we are doing above for _id. This is the current best approach. but needs another field and index to support it.

We can run an additional top level update to calculate and retain the max value of *petid* in a different collection , as long as we can specify a collection for the Mongo Query. This shoudl be bound into the same config though once multiple parallel conversion are allowed.

### Logging and output

Logging is handled by logback and configured in the logback xml either in the JAR file or supplied at runtime. By default logging data is verbose and goes to MongoSyphon.log in the directory you are running MongoSyphon.


text:
// Copyright 2017 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include "third_party/blink/renderer/modules/mediastream/video_track_adapter.h"

#include <limits>

#include "base/callback_helpers.h"
#include "base/run_loop.h"
#include "base/synchronization/waitable_event.h"
#include "base/test/bind.h"
#include "base/threading/thread.h"
#include "media/base/limits.h"
#include "testing/gtest/include/gtest/gtest.h"
#include "third_party/blink/public/web/web_heap.h"
#include "third_party/blink/renderer/modules/mediastream/mock_encoded_video_frame.h"
#include "third_party/blink/renderer/modules/mediastream/mock_media_stream_video_source.h"
#include "third_party/blink/renderer/modules/mediastream/video_track_adapter_settings.h"
#include "third_party/blink/renderer/platform/testing/io_task_runner_testing_platform_support.h"
#include "third_party/blink/renderer/platform/testing/video_frame_utils.h"
#include "third_party/webrtc_overrides/metronome_provider.h"
#include "third_party/webrtc_overrides/metronome_source.h"

namespace blink {

// Most VideoTrackAdapter functionality is tested in MediaStreamVideoSourceTest.
// These tests focus on the computation of cropped frame sizes in edge cases
// that cannot be easily reproduced by a mocked video source, such as tests
// involving frames of zero size.
// Such frames can be produced by sources in the wild (e.g., element capture).

// Test that cropped sizes with zero-area input frames are correctly computed.
// Aspect ratio limits should be ignored.
TEST(VideoTrackAdapterTest, ZeroInputArea) {
  const int kMaxWidth = 640;
  const int kMaxHeight = 480;
  const int kSmallDimension = 300;
  const int kLargeDimension = 1000;
  static_assert(kSmallDimension < kMaxWidth && kSmallDimension < kMaxHeight,
                "kSmallDimension must be less than kMaxWidth and kMaxHeight");
  static_assert(
      kLargeDimension > kMaxWidth && kLargeDimension > kMaxHeight,
      "kLargeDimension must be greater than kMaxWidth and kMaxHeight");
  const VideoTrackAdapterSettings kVideoTrackAdapterSettings(
      gfx::Size(kMaxWidth, kMaxHeight), 0.1 /* min_aspect_ratio */,
      2.0 /* max_aspect_ratio */, 0.0 /* max_frame_rate */);
  const bool kIsRotatedValues[] = {true, false};

  for (bool is_rotated : kIsRotatedValues) {
    gfx::Size desired_size;

    VideoTrackAdapter::CalculateDesiredSize(
        is_rotated, gfx::Size(0, 0), kVideoTrackAdapterSettings, &desired_size);
    EXPECT_EQ(desired_size.width(), 0);
    EXPECT_EQ(desired_size.height(), 0);

    // Zero width.
    VideoTrackAdapter::CalculateDesiredSize(
        is_rotated, gfx::Size(0, kSmallDimension), kVideoTrackAdapterSettings,
        &desired_size);
    EXPECT_EQ(desired_size.width(), 0);
    EXPECT_EQ(desired_size.height(), kSmallDimension);

    // Zero height.
    VideoTrackAdapter::CalculateDesiredSize(
        is_rotated, gfx::Size(kSmallDimension, 0), kVideoTrackAdapterSettings,
        &desired_size);
    EXPECT_EQ(desired_size.width(), kSmallDimension);
    EXPECT_EQ(desired_size.height(), 0);

    // Requires "cropping" of height.
    VideoTrackAdapter::CalculateDesiredSize(
        is_rotated, gfx::Size(0, kLargeDimension), kVideoTrackAdapterSettings,
        &desired_size);
    EXPECT_EQ(desired_size.width(), 0);
    EXPECT_EQ(desired_size.height(), is_rotated ? kMaxWidth : kMaxHeight);

    // Requires "cropping" of width.
    VideoTrackAdapter::CalculateDesiredSize(
        is_rotated, gfx::Size(kLargeDimension, 0), kVideoTrackAdapterSettings,
        &desired_size);
    EXPECT_EQ(desired_size.width(), is_rotated ? kMaxHeight : kMaxWidth);
    EXPECT_EQ(desired_size.height(), 0);
  }
}

// Test that zero-size cropped areas are correctly computed. Aspect ratio
// limits should be ignored.
TEST(VideoTrackAdapterTest, ZeroOutputArea) {
  const double kMinAspectRatio = 0.1;
  const double kMaxAspectRatio = 2.0;
  const int kInputWidth = 640;
  const int kInputHeight = 480;
  const int kSmallMaxDimension = 300;
  const int kLargeMaxDimension = 1000;
  static_assert(
      kSmallMaxDimension < kInputWidth && kSmallMaxDimension < kInputHeight,
      "kSmallMaxDimension must be less than kInputWidth and kInputHeight");
  static_assert(
      kLargeMaxDimension > kInputWidth && kLargeMaxDimension > kInputHeight,
      "kLargeMaxDimension must be greater than kInputWidth and kInputHeight");

  gfx::Size desired_size;

  VideoTrackAdapter::CalculateDesiredSize(
      false /* is_rotated */, gfx::Size(kInputWidth, kInputHeight),
      VideoTrackAdapterSettings(gfx::Size(0, 0), kMinAspectRatio,
                                kMaxAspectRatio, 0.0 /* max_frame_rate */),
      &desired_size);
  EXPECT_EQ(desired_size.width(), 0);
  EXPECT_EQ(desired_size.height(), 0);

  // Width is cropped to zero.
  VideoTrackAdapter::CalculateDesiredSize(
      false /* is_rotated */, gfx::Size(kInputWidth, kInputHeight),
      VideoTrackAdapterSettings(gfx::Size(0, kLargeMaxDimension), 0.0,
                                HUGE_VAL,  // kMinAspectRatio, kMaxAspectRatio,
                                0.0 /* max_frame_rate */),
      &desired_size);
  EXPECT_EQ(desired_size.width(), 0);
  EXPECT_EQ(desired_size.height(), kInputHeight);

  // Requires "cropping" of width and height.
  VideoTrackAdapter::CalculateDesiredSize(
      false /* is_rotated */, gfx::Size(kInputWidth, kInputHeight),
      VideoTrackAdapterSettings(gfx::Size(0, kSmallMaxDimension),
                                kMinAspectRatio, kMaxAspectRatio,
                                0.0 /* max_frame_rate */),
      &desired_size);
  EXPECT_EQ(desired_size.width(), 0);
  EXPECT_EQ(desired_size.height(), kSmallMaxDimension);

  // Height is cropped to zero.
  VideoTrackAdapter::CalculateDesiredSize(
      false /* is_rotated */, gfx::Size(kInputWidth, kInputHeight),
      VideoTrackAdapterSettings(gfx::Size(kLargeMaxDimension, 0),
                                kMinAspectRatio, kMaxAspectRatio,
                                0.0 /* max_frame_rate */),
      &desired_size);
  EXPECT_EQ(desired_size.width(), kInputWidth);
  EXPECT_EQ(desired_size.height(), 0);

  // Requires "cropping" of width and height.
  VideoTrackAdapter::CalculateDesiredSize(
      false /* is_rotated */, gfx::Size(kInputWidth, kInputHeight),
      VideoTrackAdapterSettings(
          gfx::Size(kSmallMaxDimension /* max_width */, 0 /* max_height */),
          kMinAspectRatio, kMaxAspectRatio, 0.0 /* max_frame_rate */),
      &desired_size);
  EXPECT_EQ(desired_size.width(), kSmallMaxDimension);
  EXPECT_EQ(desired_size.height(), 0);
}

// Test that large frames are handled correctly.
TEST(VideoTrackAdapterTest, LargeFrames) {
  const int kInputWidth = std::numeric_limits<int>::max();
  const int kInputHeight = std::numeric_limits<int>::max();
  const int kMaxWidth = std::numeric_limits<int>::max();
  const int kMaxHeight = std::numeric_limits<int>::max();

  gfx::Size desired_size;

  // If a target size is provided in VideoTrackAdapterSettings, rescaling is
  // allowed and frames must be clamped to the maximum allowed size, even if the
  // target exceeds the system maximum.
  bool success = VideoTrackAdapter::CalculateDesiredSize(
      false /* is_rotated */, gfx::Size(kInputWidth, kInputHeight),
      VideoTrackAdapterSettings(gfx::Size(kMaxWidth, kMaxHeight),
                                0.0 /* max_frame_rate */),
      &desired_size);
  EXPECT_TRUE(success);
  EXPECT_EQ(desired_size.width(), media::limits::kMaxDimension);
  EXPECT_EQ(desired_size.height(), media::limits::kMaxDimension);

  // If no target size is provided in VideoTrackAdapterSettings, rescaling is
  // disabled and |desired_size| is left unmodified.
  desired_size.set_width(0);
  desired_size.set_height(0);
  success = VideoTrackAdapter::CalculateDesiredSize(
      false /* is_rotated */, gfx::Size(kInputWidth, kInputHeight),
      VideoTrackAdapterSettings(), &desired_size);
  EXPECT_FALSE(success);
  EXPECT_EQ(desired_size.width(), 0);
  EXPECT_EQ(desired_size.height(), 0);
}

// Test that regular frames are not rescaled if settings do not specify a target
// resolution.
TEST(VideoTrackAdapterTest, NoRescaling) {
  const int kInputWidth = 640;
  const int kInputHeight = 480;

  // No target size,
  gfx::Size desired_size;
  bool success = VideoTrackAdapter::CalculateDesiredSize(
      false /* is_rotated */, gfx::Size(kInputWidth, kInputHeight),
      VideoTrackAdapterSettings(), &desired_size);
  EXPECT_TRUE(success);
  EXPECT_EQ(desired_size.width(), kInputWidth);
  EXPECT_EQ(desired_size.height(), kInputHeight);
}

class VideoTrackAdapterFixtureTest : public ::testing::Test {
 public:
  VideoTrackAdapterFixtureTest()
      : testing_render_thread_("TestingRenderThread"),
        metronome_provider_(base::MakeRefCounted<MetronomeProvider>()),
        frame_received_(base::WaitableEvent::ResetPolicy::MANUAL,
                        base::WaitableEvent::InitialState::NOT_SIGNALED) {}
  ~VideoTrackAdapterFixtureTest() override = default;

 protected:
  void SetUp() override { testing_render_thread_.Start(); }

  void TearDown() override {
    if (track_added_) {
      testing_render_thread_.task_runner()->PostTask(
          FROM_HERE, base::BindOnce(&VideoTrackAdapter::RemoveTrack, adapter_,
                                    null_track_.get()));
    }
    testing_render_thread_.Stop();
  }

  void CreateAdapter(media::VideoCaptureFormat capture_format) {
    mock_source_ =
        std::make_unique<MockMediaStreamVideoSource>(capture_format, false);
    // Create the VideoTrackAdapter instance on |testing_render_thread_|.
    base::WaitableEvent adapter_created(
        base::WaitableEvent::ResetPolicy::MANUAL,
        base::WaitableEvent::InitialState::NOT_SIGNALED);
    testing_render_thread_.task_runner()->PostTask(
        FROM_HERE, base::BindLambdaForTesting([&]() {
          adapter_ = base::MakeRefCounted<VideoTrackAdapter>(
              platform_support_->GetIOTaskRunner(), metronome_provider_,
              mock_source_->GetWeakPtr());
          adapter_created.Signal();
        }));
    adapter_created.Wait();
  }

  // Create or re-configure the dummy |null_track_| with the given
  // |adapter_settings|.
  void ConfigureTrack(const VideoTrackAdapterSettings& adapter_settings) {
    if (!track_added_) {
      AddTrackInternal(null_track_.get(), adapter_settings);
      track_added_ = true;
    } else {
      testing_render_thread_.task_runner()->PostTask(
          FROM_HERE,
          base::BindOnce(&VideoTrackAdapter::ReconfigureTrack, adapter_,
                         null_track_.get(), adapter_settings));
    }
  }

  void AddTrackInternal(MediaStreamVideoTrack* track,
                        const VideoTrackAdapterSettings& adapter_settings) {
    testing_render_thread_.task_runner()->PostTask(
        FROM_HERE,
        base::BindOnce(
            &VideoTrackAdapter::AddTrack, adapter_, track,
            base::BindRepeating(&VideoTrackAdapterFixtureTest::OnFrameDelivered,
                                base::Unretained(this)),
            base::BindRepeating(
                &VideoTrackAdapterFixtureTest::OnEncodedVideoFrameDelivered,
                base::Unretained(this)),
            base::DoNothing(), base::DoNothing(), adapter_settings));
  }

  void StartFrameMonitoring() {
    base::WaitableEvent frame_monitoring_started;
    testing_render_thread_.task_runner()->PostTask(
        FROM_HERE,
        base::BindOnce(
            [](scoped_refptr<VideoTrackAdapter> adapter,
               scoped_refptr<base::SequencedTaskRunner> io_task_runner,
               base::WaitableEvent* frame_monitoring_started) {
              adapter->StartFrameMonitoring(
                  30.0, base::BindRepeating([](bool mute_state) {}));
              // Signal the event on the IO thread because that is where frame
              // monitoring starts internally.
              io_task_runner->PostTask(
                  FROM_HERE,
                  base::BindOnce(&base::WaitableEvent::Signal,
                                 base::Unretained(frame_monitoring_started)));
            },
            adapter_, platform_support_->GetIOTaskRunner(),
            base::Unretained(&frame_monitoring_started)));
    frame_monitoring_started.Wait();
  }

  void StopFrameMonitoring() {
    base::WaitableEvent frame_monitoring_stopped;
    testing_render_thread_.task_runner()->PostTask(
        FROM_HERE,
        base::BindOnce(
            [](scoped_refptr<VideoTrackAdapter> adapter,
               scoped_refptr<base::SequencedTaskRunner> io_task_runner,
               base::WaitableEvent* frame_monitoring_stopped) {
              adapter->StopFrameMonitoring();
              // Signal the event on the IO thread because that is where frame
              // monitoring stops internally.
              io_task_runner->PostTask(
                  FROM_HERE,
                  base::BindOnce(&base::WaitableEvent::Signal,
                                 base::Unretained(frame_monitoring_stopped)));
            },
            adapter_, platform_support_->GetIOTaskRunner(),
            base::Unretained(&frame_monitoring_stopped)));
    frame_monitoring_stopped.Wait();
  }

  bool IsMetronomeSourceActive(
      scoped_refptr<MetronomeSource> metronome_source) {
    bool is_active;
    base::WaitableEvent event;
    // It's possible to check IsActive() from any thread, but because stopping
    // a timer unsubscibes from the metronome asynchronously, checking for
    // activity on the IO thread ensures any pending listeners have been
    // removed before getting the status.
    platform_support_->GetIOTaskRunner()->PostTask(
        FROM_HERE, base::BindOnce(
                       [](scoped_refptr<MetronomeSource> metronome_source,
                          bool* is_active, base::WaitableEvent* event) {
                         *is_active = metronome_source->IsActive();
                         event->Signal();
                       },
                       metronome_source, base::Unretained(&is_active),
                       base::Unretained(&event)));
    event.Wait();
    return is_active;
  }

  void SetFrameValidationCallback(VideoCaptureDeliverFrameCB callback) {
    frame_validation_callback_ = std::move(callback);
  }

  // Deliver |frame| to |adapter_| and wait until OnFrameDelivered signals that
  // it receives the processed frame.
  void DeliverAndValidateFrame(scoped_refptr<media::VideoFrame> frame,
                               base::TimeTicks estimated_capture_time) {
    auto deliver_frame = [&]() {
      platform_support_->GetIOTaskRunner()->PostTask(
          FROM_HERE,
          base::BindOnce(&VideoTrackAdapter::DeliverFrameOnIO, adapter_, frame,
                         std::vector<scoped_refptr<media::VideoFrame>>(),
                         estimated_capture_time));
    };

    frame_received_.Reset();
    // Bounce the call to DeliverFrameOnIO off |testing_render_thread_| to
    // synchronize with the AddTrackOnIO / ReconfigureTrackOnIO that would be
    // invoked through ConfigureTrack.
    testing_render_thread_.task_runner()->PostTask(
        FROM_HERE, base::BindLambdaForTesting(deliver_frame));
    frame_received_.Wait();
  }

  void OnFrameDelivered(
      scoped_refptr<media::VideoFrame> frame,
      std::vector<scoped_refptr<media::VideoFrame>> scaled_frames,
      base::TimeTicks estimated_capture_time) {
    if (frame_validation_callback_) {
      frame_validation_callback_.Run(frame, {}, estimated_capture_time);
    }
    frame_received_.Signal();
  }

  MOCK_METHOD2(OnEncodedVideoFrameDelivered,
               void(scoped_refptr<EncodedVideoFrame>,
                    base::TimeTicks estimated_capture_time));

  ScopedTestingPlatformSupport<IOTaskRunnerTestingPlatformSupport>
      platform_support_;
  base::Thread testing_render_thread_;
  const scoped_refptr<MetronomeProvider> metronome_provider_;
  std::unique_ptr<MockMediaStreamVideoSource> mock_source_;
  scoped_refptr<VideoTrackAdapter> adapter_;

  base::WaitableEvent frame_received_;
  VideoCaptureDeliverFrameCB frame_validation_callback_;

  // For testing we use a nullptr for MediaStreamVideoTrack.
  std::unique_ptr<MediaStreamVideoTrack> null_track_;
  bool track_added_ = false;
};

// Verifies that the metronome is used while frame monioring. For testing of the
// mute detection logic that is based on this frame monitoring, see
// MediaStreamVideoSourceTest.MutedSource instead.
TEST_F(VideoTrackAdapterFixtureTest, MetronomeIsUsedWhileFrameMonitoring) {
  scoped_refptr<MetronomeSource> metronome_source =
      base::MakeRefCounted<MetronomeSource>(base::Hertz(64));
  metronome_provider_->OnStartUsingMetronome(metronome_source);

  const media::VideoCaptureFormat stream_format(gfx::Size(1280, 960), 30.0,
                                                media::PIXEL_FORMAT_NV12);
  CreateAdapter(stream_format);

  EXPECT_FALSE(IsMetronomeSourceActive(metronome_source));
  StartFrameMonitoring();
  EXPECT_TRUE(IsMetronomeSourceActive(metronome_source));
  StopFrameMonitoring();
  EXPECT_FALSE(IsMetronomeSourceActive(metronome_source));
}

TEST_F(VideoTrackAdapterFixtureTest, DeliverFrame_GpuMemoryBuffer) {
  // Attributes for the original input frame.
  const gfx::Size kCodedSize(1280, 960);
  const gfx::Rect kVisibleRect(0, 120, 1280, 720);
  const gfx::Size kNaturalSize(1280, 720);
  const double kFrameRate = 30.0;
  auto gmb_frame =
      CreateTestFrame(kCodedSize, kVisibleRect, kNaturalSize,
                      media::VideoFrame::STORAGE_GPU_MEMORY_BUFFER);

  // Initialize the VideoTrackAdapter to handle GpuMemoryBuffer. NV12 is the
  // only pixel format supported at the moment.
  const media::VideoCaptureFormat stream_format(kCodedSize, kFrameRate,
                                                media::PIXEL_FORMAT_NV12);
  CreateAdapter(stream_format);

  // Keep the desired size the same as the natural size of the original frame.
  VideoTrackAdapterSettings settings_nonscaled(kNaturalSize, kFrameRate);
  ConfigureTrack(settings_nonscaled);
  auto check_nonscaled =
      [&](scoped_refptr<media::VideoFrame> frame,
          std::vector<scoped_refptr<media::VideoFrame>> scaled_frames,
          base::TimeTicks estimated_capture_time) {
        // We should get the original frame as-is here.
        EXPECT_EQ(frame->storage_type(),
                  media::VideoFrame::STORAGE_GPU_MEMORY_BUFFER);
        EXPECT_EQ(frame->GetGpuMemoryBuffer(), gmb_frame->GetGpuMemoryBuffer());
        EXPECT_EQ(frame->coded_size(), kCodedSize);
        EXPECT_EQ(frame->visible_rect(), kVisibleRect);
        EXPECT_EQ(frame->natural_size(), kNaturalSize);
      };
  SetFrameValidationCallback(base::BindLambdaForTesting(check_nonscaled));
  DeliverAndValidateFrame(gmb_frame, base::TimeTicks());

  // Scale the original frame by a factor of 0.5x.
  const gfx::Size kDesiredSize(640, 360);
  VideoTrackAdapterSettings settings_scaled(kDesiredSize, kFrameRate);
  ConfigureTrack(settings_scaled);
  auto check_scaled =
      [&](scoped_refptr<media::VideoFrame> frame,
          std::vector<scoped_refptr<media::VideoFrame>> scaled_frames,
          base::TimeTicks estimated_capture_time) {
        // The original frame should be wrapped in a new frame, with
        // |kDesiredSize| exposed as natural size of the wrapped frame.
        EXPECT_EQ(frame->storage_type(),
                  media::VideoFrame::STORAGE_GPU_MEMORY_BUFFER);
        EXPECT_EQ(frame->GetGpuMemoryBuffer(), gmb_frame->GetGpuMemoryBuffer());
        EXPECT_EQ(frame->coded_size(), kCodedSize);
        EXPECT_EQ(frame->visible_rect(), kVisibleRect);
        EXPECT_EQ(frame->natural_size(), kDesiredSize);
      };
  SetFrameValidationCallback(base::BindLambdaForTesting(check_scaled));
  DeliverAndValidateFrame(gmb_frame, base::TimeTicks());
}

// Tests that we run the |settings_callback| for any additional tracks that
// share a VideoFrameResolutionAdapter with an existing track. This ensures that
// the additional track's default frame_size and frame_rate are updated to match
// incoming frames.
TEST_F(VideoTrackAdapterFixtureTest,
       DeliverPortraitFrame_RunSettingsCallbackForSecondTrack) {
  // Attributes for initial track's incoming frame.
  const gfx::Size kCodedSize(480, 640);
  const gfx::Rect kVisibleRect(0, 0, 480, 640);
  const gfx::Size kNaturalSize(480, 640);
  const double kFrameRate = 30.0;
  auto test_frame =
      CreateTestFrame(kCodedSize, kVisibleRect, kNaturalSize,
                      /*storage_type=*/media::VideoFrame::STORAGE_OWNED_MEMORY);

  const media::VideoCaptureFormat stream_format(kCodedSize, kFrameRate,
                                                media::PIXEL_FORMAT_I420);
  CreateAdapter(stream_format);

  // We don't provide a target size for the initial track.
  VideoTrackAdapterSettings adapter_settings(
      /*target_size=*/absl::nullopt,
      /*min_aspect_ratio=*/0.0000,
      /*max_aspect_ratio=*/640.0000, kFrameRate);
  ConfigureTrack(adapter_settings);
  // The delivered frame for the first track should have a portrait orientation.
  auto check_portrait =
      [&](scoped_refptr<media::VideoFrame> frame,
          std::vector<scoped_refptr<media::VideoFrame>> scaled_frames,
          base::TimeTicks estimated_capture_time) {
        // We should get the original frame as-is here.
        EXPECT_EQ(frame->storage_type(),
                  media::VideoFrame::STORAGE_OWNED_MEMORY);
        EXPECT_EQ(frame->coded_size(), kCodedSize);
        EXPECT_EQ(frame->visible_rect(), kVisibleRect);
        EXPECT_EQ(frame->natural_size(), kNaturalSize);
      };
  SetFrameValidationCallback(base::BindLambdaForTesting(check_portrait));
  DeliverAndValidateFrame(test_frame, base::TimeTicks());

  base::WaitableEvent settings_callback_run_;
  // This lambda callback method validates that the |settings_callback|
  // (MediaStreamVideoTrack::SetSizeAndComputedFrameRate) is run with the
  // frame_size & frame_rate stored in the adapter's |track_settings_|. These
  // values should have been set when we delivered a frame for the first
  // track.
  auto check_dimensions = [&](gfx::Size frame_size, double frame_rate) {
    EXPECT_EQ(frame_size, kNaturalSize);
    EXPECT_EQ(frame_rate, kFrameRate);
    settings_callback_run_.Signal();
  };

  // Add an additional track with the same |adapter_settings| as the first
  // track. Because of this it should share a VideoFrameResolutionAdapter with
  // the first track. The lambda callback method should run as part of the
  // VideoTrackAdapter::AddTrack logic.
  std::unique_ptr<MediaStreamVideoTrack> second_track;
  testing_render_thread_.task_runner()->PostTask(
      FROM_HERE,
      base::BindOnce(
          &VideoTrackAdapter::AddTrack, adapter_, second_track.get(),
          /*frame_callback=*/base::DoNothing(),
          /*encoded_frame_callback=*/base::DoNothing(),
          /*settings_callback=*/base::BindLambdaForTesting(check_dimensions),
          /*track_callback=*/base::DoNothing(), adapter_settings));
  settings_callback_run_.Wait();
}

class VideoTrackAdapterEncodedTest : public ::testing::Test {
 public:
  VideoTrackAdapterEncodedTest()
      : render_thread_("VideoTrackAdapterEncodedTest_RenderThread") {
    render_thread_.Start();
    auto source = std::make_unique<MockMediaStreamVideoSource>(
        media::VideoCaptureFormat(), false);
    mock_source_ = source.get();
    web_source_.Initialize(
        blink::WebString::FromASCII("source_id"),
        blink::WebMediaStreamSource::kTypeVideo,
        blink::WebString::FromASCII("DeliverEncodedVideoFrameSource"),
        false /* remote */);
    web_source_.SetPlatformSource(std::move(source));
    RunSyncOnRenderThread([&] {
      adapter_ = base::MakeRefCounted<VideoTrackAdapter>(
          platform_support_->GetIOTaskRunner(), /*metronome_provider=*/nullptr,
          mock_source_->GetWeakPtr());
    });
  }

  ~VideoTrackAdapterEncodedTest() {
    web_source_.Reset();
    WebHeap::CollectAllGarbageForTesting();
  }

  std::unique_ptr<MediaStreamVideoTrack> AddTrack() {
    auto track = std::make_unique<MediaStreamVideoTrack>(
        mock_source_, WebPlatformMediaStreamSource::ConstraintsOnceCallback(),
        true);
    RunSyncOnRenderThread([&] {
      adapter_->AddTrack(
          track.get(),
          base::BindRepeating(&VideoTrackAdapterEncodedTest::OnFrameDelivered,
                              base::Unretained(this)),
          base::BindRepeating(
              &VideoTrackAdapterEncodedTest::OnEncodedVideoFrameDelivered,
              base::Unretained(this)),
          base::DoNothing(), base::DoNothing(), VideoTrackAdapterSettings());
    });
    return track;
  }

  template <class Function>
  void RunSyncOnRenderThread(Function function) {
    base::RunLoop run_loop;
    base::OnceClosure quit_closure = run_loop.QuitClosure();
    render_thread_.task_runner()->PostTask(FROM_HERE,
                                           base::BindLambdaForTesting([&] {
                                             std::move(function)();
                                             std::move(quit_closure).Run();
                                           }));
    run_loop.Run();
  }

  MOCK_METHOD3(OnFrameDelivered,
               void(scoped_refptr<media::VideoFrame> frame,
                    std::vector<scoped_refptr<media::VideoFrame>> scaled_frames,
                    base::TimeTicks estimated_capture_time));
  MOCK_METHOD2(OnEncodedVideoFrameDelivered,
               void(scoped_refptr<EncodedVideoFrame>,
                    base::TimeTicks estimated_capture_time));

 protected:
  ScopedTestingPlatformSupport<IOTaskRunnerTestingPlatformSupport>
      platform_support_;
  base::Thread render_thread_;
  WebMediaStreamSource web_source_;
  MockMediaStreamVideoSource* mock_source_;
  scoped_refptr<VideoTrackAdapter> adapter_;
};

TEST_F(VideoTrackAdapterEncodedTest, DeliverEncodedVideoFrame) {
  auto track1 = AddTrack();
  auto track2 = AddTrack();
  EXPECT_CALL(*this, OnEncodedVideoFrameDelivered)
      .Times(2)
      .WillRepeatedly(
          testing::Invoke([&](const scoped_refptr<EncodedVideoFrame>& frame,
                              base::TimeTicks estimated_capture_time) {}));
  base::RunLoop run_loop;
  base::OnceClosure quit_closure = run_loop.QuitClosure();
  platform_support_->GetIOTaskRunner()->PostTask(
      FROM_HERE, base::BindLambdaForTesting([&]() {
        adapter_->DeliverEncodedVideoFrameOnIO(
            base::MakeRefCounted<MockEncodedVideoFrame>(), base::TimeTicks());
        std::move(quit_closure).Run();
      }));
  run_loop.Run();
  RunSyncOnRenderThread([&] {
    adapter_->RemoveTrack(track1.get());
    adapter_->RemoveTrack(track2.get());
  });
}

}  // namespace blink

redpajama_set_name: RedPajamaGithub


text:
namespace LaYumba.Functional
{
   public struct NoneType
   {
      public static readonly NoneType Default = new NoneType();
   }
}


text:
package models.service;

import javax.annotation.Nullable;

import models.generic.Unique;

/**
 * Created by daniel on 08.05.15.
 */
public interface UniqueRepository<T extends Unique> {
    @Nullable T findByUuid(String uuid);
}


text:
"""
Analyze lux sensor data and use it to predict future behavior of the light.
"""
import math
from numpy.core.numeric import NaN
from collections import deque
from sklearn.preprocessing import StandardScaler

from thingflow.base import OutputThing, FunctionFilter,\
                           SensorEvent, filtermethod
from thingflow.filters.transducer import SensorSlidingMean

from lux_time_utils import get_sunrise_sunset, time_of_day_to_zone,\
    NUM_ZONES, dt_to_minutes, minutes_to_time

MAX_TIME_INTERVAL = 60*4
EXPECTED_TIME_INTERVAL = 60

@filtermethod(OutputThing)
def fill_in_missing_times(this):
    def on_next(self, x):
        if (self.last_time is not None) and \
           (x.ts - self.last_time)>MAX_TIME_INTERVAL:
            ts = self.last_time + EXPECTED_TIME_INTERVAL
            missing = 0
            while (x.ts-ts)>EXPECTED_TIME_INTERVAL:
                if missing==0:
                    self._dispatch_next(SensorEvent(sensor_id=x.sensor_id,
                                                    ts=ts, val=NaN))
                ts += EXPECTED_TIME_INTERVAL
                missing += 1
            print("Found %s missing samples" % missing)
        self.last_time = x.ts
        self._dispatch_next(x)

    f = FunctionFilter(this, on_next=on_next,
                       name="fill_in_missing_times()")
    setattr(f, 'last_time', None)
    return f

class SensorSlidingMeanPassNaNs(SensorSlidingMean):
    """Variant of SensorSlidingMean that passes on NaN-valued events without
    including them in the history. We clear the sliding window each time
    an NaN is found.
    """
    def __init__(self, history_samples):
        super().__init__(history_samples)

    def step(self, event):
        if math.isnan(event.val):
            if len(self.history)>0:
                #self.state -= self.history.popleft().val
                self.state = None
                self.history = deque(maxlen=self.history_samples)
            return event
        else:
            return SensorSlidingMean.step(self, event)

class CaptureNaNIndexes:
    """AntEvents subscriber that watches for values that are
    NaN and tracks those indexes.
    """
    def __init__(self):
        self.idx = 0
        self.nan_indexes = []

    def on_next(self, x):
        if math.isnan(x.val):
            self.nan_indexes.append(self.idx)
        self.idx += 1

    def on_error(self, e):
        pass

    def on_completed(self):
        pass

    def replace_nans(self, array, val):
        for idx in self.nan_indexes:
            print("replacing index %s with %s" % (idx, val))
            array[idx] = val

    def new_array_replace_nans(self, array, replace_value):
        result = []
        for (idx, val) in enumerate(array):
            if idx in self.nan_indexes:
                result.append(replace_value)
            else:
                result.append(val)
        return result

class HmmScanner:
    """Scan through a set of samples in sequential order and
    build up sequences for each time zone that can be passed to
    the HMM learn. For each zone, we need to pass the fit() method
    the concatenated sequence of samples and the lengths of each
    subsequence. We break the samples into multiple subsequences
    whenever we encounter a time gap (as indicated via a NaN value)
    or when we cross between zones.
    """
    def __init__(self):
        self.length = None
        self.zone = None
        self.samples_by_zone = [[] for zone in range(NUM_ZONES)]
        self.lengths_by_zone = [[] for zone in range(NUM_ZONES)]

    def _start_sequence(self, zone, s):
        self.length = 1
        self.zone = zone
        self.samples_by_zone[zone].append(s)

    def _complete_sequence(self):
        if self.length is not None:
            assert self.zone is not None
            self.lengths_by_zone[self.zone].append(self.length)
        self.zone = None
        self.length = None

    def process_samples(self, samples, timestamps):
        for (s, t) in zip(samples, timestamps):
            s = int(s) if not math.isnan(s) else NaN
            (sunrise, sunset) = get_sunrise_sunset(t.year, t.month, t.day)
            current_zone = time_of_day_to_zone(dt_to_minutes(t), sunrise,
                                               sunset)
            if self.length is None:
                if math.isnan(s):
                    continue
                else:
                    self._start_sequence(current_zone, s)
            elif math.isnan(s):
                self._complete_sequence()
            elif self.zone != current_zone:
                self._complete_sequence()
                self._start_sequence(current_zone, s)
            else: # just extend the current sequence
                self.length += 1
                self.samples_by_zone[self.zone].append(s)
        # see if there was an in-progress sequence for which we need to add
        # the length
        self._complete_sequence()
        # sanity check
        for zone in range(NUM_ZONES):
            assert sum(self.lengths_by_zone[zone])==len(self.samples_by_zone[zone])

BACKCHECK_LENGTH = 10

class ScanState:
    """A state machine for finding sequences of on or off samples
    """
    WAITING_FOR_TRANSITION = 0
    RECORDING_LENGTH = 1
    NAN_STATE = 2
    STATE_NAMES=['WAITING', 'RECORDING' 'NAN']

    def __init__(self):
        self.state = None
        self.prev_sample = None
        self.start_zone = None
        self.start_time = None
        self.prev_zone = None
        self.length = None
        self.recorded_events = None
        # The backcheck queue maintains a queue of older samples.
        # This is used to include the sample which is BACKCHECK_LENGTH
        # samples back, which is helpful for some machine learning algorihms.
        self.backcheck_queue = []

    def add_sample(self, s, t):
        s = int(s) if not math.isnan(s) else NaN
        (sunrise, sunset) = get_sunrise_sunset(t.year, t.month, t.day)
        current_zone = time_of_day_to_zone(dt_to_minutes(t), sunrise,
                                           sunset)
        if self.state==None:
            if math.isnan(s):
                self.state = ScanState.NAN_STATE
            else:
                self.state = ScanState.WAITING_FOR_TRANSITION
            print("initial state is %s" % ScanState.STATE_NAMES[self.state])
        elif math.isnan(s):
            if self.state!=ScanState.NAN_STATE:
                print("changing to NAN_STATE at %s" % t)
            self.state = ScanState.NAN_STATE
            self.length = None
            self.start_zone = None
            self.start_time = None
            self.prev_zone = None
            self.recorded_events = None
            self.backcheck_queue = []
        elif self.state==ScanState.NAN_STATE:
            print("changing to WAITING_STATE at %s" % t)
            self.state = ScanState.WAITING_FOR_TRANSITION # got a value
        elif self.state==ScanState.WAITING_FOR_TRANSITION and (s!=self.prev_sample):
            print("changing to RECORDING_STATE(%s) at %s" % (s,t))
            self.state = ScanState.RECORDING_LENGTH
            self.start_zone = current_zone
            self.start_time = t
            self.length = 1
            back = self.backcheck_queue[0] if len(self.backcheck_queue)>0 else s
            self.recorded_events = [(s, t, current_zone, current_zone, 1, back),]
        elif self.state==ScanState.RECORDING_LENGTH:
            if s==self.prev_sample:
                self.length += 1
                back = self.backcheck_queue[0] if len(self.backcheck_queue)>0 else s
                self.recorded_events.append((s, t, self.start_zone, current_zone,
                                             self.length, back),)
            else:
                if self.prev_sample==0:
                    print("OFF sequence zone %s, length %d" % (self.start_zone,
                                                               self.length))
                    self.record_off_sequence(self.start_zone, self.start_time,
                                             self.prev_zone,
                                             self.length)
                else:
                    print("ON sequence zone %s, length %d" % (self.start_zone,
                                                              self.length))
                    self.record_on_sequence(self.start_zone, self.start_time,
                                            self.prev_zone,
                                            self.length)
                # we know this was valid, so call record_event() for each event
                for (evt_s, evt_dt, evt_start_zone, evt_current_zone,
                     evt_length, evt_back) in self.recorded_events:
                    self.record_event(evt_s, evt_dt, evt_start_zone, evt_current_zone,
                                      evt_length, evt_back)
                # reset for the new value of s
                self.length = 1
                self.start_zone = current_zone
                self.start_time = t
                back = self.backcheck_queue[0] if len(self.backcheck_queue)>0 else s
                self.recorded_events = [(s, t, current_zone, current_zone, 1, back),]
        self.prev_sample = s
        self.prev_zone = current_zone
        if not math.isnan(s):
            self.backcheck_queue.append(s)
            if len(self.backcheck_queue)>BACKCHECK_LENGTH:
                self.backcheck_queue.pop(0) # remove the oldest

    def record_event(self, s, dt, start_zone, current_zone, current_length, back_event):
        """Template method that is called when we have a valid sample we
        can use. A sample is valid if it is preceeded by zero or more samples
        of the same value and one or more samples of the other value. This
        means we can provide a correct value for the length. If the samples of
        the current value were preceeded by a NaN, we don't know for certain
        how long that value was present.
        """
        pass

    def record_on_sequence(self, start_zone, start_time, end_zone, length):
        """Template method that is called when we have a sequence of on samples
        we can use. A sequence is valid if it has one or more samples of the
        same value, both preceeded and succeeded by one or more samples of the
        oppositive value. If there is an NaN on either side, we cannot
        conclusively determine the length.
        """
        pass

    def record_off_sequence(self, start_zone, start_time, end_zone, length):
        """Template method that is called when we have a sequence of off samples
        we can use.
        """
        pass

class LengthHistogramState(ScanState):
    """Build lists of on/off lengths that can be used to compute
    histograms.
    """
    def __init__(self):
        super().__init__()
        self.on_lengths = [[] for i in range(NUM_ZONES)]
        self.off_lengths = [[] for i in range(NUM_ZONES)]

    def record_on_sequence(self, start_zone, start_time, end_zone, length):
        self.on_lengths[start_zone].append(length)

    def record_off_sequence(self, start_zone, start_time, end_zone, length):
        self.off_lengths[start_zone].append(length)

def build_length_histogram_data(samples, timestamps):
    """Given a series of samples and timestamps, find on and off
    sequences and build lists of the sequence lengths per zone. A
    'sequence' is a series of samples of the same value preceeded by at
    last one sample of the opposite value (as oppposed to a break in the
    readings). Returns a LengthHistogramState object containing on_lengths
    and off_lengths members.
    """
    state = LengthHistogramState()
    for (s, t) in zip(samples, timestamps):
        state.add_sample(s, t)
    for zone in range(NUM_ZONES):
        state.on_lengths[zone].sort()
        state.off_lengths[zone].sort()
    return state

class LightPredictionStates(ScanState):
    def __init__(self, trainer):
        super().__init__()
        self.trainer = trainer

    def record_on_sequence(self, start_zone, start_time, end_zone, length):
        self.trainer.on_lengths[start_zone].append(length)
        self.trainer.on_lengths_with_start.append((dt_to_minutes(start_time), length),)

    def record_off_sequence(self, start_zone, start_time, end_zone, length):
        self.trainer.off_lengths[start_zone].append(length)
        self.trainer.off_lengths_with_start.append((dt_to_minutes(start_time), length),)

    def record_event(self, s, dt, start_zone, current_zone, current_length, back_event):
        length_bucket = int(round(current_length/BACKCHECK_LENGTH))+1
        if s==1:
            f = (dt.hour, current_zone, 0, length_bucket, back_event)
        else:
            f = (dt.hour, current_zone, length_bucket, 0, back_event)
        self.trainer.training_features.append(self.trainer.feature_filter(f))
        self.trainer.training_targets.append(s)
        self.trainer.obs_by_zone[current_zone].append(s)

class LightPredictionTrainer:
    """This class preprocessed the data to create features for machine
    learning.
    """
    def __init__(self, feature_filter=lambda x: x):
        """The feature filter is a function that can remove elements from
        the feature tuple. By default the tuple is
        (hour, zone, len_off, len_on, back_event).
        """
        self.on_lengths = [[] for i in range(NUM_ZONES)]
        self.off_lengths = [[] for i in range(NUM_ZONES)]
        self.on_lengths_with_start = []
        self.off_lengths_with_start = []
        self.training_features = []
        self.training_targets = []
        self.scaled_features = None
        self.scaler = None
        self.feature_filter = feature_filter
        self.obs_by_zone = [[] for z in range(NUM_ZONES)]

    def _compute_lengths(self, samples, timestamps):
        state_machine = LightPredictionStates(self)
        for (s, t) in zip(samples, timestamps):
            state_machine.add_sample(s, t)
        print("self.on_lengths")
        print(self.on_lengths)
        print("self.off_lengths")
        print(self.off_lengths)

    def _compute_scaled_features(self):
        self.scaler = StandardScaler()
        self.scaled_features = self.scaler.fit_transform(self.training_features)

    def _compute_zones(self):
        """Lets compute some zones where we have the same number of samples
        per zone.
        """
        self.on_lengths_with_start.sort()
        self.off_lengths_with_start.sort()
        samples_per_zone = int(round(len(self.on_lengths_with_start)/4))
        print("Compute zones: %d total samples, try for %d samples per zone" %
              (len(self.on_lengths_with_start), samples_per_zone))
        zone_boundaries = []
        prev_btime = 0
        for boundary_idx in [samples_per_zone, 2*samples_per_zone, 3*samples_per_zone]:
            (btime, _) = self.on_lengths_with_start[boundary_idx]
            while btime==prev_btime:
                # if the same time as the last keep going
                boundary_idx += 1
                (btime, _) = self.on_lengths_with_start[boundary_idx]
            zone_boundaries.append(btime)
        for i in zone_boundaries:
            (hr, mn) = minutes_to_time(i)
            print("computed boundary: %d minutes %02d:%02d" % (i, hr, mn))

    def compute(self, samples, timestamps):
        self._compute_lengths(samples, timestamps)
        self._compute_scaled_features()
        self._compute_zones()

    def features_for_prediction(self, s, dt, length, sample_queue, scaled=True):
        minutes = dt_to_minutes(dt)
        (sunrise, sunset) = get_sunrise_sunset(dt.year, dt.month, dt.day)
        zone = time_of_day_to_zone(minutes, sunrise, sunset)
        length_bucket = int(round(length/BACKCHECK_LENGTH))+1
        hist_value = sample_queue[0] if len(sample_queue)>0 else s
        if len(sample_queue)>=BACKCHECK_LENGTH:
            sample_queue.pop(0)
        if s==1:
            raw = self.feature_filter((dt.hour, zone, 0, length_bucket, hist_value),)
        else:
            raw = self.feature_filter((dt.hour, zone, length_bucket, 0, hist_value),)
        if scaled:
            return self.scaler.transform([raw,])
        else:
            return [raw,]

    def sort_lengths(self):
        for lengths in [self.on_lengths, self.off_lengths]:
            for zone in range(NUM_ZONES):
                lengths[zone].sort()
        print("Sorted On Lengths: %s" % self.on_lengths)
        print("Sorted Off Lengths: %s" % self.off_lengths)

def lengths_to_histogram(lengths):
    sl = sorted(lengths)
    if len(sl)==0:
        return []
    max_len = sl[-1]
    hist = [0 for i in range(max_len+1)]
    for l in sl:
        hist[l] += 1
    return hist

#compute_lengths(kmeans_lux, smoothed_series_writer.index)
#ON_PROBS = compute_probs(kmeans_lux, smoothed_series_writer.index)

import random
random.seed()

class LightPredictorLengthOnly:
    """Use the length data to determine when the light should go on and off
    and how long each time.
    """
    def __init__(self, initial_state, trainer):
        self.current_state = initial_state
        self.current_length = None
        self.current_zone = None
        self.trainer = trainer

    def _choose_length(self, zone, state):
        if state==0:
            lengths = self.trainer.off_lengths
        else:
            lengths = self.trainer.on_lengths
        if len(lengths[zone])==0:
            orig_zone = zone
            while True:
                zone = (zone+1)%NUM_ZONES
                assert zone!=orig_zone
                if len(lengths[zone])>0:
                    print("No lengths for %s available for zone %s, using value from zone %s" %
                          ('ON' if state else 'Off', orig_zone, zone))
                    break
        base_length = random.choice(lengths[zone])
        # add some noise to the length, then subtract 1 since this step is already the first sample
        length = max(int(round(random.gauss(base_length, 0.1*base_length))), 1) - 1
        print("chose %s length %s, randomized to %s" %
              ('ON' if state else 'OFF', base_length, length+1))
        return length

    def predict(self, dt):
        (sunrise, sunset) = get_sunrise_sunset(dt.year, dt.month, dt.day)
        new_zone = time_of_day_to_zone(dt_to_minutes(dt), sunrise, sunset)
        if self.current_zone==None:
            self.current_zone = new_zone
        if self.current_length==None:
            self.current_length = self._choose_length(self.current_zone, self.current_state)
        elif self.current_length==0:
            self.current_state = 0 if self.current_state else 1
            self.current_length = self._choose_length(new_zone, self.current_state)
        else:
            self.current_length -= 1
        self.current_zone = new_zone
        return self.current_state


text:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

	<modelVersion>4.0.0</modelVersion>
	<packaging>war</packaging>
	<version>1.0-SNAPSHOT</version>

	<groupId>edu.vu.isis.schmear</groupId>
	<artifactId>simple-rest</artifactId>

	<developers>
		<developer>
			<id>mwalker</id>
			<name>Michael A. Walker</name>
			<email>mwalker@isis.vanderbilt.edu</email>
			<organization>Vanderbilt University: Institute for Software Integrated Systems</organization>
			<organizationUrl>http://www.isis.vanderbilt.edu</organizationUrl>
			<roles>
				<role>architect</role>
				<role>developer</role>
			</roles>
			<timezone>-6</timezone>
		</developer>
		<developer>
			<id>violettav</id>
			<name>Violetta Vylegzhanina</name>
			<email>violetta.vylegzhanina@vanderbilt.edu</email>
			<organization>Vanderbilt University: Institute for Software Integrated Systems</organization>
			<organizationUrl>http://www.isis.vanderbilt.edu</organizationUrl>
			<roles>
				<role>developer</role>
			</roles>
			<timezone>-6</timezone>
		</developer>
	</developers>

	<!-- Properties availabe to the rest of pom file -->
	<properties>

		<!-- Project Properties -->
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

		<!-- Dependency Versions -->
		<appengine.app.version>1</appengine.app.version>
		<appengine.target.version>1.8.9</appengine.target.version>
		<spring.version>4.0.1.RELEASE</spring.version>
		<jackson.version>2.2.3</jackson.version>
		<servlet-api-version>2.5</servlet-api-version>
		<jstl-version>1.2</jstl-version>
		<junit-version>4.11</junit-version>
		<mockito-version>1.9.0</mockito-version>
		<jdo-api-version>3.0.1</jdo-api-version>
		<jsr107cache-version>1.1</jsr107cache-version>
		<datanucleus-appengine-version>2.1.2</datanucleus-appengine-version>
		<datanucleus-api-jdo-version>3.1.3</datanucleus-api-jdo-version>
		<commons-io-version>2.4</commons-io-version>
		<guava-version>16.0.1</guava-version>
		<guava-collections-version>r03</guava-collections-version>
		<guava-base-version>r03</guava-base-version>
		<guava-parent-version>16.0.1</guava-parent-version>
		<guice-version>3.0</guice-version>
		<guice-multibindings-version>3.0</guice-multibindings-version>
		<gmultipart-version>0.4</gmultipart-version>
		<joda-time-version>2.3</joda-time-version>
		<persistence-api-version>1.0.2</persistence-api-version>
		<datanucleus-jodatime-version>3.2.1</datanucleus-jodatime-version>

		<!-- Plugin Versions -->
		<maven-war-plugin-version>2.3</maven-war-plugin-version>
		<maven-datanucleus-plugin-version>3.1.3</maven-datanucleus-plugin-version>
		<maven-surefire-plugin-version>2.16</maven-surefire-plugin-version>
		<datanucleus-core-version>3.1.3</datanucleus-core-version>

		<!-- Plugin Management Versions -->
		<org-eclipse-m2e-lifecycle-mapping-version>1.0.0</org-eclipse-m2e-lifecycle-mapping-version>
		<maven-datanucleus-plugin-version>3.1.3</maven-datanucleus-plugin-version>
	</properties>

	<repositories>
		<repository>
			<id>gmultipart</id>
			<url>http://gmultipart.googlecode.com/svn/repo/m2</url>
		</repository>
	</repositories>

	<dependencies>
		<!-- Compile/runtime dependencies -->

		<dependency>
			<groupId>com.google.appengine</groupId>
			<artifactId>appengine-api-1.0-sdk</artifactId>
			<version>${appengine.target.version}</version>
		</dependency>

		<dependency>
			<groupId>javax.servlet</groupId>
			<artifactId>servlet-api</artifactId>
			<version>${servlet-api-version}</version>
			<scope>provided</scope>
		</dependency>

		<dependency>
			<groupId>jstl</groupId>
			<artifactId>jstl</artifactId>
			<version>${jstl-version}</version>
		</dependency>

		<!-- Test Dependencies -->

		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>${junit-version}</version>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.mockito</groupId>
			<artifactId>mockito-all</artifactId>
			<version>${mockito-version}</version>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-test</artifactId>
			<version>${spring.version}</version>
			<scope>test</scope>
		</dependency>

		<!-- App Engine stuff ( not yet used, but for future testing ) -->

		<dependency>
			<groupId>com.google.appengine</groupId>
			<artifactId>appengine-testing</artifactId>
			<version>${appengine.target.version}</version>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>com.google.appengine</groupId>
			<artifactId>appengine-api-stubs</artifactId>
			<version>${appengine.target.version}</version>
			<scope>test</scope>
		</dependency>

		<!-- Spring JSON stuff -->
		<!-- not 'directly' referenced, but used at runtime to make JSON objects -->

		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-core</artifactId>
			<version>${jackson.version}</version>
		</dependency>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-databind</artifactId>
			<version>${jackson.version}</version>
		</dependency>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-annotations</artifactId>
			<version>${jackson.version}</version>
		</dependency>

		<!-- datastore related -->

		<dependency>
			<groupId>javax.jdo</groupId>
			<artifactId>jdo-api</artifactId>
			<version>${jdo-api-version}</version>
		</dependency>

		<dependency>
			<groupId>net.sf.jsr107cache</groupId>
			<artifactId>jsr107cache</artifactId>
			<version>${jsr107cache-version}</version>
		</dependency>

		<dependency>
			<groupId>com.google.appengine.orm</groupId>
			<artifactId>datanucleus-appengine</artifactId>
			<version>${datanucleus-appengine-version}</version>
			<exclusions>
				<exclusion>
					<groupId>org.datanucleus</groupId>
					<artifactId>datanucleus-enhancer</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

		<dependency>
			<groupId>org.datanucleus</groupId>
			<artifactId>datanucleus-api-jdo</artifactId>
			<version>${datanucleus-api-jdo-version}</version>
		</dependency>

		<!-- Spring persistence related -->

		<dependency>
			<groupId>commons-io</groupId>
			<artifactId>commons-io</artifactId>
			<version>${commons-io-version}</version>
		</dependency>

		<dependency>
			<groupId>com.google.inject.extensions</groupId>
			<artifactId>guice-multibindings</artifactId>
			<version>${guice-multibindings-version}</version>
		</dependency>

		<!-- Spring Dependencies -->

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-aop</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-context</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-beans</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-context-support</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-expression</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-web</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-webmvc</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<!-- other misc dependencies -->

		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava</artifactId>
			<version>${guava-version}</version>
		</dependency>

		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava-collections</artifactId>
			<version>${guava-collections-version}</version>
		</dependency>

		<dependency>
			<groupId>com.google.inject</groupId>
			<artifactId>guice</artifactId>
			<version>${guice-version}</version>
		</dependency>

		<dependency>
			<groupId>gmultipart</groupId>
			<artifactId>gmultipart</artifactId>
			<version>${gmultipart-version}</version>
		</dependency>

		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava-base</artifactId>
			<version>${guava-base-version}</version>
		</dependency>

		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava-parent</artifactId>
			<version>${guava-parent-version}</version>
			<type>pom</type>
		</dependency>

		<dependency>
			<groupId>joda-time</groupId>
			<artifactId>joda-time</artifactId>
			<version>${joda-time-version}</version>
		</dependency>

		<dependency>
			<groupId>javax.persistence</groupId>
			<artifactId>persistence-api</artifactId>
			<version>${persistence-api-version}</version>
		</dependency>

		<dependency>
			<groupId>org.datanucleus</groupId>
			<artifactId>datanucleus-jodatime</artifactId>
			<version>${datanucleus-jodatime-version}</version>
		</dependency>

	</dependencies>

	<build>
		<outputDirectory>${project.build.directory}/${project.build.finalName}/WEB-INF/classes</outputDirectory>
		<plugins>

			<!-- maven compiler -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<version>2.5.1</version>
				<artifactId>maven-compiler-plugin</artifactId>
				<configuration>
					<source>1.7</source>
					<target>1.7</target>
				</configuration>
			</plugin>

			<!-- maven war(servlet) compiler -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-war-plugin</artifactId>
				<version>${maven-war-plugin-version}</version>
				<configuration>
					<archiveClasses>true</archiveClasses>
					<webResources>
						<!-- in order to interpolate version from pom into appengine-web.xml -->
						<resource>
							<directory>${basedir}/src/main/webapp/WEB-INF</directory>
							<filtering>true</filtering>
							<targetPath>WEB-INF</targetPath>
						</resource>
					</webResources>
				</configuration>
			</plugin>

			<!-- google app engine -->
			<plugin>
				<groupId>com.google.appengine</groupId>
				<artifactId>appengine-maven-plugin</artifactId>
				<version>${appengine.target.version}</version>
				<!-- <configuration> <jvmFlags> <jvmFlag>-Xdebug</jvmFlag> <jvmFlag>-Xrunjdwp:transport=dt_socket,address=1044,server=y,suspend=y
					</jvmFlag> </jvmFlags> <disableUpdateCheck>true</disableUpdateCheck> </configuration> -->
			</plugin>

			<!-- Enhance all classes found within a entities folder. -->
			<plugin>
				<groupId>org.datanucleus</groupId>
				<artifactId>maven-datanucleus-plugin</artifactId>
				<version>${maven-datanucleus-plugin-version}</version>
				<configuration>
					<fork>false</fork>
					<mappingIncludes>/com/walkernation/rest/domain/*.class</mappingIncludes>
					<verbose>true</verbose>
					<enhancerName>ASM</enhancerName>
					<api>JDO</api>
				</configuration>
				<dependencies>
					<dependency>
						<groupId>org.datanucleus</groupId>
						<artifactId>datanucleus-core</artifactId>
						<version>${datanucleus-core-version}</version>
					</dependency>
				</dependencies>
				<executions>
					<execution>
						<phase>compile</phase>
						<goals>
							<goal>enhance</goal>
						</goals>
					</execution>
				</executions>
			</plugin>

			<!-- Exclude Integration Tests during normal 'test' phase -->
			<!-- Otherwise application could never be built -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>${maven-surefire-plugin-version}</version>
				<configuration>
					<excludes>
						<exclude>**/*IntegrationTest.java</exclude>
					</excludes>
				</configuration>
				<executions>
					<execution>
						<id>integration-test</id>
						<goals>
							<goal>test</goal>
						</goals>
						<phase>integration-test</phase>
						<configuration>
							<excludes>
								<exclude>none</exclude>
							</excludes>
							<includes>
								<include>**/*IntegrationTest.java</include>
							</includes>
						</configuration>
					</execution>
				</executions>
			</plugin>

		</plugins>

		<!-- Configures plugins referenced from the plugins element in child projects. -->
		<!-- Might affect plugins in 'this' build, not 100% sure, so don't touch -->
		<pluginManagement>
			<plugins>

				<plugin>
					<groupId>org.eclipse.m2e</groupId>
					<artifactId>lifecycle-mapping</artifactId>
					<version>${org-eclipse-m2e-lifecycle-mapping-version}</version>
					<configuration>
						<lifecycleMappingMetadata>
							<pluginExecutions>
								<pluginExecution>
									<pluginExecutionFilter>
										<groupId>org.datanucleus</groupId>
										<artifactId>maven-datanucleus-plugin</artifactId>
										<versionRange>${maven-datanucleus-plugin-version}</versionRange>
										<goals>
											<goal>enhance</goal>
										</goals>
									</pluginExecutionFilter>
									<action>
										<execute>
											<runOnIncremental>false</runOnIncremental>
										</execute>
									</action>
								</pluginExecution>
							</pluginExecutions>
						</lifecycleMappingMetadata>
					</configuration>
				</plugin>

			</plugins>
		</pluginManagement>
	</build>
</project>


text:
/**
 * "First, solve the problem. Then, write the code. -John Johnson"
 * "Or use Vangav M"
 * www.vangav.com
 * */

/**
 * MIT License
 *
 * Copyright (c) 2016 Vangav
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to
 * deal in the Software without restriction, including without limitation the
 * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
 * sell copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
 * IN THE SOFTWARE.
 * */

/**
 * Community
 * Facebook Group: Vangav Open Source - Backend
 *   fb.com/groups/575834775932682/
 * Facebook Page: Vangav
 *   fb.com/vangav.f
 *
 * Third party communities for Vangav Backend
 *   - play framework
 *   - cassandra
 *   - datastax
 *
 * Tag your question online (e.g.: stack overflow, etc ...) with
 *   #vangav_backend
 *   to easier find questions/answers online
 * */

package com.vangav.backend.cassandra;

/**
 * @author mustapha
 * fb.com/mustapha.abdallah
 */
/**
 * CassandraNode represents a single Cassandra node
 * e.g.: in a multi-deployment topology with 4 Cassandra nodes,
 *         each of those 4 nodes will be represented by an Object of this class
 */
public class CassandraNode {

  private final String name;
  private final String ip;

  /**
   * Constructor CassandraNode
   * @param name
   * @param ip
   * @return new CassandraNode Object
   * @throws Exception
   */
  public CassandraNode (
    final String name,
    final String ip) throws Exception {

    this.name = name;
    this.ip = ip;
  }

  /**
   * getName
   * @return this node's name
   */
  public final String getName () {

    return this.name;
  }

  /**
   * getIp
   * @return this node's name
   */
  public final String getIp () {

    return this.ip;
  }

  @Override
  public String toString () {

    return
      "Cassandra node, name ["
      + this.name
      + "] ip ["
      + this.ip
      + "]";
  }
}


text:
/**
 * check if element is enabled
 *
 * @param {String}  selector  CSS selector of desired element
 * @param {Boolean} reverse   if true, function waits until element is disabled
 *
 * @see  waitForVisible
 */

/* global document,window */
module.exports = function(selector, reverse) {

    var cb = arguments[arguments.length - 1];
    var interval = setInterval(function() {

        /**
         * query element in every interval in case it gets inserted into
         * the DOM
         */
        var elements = document.querySelectorAll(selector);

        for (var i = 0; i < elements.length; ++i) {
            var elem = elements[i];

            if ((!reverse && typeof elem.disabled === 'boolean' && !elem.disabled) ||
                ( reverse && typeof elem.disabled === 'boolean' &&  elem.disabled)) {
                window.clearInterval(interval);
                return cb(true);
            }
        }
    }, 100);
};


text:
<?php

namespace Carnage\Cqrs\Event\Projection;

interface PostRebuildInterface
{
    public function postRebuild();
}


text:
//::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
/** @author  Matthew Saltz, John Miller
 *  @version 1.2
 *  @date    Thu Jul 25 11:28:31 EDT 2013
 *  @see     LICENSE (MIT style license file).
 */

package scalation.graphalytics

import collection.immutable.{Set => SET}

import scalation.util.time

//::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
/** This `GraphSimIso` object provides an implementation for Subgraph Isomorphism
 *  that uses an adjacency set version of Ullmann's Algorithm.
 *  @param g  the data graph  G(V, E, l)
 *  @param q  the query graph Q(U, D, k)
 */
class GraphSimIso (g: Graph, q: Graph)
      extends GraphMatcher (g, q)
{
    private val sims         = new GraphSim2 (g, q)    // object for Dual Simulation algorithm
    private var matches      = SET [Array [SET [Int]]] ()   // initialize matches to empty
    private var noBijections = true                    // no results yet

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /** Apply the Ullmann's Subgraph Isomorphism pattern matching algorithm to
     *  find the mappings from the query graph 'q' to the data graph 'g'.
     *  These are represented by a multi-valued function 'phi' that maps each
     *  query graph vertex 'u' to a set of data graph vertices '{v}'.
     */
    def mappings (): Array [SET [Int]] =
    {
        var psi: SET [Array [Int]] = null              // mappings from Dual Simulation
        if (noBijections) psi = bijections ()          // if no results, create them
        merge (psi)                                    // merge bijections to create mappings
    } // mappings

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /** Apply the Ullmann's Subgraph Isomorphism algorithm to find subgraphs of data
     *  graph 'g' that isomorphically match query graph 'q'.  These are represented
     *  by a set of single-valued bijective functions {'psi'} where each 'psi'
     *  function maps each query graph vertex 'u' to a data graph vertices 'v'.
     */
    override def bijections (): SET [Array [Int]] =
    {
        val phi = sims.feasibleMates ()                          // initial mappings from label match
        GraphSimIso (sims.saltzGraphSim (phi), makeOrder (), 0)  // recursively find all bijections
        val psi = simplify (matches)                             // pull bijections out matches
        noBijections = false                                     // results now available
        psi                                                      // return the set of bijections
    } // bijections

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /** Refine the mappings 'phi' using the Ullmann Subgraph Isomorphism algorithm.
     *  @param phi    array of mappings from a query vertex u_q to { graph vertices v_g }
     *  @param depth  the depth of recursion
     */
    private def GraphSimIso (phi: Array [SET [Int]], ordering: Array [Int], depth: Int)
    {
        if (depth == q.size) {
            if (! phi.isEmpty) {
                matches += phi
                if (matches.size % CHECK == 0) println ("GraphSimIso: matches so far = " + matches.size)
            } // if
        } else if (! phi.isEmpty) {
            for (i <- phi (ordering (depth)) if (! contains (phi, i, ordering,
              depth))) {
                val phiCopy = phi.map (x => x)                           // make a copy of phi
                phiCopy (ordering (depth)) = SET [Int] (i)               // isolate vertex i
                if (matches.size >= LIMIT) return                        // quit if at LIMIT
                // solve recursively for the next depth
                GraphSimIso (sims.saltzGraphSim (phiCopy), ordering, depth + 1)
            } // for
        } // if
    } // GraphSimIso

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /** Establish a vertex order from largest to smallest adjacency set ('ch') size.
     */
    def makeOrder (): Array [Int] = q.ch.zipWithIndex.sortBy (-1 * _._1.size).map (_._2)

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /** Is vertex j contained in any phi(i) for the previous depths?
     *  @param phi       array of mappings from a query vertex u_q to { graph vertices v_g }
     *  @param j         the vertex j to check
     *  @param ordering  the 'ch' size based ordering of the vertices
     *  @param depth     the current depth of recursion
     */
    private def contains (phi: Array [SET [Int]], j: Int, ordering: Array [Int], depth: Int): Boolean =
    {
        for (i <- 0 until depth) if (phi (ordering(i)) contains j) return true
        false
    } // contains

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /** Create an array to hold matches for each vertex 'u' in the query graph
     *  'q' and initialize it to contain all empty sets.  Then for each bijection,
     *  add each element of the bijection to its corresponding match set.
     *  @param psi  the set of bijections
     */
    private def merge (psi: SET [Array [Int]]): Array [SET [Int]] =
    {
        val matches = Array.ofDim [SET [Int]] (q.size).map (_ => SET [Int] ())
        for (b <- bijections; i <- b.indices) matches(i) += b(i)
        matches
    } // merge

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /** Pull the bijections out of the complete match set.
     *  @param matches  the complete match set embedding all bijections
     */
    private def simplify (matches: SET [Array [SET [Int]]]): SET [Array [Int]] =
    {
        matches.map (m => m.map (set => set.iterator.next))
    } // simplify

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /** Get the count of the number of matches.
     */
    def getMatches (): Int = matches.size

} // GraphSimIso class

//::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
/** The `GraphSimIsoTest` object is used to test the `GraphSimIso` class.
 */
object GraphSimIsoTest extends App
{
    val gSize     = 1000         // size of the data graph
    val qSize     =   10         // size of the query graph
    val nLabels   =  100         // number of distinct labels
    val gAvDegree =    5         // average vertex out degree for data graph
    val qAvDegree =    2         // average vertex out degree for query graph

    val g = GraphGen.genRandomGraph (gSize, nLabels, gAvDegree)
    val q = GraphGen.genBFSQuery (qSize, qAvDegree, g)

    val matcher = new GraphSimIso (g, q)                    // GraphSimIso Subgraph Isomorphism Pattern Matcher
    for (i <- 0 until 3) {
        val psi = time { matcher.bijections () }            // time the matcher
	println ("Number of Matches: " + matcher.getMatches)
	for (p <- psi) println (p.deep)
    } // for

} // GraphSimIsoTest