text:
<!DOCTYPE html>

    <meta content='Introduction - Django' property='og:title' />
    <title>Introduction - Django</title>
    <link rel="canonical" href="http://ngoctran.me/zebra//2016/08/31/introduction/" />
<meta charset="UTF-8" />

<meta name="viewport" content="width=device-width" />
<link rel="shortcut icon"
	href="http://ngoctran.me/zebra//images/favicon.jpg" />

<link rel='stylesheet' id='foundation-css'  href='http://ngoctran.me/zebra//stylesheets/foundation.css?ver=4.5.3' type='text/css' media='all' />
<link rel='stylesheet' id='aude-layout-css'  href='http://ngoctran.me/zebra//stylesheets/style.css' type='text/css' media='all' />
<link rel='stylesheet' id='foundation-css'  href='http://ngoctran.me/zebra//stylesheets/syntax.css' type='text/css' media='all' />

Django

A zebra does not change its spots

             Home

             Sample Page

    											<time class="dateline" datetime="August 31, 2016">August 31, 2016</time>

Introduction

Morbi venenatis enim sed dui elementum pellentesque. Sed sit amet ex scelerisque mi luctus vehicula. Nullam libero metus, consectetur in est eget, consectetur viverra dui.

Block quotes

Fusce blandit, ipsum et rutrum efficitur, ipsum nibh bibendum justo, a gravida enim dui ut urna. Nunc sit amet tellus in ex venenatis eleifend at eu velit. Cras tincidunt luctus magna, pellentesque imperdiet purus semper a. Donec ut nisi sem.

In eu volutpat odio, ac feugiat nulla. Nam non nisi dui. In pellentesque pretium bibendum. Nullam volutpat purus vel congue venenatis. In hac habitasse platea dictumst. Donec auctor dignissim justo, volutpat consectetur leo. Cras vitae diam nec dui gravida interdum. Donec aliquam in ante a auctor. Vivamus sagittis diam et diam lobortis, at convallis tortor tincidunt. Donec sit amet sagittis metus, et pharetra metus. Praesent semper eget risus vel ultrices.

Code listing

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer sem risus, lobortis sed elit quis, facilisis eleifend ex.

/**
 * The HelloWorldApp class implements an application that
 * simply displays "Hello World!" to the standard output.
 */
class HelloWorldApp {
    public static void main(String[] args) {
        System.out.println("Hello World!"); //Display the string.
    }
}

Lists

Aliquam leo metus, vulputate vel dignissim et, sagittis eget enim. Sed porttitor mi id diam facilisis, quis dapibus dolor pellentesque. In urna libero, efficitur ac pulvinar quis, semper nec nisi.

Praesent commodo cursus magna, vel scelerisque nisl consectetur et.

Donec id elit non mi porta gravida at eget metus.

Nulla vitae elit libero, a pharetra augue.

Images

Vestibulum in odio sit amet leo fringilla cursus. Pellentesque nec odio libero. Sed metus ante, congue non eros vitae, luctus mollis diam. Vivamus eu justo vel mauris auctor ultrices. Vivamus sed enim lobortis, finibus dui mollis, ultricies velit. In vitae elementum mauris, iaculis blandit nulla

    Sample 600 x 400 image

List

Phasellus erat lorem, imperdiet in ultricies sed, ultricies ac neque. Nulla blandit quam et turpis sollicitudin feugiat. Praesent id turpis ut diam facilisis elementum.

First ordered list item

Another item

Unordered sub-list.

Actual numbers don't matter, just that it's a number

Ordered sub-list

And another item.

The end.

Please enable JavaScript to view the comments powered by Disqus.

No Rights Reserved - Information is FREE

redpajama_set_name: RedPajamaGithub


text:
from base_regression import BasicRegression

import numpy as np

def sigmoid(s):
    return 1 / (1 + np.exp(-s))

def softmax(z):
    # Avoid numerical overflow by removing max
    # See: http://cs231n.github.io/linear-classify/
    e = np.exp(z - np.max(z))
    return e / np.sum(e)

class LogisticRegression(BasicRegression):
    """Binary logistic regression with gradient descent optimizer."""
    def _loss(self, theta):
        X, y, C = self.X, self.y, self.C

        predict = sigmoid(X.dot(theta))
        # prevent overflow or underflow
        predict = np.clip(predict, 1e-15, 1 - 1e-15)
        loss = - np.mean(np.sum(y * np.log(predict) + (1 - y) * np.log(1 - predict)))
        grad = (1.0 / self.n_samples) * (X.T.dot(predict - y))

        # Add regularization to the loss.
        loss += 0.5 * C * np.sum(theta * theta)
        grad += C * theta

        return loss, grad

    def _predict(self, X=None, threshold=0.5):
        """Predict class labels for samples in X."""
        X = self._add_intercept(X)
        predict = sigmoid(X.dot(self.theta))
        return np.where(predict > threshold , 1, 0)

class Softmax(BasicRegression):
    """Multi-class logistic regression with gradient descent optimizer."""
    def init_weights(self):
        """Mutli-class weights"""
        self.num_classes = np.max(self.y) + 1 # assume y takes values 0...K-1 where K is number of classes
        return np.random.normal(size=(self.n_features + 1, self.num_classes), scale=0.5)

    def _loss(self, theta):
        """
        Naive implementation for softmax loss due to its simplicity and readibility
        For vectorized version: https://github.com/transedward/cs231n/blob/master/assignment1/cs231n/classifiers/softmax.py#L62
        """
        X, y, C, n_samples, n_features = self.X, self.y, self.C, self.n_samples, self.n_features
        loss = 0.0
        grad = np.zeros_like(theta)
        for i in range(n_samples):
            scores = X[i].dot(theta)
            probability = softmax(scores)
            loss += -np.log(probability[y[i]])
            # gradient for softmax loss
            # http://ufldl.stanford.edu/wiki/index.php/Softmax_Regression
            for j in range(self.num_classes):
                grad[:, j] += (probability[j] - (j == y[i])) * X[i, :]

        # Right now the loss is a sum over all training examples, but we want it
        # to be an average instead so we divide by n_samples.
        loss /= n_samples
        grad /= n_samples

        # Add regularization to the loss.
        loss += 0.5 * C * np.sum(theta * theta)
        grad += C * theta

        return loss, grad

    def _predict(self, X=None):
        """Predict class labels for samples in X."""
        X = self._add_intercept(X)
        scores = X.dot(self.theta)
        predict = np.argmax(scores, axis=1)
        return predict


text:
#include "stdafx.h"
#include "http_response.h"

vds::http_response::http_response(
  http_message message)
{
  auto p = message.headers().begin();
  auto items = split_string(*p++, ' ', true, true);

  this->protocol_ = items.front();
  items.pop_front();

  this->code_ = atoi(items.front().c_str());
  items.pop_front();

  this->comment_ = items.front();
  items.pop_front();
  while(!items.empty()) {
    this->comment_ += ' ';
    this->comment_ += items.front();
    items.pop_front();
  }

  while (message.headers().end() != p) {
    this->headers_.push_back(*p++);
  }
}

vds::http_response::http_response(
  int code,
  const std::string & comment)
: protocol_("HTTP/1.0"), code_(code), comment_(comment)
{
}

vds::async_task<vds::expected<void>> vds::http_response::simple_text_response(
  const std::shared_ptr<http_async_serializer> & output_stream,
  const std::string & body,
  const std::string & content_type /*= "text/html; charset=utf-8"*/,
  int result_code /*= HTTP_OK*/,
  const std::string & message /*= "OK"*/)
{
  std::list<std::string> headers;
  headers.push_back("HTTP/1.0 " + std::to_string(result_code) + " " + message);
  if (!content_type.empty()) {
    headers.push_back("Content-Type:" + content_type);
  }
  headers.push_back("Content-Length:" + std::to_string(body.length()));

  GET_EXPECTED_ASYNC(stream, co_await output_stream->start_message(headers));
  CHECK_EXPECTED_ASYNC(co_await stream->write_async((const uint8_t *)body.c_str(), body.length()));
  CHECK_EXPECTED_ASYNC(co_await stream->write_async(nullptr, 0));

  co_return expected<void>();
}

vds::async_task<vds::expected<void>> vds::http_response::simple_text_response(
  const std::shared_ptr<http_async_serializer> & output_stream,
  const std::shared_ptr<stream_input_async<uint8_t>> & body,
  uint64_t body_size,
  const std::string & content_type,
  int result_code,
  const std::string& message) {

  std::list<std::string> headers;
  headers.push_back("HTTP/1.0 " + std::to_string(result_code) + " " + message);
  if (!content_type.empty()) {
    headers.push_back("Content-Type:" + content_type);
  }
  headers.push_back("Content-Length:" + std::to_string(body_size));

  GET_EXPECTED_ASYNC(stream, co_await output_stream->start_message(headers));
  CHECK_EXPECTED_ASYNC(co_await body->copy_to(stream));

  co_return expected<void>();
}

vds::async_task<vds::expected<void>> vds::http_response::redirect(
  const std::shared_ptr<http_async_serializer> & output_stream,
  const std::string& location) {
  std::list<std::string> headers;
  headers.push_back("HTTP/1.0 302 Found");
  headers.push_back("Location:" + location);

  GET_EXPECTED_ASYNC(stream, co_await output_stream->start_message(headers));
  CHECK_EXPECTED_ASYNC(co_await stream->write_async(nullptr, 0));

  co_return expected<void>();
}

vds::async_task<vds::expected<void>> vds::http_response::status_response(
  const std::shared_ptr<http_async_serializer> & output_stream,
  int result_code,
    const std::string & message)
{
  std::list<std::string> headers;
  headers.push_back("HTTP/1.0 " + std::to_string(result_code) + " " + message);

  GET_EXPECTED_ASYNC(stream, co_await output_stream->start_message(headers));
  CHECK_EXPECTED_ASYNC(co_await stream->write_async(nullptr, 0));

  co_return expected<void>();
}

vds::async_task<vds::expected<void>> vds::http_response::file_response(
  const std::shared_ptr<http_async_serializer> & output_stream,
  const filename & body_file,
  const std::string & out_filename,
  const std::string & content_type,
  int result_code /*= HTTP_OK*/,
    const std::string & message /*= "OK"*/){

  GET_EXPECTED_ASYNC(body_size, file::length(body_file));

  std::list<std::string> headers;
  headers.push_back("HTTP/1.0 " + std::to_string(result_code) + " " + message);
  headers.push_back("Content-Type:" + content_type);
  headers.push_back("Content-Length:" + std::to_string(body_size));
  headers.push_back("Content-Disposition:attachment; filename=\"" + out_filename + "\"");

  auto f = std::make_shared<file_stream_input_async>();
  CHECK_EXPECTED_ASYNC(f->open(body_file));

  GET_EXPECTED_ASYNC(stream, co_await output_stream->start_message(headers));
  CHECK_EXPECTED_ASYNC(co_await f->copy_to(stream));

  co_return expected<void>();
}

vds::async_task<vds::expected<void>> vds::http_response::file_response(
  const std::shared_ptr<http_async_serializer> & output_stream,
  const std::shared_ptr<stream_input_async<uint8_t>>& body,
  uint64_t body_size,
  const std::string & filename,
  const const_data_buffer & file_hash,
  const std::string & content_type,
  int result_code,
  const std::string & message)
{
  std::list<std::string> headers;
  headers.push_back("HTTP/1.0 " + std::to_string(result_code) + " " + message);
  headers.push_back("Content-Type:" + content_type);
  headers.push_back("Content-Length:" + std::to_string(body_size));
  headers.push_back("Content-Disposition:attachment; filename=\"" + filename + "\"");
  if(0 < file_hash.size()) {
    headers.push_back("X-VDS-SHA256:" + base64::from_bytes(file_hash));
  }

  GET_EXPECTED_ASYNC(stream, co_await output_stream->start_message(headers));
  CHECK_EXPECTED_ASYNC(co_await body->copy_to(stream));

  co_return expected<void>();
}

vds::async_task<vds::expected<void>> vds::http_response::file_response(
  const std::shared_ptr<http_async_serializer> & output_stream,
  const const_data_buffer& body,
  const std::string& filename,
  const const_data_buffer & file_hash,
  const std::string& content_type,
  int result_code,
  const std::string& message) {

  return file_response(
    output_stream,
    std::make_shared<buffer_stream_input_async>(body),
    body.size(),
    filename,
    file_hash,
    content_type,
    result_code,
    message);
}


text:
<?php
namespace Infomaniac\AMF;

interface ISerializable
{
    /**
     * Return an associative array of class properties
     *
     * @return array
     */
    public function export();

    /**
     * Import data from an external source into this class
     *
     * @param $data mixed
     */
    public function import($data);
}


text:
<link rel="import" href="../bower_components/polymer/polymer.html">
<link rel="import" href="../bower_components/iron-ajax/iron-ajax.html">
<link rel="import" href="../bower_components/google-apis/google-legacy-loader.html">

<dom-module id="chromedash-feature-timeline">
<link rel="import" type="css" href="../css/elements/chromedash-timeline.css">
<template>
  <google-legacy-loader on-api-load="_chartAPILoaded"></google-legacy-loader>
  <select value="{{selectedBucketId::change}}">
    <option disabled value="1">Select a property</option>
    <template is="dom-repeat" items="{{props}}" as="bucket">
      <option value="{{bucket.0}}">{{bucket.1}}</option>
    </template>
  </select>

</template>

</dom-module>


text:
import { SimpleSchema } from 'meteor/aldeed:simple-schema';
import {
    studyDesignOptions,
    exposureAssessmentTypeOptions,
    ratings,
    biasDirection,
    biasDirectionPopoverText,
    ratingRationalePopoverText,
} from './constants';

export default {
    // #1: General
    referenceID: {
        label: 'Reference',
        type: SimpleSchema.RegEx.Id,
    },
    additionalReferences: {
        label: 'References',
        type: [SimpleSchema.RegEx.Id],
        minCount: 0,
        popoverText: 'References of earlier updates or related publications, such as publications on exposure assessment or other relevant findings from the study population',
    },
    studyDesign: {
        label: 'Study design',
        allowedValues: studyDesignOptions,
        type: String,
        popoverText: 'Study-design used for evaluation (Cohort, Case-Control, Nested-Case Control, etc.)',
    },
    location: {
        label: 'Location',
        type: String,
        optional: true,
        popoverText: 'Country; other information (e.g., region, state, province, city) if important',
    },
    enrollmentDates: {
        label: 'Enrollment dates',
        type: String,
        optional: true,
        popoverText: 'Enrollment date for cohort or case-controls',
    },
    // #2: Population
    populationEligibility: {
        label: 'Eligibility criteria',
        type: String,
        textAreaRows: 3,
        popoverText: 'Describe the inclusion/exclusion criteria  (e.g., age, sex, length of employment) for selecting the cohort. Provide information on the subcohort (exposed) as well as the total cohort',
        optional: true,
    },
    cohortPopulationSize: {
        label: 'Population size',
        type: String,
        popoverText: 'Number enrolled for cohort studies which were included in the analysis after exclusions',
        optional: true,
    },
    referentGroup: {
        label: 'Referent group type',
        type: String,
        popoverText: 'i.e., external vs. internal analysis (or NR if not reported)',
        optional: true,
    },
    lossToFollowUp: {
        label: 'Loss to follow up',
        type: String,
        popoverText: 'The proportion of enrolled subjects whose status was unknown at the end of follow-up',
        optional: true,
    },
    smrAllCauses: {
        label: 'SMR/SIR all causes/cases',
        type: String,
        popoverText: '<ADD>',
        optional: true,
    },
    otherPopulationDescriptors: {
        label: 'Other population descriptors',
        type: String,
        optional: true,
        textAreaRows: 5,
        popoverText: 'Provide a short definition to describe the cohort or case-control study - Nurses heath study, type of workers/geographical location',
    },
    selectionBiasRating: {
        label: 'Selection bias rating',
        type: String,
        allowedValues: ratings,
        popoverText: ratingRationalePopoverText,
        biasField: true,
        biasSummary: 'Population',
    },
    selectionBiasDirection:{
        label: 'Bias direction',
        type: String,
        allowedValues: biasDirection,
        popoverText: biasDirectionPopoverText,
        biasField: true,
    },
    selectionBiasRationale: {
        label: 'Selection bias rationale',
        type: String,
        optional: true,
        textAreaRows: 2,
        popoverText: ratingRationalePopoverText,
        biasField: true,
    },
    selectionDescriptionCases: {
        label: 'Selection description',
        type: String,
        popoverText: 'Describe the inclusion/exclusion criteria for selecting the cases (such as age, dates sex) and the source population (such as geographical location, cancer registry)',
        optional: true,
    },
    selectionDescriptionControls: {
        label: 'Selection description',
        type: String,
        popoverText: 'Describe the inclusion/exclusion criteria for selecting selection referent (such as age, dates sex) and the source population (such as geographical location, cancer registry)',
        optional: true,
    },
    populationSizeCases: {
        label: 'Population size',
        type: String,
        popoverText: 'Number of cases/controls in case-control study which were included in the analysis after exclusions',
        optional: true,
    },
    populationSizeControls: {
        label: 'Population size',
        type: String,
        popoverText: 'Number of cases/controls in case-control study which were included in the analysis after exclusions',
        optional: true,
    },
    responseRateCases: {
        label: 'Response rate',
        type: String,
        popoverText: 'Percent of eligible participants included, NR if not-reported',
        optional: true,
    },
    responseRateControls: {
        label: 'Response rate',
        type: String,
        popoverText: 'Percent of eligible participants included, NR if not-reported',
        optional: true,
    },
    // #3: Exposure/outcome
    exposureAssessmentType: {
        label: 'Exposure assessment type',
        type: String,
        allowedValues: exposureAssessmentTypeOptions,
        popoverText: 'Which method was used to estimate agent-exposure?',
        optional: true,
    },
    exposureAssessmentNotes: {
        label: 'Exposure assessment notes',
        type: String,
        textAreaRows: 8,
        popoverText: 'Approach used to estimate exposure, if any (quantitative measurement, JEM questionnaire, expert judgment, biomonitoring, other)',
        optional: true,
    },
    exposureAssessmentRating: {
        label: 'Exposure assessment rating',
        type: String,
        allowedValues: ratings,
        popoverText: ratingRationalePopoverText,
        biasField: true,
        biasSummary: 'Exposure',
    },
    exposureAssessmentDirection:{
        label: 'Bias direction',
        type: String,
        allowedValues: biasDirection,
        popoverText: biasDirectionPopoverText,
        biasField: true,
    },
    exposureAssessmentRationale: {
        label: 'Exposure assessment rationale',
        type: String,
        textAreaRows: 5,
        popoverText: ratingRationalePopoverText,
        optional: true,
        biasField: true,
    },
    exposureMissingData: {
        label: 'Exposure missing data',
        type: String,
        optional: true,
        textAreaRows: 5,
        popoverText: 'Note any missing exposure data for exposure to the candidate substance exposure (includes number of workers sampled if personal sampling used).',
    },
    outcomeDataSource: {
        label: 'Outcome source & methods',
        type: String,
        optional: true,
        textAreaRows: 6,
        popoverText: 'Method of follow-up or source of outcome data (e.g. linkage to national cancer registry, searches of death certificates, insurance records, etc.). Also note methods used to diagnosis disease and classification systems for disease endpoints (such as ICD code version).',
    },
    outcomeMissingData: {
        label: 'Outcome missing data',
        type: String,
        optional: true,
        textAreaRows: 6,
        popoverText: 'Note any missing data on disease.',
    },
    outcomeAssessmentRating: {
        label: 'Outcome assessment rating',
        type: String,
        allowedValues: ratings,
        popoverText: ratingRationalePopoverText,
        biasField: true,
        biasSummary: 'Exposure',
    },
    outcomeAssessmentDirection:{
        label: 'Bias direction',
        type: String,
        allowedValues: biasDirection,
        popoverText: biasDirectionPopoverText,
        biasField: true,
    },
    outcomeAssessmentRationale: {
        label: 'Outcome assessment rationale',
        type: String,
        optional: true,
        textAreaRows: 3,
        popoverText: ratingRationalePopoverText,
        biasField: true,
    },
    confidenceInEvidence: {
        label: 'Confidence in evidence',
        type: String,
        optional: true,
        textAreaRows: 3,
        popoverText: 'For each outcome (type of cancer), please rate the confidence of the evidence (evidence, some evidence, null, inconclusive, other) based on guidance handbook and/or protocol; please include the cancer type in your description',
        biasField: false,
    },
    // #4: Analysis/reporting
    analyticalMethods: {
        label: 'Analytical methods',
        type: String,
        optional: true,
        textAreaRows: 7,
        popoverText: 'Summarizes the statistical methods used and analyses conducted including information such as the models used, internal/external analyses, exposure-response relationships and statistics to calculate trends',
    },
    analysisRating: {
        label: 'Analysis rating',
        type: String,
        allowedValues: ratings,
        popoverText: ratingRationalePopoverText,
        biasField: true,
        biasSummary: 'Analysis',
    },
    analysisDirection:{
        label: 'Bias direction',
        type: String,
        allowedValues: biasDirection,
        popoverText: biasDirectionPopoverText,
        biasField: true,
    },
    analysisRationale: {
        label: 'Analysis rationale',
        type: String,
        optional: true,
        textAreaRows: 4,
        popoverText: ratingRationalePopoverText,
        biasField: true,
    },
    selectiveReportingRating: {
        label: 'Selective reporting rating',
        type: String,
        allowedValues: ratings,
        popoverText: ratingRationalePopoverText,
        biasField: true,
        biasSummary: 'Analysis',
    },
    selectiveReportingDirection:{
        label: 'Bias direction',
        type: String,
        allowedValues: biasDirection,
        popoverText: biasDirectionPopoverText,
        biasField: true,
    },
    selectiveReportingRationale: {
        label: 'Selective reporting rationale',
        type: String,
        optional: true,
        textAreaRows: 4,
        popoverText: ratingRationalePopoverText,
        biasField: true,
    },
    // #5: Sensitivity
    exposureLevel: {
        label: 'Exposure level(s)',
        type: String,
        optional: true,
        popoverText: 'Includes any measurements of levels (area versus population) or biomonitoring data (including time period) of exposure   Also include information on exposure duration, range, etc.',
    },
    exposedCasesOrPower: {
        label: 'Exposed cases or power',
        type: String,
        optional: true,
        textAreaRows: 4,
        popoverText: 'Include information related to statistical power such as exposed cases, prevalence of exposure, etc.',
    },
    followUp: {
        label: 'Follow-up',
        type: String,
        popoverText: 'Include calendar time period, average or range of follow up (years or person years) if available.',
        optional: true,
    },
    sensitivityOther: {
        label: 'Other sensitivity notes',
        type: String,
        optional: true,
        textAreaRows: 4,
        popoverText: 'Include any other types of information (such as analyses) that may effect the sensitivity of the study to detect a true effect.  See RoC handbook/protocol for candidate substances.',
    },
    sensitivityRating: {
        label: 'Sensitivity rating',
        type: String,
        allowedValues: ratings,
        popoverText: ratingRationalePopoverText,
        biasField: true,
        biasSummary: 'Sensitivity',
    },
    sensitivityDirection:{
        label: 'Bias direction',
        type: String,
        allowedValues: biasDirection,
        popoverText: biasDirectionPopoverText,
        biasField: true,
    },
    sensitivityRatingRationale: {
        label: 'Sensitivity rating rationale',
        type: String,
        optional: true,
        textAreaRows: 4,
        popoverText: ratingRationalePopoverText,
        biasField: true,
    },
    // #6: Study judgment
    strengths: {
        label: 'Principal strengths',
        type: String,
        optional: true,
        popoverText: ratingRationalePopoverText + '; note major strengths.',
        textAreaRows: 4,
        biasField: true,
    },
    limitations: {
        label: 'Principal limitations',
        type: String,
        optional: true,
        popoverText: ratingRationalePopoverText + '; note major limitations.',
        textAreaRows: 4,
        biasField: true,
    },
    overallUtility: {
        label: 'Overall utility',
        type: String,
        allowedValues: ratings,
        popoverText: ratingRationalePopoverText,
        biasField: true,
        biasSummary: 'Study judgement',
    },
};


text:
WindowSelect::WindowSelect(float _left, float _top, float _width, float _height,
            std::function<void(int)>cb,
            std::function<void(int)>ch,
            std::vector<std::wstring> _items):Window(_left, _top, _width, _height){
    choose_callback = cb;
    change_callback = ch;
    items = _items;
    select_index = 0;
    this->Update();
    return;
}

WindowSelect::~WindowSelect(){
    return;
}

void WindowSelect::InputEvent(Input inp){
    bool up = false, down = false;
    if(inp.type == INPUT_NORMAL_KEY_DOWN){
        if(inp.normal_key == 27){
            choose_callback(-1);
            return;
        }else if(inp.normal_key == 13){
            AudioSystem::PlaySE("item_choose.ogg");
            choose_callback(select_index);
            this->Update();
            return;
        }
    }
    int dir = Input2Dir(inp);

    if(dir == Constant::DIR_UP and select_index > 0){
        select_index--;
        AudioSystem::PlaySE("item_switch.ogg");
        this->Update();
    }
    if(dir == Constant::DIR_DOWN and select_index+1 < items.size()){
        select_index++;
        AudioSystem::PlaySE("item_switch.ogg");
        this->Update();
    }
    return;
}

void WindowSelect::Update(){
    Clear();
    change_callback(select_index);
    for(int lx = 0;lx < items.size();lx++){
        DrawWText(0.1, 0.16*lx + 0.135, items[lx].c_str());
        if(lx == select_index)
            DrawBox(0.08, 0.16*lx + 0.03, 0.4, 0.14, {0.7, 0.3, 0.4, 0.5});
    }
    return;
}


text:
"""Train simple fastText-style classifier.

Inputs:
  words - text to classify
  ngrams - n char ngrams for each word in words
  labels - output classes to classify

Model:
  word embedding
  ngram embedding
  LogisticRegression classifier of embeddings to labels
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import inputs
import sys
import tensorflow as tf
from tensorflow.contrib.layers import feature_column
from tensorflow.contrib.learn.python.learn.estimators.run_config import RunConfig

tf.flags.DEFINE_string("train_records", None,
                       "Training file pattern for TFRecords, can use wildcards")
tf.flags.DEFINE_string("eval_records", None,
                       "Evaluation file pattern for TFRecords, can use wildcards")
tf.flags.DEFINE_string("predict_records", None,
                       "File pattern for TFRecords to predict, can use wildcards")
tf.flags.DEFINE_string("label_file", None, "File containing output labels")
tf.flags.DEFINE_integer("num_labels", None, "Number of output labels")
tf.flags.DEFINE_string("vocab_file", None, "Vocabulary file, one word per line")
tf.flags.DEFINE_integer("vocab_size", None, "Number of words in vocabulary")
tf.flags.DEFINE_integer("num_oov_vocab_buckets", 20,
                        "Number of hash buckets to use for OOV words")
tf.flags.DEFINE_string("model_dir", ".",
                       "Output directory for checkpoints and summaries")
tf.flags.DEFINE_string("export_dir", None, "Directory to store savedmodel")

tf.flags.DEFINE_integer("embedding_dimension", 10, "Dimension of word embedding")
tf.flags.DEFINE_boolean("use_ngrams", False, "Use character ngrams in embedding")
tf.flags.DEFINE_integer("num_ngram_buckets", 1000000,
                        "Number of hash buckets for ngrams")
tf.flags.DEFINE_integer("ngram_embedding_dimension", 10, "Dimension of word embedding")

tf.flags.DEFINE_float("learning_rate", 0.001, "Learning rate for training")
tf.flags.DEFINE_float("clip_gradient", 5.0, "Clip gradient norm to this ratio")
tf.flags.DEFINE_integer("batch_size", 128, "Training minibatch size")
tf.flags.DEFINE_integer("train_steps", 1000,
                        "Number of train steps, None for continuous")
tf.flags.DEFINE_integer("eval_steps", 100, "Number of eval steps")
tf.flags.DEFINE_integer("num_epochs", None, "Number of training data epochs")
tf.flags.DEFINE_integer("checkpoint_steps", 1000,
                        "Steps between saving checkpoints")
tf.flags.DEFINE_integer("num_threads", 1, "Number of reader threads")
tf.flags.DEFINE_boolean("log_device_placement", False, "log where ops are located")
tf.flags.DEFINE_boolean("horovod", False,
                        "Run across multiple GPUs using Horovod MPI. https://github.com/uber/horovod")
tf.flags.DEFINE_boolean("debug", False, "Debug")
FLAGS = tf.flags.FLAGS

if FLAGS.horovod:
    try:
        import horovod.tensorflow as hvd
    except ImportError, e:
        print(e)
        print("Make sure Horovod is installed: https://github.com/uber/horovod")
        sys.exit(1)
    hvd.init()

def InputFn(mode, input_file):
    return inputs.InputFn(
        mode, FLAGS.use_ngrams, input_file, FLAGS.vocab_file, FLAGS.vocab_size,
        FLAGS.embedding_dimension, FLAGS.num_oov_vocab_buckets,
        FLAGS.label_file, FLAGS.num_labels,
        FLAGS.ngram_embedding_dimension, FLAGS.num_ngram_buckets,
        FLAGS.batch_size, FLAGS.num_epochs, FLAGS.num_threads)

def Exports(probs, embedding):
    exports = {
        "proba": tf.estimator.export.ClassificationOutput(scores=probs),
        "embedding": tf.estimator.export.RegressionOutput(value=embedding),
        tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: \
            tf.estimator.export.ClassificationOutput(scores=probs),
    }
    return exports

def FastTextEstimator(model_dir, config=None):
    params = {
        "learning_rate": FLAGS.learning_rate,
    }
    def model_fn(features, labels, mode, params):
        features["text"] = tf.sparse_tensor_to_dense(features["text"],
                                                     default_value=" ")
        if FLAGS.use_ngrams:
            features["ngrams"] = tf.sparse_tensor_to_dense(features["ngrams"],
                                                           default_value=" ")
        text_lookup_table = tf.contrib.lookup.index_table_from_file(
            FLAGS.vocab_file, FLAGS.num_oov_vocab_buckets, FLAGS.vocab_size)
        text_ids = text_lookup_table.lookup(features["text"])
        text_embedding_w = tf.Variable(tf.random_uniform(
            [FLAGS.vocab_size + FLAGS.num_oov_vocab_buckets, FLAGS.embedding_dimension],
            -0.1, 0.1))
        text_embedding = tf.reduce_mean(tf.nn.embedding_lookup(
            text_embedding_w, text_ids), axis=-2)
        input_layer = text_embedding
        if FLAGS.use_ngrams:
            ngram_hash = tf.string_to_hash_bucket(features["ngrams"],
                                                  FLAGS.num_ngram_buckets)
            ngram_embedding_w = tf.Variable(tf.random_uniform(
                [FLAGS.num_ngram_buckets, FLAGS.ngram_embedding_dimension], -0.1, 0.1))
            ngram_embedding = tf.reduce_mean(tf.nn.embedding_lookup(
                ngram_embedding_w, ngram_hash), axis=-2)
            ngram_embedding = tf.expand_dims(ngram_embedding, -2)
            input_layer = tf.concat([text_embedding, ngram_embedding], -1)
        num_classes = FLAGS.num_labels
        logits = tf.contrib.layers.fully_connected(
            inputs=input_layer, num_outputs=num_classes,
            activation_fn=None)
        predictions = tf.argmax(logits, axis=-1)
        probs = tf.nn.softmax(logits)
        loss, train_op = None, None
        metrics = {}
        if mode != tf.estimator.ModeKeys.PREDICT:
            label_lookup_table = tf.contrib.lookup.index_table_from_file(
                FLAGS.label_file, vocab_size=FLAGS.num_labels)
            labels = label_lookup_table.lookup(labels)
            loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(
                labels=labels, logits=logits))
            opt = tf.train.AdamOptimizer(params["learning_rate"])
            if FLAGS.horovod:
                opt = hvd.DistributedOptimizer(opt)
            train_op = opt.minimize(loss, global_step=tf.train.get_global_step())
            metrics = {
                "accuracy": tf.metrics.accuracy(labels, predictions)
            }
        exports = {}
        if FLAGS.export_dir:
            exports = Exports(probs, text_embedding)
        return tf.estimator.EstimatorSpec(
            mode, predictions=predictions, loss=loss, train_op=train_op,
            eval_metric_ops=metrics, export_outputs=exports)
    session_config = tf.ConfigProto(
        log_device_placement=FLAGS.log_device_placement)
    if FLAGS.horovod:
        session_config.gpu_options.visible_device_list = str(hvd.local_rank())
    config = tf.contrib.learn.RunConfig(
        save_checkpoints_secs=None,
        save_checkpoints_steps=FLAGS.checkpoint_steps,
        session_config=session_config)
    return tf.estimator.Estimator(model_fn=model_fn, model_dir=model_dir,
                                  params=params, config=config)

def FastTrain():
    print("FastTrain", FLAGS.train_steps)
    estimator = FastTextEstimator(FLAGS.model_dir)
    print("TEST" + FLAGS.train_records)
    train_input = InputFn(tf.estimator.ModeKeys.TRAIN, FLAGS.train_records)
    print("STARTING TRAIN")
    hooks = None
    if FLAGS.horovod:
        hooks = [hvd.BroadcastGlobalVariablesHook(0)]
    estimator.train(input_fn=train_input, steps=FLAGS.train_steps, hooks=hooks)
    print("TRAIN COMPLETE")
    if not FLAGS.horovod or hvd.rank() == 0:
        print("EVALUATE")
        eval_input = InputFn(tf.estimator.ModeKeys.EVAL, FLAGS.eval_records)
        #eval_metrics = { "accuracy": tf.metrics.accuracy(labels, predictions) }
        result = estimator.evaluate(input_fn=eval_input, steps=FLAGS.eval_steps, hooks=None)
        print(result)
        print("DONE")
        if FLAGS.export_dir:
            print("EXPORTING")
            estimator.export_savedmodel(FLAGS.export_dir,
                                        inputs.ServingInputFn(FLAGS.use_ngrams))

def main(_):
    if not FLAGS.vocab_size:
        FLAGS.vocab_size = len(open(FLAGS.vocab_file).readlines())
    if not FLAGS.num_labels:
        FLAGS.num_labels = len(open(FLAGS.label_file).readlines())
    if FLAGS.horovod:
        nproc = hvd.size()
        total = FLAGS.train_steps
        FLAGS.train_steps = total / nproc
        print("Running %d steps on each of %d processes for %d total" % (
            FLAGS.train_steps, nproc, total))
    FastTrain()

if __name__ == '__main__':
    if FLAGS.debug:
        tf.logging.set_verbosity(tf.logging.DEBUG)
    tf.app.run()


text:
import copy

from requests_mock.contrib import fixture

from openstackclient.object.v1 import container as container_cmds
from openstackclient.tests.object.v1 import fakes as object_fakes

class TestContainerAll(object_fakes.TestObjectv1):

    def setUp(self):
        super(TestContainerAll, self).setUp()

        self.requests_mock = self.useFixture(fixture.Fixture())

class TestContainerCreate(TestContainerAll):

    columns = (
        'account',
        'container',
        'x-trans-id',
    )

    def setUp(self):
        super(TestContainerCreate, self).setUp()

        # Get the command object to test
        self.cmd = container_cmds.CreateContainer(self.app, None)

    def test_object_create_container_single(self):
        self.requests_mock.register_uri(
            'PUT',
            object_fakes.ENDPOINT + '/ernie',
            headers={'x-trans-id': '314159'},
            status_code=200,
        )

        arglist = [
            'ernie',
        ]
        verifylist = [(
            'containers', ['ernie'],
        )]
        parsed_args = self.check_parser(self.cmd, arglist, verifylist)

        # In base command class ShowOne in cliff, abstract method take_action()
        # returns a two-part tuple with a tuple of column names and a tuple of
        # data to be shown.
        columns, data = self.cmd.take_action(parsed_args)

        self.assertEqual(self.columns, columns)
        datalist = [(
            object_fakes.ACCOUNT_ID,
            'ernie',
            '314159',
        )]
        self.assertEqual(datalist, list(data))

    def test_object_create_container_more(self):
        self.requests_mock.register_uri(
            'PUT',
            object_fakes.ENDPOINT + '/ernie',
            headers={'x-trans-id': '314159'},
            status_code=200,
        )
        self.requests_mock.register_uri(
            'PUT',
            object_fakes.ENDPOINT + '/bert',
            headers={'x-trans-id': '42'},
            status_code=200,
        )

        arglist = [
            'ernie',
            'bert',
        ]
        verifylist = [(
            'containers', ['ernie', 'bert'],
        )]
        parsed_args = self.check_parser(self.cmd, arglist, verifylist)

        # In base command class ShowOne in cliff, abstract method take_action()
        # returns a two-part tuple with a tuple of column names and a tuple of
        # data to be shown.
        columns, data = self.cmd.take_action(parsed_args)

        self.assertEqual(self.columns, columns)
        datalist = [
            (
                object_fakes.ACCOUNT_ID,
                'ernie',
                '314159',
            ),
            (
                object_fakes.ACCOUNT_ID,
                'bert',
                '42',
            ),
        ]
        self.assertEqual(datalist, list(data))

class TestContainerDelete(TestContainerAll):

    def setUp(self):
        super(TestContainerDelete, self).setUp()

        # Get the command object to test
        self.cmd = container_cmds.DeleteContainer(self.app, None)

    def test_object_delete_container_single(self):
        self.requests_mock.register_uri(
            'DELETE',
            object_fakes.ENDPOINT + '/ernie',
            status_code=200,
        )

        arglist = [
            'ernie',
        ]
        verifylist = [(
            'containers', ['ernie'],
        )]
        parsed_args = self.check_parser(self.cmd, arglist, verifylist)

        # Command.take_action() returns None
        ret = self.cmd.take_action(parsed_args)
        self.assertIsNone(ret)

    def test_object_delete_container_more(self):
        self.requests_mock.register_uri(
            'DELETE',
            object_fakes.ENDPOINT + '/ernie',
            status_code=200,
        )
        self.requests_mock.register_uri(
            'DELETE',
            object_fakes.ENDPOINT + '/bert',
            status_code=200,
        )

        arglist = [
            'ernie',
            'bert',
        ]
        verifylist = [(
            'containers', ['ernie', 'bert'],
        )]
        parsed_args = self.check_parser(self.cmd, arglist, verifylist)

        # Command.take_action() returns None
        ret = self.cmd.take_action(parsed_args)
        self.assertIsNone(ret)

class TestContainerList(TestContainerAll):

    columns = ('Name',)

    def setUp(self):
        super(TestContainerList, self).setUp()

        # Get the command object to test
        self.cmd = container_cmds.ListContainer(self.app, None)

    def test_object_list_containers_no_options(self):
        return_body = [
            copy.deepcopy(object_fakes.CONTAINER),
            copy.deepcopy(object_fakes.CONTAINER_3),
            copy.deepcopy(object_fakes.CONTAINER_2),
        ]
        self.requests_mock.register_uri(
            'GET',
            object_fakes.ENDPOINT + '?format=json',
            json=return_body,
            status_code=200,
        )

        arglist = []
        verifylist = []
        parsed_args = self.check_parser(self.cmd, arglist, verifylist)

        # Lister.take_action() returns two tuples
        columns, data = self.cmd.take_action(parsed_args)

        self.assertEqual(self.columns, columns)
        datalist = [
            (object_fakes.container_name, ),
            (object_fakes.container_name_3, ),
            (object_fakes.container_name_2, ),
        ]
        self.assertEqual(datalist, list(data))

    def test_object_list_containers_prefix(self):
        return_body = [
            copy.deepcopy(object_fakes.CONTAINER),
            copy.deepcopy(object_fakes.CONTAINER_3),
        ]
        self.requests_mock.register_uri(
            'GET',
            object_fakes.ENDPOINT + '?format=json&prefix=bit',
            json=return_body,
            status_code=200,
        )

        arglist = [
            '--prefix', 'bit',
        ]
        verifylist = [
            ('prefix', 'bit'),
        ]
        parsed_args = self.check_parser(self.cmd, arglist, verifylist)

        # Lister.take_action() returns two tuples
        columns, data = self.cmd.take_action(parsed_args)

        self.assertEqual(self.columns, columns)
        datalist = [
            (object_fakes.container_name, ),
            (object_fakes.container_name_3, ),
        ]
        self.assertEqual(datalist, list(data))

class TestContainerSave(TestContainerAll):

    def setUp(self):
        super(TestContainerSave, self).setUp()

        # Get the command object to test
        self.cmd = container_cmds.SaveContainer(self.app, None)

# TODO(dtroyer): need to mock out object_lib.save_object() to test this
#     def test_object_save_container(self):
#         return_body = [
#             copy.deepcopy(object_fakes.OBJECT),
#             copy.deepcopy(object_fakes.OBJECT_2),
#         ]
#         # Initial container list request
#         self.requests_mock.register_uri(
#             'GET',
#             object_fakes.ENDPOINT + '/oscar?format=json',
#             json=return_body,
#             status_code=200,
#         )
#         # Individual object save requests
#         self.requests_mock.register_uri(
#             'GET',
#             object_fakes.ENDPOINT + '/oscar/' + object_fakes.object_name_1,
#             json=object_fakes.OBJECT,
#             status_code=200,
#         )
#         self.requests_mock.register_uri(
#             'GET',
#             object_fakes.ENDPOINT + '/oscar/' + object_fakes.object_name_2,
#             json=object_fakes.OBJECT_2,
#             status_code=200,
#         )
#
#         arglist = [
#             'oscar',
#         ]
#         verifylist = [(
#             'container', 'oscar',
#         )]
#         parsed_args = self.check_parser(self.cmd, arglist, verifylist)
#
#         # Command.take_action() returns None
#         ret = self.cmd.take_action(parsed_args)
#         self.assertIsNone(ret)

class TestContainerShow(TestContainerAll):

    def setUp(self):
        super(TestContainerShow, self).setUp()

        # Get the command object to test
        self.cmd = container_cmds.ShowContainer(self.app, None)

    def test_object_show_container(self):
        headers = {
            'x-container-object-count': '42',
            'x-container-bytes-used': '123',
            'x-container-read': 'qaz',
            'x-container-write': 'wsx',
            'x-container-sync-to': 'edc',
            'x-container-sync-key': 'rfv',
        }
        self.requests_mock.register_uri(
            'HEAD',
            object_fakes.ENDPOINT + '/ernie',
            headers=headers,
            status_code=200,
        )

        arglist = [
            'ernie',
        ]
        verifylist = [(
            'container', 'ernie',
        )]
        parsed_args = self.check_parser(self.cmd, arglist, verifylist)

        # In base command class ShowOne in cliff, abstract method take_action()
        # returns a two-part tuple with a tuple of column names and a tuple of
        # data to be shown.
        columns, data = self.cmd.take_action(parsed_args)

        collist = (
            'account',
            'bytes_used',
            'container',
            'object_count',
            'read_acl',
            'sync_key',
            'sync_to',
            'write_acl',
        )
        self.assertEqual(collist, columns)
        datalist = [
            object_fakes.ACCOUNT_ID,
            '123',
            'ernie',
            '42',
            'qaz',
            'rfv',
            'edc',
            'wsx',
        ]
        self.assertEqual(datalist, list(data))


text:
kalman
====

Kalman filtering and smoothing implemented in Rust.

## License
See [here](LICENSE).


text:
#ifndef MSGPACK_PREDEF_COMPILER_METAWARE_H
#define MSGPACK_PREDEF_COMPILER_METAWARE_H

#include <msgpack/predef/version_number.h>
#include <msgpack/predef/make.h>

/*`
[heading `MSGPACK_COMP_HIGHC`]

MetaWare High C/C++ compiler.

[table
    [[__predef_symbol__] [__predef_version__]]

    [[`__HIGHC__`] [__predef_detection__]]
    ]
 */

#define MSGPACK_COMP_HIGHC MSGPACK_VERSION_NUMBER_NOT_AVAILABLE

#if defined(__HIGHC__)
#   define MSGPACK_COMP_HIGHC_DETECTION MSGPACK_VERSION_NUMBER_AVAILABLE
#endif

#ifdef MSGPACK_COMP_HIGHC_DETECTION
#   if defined(MSGPACK_PREDEF_DETAIL_COMP_DETECTED)
#       define MSGPACK_COMP_HIGHC_EMULATED MSGPACK_COMP_HIGHC_DETECTION
#   else
#       undef MSGPACK_COMP_HIGHC
#       define MSGPACK_COMP_HIGHC MSGPACK_COMP_HIGHC_DETECTION
#   endif
#   define MSGPACK_COMP_HIGHC_AVAILABLE
#   include <msgpack/predef/detail/comp_detected.h>
#endif

#define MSGPACK_COMP_HIGHC_NAME "MetaWare High C/C++"

#include <msgpack/predef/detail/test.h>
MSGPACK_PREDEF_DECLARE_TEST(MSGPACK_COMP_HIGHC,MSGPACK_COMP_HIGHC_NAME)

#ifdef MSGPACK_COMP_HIGHC_EMULATED
#include <msgpack/predef/detail/test.h>
MSGPACK_PREDEF_DECLARE_TEST(MSGPACK_COMP_HIGHC_EMULATED,MSGPACK_COMP_HIGHC_NAME)
#endif

#endif


text:
require 'yaml'
require 'logger'
require 'optparse'
require 'memcached'
require 'cgi'
require 'lib/lamed/helper'
require 'lib/lamed/object_loader'
require 'lib/lamed/aws/aws'
require 'lib/lamed/aws/authentication'
require 'lib/lamed/aws/sqs'

module Lamed

  # Build configurations from yaml files
  if defined?(ROOT)
    OPTIONS = {
      :env      => (ENV['RACK_ENV'] || "development").to_sym,
      :config   => ROOT + '/conf/',
      :log_path => ROOT + '/logs',
      :verbose  => false
    }

    if File.exists?(File.join(ROOT,'/conf/config.yml'))
      SYS_OPTIONS = YAML.load_file(OPTIONS[:config] + "config.yml")[OPTIONS[:env].to_s].inject({}) {|h,(k,v)| h[k.intern] = v; h}
    end
    if File.exists?(ROOT + '/conf/database.yml')
    DB_OPTIONS = YAML.load_file(OPTIONS[:config] + "database.yml")[OPTIONS[:env].to_s].inject({}) {|h,(k,v)| h[k.intern] = v; h}
    end
    if defined?(SYS_OPTIONS)
      @sys_options = SYS_OPTIONS
      log_path =@sys_options[:logs] || OPTIONS[:log_path]
      OPTIONS[:rotate] = !@sys_options.nil? ? @sys_options[:rotate] : nil
      @log_path = File.join(log_path, OPTIONS[:env].to_s  + ".log")
    end
  end

  class << self
    # Set up logging
    def initialize_logger
      if defined?(SYS_OPTIONS)
        begin
          logs = Logger.new(@log_path, OPTIONS[:rotate])
          case OPTIONS[:env]
          when :development
            logs.level = Logger::DEBUG
          when :test
            logs.level = Logger::DEBUG
          when :staging
            logs.level = Logger::DEBUG
          when :production
            logs.level = Logger::WARN
          end
        rescue
          logs = nil
        end
      end
      if logs.nil?
        puts "The log path #{@log_path} does not exist or is not writable."
        puts "Log messages will be sent to STANDARD ERROR"
        logs = Logger.new(STDERR)
      end
      @logger = logs
      logs.warn("Logging level is set to #{@logger.level}")
      return @logger
    end

    if defined?(ROOT)
      require 'lib/lamed/model'
    end

    def logger
      @logger
    end

    def opts
      @opts = SYS_OPTIONS
    end

    def load_lib
      Dir[ROOT + '/lib/**/*.rb'].each {|f| load f}
    end

    def load_controller
      ObjectLoader.load_controller_object
    end

    def load_model
     ObjectLoader.load_model_object
    end
  end

  if defined?(SYS_OPTIONS)
    load_model
    require 'lib/lamed/controller'
    load_controller
    load_lib
  end
  initialize_logger

end


text:
using base::HistogramBase;
using base::HistogramSamples;
using base::StatisticsRecorder;
using chrome::spellcheck_common::WordList;
using chrome::spellcheck_common::WordSet;

namespace {

// Get all sync data for the custom dictionary without limiting to maximum
// number of syncable words.
syncer::SyncDataList GetAllSyncDataNoLimit(
    const SpellcheckCustomDictionary* dictionary) {
  syncer::SyncDataList data;
  std::string word;
  const WordSet& words = dictionary->GetWords();
  for (WordSet::const_iterator it = words.begin(); it != words.end(); ++it) {
    word = *it;
    sync_pb::EntitySpecifics specifics;
    specifics.mutable_dictionary()->set_word(word);
    data.push_back(syncer::SyncData::CreateLocalData(word, word, specifics));
  }
  return data;
}

}  // namespace

static BrowserContextKeyedService* BuildSpellcheckService(
    content::BrowserContext* profile) {
  return new SpellcheckService(static_cast<Profile*>(profile));
}

class SpellcheckCustomDictionaryTest : public testing::Test {
 protected:
  virtual void SetUp() OVERRIDE {
    // Use SetTestingFactoryAndUse to force creation and initialization.
    SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
        &profile_, &BuildSpellcheckService);

    StatisticsRecorder::Initialize();
  }

  // A wrapper around SpellcheckCustomDictionary::LoadDictionaryFile private
  // function to avoid a large number of FRIEND_TEST declarations in
  // SpellcheckCustomDictionary.
  chrome::spellcheck_common::WordList LoadDictionaryFile(
      const base::FilePath& path) {
    return SpellcheckCustomDictionary::LoadDictionaryFile(path);
  }

  // A wrapper around SpellcheckCustomDictionary::UpdateDictionaryFile private
  // function to avoid a large number of FRIEND_TEST declarations in
  // SpellcheckCustomDictionary.
  void UpdateDictionaryFile(
      const SpellcheckCustomDictionary::Change& dictionary_change,
      const base::FilePath& path) {
    SpellcheckCustomDictionary::UpdateDictionaryFile(dictionary_change, path);
  }

  // A wrapper around SpellcheckCustomDictionary::OnLoaded private method to
  // avoid a large number of FRIEND_TEST declarations in
  // SpellcheckCustomDictionary.
  void OnLoaded(
      SpellcheckCustomDictionary& dictionary,
      const chrome::spellcheck_common::WordList& custom_words) {
    dictionary.OnLoaded(custom_words);
  }

  // A wrapper around SpellcheckCustomDictionary::Apply private method to avoid
  // a large number of FRIEND_TEST declarations in SpellcheckCustomDictionary.
  void Apply(
      SpellcheckCustomDictionary& dictionary,
      const SpellcheckCustomDictionary::Change& change) {
    return dictionary.Apply(change);
  }

  content::TestBrowserThreadBundle thread_bundle_;

  TestingProfile profile_;
  net::TestURLFetcherFactory fetcher_factory_;
};

// A wrapper around SpellcheckCustomDictionary that does not own the wrapped
// object. An instance of this class can be inside of a scoped pointer safely
// while the dictionary is managed by another scoped pointer.
class SyncChangeProcessorDelegate : public syncer::SyncChangeProcessor {
 public:
  explicit SyncChangeProcessorDelegate(SpellcheckCustomDictionary* dictionary)
      : dictionary_(dictionary) {}
  virtual ~SyncChangeProcessorDelegate() {}

  // Overridden from syncer::SyncChangeProcessor:
  virtual syncer::SyncError ProcessSyncChanges(
      const tracked_objects::Location& from_here,
      const syncer::SyncChangeList& change_list) OVERRIDE {
    return dictionary_->ProcessSyncChanges(from_here, change_list);
  }

  virtual syncer::SyncDataList GetAllSyncData(syncer::ModelType type) const
      OVERRIDE {
    return syncer::SyncDataList();
  }

 private:
  SpellcheckCustomDictionary* dictionary_;
  DISALLOW_COPY_AND_ASSIGN(SyncChangeProcessorDelegate);
};

// An implementation of SyncErrorFactory that does not upload the error message
// and updates an outside error counter. This lets us know the number of error
// messages in an instance of this class after that instance is deleted.
class SyncErrorFactoryStub : public syncer::SyncErrorFactory {
 public:
  explicit SyncErrorFactoryStub(int* error_counter)
      : error_counter_(error_counter) {}
  virtual ~SyncErrorFactoryStub() {}

  // Overridden from syncer::SyncErrorFactory:
  virtual syncer::SyncError CreateAndUploadError(
      const tracked_objects::Location& location,
      const std::string& message) OVERRIDE {
    (*error_counter_)++;
    return syncer::SyncError(location,
                             syncer::SyncError::DATATYPE_ERROR,
                             message,
                             syncer::DICTIONARY);
  }

 private:
  int* error_counter_;
  DISALLOW_COPY_AND_ASSIGN(SyncErrorFactoryStub);
};

// Counts the number of notifications for dictionary load and change.
class DictionaryObserverCounter : public SpellcheckCustomDictionary::Observer {
 public:
  DictionaryObserverCounter() : loads_(0), changes_(0) {}
  virtual ~DictionaryObserverCounter() {}

  int loads() const { return loads_; }
  int changes() const { return changes_; }

  // Overridden from SpellcheckCustomDictionary::Observer:
  virtual void OnCustomDictionaryLoaded() OVERRIDE { loads_++; }
  virtual void OnCustomDictionaryChanged(
      const SpellcheckCustomDictionary::Change& change) OVERRIDE { changes_++; }

 private:
  int loads_;
  int changes_;
  DISALLOW_COPY_AND_ASSIGN(DictionaryObserverCounter);
};

TEST_F(SpellcheckCustomDictionaryTest, SaveAndLoad) {
  base::FilePath path =
      profile_.GetPath().Append(chrome::kCustomDictionaryFileName);
  WordList loaded_custom_words = LoadDictionaryFile(path);

  // The custom word list should be empty now.
  WordList expected;
  EXPECT_EQ(expected, loaded_custom_words);

  SpellcheckCustomDictionary::Change change;
  change.AddWord("bar");
  change.AddWord("foo");

  UpdateDictionaryFile(change, path);
  expected.push_back("bar");
  expected.push_back("foo");

  // The custom word list should include written words.
  loaded_custom_words = LoadDictionaryFile(path);
  EXPECT_EQ(expected, loaded_custom_words);

  change = SpellcheckCustomDictionary::Change();
  change.RemoveWord("bar");
  change.RemoveWord("foo");
  UpdateDictionaryFile(change, path);
  loaded_custom_words = LoadDictionaryFile(path);
  expected.clear();
  EXPECT_EQ(expected, loaded_custom_words);
}

TEST_F(SpellcheckCustomDictionaryTest, MultiProfile) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  WordSet expected1;
  WordSet expected2;

  custom_dictionary->AddWord("foo");
  custom_dictionary->AddWord("bar");
  expected1.insert("foo");
  expected1.insert("bar");

  custom_dictionary2->AddWord("hoge");
  custom_dictionary2->AddWord("fuga");
  expected2.insert("hoge");
  expected2.insert("fuga");

  WordSet actual1 = custom_dictionary->GetWords();
  EXPECT_EQ(actual1, expected1);

  WordSet actual2 = custom_dictionary2->GetWords();
  EXPECT_EQ(actual2, expected2);
}

// Legacy empty dictionary should be converted to new format empty dictionary.
TEST_F(SpellcheckCustomDictionaryTest, LegacyEmptyDictionaryShouldBeConverted) {
  base::FilePath path =
      profile_.GetPath().Append(chrome::kCustomDictionaryFileName);

  std::string content;
  file_util::WriteFile(path, content.c_str(), content.length());
  WordList loaded_custom_words = LoadDictionaryFile(path);
  EXPECT_TRUE(loaded_custom_words.empty());
}

// Legacy dictionary with two words should be converted to new format dictionary
// with two words.
TEST_F(SpellcheckCustomDictionaryTest,
       LegacyDictionaryWithTwoWordsShouldBeConverted) {
  base::FilePath path =
      profile_.GetPath().Append(chrome::kCustomDictionaryFileName);

  std::string content = "foo\nbar\nfoo\n";
  file_util::WriteFile(path, content.c_str(), content.length());
  WordList loaded_custom_words = LoadDictionaryFile(path);
  WordList expected;
  expected.push_back("bar");
  expected.push_back("foo");
  EXPECT_EQ(expected, loaded_custom_words);
}

// Illegal words should be removed. Leading and trailing whitespace should be
// trimmed.
TEST_F(SpellcheckCustomDictionaryTest,
       IllegalWordsShouldBeRemovedFromDictionary) {
  base::FilePath path =
      profile_.GetPath().Append(chrome::kCustomDictionaryFileName);

  std::string content = "foo\n foo bar \n\n \nbar\n"
      "01234567890123456789012345678901234567890123456789"
      "01234567890123456789012345678901234567890123456789";
  file_util::WriteFile(path, content.c_str(), content.length());
  WordList loaded_custom_words = LoadDictionaryFile(path);
  WordList expected;
  expected.push_back("bar");
  expected.push_back("foo");
  expected.push_back("foo bar");
  EXPECT_EQ(expected, loaded_custom_words);
}

// Write to dictionary should backup previous version and write the word to the
// end of the dictionary. If the dictionary file is corrupted on disk, the
// previous version should be reloaded.
TEST_F(SpellcheckCustomDictionaryTest, CorruptedWriteShouldBeRecovered) {
  base::FilePath path =
      profile_.GetPath().Append(chrome::kCustomDictionaryFileName);

  std::string content = "foo\nbar";
  file_util::WriteFile(path, content.c_str(), content.length());
  WordList loaded_custom_words = LoadDictionaryFile(path);
  WordList expected;
  expected.push_back("bar");
  expected.push_back("foo");
  EXPECT_EQ(expected, loaded_custom_words);

  SpellcheckCustomDictionary::Change change;
  change.AddWord("baz");
  UpdateDictionaryFile(change, path);
  content.clear();
  base::ReadFileToString(path, &content);
  content.append("corruption");
  file_util::WriteFile(path, content.c_str(), content.length());
  loaded_custom_words = LoadDictionaryFile(path);
  EXPECT_EQ(expected, loaded_custom_words);
}

TEST_F(SpellcheckCustomDictionaryTest,
       GetAllSyncDataAccuratelyReflectsDictionaryState) {
  SpellcheckCustomDictionary* dictionary =
      SpellcheckServiceFactory::GetForContext(
          &profile_)->GetCustomDictionary();

  syncer::SyncDataList data = dictionary->GetAllSyncData(syncer::DICTIONARY);
  EXPECT_TRUE(data.empty());

  EXPECT_TRUE(dictionary->AddWord("bar"));
  EXPECT_TRUE(dictionary->AddWord("foo"));

  data = dictionary->GetAllSyncData(syncer::DICTIONARY);
  EXPECT_EQ(2UL, data.size());
  std::vector<std::string> words;
  words.push_back("bar");
  words.push_back("foo");
  for (size_t i = 0; i < data.size(); i++) {
    EXPECT_TRUE(data[i].GetSpecifics().has_dictionary());
    EXPECT_EQ(syncer::DICTIONARY, data[i].GetDataType());
    EXPECT_EQ(words[i], data[i].GetTag());
    EXPECT_EQ(words[i], data[i].GetSpecifics().dictionary().word());
  }

  EXPECT_TRUE(dictionary->RemoveWord("bar"));
  EXPECT_TRUE(dictionary->RemoveWord("foo"));

  data = dictionary->GetAllSyncData(syncer::DICTIONARY);
  EXPECT_TRUE(data.empty());
}

TEST_F(SpellcheckCustomDictionaryTest, GetAllSyncDataHasLimit) {
  SpellcheckCustomDictionary* dictionary =
      SpellcheckServiceFactory::GetForContext(
          &profile_)->GetCustomDictionary();

  SpellcheckCustomDictionary::Change change;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS - 1;
       i++) {
    change.AddWord("foo" + base::Uint64ToString(i));
  }
  Apply(*dictionary, change);
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS - 1,
            dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS - 1,
            dictionary->GetAllSyncData(syncer::DICTIONARY).size());

  dictionary->AddWord("baz");
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            dictionary->GetAllSyncData(syncer::DICTIONARY).size());

  dictionary->AddWord("bar");
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            dictionary->GetAllSyncData(syncer::DICTIONARY).size());

  dictionary->AddWord("snafoo");
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 2,
            dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            dictionary->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, ProcessSyncChanges) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* dictionary =
      spellcheck_service->GetCustomDictionary();

  dictionary->AddWord("foo");
  dictionary->AddWord("bar");

  syncer::SyncChangeList changes;
  {
    // Add existing word.
    std::string word = "foo";
    sync_pb::EntitySpecifics specifics;
    specifics.mutable_dictionary()->set_word(word);
    changes.push_back(syncer::SyncChange(
        FROM_HERE,
        syncer::SyncChange::ACTION_ADD,
        syncer::SyncData::CreateLocalData(word, word, specifics)));
  }
  {
    // Add invalid word. This word is too long.
    std::string word = "01234567890123456789012345678901234567890123456789"
        "01234567890123456789012345678901234567890123456789";
    sync_pb::EntitySpecifics specifics;
    specifics.mutable_dictionary()->set_word(word);
    changes.push_back(syncer::SyncChange(
        FROM_HERE,
        syncer::SyncChange::ACTION_ADD,
        syncer::SyncData::CreateLocalData(word, word, specifics)));
  }
  {
    // Add valid word.
    std::string word = "baz";
    sync_pb::EntitySpecifics specifics;
    specifics.mutable_dictionary()->set_word(word);
    changes.push_back(syncer::SyncChange(
        FROM_HERE,
        syncer::SyncChange::ACTION_ADD,
        syncer::SyncData::CreateLocalData(word, word, specifics)));
  }
  {
    // Remove missing word.
    std::string word = "snafoo";
    sync_pb::EntitySpecifics specifics;
    specifics.mutable_dictionary()->set_word(word);
    changes.push_back(syncer::SyncChange(
        FROM_HERE,
        syncer::SyncChange::ACTION_DELETE,
        syncer::SyncData::CreateLocalData(word, word, specifics)));
  }
  {
    // Remove existing word.
    std::string word = "bar";
    sync_pb::EntitySpecifics specifics;
    specifics.mutable_dictionary()->set_word(word);
    changes.push_back(syncer::SyncChange(
        FROM_HERE,
        syncer::SyncChange::ACTION_DELETE,
        syncer::SyncData::CreateLocalData(word, word, specifics)));
  }

  EXPECT_FALSE(dictionary->ProcessSyncChanges(FROM_HERE, changes).IsSet());

  const WordSet& words = dictionary->GetWords();
  EXPECT_EQ(2UL, words.size());
  EXPECT_EQ(0UL, words.count("bar"));
  EXPECT_EQ(1UL, words.count("foo"));
  EXPECT_EQ(1UL, words.count("baz"));
}

TEST_F(SpellcheckCustomDictionaryTest, MergeDataAndStartSyncing) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  SpellcheckCustomDictionary::Change change;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS / 2;
       ++i) {
    change.AddWord("foo" + base::Uint64ToString(i));
  }
  Apply(*custom_dictionary, change);

  SpellcheckCustomDictionary::Change change2;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS / 2;
       ++i) {
    change2.AddWord("bar" + base::Uint64ToString(i));
  }
  Apply(*custom_dictionary2, change2);

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  WordSet words = custom_dictionary->GetWords();
  WordSet words2 = custom_dictionary2->GetWords();
  EXPECT_EQ(words.size(), words2.size());
  EXPECT_EQ(words, words2);
}

TEST_F(SpellcheckCustomDictionaryTest, DictionaryTooBigBeforeSyncing) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  SpellcheckCustomDictionary::Change change;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1;
       ++i) {
    change.AddWord("foo" + base::Uint64ToString(i));
  }
  Apply(*custom_dictionary, change);

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_FALSE(custom_dictionary->IsSyncing());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetWords().size());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary->GetAllSyncData(syncer::DICTIONARY).size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, DictionaryTooBigAndServerFull) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  SpellcheckCustomDictionary::Change change;
  SpellcheckCustomDictionary::Change change2;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS;
       ++i) {
    change.AddWord("foo" + base::Uint64ToString(i));
    change2.AddWord("bar" + base::Uint64ToString(i));
  }
  change.AddWord("foo");
  Apply(*custom_dictionary, change);
  Apply(*custom_dictionary2, change2);

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetWords().size());

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_FALSE(custom_dictionary->IsSyncing());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS * 2 + 1,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetWords().size());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary->GetAllSyncData(syncer::DICTIONARY).size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, ServerTooBig) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  SpellcheckCustomDictionary::Change change;
  SpellcheckCustomDictionary::Change change2;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1;
       ++i) {
    change.AddWord("foo" + base::Uint64ToString(i));
    change2.AddWord("bar" + base::Uint64ToString(i));
  }
  Apply(*custom_dictionary, change);
  Apply(*custom_dictionary2, change2);

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            custom_dictionary2->GetWords().size());

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      GetAllSyncDataNoLimit(custom_dictionary2),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_FALSE(custom_dictionary->IsSyncing());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS * 2 + 2,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            custom_dictionary2->GetWords().size());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary->GetAllSyncData(syncer::DICTIONARY).size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, DictionaryTooBigToStartSyncing) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  SpellcheckCustomDictionary::Change change;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS - 1;
       ++i) {
    change.AddWord("foo" + base::Uint64ToString(i));
  }
  Apply(*custom_dictionary, change);

  custom_dictionary2->AddWord("bar");
  custom_dictionary2->AddWord("baz");

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_FALSE(custom_dictionary->IsSyncing());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetWords().size());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary->GetAllSyncData(syncer::DICTIONARY).size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, DictionaryTooBigToContiueSyncing) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  SpellcheckCustomDictionary::Change change;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS - 1;
       ++i) {
    change.AddWord("foo" + base::Uint64ToString(i));
  }
  Apply(*custom_dictionary, change);

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  custom_dictionary->AddWord("bar");
  EXPECT_EQ(0, error_counter);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  custom_dictionary->AddWord("baz");
  EXPECT_EQ(0, error_counter);
  EXPECT_FALSE(custom_dictionary->IsSyncing());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetWords().size());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary->GetAllSyncData(syncer::DICTIONARY).size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, LoadAfterSyncStart) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  custom_dictionary->AddWord("foo");

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  WordList custom_words;
  custom_words.push_back("bar");
  OnLoaded(*custom_dictionary, custom_words);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  EXPECT_EQ(2UL, custom_dictionary->GetWords().size());
  EXPECT_EQ(2UL, custom_dictionary2->GetWords().size());

  EXPECT_EQ(2UL, custom_dictionary->GetAllSyncData(syncer::DICTIONARY).size());
  EXPECT_EQ(2UL, custom_dictionary2->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, LoadAfterSyncStartTooBigToSync) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  custom_dictionary->AddWord("foo");

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  WordList custom_words;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS;
       ++i) {
    custom_words.push_back("foo" + base::Uint64ToString(i));
  }
  OnLoaded(*custom_dictionary, custom_words);
  EXPECT_EQ(0, error_counter);
  EXPECT_FALSE(custom_dictionary->IsSyncing());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS + 1,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetWords().size());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary->GetAllSyncData(syncer::DICTIONARY).size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            custom_dictionary2->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, LoadDuplicatesAfterSync) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  WordList to_add;
  for (size_t i = 0;
       i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS / 2;
       ++i) {
    to_add.push_back("foo" + base::Uint64ToString(i));
  }
  Apply(*custom_dictionary, SpellcheckCustomDictionary::Change(to_add));

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  OnLoaded(*custom_dictionary, to_add);
  EXPECT_EQ(0, error_counter);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS / 2,
            custom_dictionary->GetWords().size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS / 2,
            custom_dictionary2->GetWords().size());

  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS / 2,
            custom_dictionary->GetAllSyncData(syncer::DICTIONARY).size());
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS / 2,
            custom_dictionary2->GetAllSyncData(syncer::DICTIONARY).size());
}

TEST_F(SpellcheckCustomDictionaryTest, DictionaryLoadNotification) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();

  DictionaryObserverCounter observer;
  custom_dictionary->AddObserver(&observer);

  WordList custom_words;
  custom_words.push_back("foo");
  custom_words.push_back("bar");
  OnLoaded(*custom_dictionary, custom_words);

  EXPECT_GE(observer.loads(), 1);
  EXPECT_LE(observer.loads(), 2);
  EXPECT_EQ(0, observer.changes());

  custom_dictionary->RemoveObserver(&observer);
}

TEST_F(SpellcheckCustomDictionaryTest, DictionaryAddWordNotification) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();

  OnLoaded(*custom_dictionary, WordList());

  DictionaryObserverCounter observer;
  custom_dictionary->AddObserver(&observer);

  EXPECT_TRUE(custom_dictionary->AddWord("foo"));
  EXPECT_TRUE(custom_dictionary->AddWord("bar"));
  EXPECT_FALSE(custom_dictionary->AddWord("bar"));

  EXPECT_EQ(2, observer.changes());

  custom_dictionary->RemoveObserver(&observer);
}

TEST_F(SpellcheckCustomDictionaryTest, DictionaryRemoveWordNotification) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();

  OnLoaded(*custom_dictionary, WordList());

  EXPECT_TRUE(custom_dictionary->AddWord("foo"));
  EXPECT_TRUE(custom_dictionary->AddWord("bar"));

  DictionaryObserverCounter observer;
  custom_dictionary->AddObserver(&observer);

  EXPECT_TRUE(custom_dictionary->RemoveWord("foo"));
  EXPECT_TRUE(custom_dictionary->RemoveWord("bar"));
  EXPECT_FALSE(custom_dictionary->RemoveWord("baz"));

  EXPECT_EQ(2, observer.changes());

  custom_dictionary->RemoveObserver(&observer);
}

TEST_F(SpellcheckCustomDictionaryTest, DictionarySyncNotification) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  TestingProfile profile2;
  SpellcheckService* spellcheck_service2 =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &profile2, &BuildSpellcheckService));
  SpellcheckCustomDictionary* custom_dictionary2 =
      spellcheck_service2->GetCustomDictionary();

  OnLoaded(*custom_dictionary, WordList());
  OnLoaded(*custom_dictionary2, WordList());

  custom_dictionary->AddWord("foo");
  custom_dictionary->AddWord("bar");
  custom_dictionary2->AddWord("foo");
  custom_dictionary2->AddWord("baz");

  DictionaryObserverCounter observer;
  custom_dictionary->AddObserver(&observer);

  DictionaryObserverCounter observer2;
  custom_dictionary2->AddObserver(&observer2);

  int error_counter = 0;
  EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
      syncer::DICTIONARY,
      custom_dictionary2->GetAllSyncData(syncer::DICTIONARY),
      scoped_ptr<syncer::SyncChangeProcessor>(
          new SyncChangeProcessorDelegate(custom_dictionary2)),
      scoped_ptr<syncer::SyncErrorFactory>(
          new SyncErrorFactoryStub(&error_counter))).error().IsSet());
  EXPECT_EQ(0, error_counter);
  EXPECT_TRUE(custom_dictionary->IsSyncing());

  EXPECT_EQ(1, observer.changes());
  EXPECT_EQ(1, observer2.changes());

  custom_dictionary->RemoveObserver(&observer);
  custom_dictionary2->RemoveObserver(&observer2);
}

// The server has maximum number of words and the client has maximum number of
// different words before association time. No new words should be pushed to the
// sync server upon association. The client should accept words from the sync
// server, however.
TEST_F(SpellcheckCustomDictionaryTest, DictionarySyncLimit) {
  TestingProfile server_profile;
  SpellcheckService* server_spellcheck_service =
      static_cast<SpellcheckService*>(
          SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
              &server_profile, &BuildSpellcheckService));

  // Here, |server_custom_dictionary| plays the role of the sync server.
  SpellcheckCustomDictionary* server_custom_dictionary =
      server_spellcheck_service->GetCustomDictionary();

  // Upload the maximum number of words to the sync server.
  {
    SpellcheckService* spellcheck_service =
        SpellcheckServiceFactory::GetForContext(&profile_);
    SpellcheckCustomDictionary* custom_dictionary =
        spellcheck_service->GetCustomDictionary();

    SpellcheckCustomDictionary::Change change;
    for (size_t i = 0;
         i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS;
         ++i) {
      change.AddWord("foo" + base::Uint64ToString(i));
    }
    Apply(*custom_dictionary, change);

    int error_counter = 0;
    EXPECT_FALSE(custom_dictionary->MergeDataAndStartSyncing(
        syncer::DICTIONARY,
        server_custom_dictionary->GetAllSyncData(syncer::DICTIONARY),
        scoped_ptr<syncer::SyncChangeProcessor>(
            new SyncChangeProcessorDelegate(server_custom_dictionary)),
        scoped_ptr<syncer::SyncErrorFactory>(
            new SyncErrorFactoryStub(&error_counter))).error().IsSet());
    EXPECT_EQ(0, error_counter);
    EXPECT_TRUE(custom_dictionary->IsSyncing());
    EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
              custom_dictionary->GetWords().size());
  }

  // The sync server now has the maximum number of words.
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            server_custom_dictionary->GetWords().size());

  // Associate the sync server with a client that also has the maximum number of
  // words, but all of these words are different from the ones on the sync
  // server.
  {
    TestingProfile client_profile;
    SpellcheckService* client_spellcheck_service =
        static_cast<SpellcheckService*>(
            SpellcheckServiceFactory::GetInstance()->SetTestingFactoryAndUse(
                &client_profile, &BuildSpellcheckService));

    // Here, |client_custom_dictionary| plays the role of the client.
    SpellcheckCustomDictionary* client_custom_dictionary =
        client_spellcheck_service->GetCustomDictionary();

    // Add the maximum number of words to the client. These words are all
    // different from those on the server.
    SpellcheckCustomDictionary::Change change;
    for (size_t i = 0;
         i < chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS;
         ++i) {
      change.AddWord("bar" + base::Uint64ToString(i));
    }
    Apply(*client_custom_dictionary, change);

    // Associate the server and the client.
    int error_counter = 0;
    EXPECT_FALSE(client_custom_dictionary->MergeDataAndStartSyncing(
        syncer::DICTIONARY,
        server_custom_dictionary->GetAllSyncData(syncer::DICTIONARY),
        scoped_ptr<syncer::SyncChangeProcessor>(
            new SyncChangeProcessorDelegate(server_custom_dictionary)),
        scoped_ptr<syncer::SyncErrorFactory>(
            new SyncErrorFactoryStub(&error_counter))).error().IsSet());
    EXPECT_EQ(0, error_counter);
    EXPECT_FALSE(client_custom_dictionary->IsSyncing());
    EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS * 2,
              client_custom_dictionary->GetWords().size());
  }

  // The sync server should not receive more words, because it has the maximum
  // number of words already.
  EXPECT_EQ(chrome::spellcheck_common::MAX_SYNCABLE_DICTIONARY_WORDS,
            server_custom_dictionary->GetWords().size());
}

TEST_F(SpellcheckCustomDictionaryTest, RecordSizeStatsCorrectly) {
#if defined(OS_WIN)
// Failing consistently on Win7. See crbug.com/230534.
  if (base::win::GetVersion() >= base::win::VERSION_VISTA)
    return;
#endif
  // Record a baseline.
  SpellCheckHostMetrics::RecordCustomWordCountStats(123);

  // Determine if test failures are due the statistics recorder not being
  // available or because the histogram just isn't there: crbug.com/230534.
  EXPECT_TRUE(StatisticsRecorder::IsActive());

  HistogramBase* histogram =
      StatisticsRecorder::FindHistogram("SpellCheck.CustomWords");
  ASSERT_TRUE(histogram != NULL);
  scoped_ptr<HistogramSamples> baseline = histogram->SnapshotSamples();

  // Load the dictionary which should be empty.
  base::FilePath path =
      profile_.GetPath().Append(chrome::kCustomDictionaryFileName);
  WordList loaded_custom_words = LoadDictionaryFile(path);
  EXPECT_EQ(0u, loaded_custom_words.size());

  // We expect there to be an entry with 0.
  histogram =
      StatisticsRecorder::FindHistogram("SpellCheck.CustomWords");
  ASSERT_TRUE(histogram != NULL);
  scoped_ptr<HistogramSamples> samples = histogram->SnapshotSamples();

  samples->Subtract(*baseline);
  EXPECT_EQ(0,samples->sum());

  SpellcheckCustomDictionary::Change change;
  change.AddWord("bar");
  change.AddWord("foo");
  UpdateDictionaryFile(change, path);

  // Load the dictionary again and it should have 2 entries.
  loaded_custom_words = LoadDictionaryFile(path);
  EXPECT_EQ(2u, loaded_custom_words.size());

  histogram =
      StatisticsRecorder::FindHistogram("SpellCheck.CustomWords");
  ASSERT_TRUE(histogram != NULL);
  scoped_ptr<HistogramSamples> samples2 = histogram->SnapshotSamples();

  samples2->Subtract(*baseline);
  EXPECT_EQ(2,samples2->sum());
}

TEST_F(SpellcheckCustomDictionaryTest, HasWord) {
  SpellcheckService* spellcheck_service =
      SpellcheckServiceFactory::GetForContext(&profile_);
  SpellcheckCustomDictionary* custom_dictionary =
      spellcheck_service->GetCustomDictionary();
  OnLoaded(*custom_dictionary, WordList());
  EXPECT_FALSE(custom_dictionary->HasWord("foo"));
  EXPECT_FALSE(custom_dictionary->HasWord("bar"));
  custom_dictionary->AddWord("foo");
  EXPECT_TRUE(custom_dictionary->HasWord("foo"));
  EXPECT_FALSE(custom_dictionary->HasWord("bar"));
}