text:
layout: flat
title: STIX Relationships
---

Relationships are one of the features of STIX that make it so powerful for expressing cyber threat intelligence (CTI) when compared to other options.  Although each of the major components is valuable on its own and can be used independently of the others in order to express those concepts, the true power of STIX is realized when components are used in conjunction with each other to better enable contextual understanding for threat analysis. It's one thing to use STIX to describe an indicator for an IP address without context, it's another to relate that to the relevant TTP that it indicates, the TTP to the threat actor or actors that are known to use it, to the incidents where it was observed, and to courses of action that can help mitigate its impact.

STIX relationships enable all of this by defining these connection points and how to express them:

<img src="/images/stix-architecture.png" alt="STIX relationship diagram" />

## Types of Relationships

It should be noted that not all relationships in STIX are about simply conveying that two completely independent things are associated to each other. In real world CTI, relationships between the concepts represented by the STIX major components actually take on one of two flavors: association or composition.

* **Association** ("related to") relationships relate two independent components of CTI content (e.g. ThreatActor_A and TTP_B) that are associated with each other due to some contextual detail or shared affinity. Association relationships should be considered as bidirectional and there is no hard dependency on either of the components associated (i.e. conveying either one would not require inclusion of the association relationship).
* **Composition** ("includes as part of") relationships relate two components of CTI content (e.g. Indicator_X and Observable_Y) that are related to each other by the fact that one can be considered an inherent property of the other. This does not mean that the "property" component is owned and controlled by the enclosing component or that it cannot be relevant on its own or as part of other content (through referencing). It simply means that the enclosing component would be contextually incomplete without consideration of the "property" component as part of it. Composition relationships should be considered bidirectional but imply an inclusion dependency for the enclosing component (i.e. for contextual completeness if you convey Indicator_X you would also need to include its related Observable_Y).

## Specifying Relationships

In STIX, relationships of both flavors are specified from within the primary component of the relationship (for composition relationships this is the enclosing component). This is in order to lend clarity to composition relationships, to encourage specification of particular relationships that are semantically important for the context of a component, and to localize information to where it is most relevant. In other words, if you convey a component, you have conveyed its full context and don't need to chase down and convey a bunch of related content it is dependent on.

When a relationship is specified, the related component can typically be defined either embedded inline within the primary component or via a reference to a definition elsewhere. It is important to note that a relationship's nature (Association vs Composition) can typically be considered orthagonal to and independent of its specification approach (referencing vs embedding). Most association relationships (e.g. a Threat_Actor with a related TTP) can be specified either via embedding (e.g. defining the TTP inline within the Threat_Actor/Observed_TTPs structure) or via referencing. The same can be said of composition relationships. There are a small number of relationships in STIX (e.g. Related_Campaigns within Indicators) that are exceptions to this rule and can only be specified via reference due to their pure association nature.

While a relationship's nature (Association vs Composition) is inherent in the form of information it represents, its specification approach is a choice to be made with varying tradeoffs. For more information on relevant suggested practices see [Suggested Practices: Referencing vs. Embedding](../../suggested-practices).

### Mechanisms for Specifying Relationships

Almost all relationships in STIX are implemented using a similar structure to ensure consistency and ease of implementation. The structure allows
for representation of the relationship itself (of course), a label to further describe the semantics of the relationship, a confidence in
the assertion of a relationship, and an information source for the relationship assertion. These fields mean that not only can STIX
components be related together, they can be related together in a semantically meaningful way that preserves the confidence and provenance
of the relationship assertion.

### Data Model

In XML, a basic STIX relationship structure looks like this (the relationship is "Indicated_TTP", which goes from an indicator to a TTP):

{% highlight xml %}
<indicator:Indicator>
	<indicator:Indicated_TTPs>
		<indicator:Indicated_TTP>
			<stixCommon:Confidence />
			<stixCommon:Information_Source/>
			<stixCommon:Relationship />
			<stixCommon:TTP />
		</indicator:Indicated_TTP>
	</indicator:Indicated_TTPs>
</indicator:Indicator>
{% endhighlight %}

The `Confidence` field uses the STIX confidence mechanism to express confidence in the relationship assertion. For example, if the producer is not certain that a particular TTP is used by a Threat Actor they could use "Low" confidence to denote that.

The `Information Source` field, using [InformationSourceType](/data-model/{{site.current_version}}/stixCommon/InformationSourceType), is used to characterize provenance information about the relationship assertion. It can be used to indicate who, when, and how (what tools) the relationship assertion was made.

The `Relationship` field uses specifies a semantic label for what kind of relationship is being asserted. This value can be ad-hoc or reference a value from a STIX controlled vocabulary. Although no default vocabulary has currently been defined, the STIX community is currently soliciting input on potential vocabularies.

Finally, the `TTP` field (contextually specific to an Indicated_TTP relationship) contains either a reference to or a full representation of the related component. This assumes it is a full relationship type. If it is only a reference type then only a reference to the related component would be supported. The field name will always be the name of the component that the relationship is pointing to, which in this case is TTP.

If using that field as a reference, the `idref` field is used to point to the `id` of the construct that the relationship is to. Optionally, the `timestamp` field can also be used to create a reference to a specific version of the construct.

For example:

{% include start_tabs.html tabs="XML|Python" name="ref" %}{% highlight xml linenos %}
<stix:Indicators>
  <stix:Indicator id="example:indicator-8837a4b4-b682-11e3-b0f3-0800271e87d2" xsi:type='indicator:IndicatorType' timestamp="2014-03-31T00:00:00.000000Z">
    <indicator:Indicated_TTP>
      <stixCommon:TTP idref="example:ttp-883730f6-b682-11e3-b0f3-0800271e87d2" />
    </indicator:Indicated_TTP>
  </stix:Indicator>
</stix:Indicators>
<stix:TTPs>
  <stix:TTP id="example:ttp-883730f6-b682-11e3-b0f3-0800271e87d2" xsi:type='ttp:TTPType' timestamp="2014-03-31T00:00:00.000000Z"/>
</stix:TTPs>
{% endhighlight %}{% include tab_separator.html %}{% highlight python linenos %}
from stix.core import STIXPackage
from stix.indicator import Indicator
from stix.ttp import TTP

stix_package = STIXPackage()
ttp = TTP()

indicator = Indicator()
indicator.add_indicated_ttp(TTP(idref=ttp.id_))

stix_package.add_indicator(indicator)
stix_package.add_ttp(ttp)

print stix_package.to_xml()
{% endhighlight %}{% include end_tabs.html %}

In the example above, a reference is created by setting the `idref` of the relationship TTP to the `id` of the TTP related itself. Note that while the TTP definition requires the `xsi:type`, the TTP reference does not because it only uses the `idref` field. It could also have added a `timestamp` set to exactly "2014-03-31T00:00:00.000000Z" to indicate that the relationship applies to that specific version of the TTP.

Assuming that you're using a full relationship structure, you can also choose to embed the related component:

{% include start_tabs.html tabs="XML|Python" name="embed" %}{% highlight xml linenos %}
<stix:Indicators>
  <stix:Indicator id="example:indicator-3cdff264-b682-11e3-ab3e-0800271e87d2" timestamp="2014-03-31T00:00:00.000000Z" xsi:type="indicator:IndicatorType">
    <indicator:Indicated_TTP>
      <stixCommon:TTP id="example:ttp-3cdf6c54-b682-11e3-ab3e-0800271e87d2" timestamp="2014-03-31T00:00:00.000000Z" xsi:type="ttp:TTPType">
        <!-- SNIP -->
      </stixCommon:TTP>
    </indicator:Indicated_TTP>
  </stix:Indicator>
</stix:Indicators>
{% endhighlight %}{% include tab_separator.html %}{% highlight python linenos %}
from stix.core import STIXPackage
from stix.indicator import Indicator
from stix.ttp import TTP

stix_package = STIXPackage()
ttp = TTP()

indicator = Indicator()
indicator.add_indicated_ttp(ttp)

stix_package.add_indicator(indicator)
print stix_package.to_xml()
{% endhighlight %}{% include end_tabs.html %}

In this case the TTP is not defined separately at the top level, it is included in full inside the Indicated TTP. When embedding another component it does require using the `xsi:type` (because the full TTP model is being used) and allows (but does not require) the use of `id` and `timestamp` to give the construct and ID.

### Examples

#### Minimal Embed

{% include start_tabs.html tabs="XML|Python" name="min-embed" %}{% highlight xml linenos %}
<stix:Indicators>
  <stix:Indicator id="example:indicator-3cdff264-b682-11e3-ab3e-0800271e87d2" timestamp="2014-03-31T00:00:00.000000Z" xsi:type="indicator:IndicatorType">
    <indicator:Indicated_TTP>
      <stixCommon:TTP id="example:ttp-3cdf6c54-b682-11e3-ab3e-0800271e87d2" timestamp="2014-03-31T00:00:00.000000Z" xsi:type="ttp:TTPType">
        <!-- SNIP -->
      </stixCommon:TTP>
    </indicator:Indicated_TTP>
  </stix:Indicator>
</stix:Indicators>
{% endhighlight %}{% include tab_separator.html %}{% highlight python linenos %}
from stix.core import STIXPackage
from stix.indicator import Indicator
from stix.ttp import TTP

stix_package = STIXPackage()
ttp = TTP()

indicator = Indicator()
indicator.add_indicated_ttp(ttp)

stix_package.add_indicator(indicator)
print stix_package.to_xml()
{% endhighlight %}{% include end_tabs.html %}

#### Minimal Reference

{% include start_tabs.html tabs="XML|Python" name="min-ref" %}{% highlight xml linenos %}
<stix:Indicators>
  <stix:Indicator id="example:indicator-8837a4b4-b682-11e3-b0f3-0800271e87d2" xsi:type='indicator:IndicatorType' timestamp="2014-03-31T00:00:00.000000Z">
    <indicator:Indicated_TTP>
      <stixCommon:TTP idref="example:ttp-883730f6-b682-11e3-b0f3-0800271e87d2" />
    </indicator:Indicated_TTP>
  </stix:Indicator>
</stix:Indicators>
<stix:TTPs>
  <stix:TTP id="example:ttp-883730f6-b682-11e3-b0f3-0800271e87d2" xsi:type='ttp:TTPType' timestamp="2014-03-31T00:00:00.000000Z"/>
</stix:TTPs>
{% endhighlight %}{% include tab_separator.html %}{% highlight python linenos %}
from stix.core import STIXPackage
from stix.indicator import Indicator
from stix.ttp import TTP

stix_package = STIXPackage()
ttp = TTP()

indicator = Indicator()
indicator.add_indicated_ttp(TTP(idref=ttp.id_))

stix_package.add_indicator(indicator)
stix_package.add_ttp(ttp)

print stix_package.to_xml()
{% endhighlight %}{% include end_tabs.html %}

#### Use of Relationship Field

{% include start_tabs.html tabs="XML|Python" name="rel" %}{% highlight xml linenos %}
<stix:Indicators>
  <stix:Indicator id="example:indicator-7f7b073e-b683-11e3-b79d-0800271e87d2" xsi:type='indicator:IndicatorType' timestamp="2014-03-31T00:00:00.000000Z">
    <indicator:Indicated_TTP>
      <stixCommon:Relationship>Indicates Malware</stixCommon:Relationship>
      <stixCommon:TTP idref="example:ttp-7f7a4ede-b683-11e3-b79d-0800271e87d2" />
    </indicator:Indicated_TTP>
  </stix:Indicator>
</stix:Indicators>
<stix:TTPs>
  <stix:TTP id="example:ttp-7f7a4ede-b683-11e3-b79d-0800271e87d2" xsi:type='ttp:TTPType' timestamp="2014-03-31T00:00:00.000000Z"/>
</stix:TTPs>
{% endhighlight %}{% include tab_separator.html %}{% highlight python linenos %}
from stix.core import STIXPackage
from stix.indicator import Indicator
from stix.ttp import TTP
from stix.common.related import RelatedTTP

stix_package = STIXPackage()
ttp = TTP()

indicator = Indicator()
indicator.add_indicated_ttp(RelatedTTP(TTP(idref=ttp.id_),
                                       relationship="Indicates Malware"))

stix_package.add_indicator(indicator)
stix_package.add_ttp(ttp)

print stix_package.to_xml()
{% endhighlight %}{% include end_tabs.html %}

#### Use of Confidence and Information Source

{% include start_tabs.html tabs="XML|Python" name="conf" %}{% highlight xml linenos %}
<stix:Indicators>
  <stix:Indicator id="example:indicator-50787664-b684-11e3-9149-0800271e87d2" xsi:type='indicator:IndicatorType' timestamp="2014-03-31T00:00:00.000000Z">
    <indicator:Indicated_TTP>
      <stixCommon:Confidence>
        <stixCommon:Value xsi:type="stixVocabs:HighMediumLowVocab-1.0">High</stixCommon:Value>
      </stixCommon:Confidence>
      <stixCommon:Information_Source>
        <stixCommon:Identity id="example:Identity-50790476-b684-11e3-9149-0800271e87d2">
          <stixCommon:Name>Acme, Inc.</stixCommon:Name>
        </stixCommon:Identity>
      </stixCommon:Information_Source>
      <stixCommon:Relationship>Indicates Malware</stixCommon:Relationship>
      <stixCommon:TTP idref="example:ttp-5077da92-b684-11e3-9149-0800271e87d2"/>
    </indicator:Indicated_TTP>
  </stix:Indicator>
</stix:Indicators>
<stix:TTPs>
  <stix:TTP id="example:ttp-5077da92-b684-11e3-9149-0800271e87d2" xsi:type='ttp:TTPType' timestamp="2014-03-31T00:00:00.000000Z"/>
</stix:TTPs>
{% endhighlight %}{% include tab_separator.html %}{% highlight python linenos %}
from stix.core import STIXPackage
from stix.indicator import Indicator
from stix.ttp import TTP
from stix.common.related import RelatedTTP
from stix.common import Confidence, InformationSource, Identity

stix_package = STIXPackage() ttp = TTP()

indicator = Indicator()
confidence = Confidence(value="High")
info_src = InformationSource(identity=Identity(name="Acme, Inc."))
indicator.add_indicated_ttp(RelatedTTP(TTP(idref=ttp.id_),
                                       relationship="Indicates Malware",
                                       information_source=info_src,
                                       confidence=confidence))

stix_package.add_indicator(indicator)
stix_package.add_ttp(ttp)

print stix_package.to_xml()
{% endhighlight %}{% include end_tabs.html %}

## Further Reading

Many (even most) of the use-case driven idioms show specific examples of STIX relationships:

* [Campaign Victim Targeting](/documentation/idioms/victim-targeting)
* [Malware used in an Incident](/documentation/idioms/incident-malware)
* [C2 Indicator](/documentation/idioms/c2-indicator)
* [Malicious E-mail Attachment](/documentation/idioms/malicious-email-attachment)
* [TTP Leveraged by a Threat Actor](/documentation/idioms/leveraged-ttp)


text:
package org.ethereum.vm;

import org.ethereum.util.ByteUtil;
import org.ethereum.util.FastByteComparisons;

import org.spongycastle.util.Arrays;
import org.spongycastle.util.encoders.Hex;

import java.math.BigDecimal;
import java.math.BigInteger;

import java.nio.ByteBuffer;

/**
 * DataWord is the 32-byte array representation of a 256-bit number
 * Calculations can be done on this word with other DataWords
 *
 * @author Roman Mandeleil
 * @since 01.06.2014
 */
public class DataWord implements Comparable<DataWord> {

    /* Maximum value of the DataWord */
    public static final BigInteger _2_256 = BigInteger.valueOf(2).pow(256);
    public static final BigInteger MAX_VALUE = _2_256.subtract(BigInteger.ONE);
    public static final DataWord ZERO = new DataWord(new byte[32]);      // don't push it in to the stack
    public static final DataWord ZERO_EMPTY_ARRAY = new DataWord(new byte[0]);      // don't push it in to the stack

    private byte[] data = new byte[32];

    public DataWord() {
    }

    public DataWord(int num) {
        this(ByteBuffer.allocate(4).putInt(num));
    }

    public DataWord(long num) {
        this(ByteBuffer.allocate(8).putLong(num));
    }

    private DataWord(ByteBuffer buffer) {
        final ByteBuffer data = ByteBuffer.allocate(32);
        final byte[] array = buffer.array();
        System.arraycopy(array, 0, data.array(), 32 - array.length, array.length);
        this.data = data.array();
    }

    public DataWord(String data) {
        this(Hex.decode(data));
    }

    public DataWord(byte[] data) {
        if (data == null)
            this.data = ByteUtil.EMPTY_BYTE_ARRAY;
        else if (data.length <= 32)
            System.arraycopy(data, 0, this.data, 32 - data.length, data.length);
        else
            throw new RuntimeException("Data word can't exceed 32 bytes: " + data);
    }

    public byte[] getData() {
        return data;
    }

    public byte[] getNoLeadZeroesData() {
        return ByteUtil.stripLeadingZeroes(data);
    }

    public byte[] getLast20Bytes() {
        return Arrays.copyOfRange(data, 12, data.length);
    }

    public BigInteger value() {
        return new BigInteger(1, data);
    }

    /**
     * Converts this DataWord to an int, checking for lost information.
     * If this DataWord is out of the possible range for an int result
     * then an ArithmeticException is thrown.
     *
     * @return this DataWord converted to an int.
     * @throws ArithmeticException - if this will not fit in an int.
     */
    public int intValue() {
        BigDecimal tmpValue = new BigDecimal(this.value());
        if (this.bytesOccupied() > 4)
            return Integer.MAX_VALUE;
        return tmpValue.intValueExact();
    }

    /**
     * Converts this DataWord to a long, checking for lost information.
     * If this DataWord is out of the possible range for a long result
     * then an ArithmeticException is thrown.
     *
     * @return this DataWord converted to a long.
     * @throws ArithmeticException - if this will not fit in a long.
     */
    public long longValue() {
        BigDecimal tmpValue = new BigDecimal(this.value());
        return tmpValue.longValueExact();
    }

    public BigInteger sValue() {
        return new BigInteger(data);
    }

    public boolean isZero() {
        for (byte tmp : data) {
            if (tmp != 0) return false;
        }
        return true;
    }

    // only in case of signed operation
    // when the number is explicit defined
    // as negative
    public boolean isNegative() {
        int result = data[0] & 0x80;
        return result == 0x80;
    }

    public DataWord and(DataWord w2) {

        for (int i = 0; i < this.data.length; ++i) {
            this.data[i] &= w2.data[i];
        }
        return this;
    }

    public DataWord or(DataWord w2) {

        for (int i = 0; i < this.data.length; ++i) {
            this.data[i] |= w2.data[i];
        }
        return this;
    }

    public DataWord xor(DataWord w2) {

        for (int i = 0; i < this.data.length; ++i) {
            this.data[i] ^= w2.data[i];
        }
        return this;
    }

    public void negate() {

        if (this.isZero()) return;

        for (int i = 0; i < this.data.length; ++i) {
            this.data[i] = (byte) ~this.data[i];
        }

        for (int i = this.data.length - 1; i >= 0; --i) {
            this.data[i] = (byte) (1 + this.data[i] & 0xFF);
            if (this.data[i] != 0) break;
        }
    }

    public void bnot() {
        if (this.isZero()) return;
        this.data = ByteUtil.copyToArray(MAX_VALUE.subtract(this.value()));
    }

    // By   : Holger
    // From : http://stackoverflow.com/a/24023466/459349
    public void add(DataWord word) {
        byte[] result = new byte[32];
        for (int i = 31, overflow = 0; i >= 0; i--) {
            int v = (this.data[i] & 0xff) + (word.data[i] & 0xff) + overflow;
            result[i] = (byte) v;
            overflow = v >>> 8;
        }
        this.data = result;
    }

    // old add-method with BigInteger quick hack
    public void add2(DataWord word) {
        BigInteger result = value().add(word.value());
        this.data = ByteUtil.copyToArray(result.and(MAX_VALUE));
    }

    // TODO: mul can be done in more efficient way
    // TODO:     with shift left shift right trick
    // TODO      without BigInteger quick hack
    public void mul(DataWord word) {
        BigInteger result = value().multiply(word.value());
        this.data = ByteUtil.copyToArray(result.and(MAX_VALUE));
    }

    // TODO: improve with no BigInteger
    public void div(DataWord word) {

        if (word.isZero()) {
            this.and(ZERO);
            return;
        }

        BigInteger result = value().divide(word.value());
        this.data = ByteUtil.copyToArray(result.and(MAX_VALUE));
    }

    // TODO: improve with no BigInteger
    public void sDiv(DataWord word) {

        if (word.isZero()) {
            this.and(ZERO);
            return;
        }

        BigInteger result = sValue().divide(word.sValue());
        this.data = ByteUtil.copyToArray(result.and(MAX_VALUE));
    }

    // TODO: improve with no BigInteger
    public void sub(DataWord word) {
        BigInteger result = value().subtract(word.value());
        this.data = ByteUtil.copyToArray(result.and(MAX_VALUE));
    }

    // TODO: improve with no BigInteger
    public void exp(DataWord word) {
        BigInteger result = value().modPow(word.value(), _2_256);
        this.data = ByteUtil.copyToArray(result);
    }

    // TODO: improve with no BigInteger
    public void mod(DataWord word) {

        if (word.isZero()) {
            this.and(ZERO);
            return;
        }

        BigInteger result = value().mod(word.value());
        this.data = ByteUtil.copyToArray(result.and(MAX_VALUE));
    }

    public void sMod(DataWord word) {

        if (word.isZero()) {
            this.and(ZERO);
            return;
        }

        BigInteger result = sValue().abs().mod(word.sValue().abs());
        result = (sValue().signum() == -1) ? result.negate() : result;

        this.data = ByteUtil.copyToArray(result.and(MAX_VALUE));
    }

    public void addmod(DataWord word1, DataWord word2) {

        this.add(word1);
        this.mod(word2);
    }

    public void mulmod(DataWord word1, DataWord word2) {

        if (word2.isZero()) {
            this.data = new byte[32];
            return;
        }

        BigInteger result = value().multiply(word1.value()).mod(word2.value());
        this.data = ByteUtil.copyToArray(result.and(MAX_VALUE));
    }

    public String toString() {
        return Hex.toHexString(data);
    }

    public String shortHex() {
        String hexValue = Hex.toHexString(getNoLeadZeroesData()).toUpperCase();
        return "0x" + hexValue.replaceFirst("^0+(?!$)", "");
    }

    public DataWord clone() {
        return new DataWord(Arrays.clone(data));
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        DataWord dataWord = (DataWord) o;

        return java.util.Arrays.equals(data, dataWord.data);

    }

    @Override
    public int hashCode() {
        return java.util.Arrays.hashCode(data);
    }

    @Override
    public int compareTo(DataWord o) {
        if (o == null || o.getData() == null) return -1;
        int result = FastByteComparisons.compareTo(
                data, 0, data.length,
                o.getData(), 0, o.getData().length);
        // Convert result into -1, 0 or 1 as is the convention
        return (int) Math.signum(result);
    }

    public void signExtend(byte k) {
        if (0 > k || k > 31)
            throw new IndexOutOfBoundsException();
        byte mask = this.sValue().testBit((k * 8) + 7) ? (byte) 0xff : 0;
        for (int i = 31; i > k; i--) {
            this.data[31 - i] = mask;
        }
    }

    public int bytesOccupied() {
        int firstNonZero = ByteUtil.firstNonZeroByte(data);
        if (firstNonZero == -1) return 0;
        return 31 - firstNonZero + 1;
    }

    public boolean isHex(String hex) {
        return Hex.toHexString(data).equals(hex);
    }
}

redpajama_set_name: RedPajamaGithub


text:
'''
gpipe/extract_regions.py -
======================================================

:Author: Andreas Heger
:Release: $Id$
:Date: |today|
:Tags: Python

Purpose
-------

.. todo::

   describe purpose of the script.

Usage
-----

Example::

   python gpipe/extract_regions.py --help

Type::

   python gpipe/extract_regions.py --help

for command line help.

Documentation
-------------

Code
----

'''
import os
import sys
import string
import re
import getopt
import tempfile
import time
import optparse
import math

import CGAT.Experiment as E
import CGAT.Genomics as Genomics
import CGAT.IndexedFasta as IndexedFasta
import CGAT.IOTools as IOTools
import pgdb
import CGAT.Regions as Regions

USAGE = """python %s [OPTIONS]

Version: $Id: gpipe/extract_regions.py 2781 2009-09-10 11:33:14Z andreas $

get sequences/sequence regions for a list of predictions and output a fasta-formatted
file.

""" % sys.argv[0]

def getMRNAs(dbhandle, schema, options, prediction_ids):

    xfrom = ""
    xwhere = ""
    if prediction_ids == "all":
        where = ""
    elif prediction_ids == "nr":
        xfrom = "%s.redundant AS r," % schema
        where = "AND p.prediction_id = r.rep_prediction_id AND r.rep_prediction_id = r.mem_prediction_id"
    else:
        where = "AND p.prediction_id  IN ('%s')" % "','".join(
            map(str, prediction_ids))

    # select genomic location for all predictions
    # retrieve: prediction_id, sbjct_token, sbjct_strand, sbjct_genome_from,
    # sbjct_genome_to
    statement = """
    SELECT prediction_id, sbjct_token, sbjct_strand, sbjct_genome_from, sbjct_genome_to
    FROM
    %s
    %s.%s AS p
    WHERE True %s
    """ % (xfrom, schema, options.tablename_predictions,
           where)

    cc = dbhandle.cursor()
    cc.execute(statement)
    locations = cc.fetchall()
    cc.close()

    return locations

def getExons(dbhandle, schema, options, prediction_ids):

    xfrom = ""
    where = ""
    if prediction_ids == "all":
        where = ""
    elif prediction_ids == "nr":
        xfrom = "%s.redundant AS r," % schema
        where = "AND p.prediction_id = r.rep_prediction_id AND r.rep_prediction_id = r.mem_prediction_id"
    else:
        where = "AND p.prediction_id  IN ('%s')" % "','".join(
            map(str, prediction_ids))

    # select genomic location for all introns
    # filter on genome_exon_to > 0 in order to eliminate
    # all non-matching exons to query.
    # retrieve: prediction_id, sbjct_token, sbjct_strand, sbjct_genome_from,
    # sbjct_genome_to
    statement = """
    SELECT DISTINCT
    p.prediction_id, sbjct_token, sbjct_strand, genome_exon_from, genome_exon_to
    FROM
    %s.%s AS p,
    %s
    %s.%s AS e
    WHERE p.prediction_id = e.prediction_id
    %s AND
    genome_exon_to > 0
    GROUP BY p.prediction_id, sbjct_token, sbjct_strand, genome_exon_from, genome_exon_to
    ORDER BY p.prediction_id, genome_exon_from
    """ % (schema, options.tablename_predictions,
           xfrom,
           schema, options.tablename_exons,
           where)

    cc = dbhandle.cursor()
    cc.execute(statement)
    locations = cc.fetchall()
    cc.close()

    return locations

def getIntrons(dbhandle, schema, options, prediction_ids):

    result = getExons(dbhandle, schema, options, prediction_ids)

    locations = []
    last_to = None
    last_id = None
    for id, sbjct_token, sbjct_strand, sbjct_from, sbjct_to in result:
        if last_id != id:
            last_to = None
            last_id = id
        if last_to:
            locations.append(
                (id, sbjct_token, sbjct_strand, last_to, sbjct_from))
        last_to = sbjct_to

    return locations

##########################################################################

def getOrthologs(dbhandle, options, prediction_ids):

    where = "AND l.prediction_id1 IN ('%s')" % "','".join(
        map(str, prediction_ids))

    # select genomic location for all predictions
    # retrieve: prediction_id, sbjct_token, sbjct_strand, sbjct_genome_from,
    # sbjct_genome_to
    statement = """
    SELECT schema2, prediction_id2
    FROM
    %s AS l
    WHERE l.schema1 = '%s' AND l.schema2 != l.schema1
    %s
    """ % (options.tablename_orthologs, options.schema, where )

    cc = dbhandle.cursor()
    cc.execute(statement)
    result = cc.fetchall()
    cc.close()

    orthologs = {}
    for schema, prediction_id in result:
        if schema not in orthologs:
            orthologs[schema] = []
        orthologs[schema].append(prediction_id)

    return orthologs

##########################################################################

def GetIdentifierInfo(dbhandle, schema, options, prediction_ids):
    """get additional identifier info."""

    xfrom = ""
    where = ""

    if prediction_ids == "all":
        where = ""
    elif prediction_ids == "nr":
        xfrom = "%s.redundant AS r," % schema
        where = "AND p.prediction_id = r.rep_prediction_id AND r.rep_prediction_id = r.mem_prediction_id"
    else:
        where = "AND p.prediction_id  IN ('%s')" % "','".join(
            map(str, prediction_ids))

    statement = """
    SELECT p.prediction_id, g.gene_id, q.class
    FROM
    %s.%s AS p,
    %s.%s AS g,
    %s
    %s.%s AS q
    WHERE p.prediction_id = g.prediction_id AND
    p.prediction_id = q.prediction_id
    %s
    GROUP BY p.prediction_id, g.gene_id, q.class
    """ % (schema, options.tablename_predictions,
           schema, options.tablename_genes,
           xfrom,
           schema, options.tablename_quality,
           where)

    cc = dbhandle.cursor()
    cc.execute(statement)
    result = cc.fetchall()
    cc.close()

    info = {}
    for prediction_id, gene_id, quality in result:
        info[prediction_id] = (gene_id, quality)

    return info

##########################################################################
def ProcessPredictions(dbhandle, schema, options, prediction_ids, taboo_regions=None):
    """print required regions for a set of prediction_ids from a given schema."""
    # Step 2 :

    if "%s" in options.genome_file:
        fasta = IndexedFasta.IndexedFasta(options.genome_file % schema)
    else:
        fasta = IndexedFasta.IndexedFasta(options.genome_file)

    if options.type == "mrnas":
        locations = getMRNAs(dbhandle, schema, options, prediction_ids)
    elif options.type == "introns":
        locations = getIntrons(dbhandle, schema, options, prediction_ids)
    elif options.type == "exons":
        locations = getExons(dbhandle, schema, options, prediction_ids)
    elif options.type == "cds":
        locations = getExons(dbhandle, schema, options, prediction_ids)
        options.join_regions = True

    if options.loglevel >= 1:
        options.stdlog.write(
            "# %s: retrieved %i locations.\n" % (schema, len(locations)))

    if options.loglevel >= 3:
        for location in locations:
            options.stdlog.write("# %s\n" % (str(location)))

    if len(locations) == 0:
        return

    if options.id_format == "full":
        # retrieve quality and gene information
        extra_info = GetIdentifierInfo(
            dbhandle, schema, options, prediction_ids)

    contigs = set(map(lambda x: x[1], locations))

    if options.loglevel >= 1:
        options.stdlog.write("# %s: %i locations are on %i contigs.\n" %
                             (schema, len(locations), len(contigs)))
        options.stdlog.flush()

    # retrieve contig sizes
    statement = """
    SELECT sbjct_token, size FROM %s.contigs""" % ( schema )
    cc = dbhandle.cursor()
    cc.execute(statement)
    result = cc.fetchall()
    cc.close()

    contig_size = {}
    for sbjct_token, size in result:
        contig_size[sbjct_token] = size

    # sort locations by contig, that way sequence retrieval is faster
    # sort also by identifier
    locations.sort(lambda x, y: cmp((x[2], x[0]), (y[2], y[0])))
    tokens = {}
    noutput = 0
    nskipped_length = 0
    nskipped_overlap = 0
    nskipped_doubles = 0

    output_regions = set()

    last_token = None
    this_is_last = False

    if options.join_regions:
        # add dummy to locations
        locations.append(("", "", "+", 0, 0))

    for token, sbjct_token, sbjct_strand, sbjct_genome_from, sbjct_genome_to in locations:

        if token != "":

            if token not in tokens:
                tokens[token] = 0
            tokens[token] += 1

            sbjct_genome_from -= (options.extend_region -
                                  options.shorten_region)
            sbjct_genome_from = max(0, sbjct_genome_from)
            sbjct_genome_to += (options.extend_region - options.shorten_region)
            sbjct_genome_to = min(sbjct_genome_to, contig_size[sbjct_token])

            if sbjct_genome_to - sbjct_genome_from <= 0:
                continue

            lgenome = fasta.getLength(sbjct_token)

            forward_sbjct_genome_from, forward_sbjct_genome_to = Genomics.ToForwardCoordinates(sbjct_genome_from,
                                                                                sbjct_genome_to,
                                                                                sbjct_strand,
                                                                                lgenome)
            if taboo_regions:

                if options.taboo_regions == "both":
                    overlaps = taboo_regions.getOverlaps(sbjct_token, sbjct_strand,
                                                         forward_sbjct_genome_from, forward_sbjct_genome_to)
                else:
                    overlaps = taboo_regions.getOverlaps(sbjct_token, sbjct_strand,
                                                         sbjct_genome_from, sbjct_genome_to)

                if overlaps:
                    if options.loglevel >= 2:
                        options.stdlog.write(
                            "# %s eliminated due to overlap with taboo region.\n" % (token))
                    nskipped_overlap += 1
                    continue

            genomic_sequence = fasta.getSequence(sbjct_token,
                                                 sbjct_strand,
                                                 sbjct_genome_from,
                                                 sbjct_genome_to)

            if len(genomic_sequence) == 0:
                options.stderr.write("## warning: sequence for prediction %s is emtpy: %s:%s:%i:%i\n" %
                                     (str(token), sbjct_token, sbjct_strand,
                                      sbjct_genome_from, sbjct_genome_to))
                options.stderr.flush()

            # adjust sbjct_genome_to if it went outside of contig boundaries.
            sbjct_genome_to = len(genomic_sequence) + sbjct_genome_from

        do_output = False
        # decide whether segment is to be joined
        if options.join_regions:

            if last_token == token:
                do_output = False
                fragments.append(str(genomic_sequence))
                max_sbjct_genome_to = sbjct_genome_to
            else:
                if last_token is not None:
                    output_sbjct_genome_from = min_sbjct_genome_from
                    output_sbjct_genome_to = max_sbjct_genome_to
                    output_genomic_sequence = "".join(fragments)
                    output_token = last_token
                    do_output = True

                min_sbjct_genome_from = sbjct_genome_from
                max_sbjct_genome_to = sbjct_genome_to
                fragments = [str(genomic_sequence)]
                last_token = token
        else:
            output_sbjct_genome_from = sbjct_genome_from
            output_sbjct_genome_to = sbjct_genome_to
            output_genomic_sequence = str(genomic_sequence)
            output_token = token
            do_output = True

        if not do_output:
            continue

        # build output token
        if options.id_format == "id":
            output_token = str(output_token)
        elif options.id_format == "schema-id":
            output_token = "%s%s%s" % (
                schema, options.separator, str(output_token))
        elif options.id_format == "full":
            gene_id, quality = extra_info[output_token]
            output_token = options.separator.join(
                map(str, (schema, output_token, gene_id, quality)))

        if options.forward_coordinates:
            sbjct_genome_from, sbjct_genome_to = forward_sbjct_genome_from, forward_sbjct_genome_to

        # Filter output by length
        if len(output_genomic_sequence) < options.min_length:
            nskipped_length += 1
            return

        # Filter output for redundant entries
        k = "%s-%s-%i-%i" % (sbjct_token, sbjct_strand,
                             output_sbjct_genome_from, output_sbjct_genome_to)
        if k in output_regions:
            nskipped_doubles += 1
            return

        output_regions.add(k)

        if options.output_coordinate_format == "full":
            coordinates = "%s:%s:%i:%i" % (sbjct_token,
                                           sbjct_strand,
                                           output_sbjct_genome_from,
                                           output_sbjct_genome_to)

        elif options.output_coordinate_format == "long":
            coordinates = "%s:%s:%i:%i:%i" % (sbjct_token,
                                              sbjct_strand,
                                              sbjct_genome_from,
                                              sbjct_genome_to,
                                              lgenome)

        if options.output_format == "fasta":
            # print fasta formatted output
            if options.fasta_format == "id-coordinates":
                options.stdout.write(">%s %s\n%s\n" % (output_token, coordinates,
                                                       str(output_genomic_sequence)))
            elif options.fasta_format == "coordinates":
                options.stdout.write(">%s:%s:%i:%i\n%s\n" % (coordinates,
                                                             str(output_genomic_sequence)))
            elif options.fasta_format == "schema-coordinates":
                options.stdout.write(">%s|%s:%s:%i:%i\n%s\n" % (schema,
                                                                coordinates,
                                                                str(output_genomic_sequence)))

        elif options.output_format == "table":
            # print identifier and sequence in tab separated table
            options.stdout.write("%s\t%s\t%s\t%i\t%i\ts\n" % (output_token,
                                                              sbjct_token, sbjct_strand,
                                                              output_sbjct_genome_from, output_sbjct_genome_to,
                                                              str(output_genomic_sequence)))

        elif options.output_format == "region":
            options.stdout.write("%s\t%s\t%s\t%i\t%i\n" % (output_token,
                                                           sbjct_token, sbjct_strand,
                                                           output_sbjct_genome_from, output_sbjct_genome_to))

        noutput += 1

        options.stdout.flush()

    if options.loglevel >= 1:
        if not prediction_ids:
            prediction_ids = []
        options.stdlog.write("# %s: ninput=%i, npredictions=%i, nlocations=%i, noutput=%i, nskipped_length=%i, nskipped_overlap=%i, nskipped_doubles=%i\n" % (
            schema, len(prediction_ids), len(tokens), len(locations), noutput, nskipped_length, nskipped_overlap, nskipped_doubles))

def main(argv=None):
    """script main.

    parses command line options in sys.argv, unless *argv* is given.
    """

    if argv is None:
        argv = sys.argv

    parser = E.OptionParser(
        version="%prog version: $Id: gpipe/extract_regions.py 2781 2009-09-10 11:33:14Z andreas $", usage=globals()["__doc__"])

    parser.add_option("-g", "--genome-file", dest="genome_file", type="string",
                      help="pattern to look for sequence filename.")

    parser.add_option("-i", "--ids", dest="ids", type="string",
                      help="comma separated list of prediction ids. Use 'all' to use all predictions.")

    parser.add_option("-f", "--filename-ids", dest="filename_ids", type="string",
                      help="filename with prediction ids.")

    parser.add_option("-t", "--sequence-type", dest="type", type="choice",
                      choices=("mrnas", "introns", "exons", "cds"),
                      help="type to output.")

    parser.add_option("-e", "--extend-region", dest="extend_region", type="int",
                      help="regions are extended by this margin at either end.")

    parser.add_option("-r", "--shorten-region", dest="shorten_region", type="int",
                      help="regions are shortened by this margin at either end.")

    parser.add_option("-m", "--min-interval-length", dest="min_length", type="int",
                      help="minimum length of segment.")

    parser.add_option("-s", "--schema", dest="schema", type="string",
                      help="schema to take data from.")

    parser.add_option("-o", "--output-format", dest="output_format", type="choice",
                      choices=("fasta", "table", "region"),
                      help="output formats.")

    parser.add_option("--fasta-format", dest="fasta_format", type="choice",
                      choices=(
                          "id-coordinates", "coordinates", "schema-coordinates"),
                      help="output formats for fasta formatted headers.")

    parser.add_option("--orthologs", dest="orthologs", action="store_true",
                      help="lookup up orthologs of prediction ids.")

    parser.add_option("--multiple", dest="multiple", action="store_true",
                      help="""lookup up predictions in multiple species.
                       Identifiers should be given as schema|prediction_id[|additional_fields].
                       Note that the genome file locations have to be consistent.""")

    parser.add_option("--id-format", dest="id_format", type="choice",
                      choices=("id", "schema-id", "full"),
                      help="output format for ids.")

    parser.add_option("--taboo-regions", dest="taboo_regions", type="choice",
                      choices=("same", "both"),
                      help="check for overlap in same/both strands.")

    parser.add_option("--filename-taboo-regions", dest="filename_taboo_regions", type="string",
                      help="filename with information about taboo regions.")

    parser.add_option("--is-forward-coordinates", dest="forward_coordinates", action="store_true",
                      help="output coordinates are forward coordinates.")

    parser.add_option("--join-regions", dest="join_regions", action="store_true",
                      help="join regions with the same identifier.")

    parser.add_option("--output-coordinate-format", dest="output_coordinate_format", type="choice",
                      choices=("full", "long"),
                      help="""output format of coordinates. Output format is contig:strand:from:to in zero based
/forward/reverse strand coordinates in open/closed notation. 'long' includes the contig length as fifth field"""  )

    parser.set_defaults(
        genome_file="genome",
        identifiers=None,
        filename_ids="-",
        ids=None,
        extend_region=0,
        shorten_region=0,
        join_regions=False,
        tablename_predictions="predictions",
        tablename_exons="exons",
        tablename_genes="genes",
        tablename_quality="quality",
        tablename_orthologs="orthology_pairwise1v5.orthologlinks_first",
        schema=None,
        output_format="fasta",
        fasta_format="id-coordinates",
        type="mrnas",
        min_length=1,
        id_format="id",
        mmultiple=False,
        separator="|",
        filename_taboo_regions=False,
        forward_coordinates=False,
        output_coordinate_format="full",
    )

    (options, args) = E.Start(parser, add_database_options=True)

    if options.orthologs:
        options.id_format = "schema-id"

    # database handle for connecting to postgres
    dbhandle = pgdb.connect(options.psql_connection)

    # Step 1 : Input of predictions

    # read identifiers from file, command line arguments or stdin.

    if options.ids in ("all", "nr"):
        prediction_ids = options.ids
        if options.loglevel >= 1:
            options.stdlog.write("# using all prediction ids.\n")
            options.stdlog.flush()
    elif options.ids:
        prediction_ids = options.ids.split(",")
    elif len(args) > 0:
        prediction_ids = args

    elif options.filename_ids:
        prediction_ids = []

        if options.filename_ids == "-":
            prediction_ids += IOTools.ReadList(sys.stdin)[0]
        elif options.filename_ids:
            prediction_ids += IOTools.ReadList(
                open(options.filename_ids, "r"))[0]

        if len(prediction_ids) == 0:
            raise "no prediction identifiers given."

        if options.loglevel >= 1:
            options.stdlog.write(
                "# read %i prediction ids.\n" % len(prediction_ids))
            options.stdlog.flush()

    if options.filename_taboo_regions:
        # Note: the input has to be in forward coordinates in order for option
        # "both" to work.
        taboo_regions = Regions.RegionFilter()
        if options.taboo_regions == "both":
            ignore_strand = True
        else:
            ignore_strand = False
        taboo_regions.readFromFile(
            open(options.filename_taboo_regions, "r"), ignore_strand=ignore_strand)
    else:
        taboo_regions = None

    if options.orthologs:
        orthologs = getOrthologs(dbhandle, options, prediction_ids)
        for schema, ids in orthologs.items():
            ProcessPredictions(dbhandle, schema, options, ids, taboo_regions)
    elif options.multiple:
        # convert identifiers
        prediction_ids = map(
            lambda x: x.split(options.separator), prediction_ids)
        prediction_ids.sort()
        last_schema = None
        ids = []
        for x in prediction_ids:
            if x[0] != last_schema:
                ProcessPredictions(dbhandle, last_schema, options, ids)
                ids = []
                last_schema = x[0]
            ids.append(x[1])
        ProcessPredictions(dbhandle, last_schema, options, ids)
    else:
        ProcessPredictions(
            dbhandle, options.schema, options, prediction_ids, taboo_regions)

    E.Stop()

if __name__ == "__main__":
    sys.exit(main(sys.argv))


text:
package org.wso2.carbon.identity.oauth;

import java.net.URLDecoder;

public class Parameters {

    // oauth_signature
    private String oauthSignature = null;
    // oauth_nonce
    private String oauthNonce = null;
    // oauth_signature_method = HMAC-SHA1
    private String oauthSignatureMethod = null;
    // oauth_consumer_key
    private String oauthConsumerKey = null;
    // oauth_timestamp
    private String oauthTimeStamp = null;
    private String baseString = null;
    private String httpMethod = null;
    private String oauthCallback = null;
    private String scope = null;
    private String displayName = null;
    private String oauthToken = null;
    private String oauthTokenSecret = null;
    private String callbackConfirmed = null;
    private String authorizedbyUserName = null;
    private String authorizedbyUserPassword = null;
    private String oauthTokenVerifier = null;
    private String appName = null;
    private String version = null;
    private boolean accessTokenIssued;

    public boolean isAccessTokenIssued() {
        return accessTokenIssued;
    }

    public void setAccessTokenIssued(boolean accessTokenIssued) {
        this.accessTokenIssued = accessTokenIssued;
    }

    public String getVersion() {
        return version;
    }

    public void setVersion(String version) {
        this.version = version;
    }

    public String getAppName() {
        return appName;
    }

    public void setAppName(String appName) {
        this.appName = appName;
    }

    public String getOauthTokenVerifier() {
        return oauthTokenVerifier;
    }

    public void setOauthTokenVerifier(String oauthTokenVerifier) {
        this.oauthTokenVerifier = oauthTokenVerifier;
    }

    public String getAuthorizedbyUserPassword() {
        return authorizedbyUserPassword;
    }

    public void setAuthorizedbyUserPassword(String authorizedbyUserPassword) {
        this.authorizedbyUserPassword = authorizedbyUserPassword;
    }

    public String getAuthorizedbyUserName() {
        return authorizedbyUserName;
    }

    public void setAuthorizedbyUserName(String authorizedbyUserName) {
        this.authorizedbyUserName = authorizedbyUserName;
    }

    public String getOauthTokenSecret() {
        return oauthTokenSecret;
    }

    public void setOauthTokenSecret(String oauthTokenSecret) {
        this.oauthTokenSecret = oauthTokenSecret;
    }

    public String getCallbackConfirmed() {
        return callbackConfirmed;
    }

    public void setCallbackConfirmed(String callbackConfirmed) {
        this.callbackConfirmed = callbackConfirmed;
    }

    public String getOauthToken() {
        return oauthToken;
    }

    public void setOauthToken(String oauthToken) {
        this.oauthToken = oauthToken;
    }

    public String getDisplayName() {
        return displayName;
    }

    public void setDisplayName(String displayName) {
        this.displayName = displayName;
    }

    public String getScope() {
        return scope;
    }

    public void setScope(String scope) {
        this.scope = scope;
    }

    public String getOauthCallback() {
        return oauthCallback;
    }

    public void setOauthCallback(String oauthCallback) {
        this.oauthCallback = oauthCallback;
    }

    public String getBaseString() {
        return baseString;
    }

    public void setBaseString(String baseString) {
        this.baseString = baseString;
    }

    public String getHttpMethod() {
        return httpMethod;
    }

    public void setHttpMethod(String httpMethod) {
        this.httpMethod = httpMethod;
    }

    public String getOauthSignature() {
        return oauthSignature;
    }

    public void setOauthSignature(String oauthSignature) {
        this.oauthSignature = oauthSignature;
    }

    public String getOauthNonce() {
        return oauthNonce;
    }

    public void setOauthNonce(String oauthNonce) {
        this.oauthNonce = oauthNonce;
    }

    public String getOauthSignatureMethod() {
        return oauthSignatureMethod;
    }

    public void setOauthSignatureMethod(String oauthSignatureMethod) {
        this.oauthSignatureMethod = oauthSignatureMethod;
    }

    public String getOauthConsumerKey() {
        if (oauthConsumerKey != null) {
            return URLDecoder.decode(oauthConsumerKey);
        } else {
            return oauthConsumerKey;
        }
    }

    public void setOauthConsumerKey(String oauthConsumerKey) {
        this.oauthConsumerKey = oauthConsumerKey;
    }

    public String getOauthTimeStamp() {
        return oauthTimeStamp;
    }

    public void setOauthTimeStamp(String oauthTimeStamp) {
        this.oauthTimeStamp = oauthTimeStamp;
    }
}


text:
package com.topq.pagefactor;

import org.openqa.selenium.WebDriver;

public class HomePage extends AbstractSugerCrmPage {
	public HomePage(WebDriver driver_) throws Exception {
		super(driver_);
	}

	@Override
	protected void assertInPage() throws Exception {}
}


text:
{-# LANGUAGE PatternSynonyms, ForeignFunctionInterface, JavaScriptFFI #-}
module GHCJS.DOM.JSFFI.Generated.MutationRecord
       (js_getType, getType, js_getTarget, getTarget, js_getAddedNodes,
        getAddedNodes, js_getRemovedNodes, getRemovedNodes,
        js_getPreviousSibling, getPreviousSibling, js_getNextSibling,
        getNextSibling, js_getAttributeName, getAttributeName,
        js_getAttributeNamespace, getAttributeNamespace, js_getOldValue,
        getOldValue, MutationRecord, castToMutationRecord,
        gTypeMutationRecord)
       where
import Prelude ((.), (==), (>>=), return, IO, Int, Float, Double, Bool(..), Maybe, maybe, fromIntegral, round, fmap, Show, Read, Eq, Ord)
import Data.Typeable (Typeable)
import GHCJS.Types (JSRef(..), JSString, castRef)
import GHCJS.Foreign (jsNull)
import GHCJS.Foreign.Callback (syncCallback, asyncCallback, syncCallback1, asyncCallback1, syncCallback2, asyncCallback2, OnBlocked(..))
import GHCJS.Marshal (ToJSRef(..), FromJSRef(..))
import GHCJS.Marshal.Pure (PToJSRef(..), PFromJSRef(..))
import Control.Monad.IO.Class (MonadIO(..))
import Data.Int (Int64)
import Data.Word (Word, Word64)
import GHCJS.DOM.Types
import Control.Applicative ((<$>))
import GHCJS.DOM.EventTargetClosures (EventName, unsafeEventName)
import GHCJS.DOM.Enums

foreign import javascript unsafe "$1[\"type\"]" js_getType ::
        JSRef MutationRecord -> IO JSString

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.type Mozilla MutationRecord.type documentation>
getType ::
        (MonadIO m, FromJSString result) => MutationRecord -> m result
getType self
  = liftIO (fromJSString <$> (js_getType (unMutationRecord self)))

foreign import javascript unsafe "$1[\"target\"]" js_getTarget ::
        JSRef MutationRecord -> IO (JSRef Node)

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.target Mozilla MutationRecord.target documentation>
getTarget :: (MonadIO m) => MutationRecord -> m (Maybe Node)
getTarget self
  = liftIO ((js_getTarget (unMutationRecord self)) >>= fromJSRef)

foreign import javascript unsafe "$1[\"addedNodes\"]"
        js_getAddedNodes :: JSRef MutationRecord -> IO (JSRef NodeList)

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.addedNodes Mozilla MutationRecord.addedNodes documentation>
getAddedNodes ::
              (MonadIO m) => MutationRecord -> m (Maybe NodeList)
getAddedNodes self
  = liftIO ((js_getAddedNodes (unMutationRecord self)) >>= fromJSRef)

foreign import javascript unsafe "$1[\"removedNodes\"]"
        js_getRemovedNodes :: JSRef MutationRecord -> IO (JSRef NodeList)

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.removedNodes Mozilla MutationRecord.removedNodes documentation>
getRemovedNodes ::
                (MonadIO m) => MutationRecord -> m (Maybe NodeList)
getRemovedNodes self
  = liftIO
      ((js_getRemovedNodes (unMutationRecord self)) >>= fromJSRef)

foreign import javascript unsafe "$1[\"previousSibling\"]"
        js_getPreviousSibling :: JSRef MutationRecord -> IO (JSRef Node)

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.previousSibling Mozilla MutationRecord.previousSibling documentation>
getPreviousSibling ::
                   (MonadIO m) => MutationRecord -> m (Maybe Node)
getPreviousSibling self
  = liftIO
      ((js_getPreviousSibling (unMutationRecord self)) >>= fromJSRef)

foreign import javascript unsafe "$1[\"nextSibling\"]"
        js_getNextSibling :: JSRef MutationRecord -> IO (JSRef Node)

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.nextSibling Mozilla MutationRecord.nextSibling documentation>
getNextSibling :: (MonadIO m) => MutationRecord -> m (Maybe Node)
getNextSibling self
  = liftIO
      ((js_getNextSibling (unMutationRecord self)) >>= fromJSRef)

foreign import javascript unsafe "$1[\"attributeName\"]"
        js_getAttributeName ::
        JSRef MutationRecord -> IO (JSRef (Maybe JSString))

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.attributeName Mozilla MutationRecord.attributeName documentation>
getAttributeName ::
                 (MonadIO m, FromJSString result) =>
                   MutationRecord -> m (Maybe result)
getAttributeName self
  = liftIO
      (fromMaybeJSString <$>
         (js_getAttributeName (unMutationRecord self)))

foreign import javascript unsafe "$1[\"attributeNamespace\"]"
        js_getAttributeNamespace ::
        JSRef MutationRecord -> IO (JSRef (Maybe JSString))

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.attributeNamespace Mozilla MutationRecord.attributeNamespace documentation>
getAttributeNamespace ::
                      (MonadIO m, FromJSString result) =>
                        MutationRecord -> m (Maybe result)
getAttributeNamespace self
  = liftIO
      (fromMaybeJSString <$>
         (js_getAttributeNamespace (unMutationRecord self)))

foreign import javascript unsafe "$1[\"oldValue\"]" js_getOldValue
        :: JSRef MutationRecord -> IO (JSRef (Maybe JSString))

-- | <https://developer.mozilla.org/en-US/docs/Web/API/MutationRecord.oldValue Mozilla MutationRecord.oldValue documentation>
getOldValue ::
            (MonadIO m, FromJSString result) =>
              MutationRecord -> m (Maybe result)
getOldValue self
  = liftIO
      (fromMaybeJSString <$> (js_getOldValue (unMutationRecord self)))


text:
import React     from 'react';
import UserStore from '../stores/user';

const Format = {
	CSV:       'csv',
	JSON: 	   'json',
	PLAINTEXT: 'plaintext',
	IMAGE: 	   'image'
}

/**
 *
 */
export default React.createClass({
	propTypes: {
		boardID: React.PropTypes.string.isRequired
	},

	getInitialState() {
		return { format: Format.CSV }
	},

	onChange(event) {
		this.setState({ format: event.target.value });
	},

	render() {
		let id    = this.props.boardID;
		let query = `access_token=${UserStore.getToken()}
			&format=${this.state.format}`;

		let apiURL    = process.env.API_URL || 'http://localhost:9002/api';
		let exportURL = `${apiURL}/boards/${id}/export?${query}`;

		return (

				Export Format

						<select onChange={this.onChange}
								defaultValue={this.state.format}>
							{this.renderFormats()}
						</select>

						Export

		);
	},

	renderFormats() {
		return Object.keys(Format).map((key) => {
			return <option key={key} value={Format[key]}>{key}</option>;
		});
	}
});


text:
using System;
using System.Collections.Generic;
using System.Text;
using UnitDriven;

#if NUNIT
using NUnit.Framework;
using TestClass = NUnit.Framework.TestFixtureAttribute;
using TestInitialize = NUnit.Framework.SetUpAttribute;
using TestCleanup = NUnit.Framework.TearDownAttribute;
using TestMethod = NUnit.Framework.TestAttribute;
#elif MSTEST
using Microsoft.VisualStudio.TestTools.UnitTesting;
#endif

namespace Csla.Test.ValidationRules
{
#if TESTING
  [System.Diagnostics.DebuggerStepThrough]
#endif
  [TestClass]
  public class PerTypeTests : TestBase
  {
    [TestMethod]
    public void OnlySharedRules()
    {
      UnitTestContext context = GetContext();
      ApplicationContext.GlobalContext.Clear();
      ApplicationContext.GlobalContext["Shared"] = 0;

      HasOnlyPerTypeRules root = new HasOnlyPerTypeRules();
      root.Validate();
      context.Assert.AreEqual(string.Empty, root.Test, "Test string should be empty");
      context.Assert.AreEqual(1, root.BrokenRulesCollection.Count, "Broken rule count should be 1 first");
      root.Test = "test";
      context.Assert.AreEqual("test", root.Test, "Test string should be 'test'");
      context.Assert.AreEqual(0, root.BrokenRulesCollection.Count, "Broken rule count should be 0");
      root.Test = "big test";
      context.Assert.AreEqual("big test", root.Test, "Test string should be 'big test'");
      context.Assert.AreEqual(1, root.BrokenRulesCollection.Count, "Broken rule count should be 1 last");
      context.Assert.Success();
      context.Complete();
    }

    [TestMethod]
    public void StringRequired()
    {
      UnitTestContext context = GetContext();
      ApplicationContext.GlobalContext.Clear();
      ApplicationContext.GlobalContext["Shared"] = 0;

      HasPerTypeRules root = new HasPerTypeRules();
      root.Validate();
      context.Assert.AreEqual(string.Empty, root.Test, "Test string should be empty");
      context.Assert.AreEqual(1, root.BrokenRulesCollection.Count, "Broken rule count should be 1 first");
      root.Test = "test";
      context.Assert.AreEqual("test", root.Test, "Test string should be 'test'");
      context.Assert.AreEqual(0, root.BrokenRulesCollection.Count, "Broken rule count should be 0");
      root.Test = "big test";
      context.Assert.AreEqual("big test", root.Test, "Test string should be 'big test'");
      context.Assert.AreEqual(1, root.BrokenRulesCollection.Count, "Broken rule count should be 1 last");
      context.Assert.Success();

      context.Complete();
    }

    /// <summary>
    /// This is needed because in Silverlight the tests cannot be run in separate AppDomains
    /// therefore the expected value is different for subsequent runs of this test.
    /// </summary>
    private static bool _initialized = false;

    [TestMethod]
    public void NoDoubleInit()
    {
      UnitTestContext context = GetContext();
      ApplicationContext.GlobalContext.Clear();
      ApplicationContext.GlobalContext["Shared"] = 0;

      HasPerTypeRules2 root = new HasPerTypeRules2();
      root = new HasPerTypeRules2();

      int expected = (_initialized ? 0 : 1);
      int actual = (int)ApplicationContext.GlobalContext["Shared"];
      context.Assert.AreEqual(expected, actual, "Rules should init just once");

      _initialized = true;
      context.Assert.Success();
      context.Complete();
    }

    [TestMethod]
    public void LoadRuleSets()
    {
      UnitTestContext context = GetContext();
      ApplicationContext.GlobalContext.Clear();

      var root = new HasRuleSetRules();

      var d = root.GetDefaultRules();
      context.Assert.AreEqual(1, d.Length);

      root.Name = "abc";
      context.Assert.IsTrue(root.IsValid, "valid with name");
      root.Name = "123456";
      context.Assert.IsTrue(root.IsValid, "valid with long name");
      root.Name = "";
      context.Assert.IsFalse(root.IsValid, "not valid with empty name");

      var t = root.GetTestRules();
      context.Assert.AreEqual(2, t.Length);

      root.Name = "abc";
      context.Assert.IsTrue(root.IsValid, "valid with name");
      root.Name = "123456";
      context.Assert.IsFalse(root.IsValid, "not valid with too long name");
      root.Name = "";
      context.Assert.IsFalse(root.IsValid, "not valid with empty name");

      context.Assert.Success();
      context.Complete();
    }
  }
}


text:
<?php

use yii\helpers\Html;
use yii\widgets\ActiveForm;

/* @var $this yii\web\View */
/* @var $model lukisongroup\purchasing\models\salesmanorder\SoT2 */
/* @var $form yii\widgets\ActiveForm */
?>

    <?php $form = ActiveForm::begin(); ?>

    <?= $form->field($model, 'TGL')->textInput() ?>

    <?= $form->field($model, 'CUST_KD')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'CUST_KD_ALIAS')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'CUST_NM')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'KD_BARANG')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'KD_BARANG_ALIAS')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'NM_BARANG')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'SO_QTY')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'SO_TYPE')->textInput() ?>

    <?= $form->field($model, 'POS')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'KD_DIS')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'NM_DIS')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'USER_ID')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'UNIT_BARANG')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'UNIT_QTY')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'UNIT_BERAT')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'HARGA_PABRIK')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'HARGA_DIS')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'HARGA_SALES')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'NOTED')->textarea(['rows' => 6]) ?>

    <?= $form->field($model, 'HARGA_LG')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'STATUS')->textInput() ?>

    <?= $form->field($model, 'WAKTU_INPUT_INVENTORY')->textInput() ?>

    <?= $form->field($model, 'ID_GROUP')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'KODE_REF')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'SUBMIT_QTY')->textInput(['maxlength' => true]) ?>

    <?= $form->field($model, 'SUBMIT_PRICE')->textInput(['maxlength' => true]) ?>

        <?= Html::submitButton($model->isNewRecord ? 'Create' : 'Update', ['class' => $model->isNewRecord ? 'btn btn-success' : 'btn btn-primary']) ?>

    <?php ActiveForm::end(); ?>


text:
package sched // import "bosun.org/cmd/bosun/sched"

import (
	"encoding/gob"
	"fmt"
	"reflect"
	"strings"
	"sync"
	"time"

	"golang.org/x/net/context"

	"bosun.org/cmd/bosun/cache"
	"bosun.org/cmd/bosun/conf"
	"bosun.org/cmd/bosun/database"
	"bosun.org/cmd/bosun/expr"
	"bosun.org/cmd/bosun/search"
	"bosun.org/collect"
	"bosun.org/metadata"
	"bosun.org/models"
	"bosun.org/opentsdb"
	"bosun.org/slog"
	"github.com/MiniProfiler/go/miniprofiler"
	"github.com/boltdb/bolt"
	"github.com/bradfitz/slice"
	"github.com/kylebrandt/boolq"
)

func utcNow() time.Time {
	return time.Now().UTC()
}

func init() {
	gob.Register(expr.Number(0))
	gob.Register(expr.Scalar(0))
}

type Schedule struct {
	mutex         sync.Mutex
	mutexHolder   string
	mutexAquired  time.Time
	mutexWaitTime int64

	RuleConf   conf.RuleConfProvider
	SystemConf conf.SystemConfProvider
	Group      map[time.Time]models.AlertKeys

	Search *search.Search

	skipLast bool
	quiet    bool

	//channel signals an alert has added notifications, and notifications should be processed.
	nc chan interface{}
	//notifications to be sent immediately
	pendingNotifications map[*conf.Notification][]*models.IncidentState

	//unknown states that need to be notified about. Collected and sent in batches.
	pendingUnknowns map[*conf.Notification][]*models.IncidentState

	db *bolt.DB

	lastLogTimes map[models.AlertKey]time.Time
	LastCheck    time.Time

	ctx *checkContext

	DataAccess database.DataAccess

	// runnerContext is a context to track running alert routines
	runnerContext context.Context
	// cancelChecks is the function to call to cancel all alert routines
	cancelChecks context.CancelFunc
	// checksRunning waits for alert checks to finish before reloading
	// things that take significant time should be cancelled (i.e. expression execution)
	// whereas the runHistory is allowed to complete
	checksRunning sync.WaitGroup
}

func (s *Schedule) Init(systemConf conf.SystemConfProvider, ruleConf conf.RuleConfProvider, skipLast, quiet bool) error {
	//initialize all variables and collections so they are ready to use.
	//this will be called once at app start, and also every time the rule
	//page runs, so be careful not to spawn long running processes that can't
	//be avoided.
	//var err error
	s.skipLast = skipLast
	s.quiet = quiet
	s.SystemConf = systemConf
	s.RuleConf = ruleConf
	s.Group = make(map[time.Time]models.AlertKeys)
	s.pendingUnknowns = make(map[*conf.Notification][]*models.IncidentState)
	s.lastLogTimes = make(map[models.AlertKey]time.Time)
	s.LastCheck = utcNow()
	s.ctx = &checkContext{utcNow(), cache.New(0)}

	// Initialize the context and waitgroup used to gracefully shutdown bosun as well as reload
	s.runnerContext, s.cancelChecks = context.WithCancel(context.Background())
	s.checksRunning = sync.WaitGroup{}

	if s.DataAccess == nil {
		if systemConf.GetRedisHost() != "" {
			s.DataAccess = database.NewDataAccess(systemConf.GetRedisHost(), true, systemConf.GetRedisDb(), systemConf.GetRedisPassword())
		} else {
			_, err := database.StartLedis(systemConf.GetLedisDir(), systemConf.GetLedisBindAddr())
			if err != nil {
				return err
			}
			s.DataAccess = database.NewDataAccess(systemConf.GetLedisBindAddr(), false, 0, "")
		}
	}
	if s.Search == nil {
		s.Search = search.NewSearch(s.DataAccess, skipLast)
	}
	return nil
}

type checkContext struct {
	runTime    time.Time
	checkCache *cache.Cache
}

func init() {
	metadata.AddMetricMeta(
		"bosun.schedule.lock_time", metadata.Counter, metadata.MilliSecond,
		"Length of time spent waiting for or holding the schedule lock.")
	metadata.AddMetricMeta(
		"bosun.schedule.lock_count", metadata.Counter, metadata.Count,
		"Number of times the given caller acquired the lock.")
}

func (s *Schedule) Lock(method string) {
	start := utcNow()
	s.mutex.Lock()
	s.mutexAquired = utcNow()
	s.mutexHolder = method
	s.mutexWaitTime = int64(s.mutexAquired.Sub(start) / time.Millisecond) // remember this so we don't have to call put until we leave the critical section.
}

func (s *Schedule) Unlock() {
	holder := s.mutexHolder
	start := s.mutexAquired
	waitTime := s.mutexWaitTime
	s.mutexHolder = ""
	s.mutex.Unlock()
	collect.Add("schedule.lock_time", opentsdb.TagSet{"caller": holder, "op": "wait"}, waitTime)
	collect.Add("schedule.lock_time", opentsdb.TagSet{"caller": holder, "op": "hold"}, int64(time.Since(start)/time.Millisecond))
	collect.Add("schedule.lock_count", opentsdb.TagSet{"caller": holder}, 1)
}

func (s *Schedule) GetLockStatus() (holder string, since time.Time) {
	return s.mutexHolder, s.mutexAquired
}

func (s *Schedule) PutMetadata(k metadata.Metakey, v interface{}) error {

	isCoreMeta := (k.Name == "desc" || k.Name == "unit" || k.Name == "rate")
	if !isCoreMeta {
		s.DataAccess.Metadata().PutTagMetadata(k.TagSet(), k.Name, fmt.Sprint(v), utcNow())
		return nil
	}
	if k.Metric == "" {
		err := fmt.Errorf("desc, rate, and unit require metric name")
		slog.Error(err)
		return err
	}
	strVal, ok := v.(string)
	if !ok {
		err := fmt.Errorf("desc, rate, and unit require value to be string. Found: %s", reflect.TypeOf(v))
		slog.Error(err)
		return err
	}
	return s.DataAccess.Metadata().PutMetricMetadata(k.Metric, k.Name, strVal)
}

func (s *Schedule) DeleteMetadata(tags opentsdb.TagSet, name string) error {
	return s.DataAccess.Metadata().DeleteTagMetadata(tags, name)
}

func (s *Schedule) MetadataMetrics(metric string) (*database.MetricMetadata, error) {
	//denormalized metrics should give metric metadata for their undenormalized counterparts
	if strings.HasPrefix(metric, "__") {
		if idx := strings.Index(metric, "."); idx != -1 {
			metric = metric[idx+1:]
		}
	}
	mm, err := s.DataAccess.Metadata().GetMetricMetadata(metric)
	if err != nil {
		return nil, err
	}
	return mm, nil
}

func (s *Schedule) GetMetadata(metric string, subset opentsdb.TagSet) ([]metadata.Metasend, error) {
	ms := make([]metadata.Metasend, 0)
	if metric != "" {
		meta, err := s.MetadataMetrics(metric)
		if err != nil {
			return nil, err
		}
		if meta.Desc != "" {
			ms = append(ms, metadata.Metasend{
				Metric: metric,
				Name:   "desc",
				Value:  meta.Desc,
			})
		}
		if meta.Unit != "" {
			ms = append(ms, metadata.Metasend{
				Metric: metric,
				Name:   "unit",
				Value:  meta.Unit,
			})
		}
		if meta.Rate != "" {
			ms = append(ms, metadata.Metasend{
				Metric: metric,
				Name:   "rate",
				Value:  meta.Rate,
			})
		}
	} else {
		meta, err := s.DataAccess.Metadata().GetTagMetadata(subset, "")
		if err != nil {
			return nil, err
		}
		for _, m := range meta {
			tm := time.Unix(m.LastTouched, 0)
			ms = append(ms, metadata.Metasend{
				Tags:  m.Tags,
				Name:  m.Name,
				Value: m.Value,
				Time:  &tm,
			})
		}
	}
	return ms, nil
}

type States map[models.AlertKey]*models.IncidentState

type StateTuple struct {
	NeedAck       bool
	Active        bool
	Status        models.Status
	CurrentStatus models.Status
	Silenced      bool
}

// GroupStates groups by NeedAck, Active, Status, and Silenced.
func (states States) GroupStates(silenced SilenceTester) map[StateTuple]States {
	r := make(map[StateTuple]States)
	for ak, st := range states {
		sil := silenced(ak) != nil
		t := StateTuple{
			NeedAck:       st.NeedAck,
			Active:        st.IsActive(),
			Status:        st.LastAbnormalStatus,
			CurrentStatus: st.CurrentStatus,
			Silenced:      sil,
		}
		if _, present := r[t]; !present {
			r[t] = make(States)
		}
		r[t][ak] = st
	}
	return r
}

// GroupSets returns slices of TagSets, grouped by most common ancestor. Those
// with no shared ancestor are grouped by alert name.
func (states States) GroupSets(minGroup int) map[string]models.AlertKeys {
	type Pair struct {
		k, v string
	}
	groups := make(map[string]models.AlertKeys)
	seen := make(map[*models.IncidentState]bool)
	for {
		counts := make(map[Pair]int)
		for _, s := range states {
			if seen[s] {
				continue
			}
			for k, v := range s.AlertKey.Group() {
				counts[Pair{k, v}]++
			}
		}
		if len(counts) == 0 {
			break
		}
		max := 0
		var pair Pair
		for p, c := range counts {
			if c > max {
				max = c
				pair = p
			}
		}
		if max < minGroup {
			break
		}
		var group models.AlertKeys
		for _, s := range states {
			if seen[s] {
				continue
			}
			if s.AlertKey.Group()[pair.k] != pair.v {
				continue
			}
			seen[s] = true
			group = append(group, s.AlertKey)
		}
		if len(group) > 0 {
			groups[fmt.Sprintf("{%s=%s}", pair.k, pair.v)] = group
		}
	}
	// alerts
	groupedByAlert := map[string]models.AlertKeys{}
	for _, s := range states {
		if seen[s] {
			continue
		}
		groupedByAlert[s.Alert] = append(groupedByAlert[s.Alert], s.AlertKey)
	}
	for a, aks := range groupedByAlert {
		if len(aks) >= minGroup {
			group := models.AlertKeys{}
			for _, ak := range aks {
				group = append(group, ak)
			}
			groups[a] = group
		}
	}
	// ungrouped
	for _, s := range states {
		if seen[s] || len(groupedByAlert[s.Alert]) >= minGroup {
			continue
		}
		groups[string(s.AlertKey)] = models.AlertKeys{s.AlertKey}
	}
	return groups
}

func (s *Schedule) GetOpenStates() (States, error) {
	incidents, err := s.DataAccess.State().GetAllOpenIncidents()
	if err != nil {
		return nil, err
	}
	states := make(States, len(incidents))
	for _, inc := range incidents {
		states[inc.AlertKey] = inc
	}
	return states, nil
}

type StateGroup struct {
	Active        bool `json:",omitempty"`
	Status        models.Status
	CurrentStatus models.Status
	Silenced      bool
	IsError       bool                  `json:",omitempty"`
	Subject       string                `json:",omitempty"`
	Alert         string                `json:",omitempty"`
	AlertKey      models.AlertKey       `json:",omitempty"`
	Ago           string                `json:",omitempty"`
	State         *models.IncidentState `json:",omitempty"`
	Children      []*StateGroup         `json:",omitempty"`
}

type StateGroups struct {
	Groups struct {
		NeedAck      []*StateGroup `json:",omitempty"`
		Acknowledged []*StateGroup `json:",omitempty"`
	}
	TimeAndDate                   []int
	FailingAlerts, UnclosedErrors int
}

func (s *Schedule) MarshalGroups(T miniprofiler.Timer, filter string) (*StateGroups, error) {
	var silenced SilenceTester
	T.Step("Silenced", func(miniprofiler.Timer) {
		silenced = s.Silenced()
	})
	var groups map[StateTuple]States
	var err error
	status := make(States)
	t := StateGroups{
		TimeAndDate: s.SystemConf.GetTimeAndDate(),
	}
	t.FailingAlerts, t.UnclosedErrors = s.getErrorCounts()
	T.Step("Setup", func(miniprofiler.Timer) {
		status2, err2 := s.GetOpenStates()
		if err2 != nil {
			err = err2
			return
		}
		var parsedExpr *boolq.Tree
		parsedExpr, err2 = boolq.Parse(filter)
		if err2 != nil {
			err = err2
			return
		}
		for k, v := range status2 {
			a := s.RuleConf.GetAlert(k.Name())
			if a == nil {
				slog.Errorf("unknown alert %s. Force closing.", k.Name())
				if err2 = s.ActionByAlertKey("bosun", "closing because alert doesn't exist.", models.ActionForceClose, k); err2 != nil {
					slog.Error(err2)
				}
				continue
			}
			is, err2 := MakeIncidentSummary(s.RuleConf, silenced, v)
			if err2 != nil {
				err = err2
				return
			}
			match := false
			match, err2 = boolq.AskParsedExpr(parsedExpr, is)
			if err2 != nil {
				err = err2
				return
			}
			if match {
				status[k] = v
			}
		}
	})
	if err != nil {
		return nil, err
	}
	T.Step("GroupStates", func(T miniprofiler.Timer) {
		groups = status.GroupStates(silenced)
	})
	T.Step("groups", func(T miniprofiler.Timer) {
		for tuple, states := range groups {
			var grouped []*StateGroup
			switch tuple.Status {
			case models.StWarning, models.StCritical, models.StUnknown:
				var sets map[string]models.AlertKeys
				T.Step(fmt.Sprintf("GroupSets (%d): %v", len(states), tuple), func(T miniprofiler.Timer) {
					sets = states.GroupSets(s.SystemConf.GetMinGroupSize())
				})
				for name, group := range sets {
					g := StateGroup{
						Active:        tuple.Active,
						Status:        tuple.Status,
						CurrentStatus: tuple.CurrentStatus,
						Silenced:      tuple.Silenced,
						Subject:       fmt.Sprintf("%s - %s", tuple.Status, name),
					}
					for _, ak := range group {
						st := status[ak]
						st.Body = ""
						st.EmailBody = nil
						st.Attachments = nil
						g.Children = append(g.Children, &StateGroup{
							Active:   tuple.Active,
							Status:   tuple.Status,
							Silenced: tuple.Silenced,
							AlertKey: ak,
							Alert:    ak.Name(),
							Subject:  string(st.Subject),
							Ago:      marshalTime(st.Last().Time),
							State:    st,
							IsError:  !s.AlertSuccessful(ak.Name()),
						})
					}
					if len(g.Children) == 1 && g.Children[0].Subject != "" {
						g.Subject = g.Children[0].Subject
					}
					grouped = append(grouped, &g)
				}
			default:
				continue
			}
			if tuple.NeedAck {
				t.Groups.NeedAck = append(t.Groups.NeedAck, grouped...)
			} else {
				t.Groups.Acknowledged = append(t.Groups.Acknowledged, grouped...)
			}
		}
	})
	T.Step("sort", func(T miniprofiler.Timer) {
		gsort := func(grp []*StateGroup) func(i, j int) bool {
			return func(i, j int) bool {
				a := grp[i]
				b := grp[j]
				if a.Active && !b.Active {
					return true
				} else if !a.Active && b.Active {
					return false
				}
				if a.Status != b.Status {
					return a.Status > b.Status
				}
				if a.AlertKey != b.AlertKey {
					return a.AlertKey < b.AlertKey
				}
				return a.Subject < b.Subject
			}
		}
		slice.Sort(t.Groups.NeedAck, gsort(t.Groups.NeedAck))
		slice.Sort(t.Groups.Acknowledged, gsort(t.Groups.Acknowledged))
	})
	return &t, nil
}

func marshalTime(t time.Time) string {
	if t.IsZero() {
		return ""
	}
	b, _ := t.MarshalText()
	return string(b)
}

var DefaultSched = &Schedule{}

// Load loads a configuration into the default schedule.
func Load(systemConf conf.SystemConfProvider, ruleConf conf.RuleConfProvider, skipLast, quiet bool) error {
	return DefaultSched.Load(systemConf, ruleConf, skipLast, quiet)
}

// Run runs the default schedule.
func Run() error {
	return DefaultSched.Run()
}

func (s *Schedule) Load(systemConf conf.SystemConfProvider, ruleConf conf.RuleConfProvider, skipLast, quiet bool) error {
	if err := s.Init(systemConf, ruleConf, skipLast, quiet); err != nil {
		return err
	}
	if s.db == nil {
		return nil
	}
	return nil
}

func Close(reload bool) {
	DefaultSched.Close(reload)
}

func (s *Schedule) Close(reload bool) {
	s.cancelChecks()
	s.checksRunning.Wait()
	if s.skipLast || reload {
		return
	}
	err := s.Search.BackupLast()
	if err != nil {
		slog.Error(err)
	}
}

func (s *Schedule) Reset() {
	DefaultSched = &Schedule{}
}

func Reset() {
	DefaultSched.Reset()
}

func init() {
	metadata.AddMetricMeta("bosun.statefile.size", metadata.Gauge, metadata.Bytes,
		"The total size of the Bosun state file.")
	metadata.AddMetricMeta("bosun.check.duration", metadata.Gauge, metadata.Second,
		"The number of seconds it took Bosun to check each alert rule.")
	metadata.AddMetricMeta("bosun.check.err", metadata.Gauge, metadata.Error,
		"The running count of the number of errors Bosun has received while trying to evaluate an alert expression.")

	metadata.AddMetricMeta("bosun.actions", metadata.Gauge, metadata.Count,
		"The running count of actions performed by individual users (Closed alert, Acknowledged alert, etc).")
}

func (s *Schedule) ActionByAlertKey(user, message string, t models.ActionType, ak models.AlertKey) error {
	st, err := s.DataAccess.State().GetLatestIncident(ak)
	if err != nil {
		return err
	}
	if st == nil {
		return fmt.Errorf("no such alert key: %v", ak)
	}
	_, err = s.action(user, message, t, st)
	return err
}

func (s *Schedule) ActionByIncidentId(user, message string, t models.ActionType, id int64) (models.AlertKey, error) {
	st, err := s.DataAccess.State().GetIncidentState(id)
	if err != nil {
		return "", err
	}
	if st == nil {
		return "", fmt.Errorf("no incident with id: %v", id)
	}
	return s.action(user, message, t, st)
}

func (s *Schedule) action(user, message string, t models.ActionType, st *models.IncidentState) (ak models.AlertKey, e error) {
	if err := collect.Add("actions", opentsdb.TagSet{"user": user, "alert": st.AlertKey.Name(), "type": t.String()}, 1); err != nil {
		slog.Errorln(err)
	}
	defer func() {
		if e == nil {
			if err := collect.Add("actions", opentsdb.TagSet{"user": user, "alert": st.AlertKey.Name(), "type": t.String()}, 1); err != nil {
				slog.Errorln(err)
			}
			if err := s.DataAccess.Notifications().ClearNotifications(st.AlertKey); err != nil {
				e = err
			}
		}
	}()
	isUnknown := st.LastAbnormalStatus == models.StUnknown
	timestamp := utcNow()
	switch t {
	case models.ActionAcknowledge:
		if !st.NeedAck {
			return "", fmt.Errorf("alert already acknowledged")
		}
		if !st.Open {
			return "", fmt.Errorf("cannot acknowledge closed alert")
		}
		st.NeedAck = false
	case models.ActionClose:
		if st.IsActive() {
			return "", fmt.Errorf("cannot close active alert")
		}
		fallthrough
	case models.ActionForceClose:
		st.Open = false
		st.End = &timestamp
	case models.ActionForget:
		if !isUnknown {
			return "", fmt.Errorf("can only forget unknowns")
		}
		fallthrough
	case models.ActionPurge:
		return st.AlertKey, s.DataAccess.State().Forget(st.AlertKey)
	case models.ActionNote:
		// pass
	default:
		return "", fmt.Errorf("unknown action type: %v", t)
	}
	st.Actions = append(st.Actions, models.Action{
		Message: message,
		Time:    timestamp,
		Type:    t,
		User:    user,
	})
	_, err := s.DataAccess.State().UpdateIncidentState(st)
	return st.AlertKey, err
}

type IncidentStatus struct {
	IncidentID         int64
	Active             bool
	AlertKey           models.AlertKey
	Status             models.Status
	StatusTime         int64
	Subject            string
	Silenced           bool
	LastAbnormalStatus models.Status
	LastAbnormalTime   int64
	NeedsAck           bool
}

func (s *Schedule) AlertSuccessful(name string) bool {
	b, err := s.DataAccess.Errors().IsAlertFailing(name)
	if err != nil {
		slog.Error(err)
		b = true
	}
	return !b
}

func (s *Schedule) markAlertError(name string, e error) {
	d := s.DataAccess.Errors()
	if err := d.MarkAlertFailure(name, e.Error()); err != nil {
		slog.Error(err)
		return
	}

}

func (s *Schedule) markAlertSuccessful(name string) {
	if err := s.DataAccess.Errors().MarkAlertSuccess(name); err != nil {
		slog.Error(err)
	}
}

func (s *Schedule) ClearErrors(alert string) error {
	if alert == "all" {
		return s.DataAccess.Errors().ClearAll()
	}
	return s.DataAccess.Errors().ClearAlert(alert)
}

func (s *Schedule) getErrorCounts() (failing, total int) {
	var err error
	failing, total, err = s.DataAccess.Errors().GetFailingAlertCounts()
	if err != nil {
		slog.Error(err)
	}
	return
}

func (s *Schedule) GetQuiet() bool {
	return s.quiet
}


text:
package client

import (
	"net/url"
	"testing"

	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
	"github.com/GoogleCloudPlatform/kubernetes/pkg/api/resource"
	"github.com/GoogleCloudPlatform/kubernetes/pkg/api/testapi"
	"github.com/GoogleCloudPlatform/kubernetes/pkg/fields"
	"github.com/GoogleCloudPlatform/kubernetes/pkg/labels"
)

func getLimitRangesResourceName() string {
	if api.PreV1Beta3(testapi.Version()) {
		return "limitRanges"
	}
	return "limitranges"
}

func TestLimitRangeCreate(t *testing.T) {
	ns := api.NamespaceDefault
	limitRange := &api.LimitRange{
		ObjectMeta: api.ObjectMeta{
			Name: "abc",
		},
		Spec: api.LimitRangeSpec{
			Limits: []api.LimitRangeItem{
				{
					Type: api.LimitTypePod,
					Max: api.ResourceList{
						api.ResourceCPU:    resource.MustParse("100"),
						api.ResourceMemory: resource.MustParse("10000"),
					},
					Min: api.ResourceList{
						api.ResourceCPU:    resource.MustParse("0"),
						api.ResourceMemory: resource.MustParse("100"),
					},
				},
			},
		},
	}
	c := &testClient{
		Request: testRequest{
			Method: "POST",
			Path:   testapi.ResourcePath(getLimitRangesResourceName(), ns, ""),
			Query:  buildQueryValues(ns, nil),
			Body:   limitRange,
		},
		Response: Response{StatusCode: 200, Body: limitRange},
	}

	response, err := c.Setup().LimitRanges(ns).Create(limitRange)
	c.Validate(t, response, err)
}

func TestLimitRangeGet(t *testing.T) {
	ns := api.NamespaceDefault
	limitRange := &api.LimitRange{
		ObjectMeta: api.ObjectMeta{
			Name: "abc",
		},
		Spec: api.LimitRangeSpec{
			Limits: []api.LimitRangeItem{
				{
					Type: api.LimitTypePod,
					Max: api.ResourceList{
						api.ResourceCPU:    resource.MustParse("100"),
						api.ResourceMemory: resource.MustParse("10000"),
					},
					Min: api.ResourceList{
						api.ResourceCPU:    resource.MustParse("0"),
						api.ResourceMemory: resource.MustParse("100"),
					},
				},
			},
		},
	}
	c := &testClient{
		Request: testRequest{
			Method: "GET",
			Path:   testapi.ResourcePath(getLimitRangesResourceName(), ns, "abc"),
			Query:  buildQueryValues(ns, nil),
			Body:   nil,
		},
		Response: Response{StatusCode: 200, Body: limitRange},
	}

	response, err := c.Setup().LimitRanges(ns).Get("abc")
	c.Validate(t, response, err)
}

func TestLimitRangeList(t *testing.T) {
	ns := api.NamespaceDefault

	limitRangeList := &api.LimitRangeList{
		Items: []api.LimitRange{
			{
				ObjectMeta: api.ObjectMeta{Name: "foo"},
			},
		},
	}
	c := &testClient{
		Request: testRequest{
			Method: "GET",
			Path:   testapi.ResourcePath(getLimitRangesResourceName(), ns, ""),
			Query:  buildQueryValues(ns, nil),
			Body:   nil,
		},
		Response: Response{StatusCode: 200, Body: limitRangeList},
	}
	response, err := c.Setup().LimitRanges(ns).List(labels.Everything())
	c.Validate(t, response, err)
}

func TestLimitRangeUpdate(t *testing.T) {
	ns := api.NamespaceDefault
	limitRange := &api.LimitRange{
		ObjectMeta: api.ObjectMeta{
			Name:            "abc",
			ResourceVersion: "1",
		},
		Spec: api.LimitRangeSpec{
			Limits: []api.LimitRangeItem{
				{
					Type: api.LimitTypePod,
					Max: api.ResourceList{
						api.ResourceCPU:    resource.MustParse("100"),
						api.ResourceMemory: resource.MustParse("10000"),
					},
					Min: api.ResourceList{
						api.ResourceCPU:    resource.MustParse("0"),
						api.ResourceMemory: resource.MustParse("100"),
					},
				},
			},
		},
	}
	c := &testClient{
		Request:  testRequest{Method: "PUT", Path: testapi.ResourcePath(getLimitRangesResourceName(), ns, "abc"), Query: buildQueryValues(ns, nil)},
		Response: Response{StatusCode: 200, Body: limitRange},
	}
	response, err := c.Setup().LimitRanges(ns).Update(limitRange)
	c.Validate(t, response, err)
}

func TestInvalidLimitRangeUpdate(t *testing.T) {
	ns := api.NamespaceDefault
	limitRange := &api.LimitRange{
		ObjectMeta: api.ObjectMeta{
			Name: "abc",
		},
		Spec: api.LimitRangeSpec{
			Limits: []api.LimitRangeItem{
				{
					Type: api.LimitTypePod,
					Max: api.ResourceList{
						api.ResourceCPU:    resource.MustParse("100"),
						api.ResourceMemory: resource.MustParse("10000"),
					},
					Min: api.ResourceList{
						api.ResourceCPU:    resource.MustParse("0"),
						api.ResourceMemory: resource.MustParse("100"),
					},
				},
			},
		},
	}
	c := &testClient{
		Request:  testRequest{Method: "PUT", Path: testapi.ResourcePath(getLimitRangesResourceName(), ns, "abc"), Query: buildQueryValues(ns, nil)},
		Response: Response{StatusCode: 200, Body: limitRange},
	}
	_, err := c.Setup().LimitRanges(ns).Update(limitRange)
	if err == nil {
		t.Errorf("Expected an error due to missing ResourceVersion")
	}
}

func TestLimitRangeDelete(t *testing.T) {
	ns := api.NamespaceDefault
	c := &testClient{
		Request:  testRequest{Method: "DELETE", Path: testapi.ResourcePath(getLimitRangesResourceName(), ns, "foo"), Query: buildQueryValues(ns, nil)},
		Response: Response{StatusCode: 200},
	}
	err := c.Setup().LimitRanges(ns).Delete("foo")
	c.Validate(t, nil, err)
}

func TestLimitRangeWatch(t *testing.T) {
	c := &testClient{
		Request: testRequest{
			Method: "GET",
			Path:   "/api/" + testapi.Version() + "/watch/" + getLimitRangesResourceName(),
			Query:  url.Values{"resourceVersion": []string{}}},
		Response: Response{StatusCode: 200},
	}
	_, err := c.Setup().LimitRanges(api.NamespaceAll).Watch(labels.Everything(), fields.Everything(), "")
	c.Validate(t, nil, err)
}


text:
package fi.iki.elonen.integration;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.StatusLine;
import org.apache.http.client.HttpResponseException;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.BasicHttpEntity;
import org.apache.http.entity.ByteArrayEntity;
import org.apache.http.util.EntityUtils;
import org.junit.Assert;
import org.junit.Test;

import com.ericsson.research.trap.nhttpd.Request;
import com.ericsson.research.trap.nhttpd.RequestHandler;
import com.ericsson.research.trap.nhttpd.Response;
import com.ericsson.research.trap.nhttpd.StatusCodes;
import com.ericsson.research.trap.nhttpd.impl.NanoHTTPDImpl;

/**
 * @author Paul S. Hawke (paul.hawke@gmail.com) On: 5/19/13 at 5:36 PM
 */
public class POSTIntegrationTests extends IntegrationTestBase<POSTIntegrationTests.TestServer>
{

	public class BasicResponseHandler implements ResponseHandler<byte[]> {

	    /**
	     * Returns the response body as a String if the response was successful (a
	     * 2xx status code). If no response body exists, this returns null. If the
	     * response was unsuccessful (>= 300 status code), throws an
	     * {@link HttpResponseException}.
	     */
	    public byte[] handleResponse(final HttpResponse response)
	            throws HttpResponseException, IOException {
	        StatusLine statusLine = response.getStatusLine();
	        HttpEntity entity = response.getEntity();
	        if (statusLine.getStatusCode() >= 300) {
	            EntityUtils.consume(entity);
	            throw new HttpResponseException(statusLine.getStatusCode(),
	                    statusLine.getReasonPhrase());
	        }
	        return entity == null ? null : EntityUtils.toByteArray(entity);
	    }

	}

	@Test
	public void testLargePost() throws Exception
	{
		testServer.chunked = false;
		byte[] testData = new byte[128*1024];
		for (int i=0; i<testData.length; i++)
			testData[i] = (byte) (i%255);

		HttpPost post = new HttpPost("http://localhost:8192/");
		ResponseHandler<byte[]> responseHandler = new BasicResponseHandler();

		post.setEntity(new ByteArrayEntity(testData));

		byte[] responseBody = httpclient.execute(post, responseHandler);

		Assert.assertArrayEquals(testData, responseBody);
	}

	@Test
	public void testLargePostChunked() throws Exception
	{

		byte[] testData = new byte[128*1024];
		for (int i=0; i<testData.length; i++)
			testData[i] = (byte) (i%255);

		HttpPost post = new HttpPost("http://localhost:8192/");
		ResponseHandler<byte[]> responseHandler = new BasicResponseHandler();

		BasicHttpEntity entity = new BasicHttpEntity();
		entity.setChunked(true);
		entity.setContent(new ByteArrayInputStream(testData));
		post.setEntity(entity);

		byte[] responseBody = httpclient.execute(post, responseHandler);

		Assert.assertArrayEquals(testData, responseBody);
	}

	@Test
	public void testLargePostChunkedReply() throws Exception
	{

		testServer.chunked = true;
		byte[] testData = new byte[128*1024];
		for (int i=0; i<testData.length; i++)
			testData[i] = (byte) (i%255);

		HttpPost post = new HttpPost("http://localhost:8192/");
		ResponseHandler<byte[]> responseHandler = new BasicResponseHandler();

		post.setEntity(new ByteArrayEntity(testData));

		byte[] responseBody = httpclient.execute(post, responseHandler);

		Assert.assertArrayEquals(testData, responseBody);
	}

	@Test
	public void testPipelinedPost() throws Exception
	{
		for (int i=0; i<10; i++)
			testLargePost();
	}

	@Override
	public TestServer createTestServer()
	{
		return new TestServer();
	}

	public static class TestServer extends NanoHTTPDImpl implements RequestHandler
	{
		public boolean	chunked;

		public TestServer()
		{
			super(8192);
            setHandler(this);
		}

		@Override
		public void handleRequest(Request session, Response r)
		{

			try
			{
				InputStream is = session.getInputStream();
				byte[] bs = new byte[4096];
				ByteArrayOutputStream bos = new ByteArrayOutputStream();
				int read = 0;
				while ((read = is.read(bs)) > -1)
					bos.write(bs, 0, read);

				r.setStatus(StatusCodes.OK);

				if (chunked)
					r.setData(new ByteArrayInputStream(bos.toByteArray()));
				else
					r.setData(bos.toByteArray());
			}
			catch (IOException e)
			{
				throw new RuntimeException(e);
			}
		}

	}
}