text:
<?xml version="1.0" ?>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

<link rel="stylesheet" type="text/css" href="../../scineplex.css">
<title></title>
<link rel="stylesheet" href="../../Active.css" type="text/css" />
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<link rev="made" href="mailto:" />

NAME

SYNOPSIS

DESCRIPTION

OVERVIEW

ENABLING A PROFILE

THE PROFILE OBJECT

Data

Path

Special Constant

Code Reference

Subroutine Specifier

Attribute Specifier

Reference to a Scalar

Other Values

REPORTING

Report Format

Report Destination

CHILD HANDLES

PROFILE OBJECT METHODS

format

as_node_path_list

as_text

CUSTOM DATA MANIPULATION

dbi_profile_merge_nodes

CUSTOM DATA COLLECTION

Using The Path Attribute

Adding Your Own Samples

SUBCLASSING

CAVEATS

NAME

DBI::Profile - Performance profiling and benchmarking for the DBI

SYNOPSIS

The easiest way to enable DBI profiling is to set the DBI_PROFILE environment variable to 2 and then run your code as usual:

  DBI_PROFILE=2 prog.pl

This will profile your program and then output a textual summary grouped by query when the program exits. You can also enable profiling by setting the Profile attribute of any DBI handle:

  $dbh->{Profile} = 2;

Then the summary will be printed when the handle is destroyed.

Many other values apart from are possible - see "ENABLING A PROFILE" below.

DESCRIPTION

The DBI::Profile module provides a simple interface to collect and report performance and benchmarking data from the DBI.

For a more elaborate interface, suitable for larger programs, see DBI::ProfileDumper and dbiprof. For Apache/mod_perl applications see DBI::ProfileDumper::Apache.

OVERVIEW

Performance data collection for the DBI is built around several concepts which are important to understand clearly.

Method Dispatch

Every method call on a DBI handle passes through a single 'dispatch' function which manages all the common aspects of DBI method calls, such as handling the RaiseError attribute.

Data Collection

If profiling is enabled for a handle then the dispatch code takes a high-resolution timestamp soon after it is entered. Then, after calling the appropriate method and just before returning, it takes another high-resolution timestamp and calls a function to record the information. That function is passed the two timestamps plus the DBI handle and the name of the method that was called. That data about a single DBI method call is called a profile sample.

Data Filtering

If the method call was invoked by the DBI or by a driver then the call is ignored for profiling because the time spent will be accounted for by the original 'outermost' call for your code.

For example, the calls that the selectrow_arrayref() method makes to prepare() and execute() etc. are not counted individually because the time spent in those methods is going to be allocated to the selectrow_arrayref() method when it returns. If this was not done then it would be very easy to double count time spent inside the DBI.

Data Storage Tree

The profile data is accumulated as 'leaves on a tree'. The 'path' through the branches of the tree to a particular leaf is determined dynamically for each sample. This is a key feature of DBI profiling.

For each profiled method call the DBI walks along the Path and uses each value in the Path to step into and grow the Data tree.

For example, if the Path is

  [ 'foo', 'bar', 'baz' ]

then the new profile sample data will be merged into the tree at

  $h->{Profile}->{Data}->{foo}->{bar}->{baz}

But it's not very useful to merge all the call data into one leaf node (except to get an overall 'time spent inside the DBI' total). It's more common to want the Path to include dynamic values such as the current statement text and/or the name of the method called to show what the time spent inside the DBI was for.

The Path can contain some 'magic cookie' values that are automatically replaced by corresponding dynamic values when they're used. These magic cookies always start with a punctuation character.

For example a value of '!MethodName' in the Path causes the corresponding entry in the Data to be the name of the method that was called. For example, if the Path was:

  [ 'foo', '!MethodName', 'bar' ]

and the selectall_arrayref() method was called, then the profile sample data for that call will be merged into the tree at:

  $h->{Profile}->{Data}->{foo}->{selectall_arrayref}->{bar}

Profile Data

Profile data is stored at the 'leaves' of the tree as references to an array of numeric values. For example:

  [
    106,                  # 0: count of samples at this node
    0.0312958955764771,   # 1: total duration
    0.000490069389343262, # 2: first duration
    0.000176072120666504, # 3: shortest duration
    0.00140702724456787,  # 4: longest duration
    1023115819.83019,     # 5: time of first sample
    1023115819.86576,     # 6: time of last sample
  ]

After the first sample, later samples always update elements 0, 1, and 6, and may update 3 or 4 depending on the duration of the sampled call.

ENABLING A PROFILE

Profiling is enabled for a handle by assigning to the Profile attribute. For example:

  $h->{Profile} = DBI::Profile->new();

The Profile attribute holds a blessed reference to a hash object that contains the profile data and attributes relating to it.

The class the Profile object is blessed into is expected to provide at least a DESTROY method which will dump the profile data to the DBI trace file handle (STDERR by default).

All these examples have the same effect as each other:

  $h->{Profile} = 0;
  $h->{Profile} = "/DBI::Profile";
  $h->{Profile} = DBI::Profile->new();
  $h->{Profile} = {};
  $h->{Profile} = { Path => [] };

Similarly, these examples have the same effect as each other:

  $h->{Profile} = 6;
  $h->{Profile} = "6/DBI::Profile";
  $h->{Profile} = "!Statement:!MethodName/DBI::Profile";
  $h->{Profile} = { Path => [ '!Statement', '!MethodName' ] };

If a non-blessed hash reference is given then the DBI::Profile module is automatically require'd and the reference is blessed into that class.

If a string is given then it is processed like this:

    ($path, $module, $args) = split /\//, $string, 3

    @path = split /:/, $path
    @args = split /:/, $args

    eval "require $module" if $module
    $module ||= "DBI::Profile"

    $module->new( Path => \@Path, @args )

So the first value is used to select the Path to be used (see below). The second value, if present, is used as the name of a module which will be loaded and it's new method called. If not present it defaults to DBI::Profile. Any other values are passed as arguments to the new method. For example: "2/DBIx::OtherProfile/Foo:42".

Numbers can be used as a shorthand way to enable common Path values. The simplest way to explain how the values are interpreted is to show the code:

    push @Path, "DBI"           if $path_elem & 0x01;
    push @Path, "!Statement"    if $path_elem & 0x02;
    push @Path, "!MethodName"   if $path_elem & 0x04;
    push @Path, "!MethodClass"  if $path_elem & 0x08;
    push @Path, "!Caller2"      if $path_elem & 0x10;

So "2" is the same as "!Statement" and "6" (2+4) is the same as "!Statement:!Method". Those are the two most commonly used values. Using a negative number will reverse the path. Thus "-6" will group by method name then statement.

The splitting and parsing of string values assigned to the Profile attribute may seem a little odd, but there's a good reason for it. Remember that attributes can be embedded in the Data Source Name string which can be passed in to a script as a parameter. For example:

    dbi:DriverName(Profile=>2):dbname
    dbi:DriverName(Profile=>{Username}:!Statement/MyProfiler/Foo:42):dbname

And also, if the DBI_PROFILE environment variable is set then The DBI arranges for every driver handle to share the same profile object. When perl exits a single profile summary will be generated that reflects (as nearly as practical) the total use of the DBI by the application.

THE PROFILE OBJECT

The DBI core expects the Profile attribute value to be a hash reference and if the following values don't exist it will create them as needed:

Data

A reference to a hash containing the collected profile data.

Path

The Path value is a reference to an array. Each element controls the value to use at the corresponding level of the profile Data tree.

If the value of Path is anything other than an array reference, it is treated as if it was:

        [ '!Statement' ]

The elements of Path array can be one of the following types:

Special Constant

!Statement

Use the current Statement text. Typically that's the value of the Statement attribute for the handle the method was called with. Some methods, like commit() and rollback(), are unrelated to a particular statement. For those methods !Statement records an empty string.

For statement handles this is always simply the string that was given to prepare() when the handle was created. For database handles this is the statement that was last prepared or executed on that database handle. That can lead to a little 'fuzzyness' because, for example, calls to the quote() method to build a new statement will typically be associated with the previous statement. In practice this isn't a significant issue and the dynamic Path mechanism can be used to setup your own rules.

!MethodName

Use the name of the DBI method that the profile sample relates to.

!MethodClass

Use the fully qualified name of the DBI method, including the package, that the profile sample relates to. This shows you where the method was implemented. For example:

  'DBD::_::db::selectrow_arrayref' =>
      0.022902s
  'DBD::mysql::db::selectrow_arrayref' =>
      2.244521s / 99 = 0.022445s avg (first 0.022813s, min 0.022051s, max 0.028932s)

The "DBD::_::db::selectrow_arrayref" shows that the driver has inherited the selectrow_arrayref method provided by the DBI.

But you'll note that there is only one call to DBD::_::db::selectrow_arrayref but another 99 to DBD::mysql::db::selectrow_arrayref. Currently the first call doesn't record the true location. That may change.

!Caller

Use a string showing the filename and line number of the code calling the method.

!Caller2

Use a string showing the filename and line number of the code calling the method, as for !Caller, but also include filename and line number of the code that called that. Calls from DBI:: and DBD:: packages are skipped.

!File

Same as !Caller above except that only the filename is included, not the line number.

!File2

Same as !Caller2 above except that only the filenames are included, not the line number.

!Time

Use the current value of time(). Rarely used. See the more useful !Time~N below.

!Time~N

Where N is an integer. Use the current value of time() but with reduced precision. The value used is determined in this way:

    int( time() / N ) * N

This is a useful way to segregate a profile into time slots. For example:

    [ '!Time~60', '!Statement' ]

Code Reference

The subroutine is passed the handle it was called on and the DBI method name. The current Statement is in $_. The statement string should not be modified, so most subs start with local $_ = $_;
.

The list of values it returns is used at that point in the Profile Path.

The sub can 'veto' (reject) a profile sample by including a reference to undef in the returned list. That can be useful when you want to only profile statements that match a certain pattern, or only profile certain methods.

Subroutine Specifier

A Path element that begins with '&' is treated as the name of a subroutine in the DBI::ProfileSubs namespace and replaced with the corresponding code reference.

Currently this only works when the Path is specified by the DBI_PROFILE environment variable.

Also, currently, the only subroutine in the DBI::ProfileSubs namespace is '&norm_std_n3'. That's a very handy subroutine when profiling code that doesn't use placeholders. See DBI::ProfileSubs for more information.

Attribute Specifier

A string enclosed in braces, such as '{Username}
', specifies that the current value of the corresponding database handle attribute should be used at that point in the Path.

Reference to a Scalar

Specifies that the current value of the referenced scalar be used at that point in the Path. This provides an efficient way to get 'contextual' values into your profile.

Other Values

Any other values are stringified and used literally.

(References, and values that begin with punctuation characters are reserved.)

REPORTING

Report Format

The current accumulated profile data can be formatted and output using

    print $h->{Profile}->format;

To discard the profile data and start collecting fresh data you can do:

    $h->{Profile}->{Data} = undef;

The default results format looks like this:

  DBI::Profile: 0.001015s 42.7% (5 calls) programname @ YYYY-MM-DD HH:MM:SS
  '' =>
      0.000024s / 2 = 0.000012s avg (first 0.000015s, min 0.000009s, max 0.000015s)
  'SELECT mode,size,name FROM table' =>
      0.000991s / 3 = 0.000330s avg (first 0.000678s, min 0.000009s, max 0.000678s)

Which shows the total time spent inside the DBI, with a count of the total number of method calls and the name of the script being run, then a formatted version of the profile data tree.

If the results are being formatted when the perl process is exiting (which is usually the case when the DBI_PROFILE environment variable is used) then the percentage of time the process spent inside the DBI is also shown. If the process is not exiting then the percentage is calculated using the time between the first and last call to the DBI.

In the example above the paths in the tree are only one level deep and use the Statement text as the value (that's the default behaviour).

The merged profile data at the 'leaves' of the tree are presented as total time spent, count, average time spent (which is simply total time divided by the count), then the time spent on the first call, the time spent on the fastest call, and finally the time spent on the slowest call.

The 'avg', 'first', 'min' and 'max' times are not particularly useful when the profile data path only contains the statement text. Here's an extract of a more detailed example using both statement text and method name in the path:

  'SELECT mode,size,name FROM table' =>
      'FETCH' =>
          0.000076s
      'fetchrow_hashref' =>
          0.036203s / 108 = 0.000335s avg (first 0.000490s, min 0.000152s, max 0.002786s)

Here you can see the 'avg', 'first', 'min' and 'max' for the 108 calls to fetchrow_hashref() become rather more interesting. Also the data for FETCH just shows a time value because it was only called once.

Currently the profile data is output sorted by branch names. That may change in a later version so the leaf nodes are sorted by total time per leaf node.

Report Destination

The default method of reporting is for the DESTROY method of the Profile object to format the results and write them using:

    DBI->trace_msg($results, 0);  # see $ON_DESTROY_DUMP below

to write them to the DBI trace() filehandle (which defaults to STDERR). To direct the DBI trace filehandle to write to a file without enabling tracing the trace() method can be called with a trace level of 0. For example:

    DBI->trace(0, $filename);

The same effect can be achieved without changing the code by setting the DBI_TRACE environment variable to 0=filename.

The $DBI::Profile::ON_DESTROY_DUMP variable holds a code ref that's called to perform the output of the formatted results. The default value is:

  $ON_DESTROY_DUMP = sub { DBI->trace_msg($results, 0) };

Apart from making it easy to send the dump elsewhere, it can also be useful as a simple way to disable dumping results.

CHILD HANDLES

Child handles inherit a reference to the Profile attribute value of their parent. So if profiling is enabled for a database handle then by default the statement handles created from it all contribute to the same merged profile data tree.

PROFILE OBJECT METHODS

format

See "REPORTING".

as_node_path_list

  @ary = $dbh->{Profile}->as_node_path_list();
  @ary = $dbh->{Profile}->as_node_path_list($node, $path);

Returns the collected data ($dbh->{Profile}{Data}) restructured into a list of array refs, one for each leaf node in the Data tree. This 'flat' structure is often much simpler for applications to work with.

The first element of each array ref is a reference to the leaf node. The remaining elements are the 'path' through the data tree to that node.

For example, given a data tree like this:

    {key1a}{key2a}[node1]
    {key1a}{key2b}[node2]
    {key1b}{key2a}{key3a}[node3]

The as_node_path_list() method will return this list:

    [ [node1], 'key1a', 'key2a' ]
    [ [node2], 'key1a', 'key2b' ]
    [ [node3], 'key1b', 'key2a', 'key3a' ]

The nodes are ordered by key, depth-first.

The $node argument can be used to focus on a sub-tree. If not specified it defaults to $dbh->{Profile}{Data}.

The $path argument can be used to specify a list of path elements that will be added to each element of the returned list. If not specified it defaults to a a ref to an empty array.

as_text

  @txt = $dbh->{Profile}->as_text();
  $txt = $dbh->{Profile}->as_text({
      node      => undef,
      path      => [],
      separator => " > ",
      format    => '%1$s: %11$fs / %10$d = %2$fs avg (first %12$fs, min %13$fs, max %14$fs)'."\n";
      sortsub   => sub { ... },
  );

Returns the collected data ($dbh->{Profile}{Data}) reformatted into a list of formatted strings. In scalar context the list is returned as a single concatenated string.

A hashref can be used to pass in arguments, the default values are shown in the example above.

The node and <path> arguments are passed to as_node_path_list().

The separator argument is used to join the elements of the path for each leaf node.

The sortsub argument is used to pass in a ref to a sub that will order the list. The subroutine will be passed a reference to the array returned by as_node_path_list() and should sort the contents of the array in place. The return value from the sub is ignored. For example, to sort the nodes by the second level key you could use:

  sortsub => sub { my $ary=shift; @$ary = sort { $a->[2] cmp $b->[2] } @$ary }

The format argument is a sprintf format string that specifies the format to use for each leaf node. It uses the explicit format parameter index mechanism to specify which of the arguments should appear where in the string. The arguments to sprintf are:

     1:  path to node, joined with the separator
     2:  average duration (total duration/count)
         (3 thru 9 are currently unused)
    10:  count
    11:  total duration
    12:  first duration
    13:  smallest duration
    14:  largest duration
    15:  time of first call
    16:  time of first call

CUSTOM DATA MANIPULATION

Recall that $h->{Profile}->{Data}
 is a reference to the collected data. Either to a 'leaf' array (when the Path is empty, i.e., DBI_PROFILE env var is 1), or a reference to hash containing values that are either further hash references or leaf array references.

Sometimes it's useful to be able to summarise some or all of the collected data. The dbi_profile_merge_nodes() function can be used to merge leaf node values.

dbi_profile_merge_nodes

  use DBI qw(dbi_profile_merge_nodes);

  $time_in_dbi = dbi_profile_merge_nodes(my $totals=[], @$leaves);

Merges profile data node. Given a reference to a destination array, and zero or more references to profile data, merges the profile data into the destination array. For example:

  $time_in_dbi = dbi_profile_merge_nodes(
      my $totals=[],
      [ 10, 0.51, 0.11, 0.01, 0.22, 1023110000, 1023110010 ],
      [ 15, 0.42, 0.12, 0.02, 0.23, 1023110005, 1023110009 ],
  );

$totals will then contain

  [ 25, 0.93, 0.11, 0.01, 0.23, 1023110000, 1023110010 ]

and $time_in_dbi will be 0.93;

The second argument need not be just leaf nodes. If given a reference to a hash then the hash is recursively searched for for leaf nodes and all those found are merged.

For example, to get the time spent 'inside' the DBI during an http request, your logging code run at the end of the request (i.e. mod_perl LogHandler) could use:

  my $time_in_dbi = 0;
  if (my $Profile = $dbh->{Profile}) { # if DBI profiling is enabled
      $time_in_dbi = dbi_profile_merge_nodes(my $total=[], $Profile->{Data});
      $Profile->{Data} = {}; # reset the profile data
  }

If profiling has been enabled then $time_in_dbi will hold the time spent inside the DBI for that handle (and any other handles that share the same profile data) since the last request.

Prior to DBI 1.56 the dbi_profile_merge_nodes() function was called dbi_profile_merge(). That name still exists as an alias.

CUSTOM DATA COLLECTION

Using The Path Attribute

  XXX example to be added later using a selectall_arrayref call
  XXX nested inside a fetch loop where the first column of the
  XXX outer loop is bound to the profile Path using
  XXX bind_column(1, \${ $dbh->{Profile}->{Path}->[0] })
  XXX so you end up with separate profiles for each loop
  XXX (patches welcome to add this to the docs :)

Adding Your Own Samples

The dbi_profile() function can be used to add extra sample data into the profile data tree. For example:

    use DBI;
    use DBI::Profile (dbi_profile dbi_time);

    my $t1 = dbi_time(); # floating point high-resolution time

    ... execute code you want to profile here ...

    my $t2 = dbi_time();
    dbi_profile($h, $statement, $method, $t1, $t2);

The $h parameter is the handle the extra profile sample should be associated with. The $statement parameter is the string to use where the Path specifies !Statement. If $statement is undef then $h->{Statement} will be used. Similarly $method is the string to use if the Path specifies !MethodName. There is no default value for $method.

The $h->{Profile}{Path} attribute is processed by dbi_profile() in the usual way.

The $h parameter is usually a DBI handle but it can also be a reference to a hash, in which case the dbi_profile() acts on each defined value in the hash. This is an efficient way to update multiple profiles with a single sample, and is used by the DashProfiler module.

SUBCLASSING

Alternate profile modules must subclass DBI::Profile to help ensure they work with future versions of the DBI.

CAVEATS

Applications which generate many different statement strings (typically because they don't use placeholders) and profile with !Statement in the Path (the default) will consume memory in the Profile Data structure for each statement. Use a code ref in the Path to return an edited (simplified) form of the statement.

If a method throws an exception itself (not via RaiseError) then it won't be counted in the profile.

If a HandleError subroutine throws an exception (rather than returning 0 and letting RaiseError do it) then the method call won't be counted in the profile.

Time spent in DESTROY is added to the profile of the parent handle.

Time spent in DBI->*() methods is not counted. The time spent in the driver connect method, $drh->connect(), when it's called by DBI->connect is counted if the DBI_PROFILE environment variable is set.

Time spent fetching tied variables, $DBI::errstr, is counted.

Time spent in FETCH for $h->{Profile} is not counted, so getting the profile data doesn't alter it.

DBI::PurePerl does not support profiling (though it could in theory).

For asynchronous queries, time spent while the query is running on the backend is not counted.

A few platforms don't support the gettimeofday() high resolution time function used by the DBI (and available via the dbi_time() function). In which case you'll get integer resolution time which is mostly useless.

On Windows platforms the dbi_time() function is limited to millisecond resolution. Which isn't sufficiently fine for our needs, but still much better than integer resolution. This limited resolution means that fast method calls will often register as taking 0 time. And timings in general will have much more 'jitter' depending on where within the 'current millisecond' the start and and timing was taken.

This documentation could be more clear. Probably needs to be reordered to start with several examples and build from there. Trying to explain the concepts first seems painful and to lead to just as many forward references. (Patches welcome!)


text:
package de.imc.mirror.sdk.android;

import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

import org.jdom2.Document;
import org.jdom2.Element;
import org.jdom2.JDOMException;
import org.jdom2.input.SAXBuilder;
import org.jdom2.output.XMLOutputter;
import org.jivesoftware.smack.PacketInterceptor;
import org.jivesoftware.smack.PacketListener;
import org.jivesoftware.smack.XMPPConnection;
import org.jivesoftware.smack.XMPPException;
import org.jivesoftware.smack.filter.PacketFilter;
import org.jivesoftware.smack.packet.IQ;
import org.jivesoftware.smack.packet.Packet;
import org.jivesoftware.smack.packet.XMPPError;
import org.jivesoftware.smackx.pubsub.ItemPublishEvent;
import org.jivesoftware.smackx.pubsub.ItemsExtension;
import org.jivesoftware.smackx.pubsub.LeafNode;
import org.jivesoftware.smackx.pubsub.Node;
import org.jivesoftware.smackx.pubsub.PayloadItem;
import org.jivesoftware.smackx.pubsub.PubSubElementType;
import org.jivesoftware.smackx.pubsub.PubSubManager;
import org.jivesoftware.smackx.pubsub.SimplePayload;
import org.jivesoftware.smackx.pubsub.Subscription;
import org.jivesoftware.smackx.pubsub.listener.ItemEventListener;
import org.jivesoftware.smackx.pubsub.packet.PubSub;

import android.util.Log;
import de.imc.mirror.sdk.ConnectionStatus;
import de.imc.mirror.sdk.ConnectionStatusListener;
import de.imc.mirror.sdk.DataObject;
import de.imc.mirror.sdk.DataObjectFilter;
import de.imc.mirror.sdk.DataObjectListener;
import de.imc.mirror.sdk.OfflineModeHandler;
import de.imc.mirror.sdk.SerializableDataObjectFilter;
import de.imc.mirror.sdk.Space;
import de.imc.mirror.sdk.Space.PersistenceType;
import de.imc.mirror.sdk.SpaceChannel;
import de.imc.mirror.sdk.config.NamespaceConfig;
import de.imc.mirror.sdk.exceptions.ConnectionStatusException;
import de.imc.mirror.sdk.exceptions.InvalidDataException;
import de.imc.mirror.sdk.exceptions.QueryException;
import de.imc.mirror.sdk.exceptions.SpaceManagementException;
import de.imc.mirror.sdk.exceptions.SpaceManagementException.Type;
import de.imc.mirror.sdk.exceptions.UnknownEntityException;
import de.imc.mirror.sdk.android.exceptions.RequestException;
import de.imc.mirror.sdk.android.packet.DeleteRequestIQ;
import de.imc.mirror.sdk.android.packet.DeleteResponseIQ;
import de.imc.mirror.sdk.android.packet.QueryRequestIQ;
import de.imc.mirror.sdk.android.packet.QueryResponseIQ;

/**
 * Handler for the data exchange over MIRROR spaces.
 * One data handler may observe several spaces, with the restriction to one domain.
 * The class provides offline functionality, which can be accessed over the methods specified in
 * the {@link OfflineModeHandler} interface.
 * @author mach
 *
 */
public class DataHandler implements OfflineModeHandler, de.imc.mirror.sdk.DataHandler, ItemEventListener<PayloadItem<SimplePayload>> {

	private int timeout = 2000;
	private ConnectionHandler connectionHandler;
	private XMPPConnection connection;
	private ConnectionStatusListener connectionStatusListener;
	private boolean isConnectionResetted;
	private SpaceHandler spaceHandler;
	private de.imc.mirror.sdk.UserInfo userInfo;
	private List<DataObjectListener> listeners;
	private DataWrapper datawrapper;
	private List<Space> handledSpaces;
	private Mode userWantedMode;
	private Mode realMode;

	private de.imc.mirror.sdk.DataObjectFilter dataObjectFilter;

	private Map<String, RequestFuture<IQ>> pendingPersistenceServiceQueries;
	private Map<String, RequestFuture<List<PayloadItem<SimplePayload>>>> pendingPayloadRequests;
	private Map<String, RequestFuture<IQ>> pendingPublishingRequests; // <pubsub item id, request>
	private Map<String, String> publishIdMap; // <iq packet id, pubsub item id>
	private Map<String, PacketListener> pubsubServiceListeners;
	private Map<String, PubSubManager> pubsubManagers;
	private PacketInterceptor pubsubInterceptor;

	private Map<String, PacketListener> persistenceServiceListeners;

	/**
	 * Creates a new data handler.
	 * @param connectionHandler The connection handler to be used for requests.
	 * @param spaceHandler An instance of a space handler to be used for requesting space properties.
	 */
	public DataHandler(ConnectionHandler connectionHandler, SpaceHandler spaceHandler){
		if (connectionHandler == null || spaceHandler == null){
			throw new IllegalArgumentException("None of the Arguments may be null");
		}
		this.connectionHandler = connectionHandler;
		this.isConnectionResetted = true;
		this.spaceHandler = spaceHandler;

		this.timeout = connectionHandler.getConfiguration().requestTimeout();
		this.userWantedMode = Mode.OFFLINE;
		this.connection = this.connectionHandler.getXMPPConnection();
		this.listeners = new ArrayList<DataObjectListener>();
		this.handledSpaces = new ArrayList<Space>();
		this.datawrapper = DataWrapper.getInstance();

		this.pendingPayloadRequests = new HashMap<String, RequestFuture<List<PayloadItem<SimplePayload>>>>();
		this.pendingPublishingRequests = new HashMap<String, RequestFuture<IQ>>();
		this.pendingPersistenceServiceQueries = new HashMap<String, RequestFuture<IQ>>();
		this.userInfo = this.connectionHandler.getCurrentUser();
		this.dataObjectFilter = null;

		this.publishIdMap = new HashMap<String, String>();

		this.pubsubServiceListeners = new HashMap<String, PacketListener>();
		this.persistenceServiceListeners = new HashMap<String, PacketListener>();
		this.pubsubManagers = new HashMap<String, PubSubManager>();

		setPubSubPacketInterceptor();
		setConnectionStatusListener();
	}

	private void setConnectionStatusListener() {
		connectionStatusListener = new ConnectionStatusListener() {

			@Override
			public void connectionStatusChanged(ConnectionStatus newStatus) {
				switch (newStatus) {
				case OFFLINE:
					isConnectionResetted = true;
					break;
				case ONLINE:
					if (isConnectionResetted) {
						changeConnectionHandler(connectionHandler);
						for (Space space : handledSpaces) {
							registerItemEventListener(space);
						}
						isConnectionResetted = false;
					}
					try {
						sendSavedPayloads();
					} catch (UnknownEntityException e) {
						Log.w("DataHandler", "An UnknownEntityException was thrown while trying to send saved payloads.", e);
					} catch (InvalidDataException e) {
						Log.w("DataHandler", "A cached data object could not be published as it was rejected.", e);
					}
					break;
				default:
					// Do nothing.
				}
			}
		};
		connectionHandler.addConnectionStatusListener(connectionStatusListener);

		switch (connectionHandler.getStatus()) {
		case ONLINE:
			this.realMode = Mode.ONLINE;
			break;
		default:
			this.realMode = Mode.OFFLINE;
			break;
		}
	}

	/**
	 * Registers a packet listener for the given pubsub component and instantiates a pubsub manager.
	 * If the given component is already registered, nothing happens.
	 * @param componentJID JID of the pubsub service component in the XMPP network, e.g. pubsub.mydomain.com.
	 */
	private void registerPubsubService(final String componentJID) {
		if (pubsubServiceListeners.containsKey(componentJID)) {
			return;
		}

		PacketFilter packetFilter = new PacketFilter() {
			@Override
			public boolean accept(Packet packet) {
				if (!(packet instanceof IQ)) return false; // Accept only IQ packets.
				if (!componentJID.equals(packet.getFrom())) return false; // Accept only packets for the given component.
				IQ.Type type = ((IQ) packet).getType();
				if (type != IQ.Type.RESULT && type != IQ.Type.ERROR) return false;
				return true;
			}
		};

		PacketListener packetListener = new PacketListener() {

			@SuppressWarnings("unchecked")
			@Override
			public void processPacket(Packet packet) {
				String packetId = packet.getPacketID();
				if (publishIdMap.containsKey(packetId)) {
					pendingPublishingRequests.get(publishIdMap.get(packetId)).setResponse((IQ) packet);
				} else if (pendingPayloadRequests.containsKey(packetId)) {
					RequestFuture<List<PayloadItem<SimplePayload>>> spaceFuture = pendingPayloadRequests.get(packet.getPacketID());
					ItemsExtension itemsElem = (ItemsExtension) ((PubSub)packet).getExtension(PubSubElementType.ITEMS);
					spaceFuture.setResponse((List<PayloadItem<SimplePayload>>) itemsElem.getItems());
				}
			}
		};

		pubsubServiceListeners.put(componentJID, packetListener);
		connection.addPacketListener(packetListener, packetFilter);

		pubsubManagers.put(componentJID, new PubSubManager(connection, componentJID));
	}

	/**
	 * Removes a packet listener from the given pubsub component and deleted the pubsub manager instance.
	 * If the given component is not registered, nothing happens.
	 * @param componentJID JID of the pubsub service component in the XMPP network, e.g. pubsub.mydomain.com.
	 */
	private void removePubsubService(String componentJID) {
		PacketListener packetListener = pubsubServiceListeners.get(componentJID);
		if (packetListener == null) {
			return;
		}

		connection.removePacketListener(packetListener);
		pubsubServiceListeners.remove(componentJID);

		pubsubManagers.remove(componentJID);
	}

	/**
	 * Registers a packet listener for the given persistence service component.
	 * If the given component is already registered, nothing happens.
	 * @param componentJID JID of the persistence service component in the XMPP network, e.g. persistence.mydomain.com.
	 */
	private void registerPersistenceService(final String componentJID) {
		if (persistenceServiceListeners.containsKey(componentJID)) {
			return;
		}

		PacketFilter packetFilter = new PacketFilter() {
			@Override
			public boolean accept(Packet packet) {
				if (!(packet instanceof IQ)) return false; // Accept only IQ packets.
				if (!componentJID.equals(packet.getFrom())) return false; // Accept only packets for the given component.
				IQ.Type type = ((IQ) packet).getType();
				if (type != IQ.Type.RESULT && type != IQ.Type.ERROR) return false;
				return true;
			}
		};

		PacketListener packetListener = new PacketListener() {
			@Override
			public void processPacket(Packet packet) {
				String packetId = packet.getPacketID();
				if (pendingPersistenceServiceQueries.containsKey(packetId)) {
					RequestFuture<IQ> iqFuture = (RequestFuture<IQ>) pendingPersistenceServiceQueries.get(packetId);
					pendingPersistenceServiceQueries.remove(packetId);
					iqFuture.setResponse((IQ) packet);
				}
			}
		};

		persistenceServiceListeners.put(componentJID, packetListener);
		connection.addPacketListener(packetListener, packetFilter);
	}

	/**
	 * Removes a packet listener from the given persistence service component.
	 * If the given component is not registered, nothing happens.
	 * @param componentJID JID of the persistence service component in the XMPP network, e.g. persistence.mydomain.com.
	 */
	private void removePersistenceService(String componentJID) {
		PacketListener packetListener = persistenceServiceListeners.get(componentJID);
		if (packetListener == null) {
			return;
		}

		connection.removePacketListener(packetListener);
		persistenceServiceListeners.remove(componentJID);
	}

	/**
	 * Registers a packet interceptor for pubsub publishing requests.
	 * When a publishing request is intercepted, the related request future is complemented by the IQ packet id.
	 */
	private void setPubSubPacketInterceptor() {
		PacketFilter packetFilter = new PacketFilter() {
			@Override
			public boolean accept(Packet packet) {
				if (!(packet instanceof IQ)) return false;
				IQ iqPacket = (IQ) packet;
				if (iqPacket.getType() != IQ.Type.SET) return false;
				if (packet.getExtension(NamespaceConfig.XMPP_PUBSUB) == null) return false;
				return true;
			}
		};
		pubsubInterceptor = new PacketInterceptor() {
			@Override
			public void interceptPacket(Packet packet) {
				IQ iq = (IQ) packet;
				String childElement = iq.getChildElementXML();
				for (String itemId : pendingPublishingRequests.keySet()) {
					if (childElement.contains(itemId)) {
						publishIdMap.put(iq.getPacketID(), itemId);
					}
				}
			}
		};
		connection.addPacketInterceptor(pubsubInterceptor, packetFilter);
	}

	/**
	 * Adds a listener for data objects published on any space handled by this handler.
	 * @param listener Listener to add.
	 * The call-back method of listener is called each time a new item is published on the pubsub node of a MIRROR space.
	 */
	@Override
	public void addDataObjectListener(DataObjectListener listener){
		if (!listeners.contains(listener))
			listeners.add(listener);
	}

	/**
	 * Retrieves a pubsub node from the pubsub service.
	 * @param nodeId The id of the node to retrieve.
	 * @param pubsubService JID of the pubsub service component handling the the node.
	 * @return The pubsub node.
	 * @throws UnknownEntityException No node with the given ID exists.
	 */
	protected Node getNode(String nodeId, String pubsubService) throws UnknownEntityException {
		if (!pubsubManagers.containsKey(pubsubService)) {
			registerPubsubService(pubsubService);
		}
		PubSubManager manager = pubsubManagers.get(pubsubService);

		try {
			return manager.getNode(nodeId);
		} catch(XMPPException e){
			throw new UnknownEntityException("The node " + nodeId + " couldn't be retrieved.", e);
		}
	}

	/**
	 * Item handler of this data handler implementation. Checks if received items are already cached,
	 * if not, it caches them. Ultimately it sends the items to the subscribed listeners.
	 */
	@Override
	public void handlePublishedItems(ItemPublishEvent<PayloadItem<SimplePayload>> items){
		String spaceId = getSpaceId(items.getNodeId());
		if (spaceId == null) return;
		List<String> toRequest = new ArrayList<String>();
		for (PayloadItem<SimplePayload> item:items.getItems()){
        	if (item.getPayload() == null && !pendingPayloadRequests.containsKey(item.getId())){
        		toRequest.add(item.getId());
        		continue;
        	}
        	DataObject obj = parseItemToDataObject(item);
        	if (dataObjectFilter == null || dataObjectFilter.isDataObjectValid(obj)) {
				if (!datawrapper.isDataObjectAlreadyCached(item.getId())){
					datawrapper.saveDataObject(items.getNodeId(), obj, item.getId());
				}
				for (DataObjectListener listener:listeners){
					listener.handleDataObject(obj, spaceId);
				}
			}
		}
		if (!toRequest.isEmpty()){
			for (String id:toRequest){
				try {
					retrievePayload(items.getNodeId(), id);
				} catch (Exception e) {
					//Ignore
					continue;
				}
			}
		}
	}

	/**
	 * Gets the id of the space with the given nodeId.
	 * @param nodeId The nodeId to look for.
	 * @return A spaceId or null if no space was found.
	 */
	private String getSpaceId(String nodeId){
		for (Space space:handledSpaces){
			Map<String, String> prop = space.getPubSubChannel().getProperties();
			if (prop.get("node").equals(nodeId)){
				return space.getId();
			}
		}
		return null;
	}

	/**
	 * Publishes a data object on the space with the given id.
	 * ONLINE mode: The object is directly published on the related pubsub node.
	 * OFFLINE mode: The object is stored locally and published when the connection is establised again.
	 * @param object Data object to publish.
	 * @param spaceId Identifier if the space to publish.
	 * @throws UnknownEntityException A space with the given id is not known to the space handler.
	 * @throws InvalidDataException The data was rejected by the spaces service.
	 */
	@Override
	public void publishDataObject(DataObject object, String spaceId) throws UnknownEntityException, InvalidDataException {
		de.imc.mirror.sdk.android.DataObject obj = (de.imc.mirror.sdk.android.DataObject) object;
		SimplePayload payload = new SimplePayload(obj.getElementName(),
				obj.getNamespaceURI(), obj.toString());
		publish(spaceId, payload);
	}

	/**
	 * Publishes a data objects and returns the object sent over the space.
	 * Use this method to access fields set server-side like the data object identifier.
	 * @param dataObject Data object to publish.
	 * @param spaceId Identifier if the space to publish.
	 * @return Data object as published on the space.
	 * @throws UnknownEntityException A space with the given id is not known to the space handler.
	 * @throws InvalidDataException The data was rejected by the spaces service.
	 * @throws ConnectionStatusException The data handler has to be online in order to make a synchronous call.
	 */
	public DataObject publishAndRetrieveDataObject(DataObject dataObject, String spaceId) throws UnknownEntityException, InvalidDataException, ConnectionStatusException {
		SimplePayload payload = new SimplePayload(dataObject.getElement().getName(), dataObject.getNamespaceURI(), dataObject.toString());
		return publishAndRetrieve(spaceId, payload);
	}

	/**
	 * If online, the method publishs a payload.
	 * If offline, it only saves the payload in the sendcache.
	 * @param spaceId The id of the space to send the payload to.
	 * @param payload The payload to send.
	 * @throws UnknownEntityException Thrown when given space id cannot be mapped.
	 * @throws InvalidDataException The data was rejected by the spaces service.
	 * @throws RequestException The request failed.
	 */
	private void publish(String spaceId, SimplePayload payload) throws UnknownEntityException, InvalidDataException, RequestException {
		if (getMode() == Mode.ONLINE) {
			SpaceChannel channel = spaceHandler.getPubSubChannel(spaceId);
			Map<String, String> properties = channel.getProperties();
			LeafNode node = (LeafNode) getNode(properties.get("node"), properties.get("domain"));
			if (node == null) {
				throw new UnknownEntityException("There's no node with this id.");
			}
			String itemId = UUID.randomUUID().toString();
			PayloadItem<SimplePayload> item = new PayloadItem<SimplePayload>(itemId, payload);
			RequestFuture<IQ> requestFuture = new RequestFuture<IQ>();
			pendingPublishingRequests.put(itemId, requestFuture);
			node.publish(item);
			try {
				IQ response = requestFuture.get(timeout, TimeUnit.MILLISECONDS);
				XMPPError error = response.getError();
				if (error != null) {
					throw new InvalidDataException(error.getMessage());
				}
			} catch (InterruptedException e){
				throw new RequestException("Receiving a response was interrupted.", e);
			} catch (ExecutionException e) {
				throw new RequestException("Couldn't receive a response.", e);
			} catch (TimeoutException e) {
				throw new RequestException("Receiving a response timed out.", e);
			}
		}
		else {
			String id = UUID.randomUUID().toString();
			datawrapper.savePayloadToSend(userInfo.getBareJID(), id, spaceId, payload);
		}
	}

	private DataObject publishAndRetrieve(String spaceId, SimplePayload payload) throws UnknownEntityException, ConnectionStatusException, InvalidDataException {
		if (getMode() != Mode.ONLINE) {
			throw new ConnectionStatusException("The data handler has to be ONLINE to publish data objects synchronously.");
		}
		SpaceChannel channel = spaceHandler.getPubSubChannel(spaceId);
		Map<String, String> properties = channel.getProperties();
		LeafNode node = (LeafNode) getNode(properties.get("node"), properties.get("domain"));
		final String itemId = UUID.randomUUID().toString();
		final RequestFuture<DataObject> requestFuture = new RequestFuture<DataObject>();
		final PayloadItem<SimplePayload> itemToPublish = new PayloadItem<SimplePayload>(itemId, payload);

		ItemEventListener<PayloadItem<SimplePayload>> itemEventListener = new ItemEventListener<PayloadItem<SimplePayload>>() {
			@Override
			public void handlePublishedItems(ItemPublishEvent<PayloadItem<SimplePayload>> event) {
				for (PayloadItem<SimplePayload> item : event.getItems()) {
					if (itemToPublish.getId().equals(item.getId())) {
			        	DataObject dataObject = parseItemToDataObject(item);
			        	requestFuture.setResponse(dataObject);
					}
				}
			}
		};
		node.addItemEventListener(itemEventListener);

		RequestFuture<IQ> sentNotificationFuture = new RequestFuture<IQ>();
		pendingPublishingRequests.put(itemId, sentNotificationFuture);
		node.publish(itemToPublish);
		try {
			IQ response = sentNotificationFuture.get(timeout, TimeUnit.MILLISECONDS);
			XMPPError error = response.getError();
			if (error != null) {
				throw new InvalidDataException(error.getMessage());
			}
		} catch (InterruptedException e){
			throw new RequestException("Failed to sent data object: Receiving a response was interrupted.", e);
		} catch (ExecutionException e) {
			throw new RequestException("Failed to sent data object: Couldn't receive a response.", e);
		} catch (TimeoutException e) {
			throw new RequestException("Failed to sent data object: Receiving a response timed out.", e);
		}

		DataObject dataObject = null;
		try {
			dataObject = requestFuture.get(timeout, TimeUnit.MILLISECONDS);
		} catch (InterruptedException e) {
			throw new RequestException("Receiving a response was interrupted.", e);
		} catch (ExecutionException e) {
			throw new RequestException("Couldn't receive a response.", e);
		} catch (TimeoutException e) {
			throw new RequestException("The request timed out.", e);
		} finally {
			node.removeItemEventListener(itemEventListener);
		}

		return dataObject;
	}

	/**
	 * Removes a data object listener.
	 * @param listener Listener to remove.
	 */
	@Override
	public void removeDataObjectListener(DataObjectListener listener){
		listeners.remove(listener);
	}

	/**
	 * Gets the pubsub service of a pubsub node.
	 * @param nodeId The id of the node to get the pubsub service for.
	 * @return The pubsubservice.
	 */
	protected String getPubsubService(String nodeId){
		String pubsubService = null;
		List<Space> spaces = spaceHandler.getAllSpaces();
		for(Space space:spaces){
			SpaceChannel pubsub = space.getPubSubChannel();
			if (pubsub.getProperties().get("node").equalsIgnoreCase(nodeId))
				return pubsubService = pubsub.getProperties().get("domain");
		}
		return pubsubService;
	}

	/**
	 * Sends a request to the server to retrieve the payload of an item
	 * @param nodeId The id of the node the item is from.
	 * @param payloadId The id of the payload.
	 * @throws SpaceManagementException Thrown when an error occured while retrieving the payload from the server.
	 * @throws UnknownEntityException Thrown when no pubsub node could be retrieved.
	 */
	private void retrievePayload(final String nodeId, final String payloadId) throws SpaceManagementException, UnknownEntityException{
		if (getMode() == Mode.OFFLINE){
			throw new IllegalStateException("Not connected");
		}
		String pubsubService = getPubsubService(nodeId);
		if (pubsubService == null){
			return;
		}
		Node node = this.getNode(nodeId, pubsubService);
		List<Subscription> subs;
		try {
			subs = node.getSubscriptions();
		} catch (XMPPException e) {
			throw new SpaceManagementException("The Server did not respond.", Type.OTHER, e);
		}
		final String subid = subs.get(0).getId();
		IQ requestIq = new IQ() {

			@Override
			public String getChildElementXML() {
				Element childElement = new Element("pubsub", NamespaceConfig.XMPP_PUBSUB);
				Element itemsElement = new Element("items").setAttribute("node", nodeId);
				itemsElement.setAttribute("subid", subid);
				itemsElement.addContent(new Element("item").setAttribute("id", payloadId));
				childElement.addContent(itemsElement);
				XMLOutputter out = new XMLOutputter();
				return out.outputString(childElement);
			}
		};
		requestIq.setType(IQ.Type.GET);
		requestIq.setTo(pubsubService);
		requestIq.setPacketID(payloadId);
		RequestFuture<List<PayloadItem<SimplePayload>>> payloadFuture = new RequestFuture<List<PayloadItem<SimplePayload>>>();
		pendingPayloadRequests.put(payloadId, payloadFuture);
		connection.sendPacket(requestIq);
		List<PayloadItem<SimplePayload>> response = null;
		try {
			response = payloadFuture.get(timeout, TimeUnit.MILLISECONDS);
		} catch (InterruptedException e) {
			throw new SpaceManagementException("Receiving a response was interrupted.", SpaceManagementException.Type.OTHER, e);
		} catch (ExecutionException e) {
			throw new SpaceManagementException("Couldn't receive a response.", SpaceManagementException.Type.OTHER, e);
		} catch (TimeoutException e) {
			throw new SpaceManagementException("Receiving a response timed out.", SpaceManagementException.Type.OTHER, e);
		}
		if (response == null){
			return;
		}
		ItemPublishEvent<PayloadItem<SimplePayload>> event =
			new ItemPublishEvent<PayloadItem<SimplePayload>>(nodeId, response);
		handlePublishedItems(event);
	}

	/**
	 * Sends the payloads of the user, which are in the sendcache.
	 * @throws UnknownEntityException Thrown when no pubsub node could be retrieved.
	 */
	private void sendSavedPayloads() throws UnknownEntityException, InvalidDataException {
		if (datawrapper == null) return;
		Map<String, SimplePayload> payloads = datawrapper.getPayloadsToSend(userInfo.getBareJID());
		for (String id:payloads.keySet()){
			String spaceId = datawrapper.getSpaceForPayload(id);
			if (spaceId == null) continue;
			publish(spaceId, payloads.get(id));
		}
		datawrapper.clearSendCache(userInfo.getBareJID());
	}

	/**
	 * Changes the connection handler used for XMPP requests.
	 * @param connectionHandler The connection handler to be used for requests.
	 */
	public void changeConnectionHandler(ConnectionHandler connectionHandler) {
		// Unregister pubsub services.
		List<String> registeredPubsubServiceComponents = new ArrayList<String>(pubsubServiceListeners.keySet());
		for (String componentJID : registeredPubsubServiceComponents) {
			removePubsubService(componentJID);
		}
		connection.removePacketInterceptor(pubsubInterceptor);

		// Unregister persistence services.
		List<String> registeredPersistenceServiceComponents = new ArrayList<String>(persistenceServiceListeners.keySet());
		for (String componentJID : registeredPersistenceServiceComponents) {
			removePersistenceService(componentJID);
		}

		// Unregister connection status listner.
		connectionHandler.removeConnectionStatusListener(connectionStatusListener);

		// Update handler.
		this.connectionHandler = connectionHandler;
		this.connection = connectionHandler.getXMPPConnection();

		// Register pubsub services.
		for (String componentJID : registeredPubsubServiceComponents) {
			registerPubsubService(componentJID);
		}
		this.setPubSubPacketInterceptor();

		// Register persistence services.
		for (String componentJID : registeredPersistenceServiceComponents) {
			registerPersistenceService(componentJID);
		}

		// Register connection status listener.
		setConnectionStatusListener();
	}

	/**
	 * Returns the list of spaces handled by this handler.
	 * @return Unmodifiable list of spaces. May be empty.
	 */
	@Override
	public List<Space> getHandledSpaces() {
		return Collections.unmodifiableList(handledSpaces);
	}

	/**
	 * Adds a space to be observed by this data handler.
	 * If the space is already in the list of observed spaces, nothing happens.
	 * @param spaceId Identifier of the space to observe.
	 * @throws UnknownEntityException Failed to register to the pubsub node of the space.
	 */
	@Override
	public void registerSpace(String spaceId) throws UnknownEntityException {
		Space space = null;
		for (Space cachedSpace : spaceHandler.getCachedSpaces()) {
			if (spaceId.equals(cachedSpace.getId())) {
				space = cachedSpace;
				break;
			}
		}
		if (space == null) space = spaceHandler.getSpace(spaceId);
		if (space == null) {
			throw new UnknownEntityException("Space could not be found: " + spaceId);
		}
		if (handledSpaces.contains(space)) {
			return;
		} else {
			handledSpaces.add(space);
			if (getMode() == Mode.ONLINE){
				registerItemEventListener(space);
			}
		}
	}

	/**
	 * Removes a space from the list of observed spaces.
	 * If the space is not in the list of observed spaces, nothing happens.
	 * @param spaceId Identifier of the space to observe.
	 */
	@Override
	public void removeSpace(String spaceId) {
		Space space = spaceHandler.getSpace(spaceId);
		if (handledSpaces.remove(space)){
			SpaceChannel channel = space.getPubSubChannel();
			Map<String, String> properties = channel.getProperties();
			datawrapper.deleteCachedDataObjectsForSpace(properties.get("node"));
			if (getMode() == Mode.ONLINE){
				try {
					LeafNode node = (LeafNode) getNode(properties.get("node"), properties.get("domain"));
					node.removeItemEventListener(this);
				} catch (UnknownEntityException e) {
					Log.d("DATAHANDLER", "An UnknownEntityException was thrown while removing a registered space.", e);
				}
			}
		}
	}

	/**
	 * Returns a unmodifiable list of data objects previously published on the space.
	 * The data objects are retrieved from the local cache and contain the latest items published.
	 * The amount of objects cache depend on the handler implementation.
	 * @param spaceId Identifier of the space to retrieve data objects for.
	 * @return List of data objects, in reverse order of their publishing time, i.e., the latest first. May be empty.
	 * @throws UnknownEntityException A space with the given id is not known to the space handler.
	 * @deprecated Use {@link DataHandler#queryDataObjectsBySpace(String, Set)} instead.
	 */
	@Override
	@Deprecated
	public List<DataObject> retrieveDataObjects(String spaceId) throws UnknownEntityException {
		SpaceChannel channel = spaceHandler.getPubSubChannel(spaceId);
		Map<String, String> properties = channel.getProperties();
		String nodeId = properties.get("node");
		List<DataObject> objs = datawrapper.getCachedDataObjects(nodeId);
		return Collections.unmodifiableList(objs);
	}

	/**
	 * Convenience method to register the DataHandler as an ItemEventListener to a node of a space.
	 * @param space The space to which node the DataHandler should register.
	 * @return The pubsubnode of the space.
	 */
	private LeafNode registerItemEventListener(Space space) {
		SpaceChannel channel = space.getPubSubChannel();
		Map<String, String> properties = channel.getProperties();
		LeafNode node = null;
		try{
			node = (LeafNode) getNode(properties.get("node"), properties.get("domain"));
		} catch (UnknownEntityException e) {
			Log.d("DataHandler", "An Exception was thrown while trying to register the " +
					"DataHandler to the nodes of the registered Spaces.", e);
		}
		if (node != null){
			node.addItemEventListener(this);
		}
		return node;
	}

	/**
	 * Convenience method to parse an PayloadItem to an DataObject.
	 * @param item The Item to parse.
	 * @return The parsed DataObject.
	 */
	private DataObject parseItemToDataObject(PayloadItem<SimplePayload> item){
		SAXBuilder reader = new SAXBuilder();
		StringReader in = new StringReader(item.getPayload().toXML());
		Document document = null;
		try {
		document = reader.build(in);
		} catch (JDOMException e) {
			Log.d("DataHandler", "An JDOMException was thrown while parsing a newly gotten item.", e);
		} catch (IOException e) {
			Log.d("DataHandler", "An IOException was thrown while parsing a newly gotten item.", e);
		}
		if (document == null){
			return null;
		}
		Element elem = document.getRootElement();
		DataObject obj = new DataObjectBuilder(elem, item.getPayload().getNamespace()).build();
		return obj;
	}

	/**
	 * Sets the mode for the handler.
	 * @param mode Mode to set.
	 */
	@Override
	public void setMode(Mode mode){
		this.userWantedMode = mode;
		if (userWantedMode == Mode.ONLINE && this.getMode() == Mode.ONLINE) {
			try {
				sendSavedPayloads();
			} catch (UnknownEntityException e) {
				Log.w("DataHandler", "An UnknownEntityException was thrown while trying to send saved payloads.", e);
			} catch (InvalidDataException e) {
				Log.w("DataHandler", "A cached data object could not be published as it was rejected.", e);
			}
		}
	}

	/**
	 * Returns the current mode of the handler.
	 * Note: the current mode is dependant on the mode the user set and the current connectionstatus.
	 * @return Mode.ONLINE or Mode.OFFLINE.
	 */
	@Override
	public Mode getMode() {
		if (userWantedMode == Mode.OFFLINE){
			return userWantedMode;
		} else {
			return realMode;
		}
	}

	/**
	 * Deletes locally stored data.
	 */
	@Override
	public void clear() {
		datawrapper.clearDataCache();
	}

	@Override
	public DataObjectFilter getDataObjectFilter() {
		return dataObjectFilter;
	}

	@Override
	public void setDataObjectFilter(DataObjectFilter filter) {
		this.dataObjectFilter = filter;
	}

	@Override
	public DataObject queryDataObjectById(final String objectId) throws UnsupportedOperationException, ConnectionStatusException, QueryException {
		checkPersistenceServiceAvailabilty();

		IQ requestIQ = QueryRequestIQ.createQueryByObjectId(objectId);
		requestIQ.setFrom(userInfo.getFullJID());
		requestIQ.setTo(connectionHandler.getNetworkInformation().getPersistenceServiceJID());
		RequestFuture<IQ> queryFuture = new RequestFuture<IQ>();
		pendingPersistenceServiceQueries.put(requestIQ.getPacketID(), queryFuture);
		connection.sendPacket(requestIQ);

		List<DataObject> resultList = handleQueryResponse(queryFuture);
		return resultList.isEmpty() ? null : resultList.get(0);
	}

	@Override
	public List<DataObject> queryDataObjectsById(Set<String> objectIds, Set<SerializableDataObjectFilter> filters) throws UnsupportedOperationException, ConnectionStatusException, QueryException {
		checkPersistenceServiceAvailabilty();

		IQ requestIQ = QueryRequestIQ.createQueryByObjectIds(objectIds, filters);
		requestIQ.setFrom(userInfo.getFullJID());
		requestIQ.setTo(connectionHandler.getNetworkInformation().getPersistenceServiceJID());
		RequestFuture<IQ> queryFuture = new RequestFuture<IQ>();
		pendingPersistenceServiceQueries.put(requestIQ.getPacketID(), queryFuture);
		connection.sendPacket(requestIQ);

		List<DataObject> dataObjects = handleQueryResponse(queryFuture);
		return dataObjects;
	}

	/**
	 * Queries data objects persisted on a pubsub node.
	 * This method is used as fallback if no persistence service is available.
	 * @param spaceId Identifier of the space to request data objects for.
	 * @param filters Filter to apply.
	 * @return List of data objects. The list is empty if the space is not configured to persist data objects permanently.
	 * @throws QueryException Failed to retrieve data objects from pubsub node.
	 */
	private List<DataObject> queryDataObjectsFromPubSubNode(String spaceId, Set<SerializableDataObjectFilter> filters) throws QueryException {
		List<DataObject> dataObjects = new ArrayList<DataObject>();
		Space space = spaceHandler.getSpace(spaceId);
		if (space == null) {
			throw new QueryException(QueryException.Type.BAD_REQUEST, "Unknown space id: " + spaceId);
		}
		if (space.getPersistenceType() != PersistenceType.ON) {
			// nothing persisted
			return dataObjects;
		}
		SpaceChannel spaceChannel = space.getPubSubChannel();
		String nodeId = spaceChannel.getProperties().get("node");
		String pubsubJID = spaceChannel.getProperties().get("domain");
		LeafNode node;
		try {
			node = (LeafNode) this.getNode(nodeId, pubsubJID);
			datawrapper.deleteCachedDataObjectsForSpace(node.getId());
			List<PayloadItem<SimplePayload>> items = ((LeafNode) node).getItems(node.getSubscriptions().get(0).getId());
			if (node != null && items != null) {
				for (PayloadItem<SimplePayload> item : items) {
					DataObject dataObject = this.parseItemToDataObject(item);
					if (!filters.isEmpty()) {
						boolean reject = false;
						for (DataObjectFilter filter : filters) {
							if (filter.isDataObjectValid(dataObject)) {
								reject = true;
								break;
							}
						}
						if (reject) continue;
					}
					dataObjects.add(dataObject);
					if (!datawrapper.isDataObjectAlreadyCached(item.getId())){
						datawrapper.saveDataObject(node.getId(), dataObject, item.getId());
					}
				}
			}
		} catch (UnknownEntityException e) {
			throw new QueryException(QueryException.Type.FAILURE, "Failed to retrieve pubsub node.", e);
		} catch (XMPPException e) {
			throw new QueryException(QueryException.Type.FAILURE, "Failed to access pubsub node.", e);
		}
		return dataObjects;
	}

	@Override
	public List<DataObject> queryDataObjectsBySpace(String spaceId, Set<SerializableDataObjectFilter> filters) throws UnsupportedOperationException, ConnectionStatusException, QueryException {
		try {
			checkPersistenceServiceAvailabilty();
		} catch (UnsupportedOperationException e) {
			return queryDataObjectsFromPubSubNode(spaceId, filters);
		}

		IQ requestIQ = QueryRequestIQ.createQueryBySpace(spaceId, filters);
		requestIQ.setFrom(userInfo.getFullJID());
		requestIQ.setTo(connectionHandler.getNetworkInformation().getPersistenceServiceJID());

		RequestFuture<IQ> queryFuture = new RequestFuture<IQ>();
		pendingPersistenceServiceQueries.put(requestIQ.getPacketID(), queryFuture);
		connection.sendPacket(requestIQ);

		List<DataObject> dataObjects = handleQueryResponse(queryFuture);
		return dataObjects;
	}

	@Override
	public List<DataObject> queryDataObjectsBySpaces(Set<String> spaceIds, Set<SerializableDataObjectFilter> filters) throws UnsupportedOperationException, ConnectionStatusException, QueryException {
		checkPersistenceServiceAvailabilty();

		IQ requestIQ = QueryRequestIQ.createQueryBySpaces(spaceIds, filters);
		requestIQ.setFrom(userInfo.getFullJID());
		requestIQ.setTo(connectionHandler.getNetworkInformation().getPersistenceServiceJID());

		RequestFuture<IQ> queryFuture = new RequestFuture<IQ>();
		pendingPersistenceServiceQueries.put(requestIQ.getPacketID(), queryFuture);
		connection.sendPacket(requestIQ);

		List<DataObject> dataObjects = handleQueryResponse(queryFuture);
		return dataObjects;
	}

	@Override
	public boolean deleteDataObject(String objectId) throws UnsupportedOperationException, ConnectionStatusException, QueryException  {
		checkPersistenceServiceAvailabilty();

		IQ requestIq = DeleteRequestIQ.createDeleteRequest(objectId);
		requestIq.setFrom(userInfo.getFullJID());
		requestIq.setTo(connectionHandler.getNetworkInformation().getPersistenceServiceJID());

		RequestFuture<IQ> queryFuture = new RequestFuture<IQ>();
		pendingPersistenceServiceQueries.put(requestIq.getPacketID(), queryFuture);
		connection.sendPacket(requestIq);

		int numberOfDeletedObjects = handleDeleteResponse(queryFuture);
		return numberOfDeletedObjects > 0;
	}

	@Override
	public int deleteDataObjects(Set<String> objectIds) throws UnsupportedOperationException, ConnectionStatusException, QueryException {
		checkPersistenceServiceAvailabilty();

		IQ requestIq = DeleteRequestIQ.createDeleteRequest(objectIds);
		requestIq.setFrom(userInfo.getFullJID());
		requestIq.setTo(connectionHandler.getNetworkInformation().getPersistenceServiceJID());

		RequestFuture<IQ> queryFuture = new RequestFuture<IQ>();
		pendingPersistenceServiceQueries.put(requestIq.getPacketID(), queryFuture);
		connection.sendPacket(requestIq);

		int numberOfDeletedObjects = handleDeleteResponse(queryFuture);
		return numberOfDeletedObjects;
	}

	private List<DataObject> handleQueryResponse(RequestFuture<IQ> queryFuture) throws QueryException {
		List<DataObject> dataObjects;

		try {
			IQ responseIQ = queryFuture.get(2000, TimeUnit.MILLISECONDS);
			if (responseIQ.getType() == IQ.Type.ERROR) {
				XMPPError error = responseIQ.getError();
				if (XMPPError.Condition.not_allowed.toString().equalsIgnoreCase(error.getCondition())) {
					throw new QueryException(QueryException.Type.ACCESS_DENIED, "The currrent user is not allowed to access this data object.");
				} else {
					throw new QueryException(QueryException.Type.FAILURE, "Failed to perform query. " + error.getCondition() + ": " + error.getMessage());
				}
			} else if (responseIQ instanceof QueryResponseIQ) {
				dataObjects = ((QueryResponseIQ) responseIQ).getResult();
			} else {
				throw new QueryException(QueryException.Type.FAILURE, "Invalid query iq type as response: " + responseIQ.getChildElementXML());
			}
		} catch (InterruptedException e) {
			throw new QueryException(QueryException.Type.FAILURE, "The request was interrupted.", e);
		} catch (ExecutionException e) {
			throw new QueryException(QueryException.Type.FAILURE, "The request handling procedure caused an excution error.", e);
		} catch (TimeoutException e) {
			throw new QueryException(QueryException.Type.FAILURE, "The request timed out.", e);
		}

		return dataObjects;
	}

	private int handleDeleteResponse(RequestFuture<IQ> queryFuture) throws QueryException {
		int numberOfDeletedObjects;
		try {
			IQ responseIQ = queryFuture.get(2000, TimeUnit.MILLISECONDS);
			if (responseIQ.getType() == IQ.Type.ERROR) {
				XMPPError error = responseIQ.getError();
				if (XMPPError.Condition.not_allowed.toString().equalsIgnoreCase(error.getCondition())) {
					throw new QueryException(QueryException.Type.ACCESS_DENIED, "The currrent user is not allowed to access this data object.");
				} else {
					throw new QueryException(QueryException.Type.FAILURE, "Failed to perform query. " + error.getCondition() + ": " + error.getMessage());
				}
			} else if (responseIQ instanceof DeleteResponseIQ) {
				numberOfDeletedObjects = ((DeleteResponseIQ) responseIQ).getNumberOfDeletedEntries();
			} else {
				throw new QueryException(QueryException.Type.FAILURE, "Invalid query iq type as response: " + responseIQ.getChildElementXML());
			}
		} catch (InterruptedException e) {
			throw new QueryException(QueryException.Type.FAILURE, "The request was interrupted.", e);
		} catch (ExecutionException e) {
			throw new QueryException(QueryException.Type.FAILURE, "The request handling procedure caused an excution error.", e);
		} catch (TimeoutException e) {
			throw new QueryException(QueryException.Type.FAILURE, "The request timed out.", e);
		}

		return numberOfDeletedObjects;
	}

	/**
	 * Checks the availability of a persistence service and registers the service.
	 * @throws ConnectionStatusException The data handler has to be online in order to query data.
	 * @throws UnsupportedOperationException The operation is not supported.
	 */
	private void checkPersistenceServiceAvailabilty() throws ConnectionStatusException, UnsupportedOperationException {
		if (connectionHandler.getStatus() != ConnectionStatus.ONLINE) {
			throw new ConnectionStatusException("You must be online in order to query data.");
		} else {
			String persistenceServiceJID = connectionHandler.getNetworkInformation().getPersistenceServiceJID();
			if (persistenceServiceJID != null) {
				registerPersistenceService(persistenceServiceJID);
			} else {
				throw new UnsupportedOperationException("No persistence service available.");
			}
		}
	}

}

redpajama_set_name: RedPajamaGithub


text:
package com.exilant.exility.ide;

import com.exilant.exility.core.CustomCodeInterface;
import com.exilant.exility.core.DataAccessType;
import com.exilant.exility.core.DataCollection;
import com.exilant.exility.core.DbHandle;
import com.exilant.exility.core.ExilityException;
import com.exilant.exility.core.ServiceData;
import com.exilant.exility.core.ServiceInterface;
import com.exilant.exility.core.Spit;

/**
 * Implements a work flow that is modeled as a state-transition mechanism. It is
 * important to understand that we use state-transition in a strict academic
 * way. Our belief is that this modeling will be able to simplify most of our
 * application requirements. We keep the implementation fairly flexible to allow
 * any custom code that a specific application may require.
 *
 */
public class IdeService implements ServiceInterface {
	/**
	 * name is the name of the ide :-) Hope that is helpful.
	 */
	private final String name;

	private final CustomCodeInterface ideTask;

	/**
	 *
	 * @param serviceName
	 * @param ide
	 */
	public IdeService(String serviceName, CustomCodeInterface ide) {
		this.name = serviceName;
		this.ideTask = ide;
	}

	@Override
	public void execute(DataCollection dc, DbHandle handle)
			throws ExilityException {
		this.ideTask.execute(dc, handle, null, null);
	}

	@Override
	public DataAccessType getDataAccessType(DataCollection dc) {
		return this.ideTask.getDataAccessType();
	}

	@Override
	public String getName() {
		return this.name;
	}

	@Override
	public void executeAsStep(DataCollection dc, DbHandle handle)
			throws ExilityException {
		this.ideTask.execute(dc, handle, null, null);

	}

	@Override
	public boolean toBeRunInBackground() {
		return false;
	}

	@Override
	public void serve(ServiceData inData, ServiceData outData) {
		DataCollection dc = new DataCollection();
		dc.copyFrom(inData);
		DataAccessType access = this.ideTask.getDataAccessType();
		DbHandle handle = null;
		try {
			if (access != DataAccessType.NONE) {
				handle = DbHandle.borrowHandle(access, false);
				if (access == DataAccessType.AUTOCOMMIT) {
					handle.startAutoCommit();
				} else if (access == DataAccessType.READWRITE) {
					handle.beginTransaction();
				}
			}
			try {
				this.execute(dc, handle);
			} catch (ExilityException e1) {
				// need not do anything
			} catch (Exception e) {
				dc.addError(e.getMessage());
			}
			if (handle != null) {
				if (access == DataAccessType.READWRITE) {
					if (dc.hasError()) {
						handle.rollback();
					} else {
						handle.commit();
					}
				}
			}
		} catch (Exception e) {
			Spit.out(e);
			dc.addError(e.getMessage());
		}
		DbHandle.returnHandle(handle);
		dc.copyTo(outData);
	}
}


text:
use super::{Token,Characters};
use util;

use std;
use std::error::Error;

pub type Result<T> = std::result::Result<T,String>;

/// A list of symbols to be tokenized.
///
/// They are sorted from most specific to least.
/// This means that `+=` will precede `+` etc.
pub const SYMBOL_LIST: &'static [&'static str] = &[
    // Symbols with two characters
    "->",
    // Symbols with a single character
    ",", ":", "(", ")", "@", "%", "{", "}", "=",
];

pub struct Tokenizer<I: Iterator<Item=char>>
{
    chars: Characters<I>,

    preserve_comments: bool,

    peek_buf: Option<Token>,
}

impl<I> Tokenizer<I>
    where I: Iterator<Item=char>
{
    pub fn new(chars: I) -> Self {
        Tokenizer {
            chars: Characters::new(chars),
            preserve_comments: false,
            peek_buf: None,
        }
    }

    /// Allows the tokenizer to preserve comments it tokenizes.
    pub fn preserve_comments(mut self) -> Self {
        self.preserve_comments = true;
        self
    }

    pub fn peek(&mut self) -> Option<Result<Token>> {
        let token = match self.next() {
            Some(Ok(token)) => token,
            Some(Err(token)) => return Some(Err(token)),
            None => return None,
        };

        assert!(self.peek_buf.is_none());

        self.peek_buf = Some(token.clone());
        Some(Ok(token))
    }

    pub fn eat(&mut self) {
        self.next();
    }

    pub fn eat_while<P>(&mut self, mut predicate: P) -> Result<()>
        where P: FnMut(Token) -> bool {
        loop {
            match self.peek() {
                Some(Ok(tok)) => {
                    if predicate(tok) {
                        self.eat()
                    } else {
                        break;
                    }
                },
                Some(Err(e)) => return Err(e),
                None => break,
            }
        }

        Ok(())
    }

    fn eat_whitespace(&mut self) {
        self.chars.eat_while(|c| c.is_whitespace() && c != '\n')
    }

    fn next_string(&mut self) -> Option<Result<Token>> {
        self.assert('"');
        let string = self.chars.consume_while(|c| c != '\"');
        self.assert('"');

        Some(Ok(Token::String(string.collect())))
    }

    fn next_integer(&mut self) -> Option<Result<Token>> {
        let string: String = self.chars.consume_while(|c| c.is_numeric()).collect();

        let int = match i64::from_str_radix(&string, 10) {
            Ok(int) => int,
            Err(e) => return Some(Err(format!("Could not parse integer: {}",
                                              e.description()))),
        };

        Some(Ok(Token::Integer(int)))
    }

    fn next_word(&mut self) -> Option<Result<Token>> {
        let word: String = self.chars.consume_while(internal::can_word_contain).collect();

        Some(Ok(Token::Word(word)))
    }

    fn next_symbol(&mut self) -> Option<Result<Token>> {
        use std::borrow::Borrow;

        let first_char = match self.expect_something() {
            Ok(c) => c,
            Err(e) => return Some(Err(e)),
        };

        let second_char = self.chars.peek();

        let mut sym_str = format!("{}", first_char);

        if let Some(c) = second_char {
            sym_str.push(c)
        }

        while !sym_str.is_empty() {
            if SYMBOL_LIST.contains(&sym_str.borrow()) {

                // We only peek the second character, not consume it
                // if we matched with two characters, then consume it.
                if sym_str.len() == 2 {
                    self.chars.eat();
                }

                return Some(Ok(Token::symbol(sym_str)));
            } else {
                sym_str.pop();
            }
        }

        Some(Err(format!("unknown token: {}", first_char)))
    }

    fn next_comment(&mut self) -> Option<Result<Token>> {
        self.assert(';');
        let text: String = self.chars.consume_while(|c| c != '\n').collect();

        Some(Ok(Token::comment(text)))
    }

    fn eat_next_comment(&mut self) -> Option<Result<()>> {
        self.next_comment().map(|result| result.map(|_| ()))
    }

    fn assert(&mut self, expected: char) -> char {
        self.expect(expected).expect("unexpected character")
    }

    fn expect(&mut self, expected: char) -> Result<char> {
        // TODO: more specific error message
        self.expect_one_of(&[expected])
    }

    fn expect_something(&mut self) -> Result<char> {
        match self.chars.next() {
            Some(c) => Ok(c),
            None => Err(format!("expected a token but found nothing")),
        }
    }

    fn expect_one_of(&mut self, expected: &[char]) -> Result<char> {
        let expected_str = util::comma_separated_values(expected.iter());

        let next = match self.chars.next() {
            Some(c) => c,
            None => {
                return Err(format!("expected one of {} but got nothing", expected_str))
            },
        };

        if expected.iter().any(|&c| c == next) {
            Ok(next)
        } else {
            Err(format!("Expected one of {} but got {}", expected_str, next))
        }
    }
}

impl<I> Iterator for Tokenizer<I>
    where I: Iterator<Item=char>
{
    type Item = Result<Token>;

    fn next(&mut self) -> Option<Result<Token>> {
        if self.peek_buf.is_some() {
            let token = self.peek_buf.clone().unwrap();
            self.peek_buf = None;
            return Some(Ok(token));
        }

        self.eat_whitespace();

        let first_char = match self.chars.peek() {
            Some(c) => c,
            None => return None,
        };

        if first_char == '"' {
            self.next_string()
        } else if first_char == '\n' {
            self.chars.eat();
            Some(Ok(Token::new_line()))
        } else if first_char == ';' {
            if self.preserve_comments {
                self.next_comment()
            } else {

                // TODO: This could be nicer
                match self.eat_next_comment() {
                    Some(Ok(..)) => (),
                    Some(Err(e)) => return Some(Err(e)),
                    None => panic!("expected a comment to be parsed"),
                }

                // TODO: fix needless recursion
                self.next()
            }
        } else if first_char.is_numeric() {
            self.next_integer()
        } else if internal::can_word_start_with(first_char) {
            self.next_word()
        } else {
            // Try to parse the token as a symbol.
            // This is our final fallback.
            self.next_symbol()
        }
    }
}

mod internal
{
    pub fn can_word_start_with(c: char) -> bool {
        c.is_alphabetic() || c == '_'
    }

    pub fn can_word_contain(c: char) -> bool {
        can_word_start_with(c) || c.is_numeric()
    }
}

#[allow(unused_imports)]
mod test
{
    use read::{Token,Tokenizer};

    /// Expects a mapping from a string of tokens into
    /// a list of tokens with the default tokenizer.
    macro_rules! expect_mapping {
        ($input:expr => $( $output:expr ),* ) => {
            expect_mapping_with!(Tokenizer::new($input.chars()) =>
                                 $( $output ),*);
        }
    }

    /// Expects a mapping from a string of tokens into
    /// a list of tokens, given a specific tokenizer.
    macro_rules! expect_mapping_with {
        // TODO: make the macro take several arguments
        // for several outputs
        ($tokenizer:expr => $( $output:expr ),* ) => {
            {
                let mut tokenizer = $tokenizer;

                $(
                    let token = tokenizer.next().unwrap().unwrap();
                    assert!($output == token,
                            format!("expected {} but got {}",
                                    $output, token));
                )*
            }
        }
    }

    #[test]
    fn test_string() {
        expect_mapping!("\"hello\"" =>  Token::string("hello"),
                                        Token::new_line());

        expect_mapping!("\"hello abc\"" => Token::string("hello abc"),
                                           Token::new_line());

        expect_mapping!("\"hello world\"  \"it is me\"" =>
                        Token::string("hello world"),
                        Token::string("it is me"),
                        Token::new_line());
    }

    #[test]
    fn test_integer() {
        expect_mapping!("123" => Token::integer(123),
                                 Token::new_line());

        expect_mapping!("0982" => Token::integer(982),
                                  Token::new_line());

        expect_mapping!("333 662" => Token::integer(333),
                                     Token::integer(662),
                                     Token::new_line());

        expect_mapping!("1 2 3 4" => Token::integer(1),
                                     Token::integer(2),
                                     Token::integer(3),
                                     Token::integer(4),
                                     Token::new_line());
    }

    #[test]
    fn test_word() {
        expect_mapping!("hello" => Token::word("hello"));

        expect_mapping!("ab cd" => Token::word("ab"),
                                   Token::word("cd"),
                                   Token::new_line());

        expect_mapping!("a b c d" => Token::word("a"),
                                     Token::word("b"),
                                     Token::word("c"),
                                     Token::word("d"),
                                     Token::new_line());
    }

    #[test]
    fn test_comments_not_preserved_by_default() {
        expect_mapping!("hello ; there" => Token::word("hello"),
                                           Token::new_line());

        expect_mapping!(";why hello" => Token::new_line());
    }

    #[test]
    fn test_comments() {
        expect_mapping_with!(Tokenizer::new("hello world ; this is me".chars())
                             .preserve_comments()
                             => Token::word("hello"),
                                Token::word("world"),
                                Token::comment(" this is me"),
                                Token::new_line());

        expect_mapping_with!(Tokenizer::new(";this is a test".chars())
                             .preserve_comments()
                             => Token::comment("this is a test"),
                                Token::new_line());
    }

    #[test]
    fn test_symbols() {
        expect_mapping!("(" => Token::symbol("("),
                               Token::new_line());

        expect_mapping!(")" => Token::symbol(")"));
        expect_mapping!("{ }" => Token::symbol("{"), Token::symbol("}"));
        expect_mapping!(":" => Token::symbol(":"));
        expect_mapping!("," => Token::symbol(","));
        expect_mapping!("@%" => Token::symbol("@"), Token::symbol("%"));
        expect_mapping!(":=" => Token::symbol(":"), Token::symbol("="));
        expect_mapping!("->" => Token::symbol("->"));
        expect_mapping!("(->)" => Token::symbol("("), Token::symbol("->"),
                                  Token::symbol(")"));
    }

    #[test]
    fn test_multiple() {
        expect_mapping!("12 bark \"earth\"" => Token::integer(12),
                                               Token::word("bark"),
                                               Token::string("earth"),
                                               Token::new_line());

        expect_mapping!("a      23 gg \"a\"" => Token::word("a"),
                                                Token::integer(23),
                                                Token::word("gg"),
                                                Token::string("a"),
                                                Token::new_line());

        expect_mapping!("4:3{2\n}" => Token::integer(4),
                                      Token::symbol(":"),
                                      Token::integer(3),
                                      Token::symbol("{"),
                                      Token::integer(2),
                                      Token::new_line(),
                                      Token::symbol("}"),
                                      Token::new_line());
    }

    #[test]
    fn test_whitespace() {
        expect_mapping!("a\tb\t123\n\"qwer\"\n" => Token::word("a"),
                                                   Token::word("b"),
                                                   Token::integer(123),
                                                   Token::new_line(),
                                                   Token::string("qwer"),
                                                   Token::new_line(),
                                                   Token::new_line());
    }

    #[test]
    fn test_peek() {
        let mut tokenizer = Tokenizer::new("a b c".chars());

        assert_eq!(tokenizer.peek().unwrap().unwrap(),
                   Token::word("a"));

        assert_eq!(tokenizer.peek().unwrap().unwrap(),
                   Token::word("a"));

        assert_eq!(tokenizer.next().unwrap().unwrap(),
                   Token::word("a"));

        assert_eq!(tokenizer.peek().unwrap().unwrap(),
                   Token::word("b"));

        assert_eq!(tokenizer.next().unwrap().unwrap(),
                   Token::word("b"));

        assert_eq!(tokenizer.next().unwrap().unwrap(),
                   Token::word("c"));

        assert_eq!(tokenizer.next().unwrap().unwrap(),
                   Token::new_line());
    }
}


text:
using PayMe.Apps.Helpers;

namespace PayMe.Apps.ViewModels
{
    public class BaseViewModel : ObservableObject
    {

    }
}


text:
#ifndef ACTIVITYFACTORGENERATOR_H_INCLUDED
#define ACTIVITYFACTORGENERATOR_H_INCLUDED

#include <traceGenerator.h>
#include <fstream>

namespace meta {

	/*!
		Base class for generating traces
	*/
	class ActivityFactorGenerator: public TraceGenerator {
	public:

	public:
		ActivityFactorGenerator();
		virtual ~ActivityFactorGenerator();

		/*! \brief called when a traced kernel is launched to retrieve some
				parameters from the kernel
		*/
		virtual void initialize(const executive::ExecutableKernel& kernel);

		/*! \brief Called whenever an event takes place.

			Note, the const reference 'event' is only valid until event()
			returns
		*/
		virtual void event(const TraceEvent & event);

		/*! \brief Called when a kernel is finished. There will be no more
				events for this kernel.
		*/
		virtual void finish();

	protected:
		size_t invocations;
		std::string kernelName;
		ir::Dim3 blockDim;
		size_t dynamicInstructions;
		size_t threadInstructions;
	};
}

#endif


text:
<?php

namespace OCP\Files\External\Config;

use OCP\Files\External\Backend\Backend;

/**
 * Provider of external storage backends
 *
 * @since 10.0
 */
interface IBackendProvider {

	/**
	 * @since 9.1.0
	 * @return Backend[]
	 */
	public function getBackends();
}


text:
package cmd

import (
	"net/http/httptest"
	"net/url"
	"testing"

	"github.com/stretchr/testify/assert"

	"github.com/ory/x/pagination/tokenpagination"
)

func TestFatal(t *testing.T) {
	oldOsExit := osExit
	defer func() { osExit = oldOsExit }()

	var got int
	myExit := func(code int) {
		got = code
	}

	osExit = myExit
	fatal("Fatal message")
	if exp := 1; got != exp {
		t.Errorf("Expected exit code: %d, got: %d", exp, got)
	}
}

func TestGetPageToken(t *testing.T) {
	u, _ := url.Parse("https://example.com/foobar")
	rec := httptest.NewRecorder()
	tokenpagination.PaginationHeader(rec, u, 100, 3, 10)
	assert.Equal(t, `eyJwYWdlIjoiNDAiLCJ2IjoxfQ`, getPageToken(rec.Result()), rec.Result().Header.Get("Link"))
}