text:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://docbook.org/xml/4.5/docbookx.dtd">

<article>

  <articleinfo>
    <title>Ach IPC User Manual</title>
    <author>
       <firstname>Neil</firstname>
       <surname>Dantam</surname>
       <othername role='mi'>T</othername>
       <affiliation>
          <!-- Valid email...spamblock/scramble if so desired -->
          <address><email>ntd at rice dot edu</email></address>
       </affiliation>
     </author>

     <!-- All dates specified in ISO "YYYY-MM-DD" format -->
     <pubdate>2015-02-02</pubdate>

     <copyright>
       <year>2012-2014</year>
       <holder>Georgia Tech Research Corporation</holder>
     </copyright>

     <copyright>
       <year>2015</year>
       <holder>Rice University</holder>
     </copyright>

     <!-- Most recent revision goes at the top; list in descending order -->
     <revhistory>
       <revision>
          <revnumber>2.0.0</revnumber>
          <date>2015-02-02</date>
          <authorinitials>NTD</authorinitials>
          <revremark>
            New Release with kernel channels and improved mDNS
            integration
          </revremark>
       </revision>
       <revision>
          <revnumber>1.2.1</revnumber>
          <date>2014-04-07</date>
          <authorinitials>NTD</authorinitials>
          <revremark>New Release with achd period, SSH example,
          ACH_OVERFLOW clarification</revremark>
       </revision>
       <revision>
          <revnumber>1.2.0</revnumber>
          <date>2013-06-14</date>
          <authorinitials>NTD</authorinitials>
          <revremark>New Release with ach_cancel, achcop, achlog</revremark>
       </revision>
       <revision>
          <revnumber>1.1.0-1</revnumber>
          <date>2013-04-16</date>
          <authorinitials>NTD</authorinitials>
          <revremark>Few fixes and add Xenomai note</revremark>
       </revision>
       <revision>
          <revnumber>1.1.0</revnumber>
          <date>2013-02-12</date>
          <authorinitials>NTD</authorinitials>
          <revremark>Initial Release</revremark>
       </revision>
     </revhistory>

     <abstract>
        <para>
          This is the manual for the Ach IPC library.
        </para>

         <para>
           Ach is an Inter-Process Communication (IPC) mechanism and
           library, intended for communication in real-time systems
           that sample data from physical processes. Ach eliminates
           the Head-of-Line Blocking problem for applications that
           always require access to the newest message. Source code is
           provided under the 2-clause BSD license.
         </para>

     </abstract>
  </articleinfo>

  <sect1>
    <title>
      Introduction
    </title>

    <para>
      Ach provides a message bus or publish-subscribe style of
      communication between multiple writers and multiple readers.  A
      real-time system will generally have multiple Ach channels
      across which individual data samples are published.  The
      messages sent on a channel are simple byte arrays, so arbitrary
      data may be transmitted such as text, images, and binary control
      messages.  Each channel is implemented as two circular buffers,
      (1) a data buffer with variable sized entries and (2) an index
      buffer with fixed-size elements indicating the offsets into the
      data buffer. These two circular buffers are written in a
      channel-specific POSIX shared memory file. This frees users from
      managing synchronization, which is contained within the Ach
      library.
    </para>

    <para>The Ach interface consists of the following procedures:</para>

    <variablelist>
      <varlistentry>
        <term><function>ach_create</function></term>
        <listitem>
          <para>Create the shared memory region and initialize its data structures</para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><function>ach_open</function></term>
        <listitem>
          <para>Open the shared memory file and initialize process local channel counters</para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><function>ach_put</function></term>
        <listitem><para>Insert a new message into the channel</para></listitem>
      </varlistentry>
      <varlistentry>
        <term><function>ach_get</function></term>
        <listitem><para>Receive a message from the channel</para></listitem>
      </varlistentry>
      <varlistentry>
        <term><function>ach_close</function></term>
        <listitem><para>Close the shared memory file</para></listitem>
      </varlistentry>
    </variablelist>

    <para>
      Channels must be created before they can be opened. Creation may
      be done directly by either the reading or writing process, or it
      may be done via the shell command, <command>ach mk
      channel_name</command>, before the reader or writer start. After
      the channel is created, each reader or writer must open the
      channel before it can individually get or put messages.
    </para>
  </sect1>

  <sect1>
    <title>
      Libach API
    </title>

    <sect2><title>Status Codes</title>
    <para>The Ach API functions use return status codes to indicate either
    successful completion or an error condition.  The following codes are defined:</para>
    <variablelist>
      <varlistentry><term><constant>ACH_OK</constant></term>
      <listitem><para>Operation completed successfully.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_OVERFLOW</constant></term>
      <listitem><para>Destination was too small. If returned from
      <function>ach_put</function>, the channel is too small to hold
      the message you attempted to send. If returned from
      <function>ach_get</function>, the buffer you passed to receive
      the message was too small to hold the result.  If the channel is
      too small, you should specify a larger nominal message size when
      creating the channel.  If the buffer is to small, pass in a
      bigger buffer.</para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_INVALID_NAME</constant></term>
      <listitem><para>An invalid channel name.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_BAD_SHM_FILE</constant></term>
      <listitem><para>An invalid channel file.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_FAILED_SYSCALL</constant></term>
      <listitem><para>A system call was unsuccessful.  Check the
      <varname>errno</varname> to determine what went wrong.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_STALE_FRAMES</constant></term>
      <listitem><para>No new data has be published to channel.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_MISSED_FRAME</constant></term>
      <listitem><para>The receiver has skipped over some frames (which
      may no longer be stored in the channel).  You will still get
      either the latest or the oldest frame, depending on which you
      requested.  </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_TIMEOUT</constant></term>
      <listitem><para>Timeout occurred.  No data received.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_CANCELED</constant></term>
      <listitem><para>Message wait was canceled.  </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_EEXIST</constant></term>
      <listitem><para>Channel already exists.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_ENOENT</constant></term>
      <listitem><para>Channel does not exist.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_EACCES</constant></term>
      <listitem><para>No permission to access the channel file.  You
      will want to either run as the appropriate user or set the
      channel permissions appropriately.  </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_EINVAL</constant></term>
      <listitem><para>An invalid parameter was passed.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_CORRUPT</constant></term>
      <listitem><para>Corruption of the channel file detected.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_BAD_HEADER</constant></term>
      <listitem><para>Invalid header line in the network protocol.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_FAULT</constant></term>
      <listitem><para>Memory fault in a system call.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_EINTR</constant></term>
      <listitem><para>System call interrupted.  This code is only used
      internally and not returned to library clients.
      </para></listitem>
      </varlistentry>
      <varlistentry><term><constant>ACH_BUG</constant></term>
      <listitem><para>An unexpected or inconsistent condition has
      occurred.  If the software is correct, you should never see this
      code.  If you do see this code, please contact the authors.
      </para></listitem>
      </varlistentry>
    </variablelist>
    </sect2>

    <sect2><title>Creating a channel</title>
    <para>
      You can create a channel by directly calling the<function>
      ach_create</function>; however, it is generally preferable to
      call the <command>ach</command> (see <link
      linkend='achtool'>also</link>) from the shell as this is less
      likely to result in races between multiple processes needing to
      access the channel.
    </para>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_create</function></funcdef>
        <paramdef>const char *<parameter>channel_name</parameter></paramdef>
        <paramdef>size_t <parameter>frame_cnt</parameter></paramdef>
        <paramdef>size_t <parameter>frame_size</parameter></paramdef>
        <paramdef>ach_create_attr_t *<parameter>attr</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>
    <para>
      Channels are created using the <function>ach_create</function>
      function.  A channel must be created before it can be opened.
      This function will create and initialize a POSIX shared memory
      file for the channel.  The channel can then be opened by passing
      the same <varname>channel_name</varname> to
      <function>ach_open</function>.
    </para>

      <example><title>Creating a Channel</title>
      <programlisting language="C">
enum ach_status r = ach_create( "my_channel", 10, 512, NULL );
if( ACH_OK != r ) {
    fprintf( stderr, "Could not create channel: %s\n", ach_result_to_string(r) );
    exit(EXIT_FAILURE);
}
      </programlisting>
      </example>

      <para>Note that the message size given here is not a strict
      constraint.  Individual messages are allowed to be smaller or
      larger than this value.  The only hard constraint is that a
      total of frame_cnt*frame_size is allocated for all message data
      in the channel.  Thus, the total data required by all buffered
      messages cannot exceed this value (older messages are
      overwritten), and no individual message may be larger than this
      value. </para>
    </sect2>

    <sect2><title>Opening a channel</title>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_open</function></funcdef>
        <paramdef>ach_channel_t *<parameter>channel</parameter></paramdef>
        <paramdef>const char *<parameter>channel_name</parameter></paramdef>
        <paramdef>ach__attr_t *<parameter>attr</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>
    <para>Opens a channel for use within this process.  The channel
    must be opened before messages can be sent or received</para>

      <example><title>Opening a Channel</title>

      <programlisting language="C">
ach_channel_t channel;
enum ach_status r = ach_open( &channel, "my_channel", NULL );
if( ACH_OK != r ) {
    syslog( LOG_ERR, "Could not open channel: %s\n", ach_result_to_string(r) );
    exit(EXIT_FAILURE);
}
      </programlisting>

      </example>

      <warning>
        <para>
          Channels, of type <type>ach_channel_t</type>, should always
          be passed by reference.  Copying or passing by value types
          of <type>ach_channel_t</type> may result in undefined
          behavior (and will probably not do what you want).  Always
          pass the pointer to <type>ach_channel_t</type> instead.
        </para>
      </warning>

    </sect2>

    <sect2><title>Sending Data</title>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_put</function></funcdef>
        <paramdef>ach_channel_t *<parameter>channel</parameter></paramdef>
        <paramdef>const void *<parameter>buf</parameter></paramdef>
        <paramdef>size_t <parameter>cnt</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>
    <para>The <function>ach_put</function>
    <indexterm><primary>ach_put</primary></indexterm> function writes
    a new message to the channel.  This will go in the next open space
    in the circular array.  If there is insufficient unused space in
    the circular array, then the oldest entry or entries will be
    overwritten.  If the entire circular array is too small for the
    new message, the message is not written and
    <constant>ACH_OVERFLOW</constant> is returned.</para>

      <example><title>Opening a Channel</title>

      <programlisting language="C">
struct my_msg_type my_msg;
/* Fill in my_msg with useful data */
enum ach_status r = ach_put( &channel, &my_msg, sizeof(my_msg) );
if( ACH_OK != r ) {
    syslog( LOG_ERR, "Could not put data: %s\n", ach_result_to_string(r) );
}
      </programlisting>

      </example>

    </sect2>

    <sect2><title>Receiving Data</title>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_get</function></funcdef>
        <paramdef>ach_channel_t *<parameter>channel</parameter></paramdef>
        <paramdef>void *<parameter>buf</parameter></paramdef>
        <paramdef>size_t <parameter>buf_size</parameter></paramdef>
        <paramdef>size_t *<parameter>frame_size</parameter></paramdef>
        <paramdef>const struct timespec * restrict <parameter>abstime</parameter></paramdef>
        <paramdef>int <parameter>options</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>
    <para>The <function>ach_get</function> is used to received data
    from a channel.  The <varname>options</varname> parameter controls
    the behavior of this function and is the bitwise or of the
    following values.</para>

    <variablelist>

      <varlistentry><term><constant>ACH_O_NONBLOCK</constant></term>
      <listitem><para>
        Return immediately with ACH_STALE_FRAMES if no new data is
        present in the channel.  Exclusive with ACH_O_WAIT.
      </para></listitem>
      </varlistentry>

      <varlistentry><term><constant>ACH_O_WAIT</constant></term>
      <listitem><para>
        Wait until a new message is posted. Default behavior without
        this flag is to return immediately.  Exclusive with
        ACH_O_NONBLOCK.
      </para></listitem>
      </varlistentry>

      <varlistentry><term><constant>ACH_O_FIRST</constant></term>
      <listitem><para>
        Return the oldest message in the channel.  Exclusive with
        ACH_O_LAST.
      </para></listitem>
      </varlistentry>

      <varlistentry><term><constant>ACH_O_LAST</constant></term>
      <listitem><para>
        Return the newest message.  Exclusive with ACH_O_FIRST.
      </para></listitem>
      </varlistentry>

      <varlistentry><term><constant>ACH_O_ABSTIME</constant></term>
      <listitem><para>
        Timeout is an absolute time.  Exclusive with ACH_O_RELTIME.
      </para></listitem>
      </varlistentry>

      <varlistentry><term><constant>ACH_O_RELTIME</constant></term>
      <listitem><para>
        Timeout is a relative time.  Exclusive with ACH_O_ABSTIME.
      </para></listitem>
      </varlistentry>

      <varlistentry><term><constant>ACH_O_COPY</constant></term>
      <listitem><para>Copy the message to the buffer, even if we have
      already seen it.  Only valid when
      <constant>ACH_O_WAIT</constant> is not given.  Default behavior
      without this flag is to return
      <constant>ACH_STALE_FRAMES</constant> when there are no new
      messages.</para></listitem>
      </varlistentry>

    </variablelist>

      <example><title>Poll for oldest message</title>
      <para>This will get the next unseen message from the channel.</para>
      <programlisting language="C">
size_t frame_size;
enum ach_status r;
r  = ach_get( &channel, &my_msg, my_msg_size, &frame_size,
              NULL, ACH_O_NONBLOCK | ACH_O_FIRST );
if( ACH_MISSED_FRAME == r ) {
    printf("Missed a/some messages(s)\n");
} else if( ACH_STALE_FRAMES == r ) {
    printf("No new data\n");
} else if( ACH_OK != r ) {
    syslog( LOG_ERR, "Unable to get a message: %s\n", ach_result_to_string(r) );
} else if( frame_size != sizeof(my_msg) ) {
    syslog( LOG_WARNING, "Unexpected message size %" PRIuPTR ", expecting %" PRIuPTR "\n",
            frame_size, sizeof(my_msg) );
}
      </programlisting>
      </example>

      <example><title>Wait for newest message</title>
      <para>This will get the newest message from the channel.  If no
      unseen messages are in the channel, it will wait forever until
      one is posted. </para>
      <programlisting language="C">
enum ach_status r = ach_get( &channel, &my_msg, my_msg_size, &frame_size, NULL, ACH_O_WAIT | ACH_O_LAST );
      </programlisting>
      </example>

      <example><title>Timed wait for newest message</title>
      <para>This will get the newest message from the channel.  If no
      unseen messages are in the channel, it will up to one second until
      one is posted. </para>
      <programlisting language="C">
struct timespec t;
clockid_t clock;
enum ach_status r;
/* Get the channel clock */
r = ach_channel_clock(&channel,&clock);
if( ACH_OK != r ) {
  fprintf(stderr, "Could not get channel clock: %s\n",
          ach_result_to_string(r));
  exit(EXIT_FAILURE);
}
/* Get current time and set the timeout. */
clock_gettime( clock, &t );
t.tv_sec += 1;
/* Get the message */
enum ach_status r = ach_get( &channel, &my_msg, my_msg_size, &frame_size, &t,
                             ACH_O_WAIT | ACH_O_LAST | ACH_O_ABSTIME );
if( ACH_TIMEOUT == r ) {
  fprintf(stdout, "call to ach_get timed out\n");
}
      </programlisting>
      </example>

      <note><title>Does anybody know what time it is?</title>
      <para>While ach channels default to using
      <constant>CLOCK_MONOTONIC</constant> for timed waits, the issue
      of determining the correct time is fraught with complications.
      Traditionally, unix time -- seconds since January 1, 1970 -- has
      been defined based on UTC.  This is given to nanosecond
      precision with <constant>CLOCK_REALTIME</constant> (nothing to
      do with real-time-as-in-low-latency-or-motion-control).
      However, UTC and thus typically
      <constant>CLOCK_REALTIME</constant> is discontinuous.  It may be
      reset by the operator, by the NTP daemon, and it is
      <emphasis>defined</emphasis> as discontinous when leap seconds
      occur.  This is bad when one wants to use the clock for periodic
      events as is likely for motion control.
      <constant>CLOCK_MONOTONIC</constant> is not discontinous, but
      even it has issues.  The NTP daemon may slew this clock,
      speeding it up or slowing it down on Linux by 0.5ms per second.
      Linux does provide the <constant>CLOCK_MONOTONIC_RAW</constant>
      which is not slewed, but is also not portable.
      </para>
      </note>

    </sect2>

    <sect2><title>Cancel message wait</title>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_cancel</function></funcdef>
        <paramdef>ach_channel_t *<parameter>channel</parameter></paramdef>
        <paramdef>const ach_cancel_attr_t *<parameter>attr</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>

    <para>
      To interrupt an in-progress call to <function>ach_get</function>
      that has blocked waiting for a new message, use the
      <function>ach_cancel</function> function.  This will cause
      <function>ach_get</function> to return
      <constant>ACH_CANCELED</constant>.
    </para>

      <example><title>Cancel waits upon a signal</title>
      <para>
        This will install a signal handler to cancel waits when a
        signal is received.
      </para>
      <programlisting language="C">
ach_channel_t channel;

...

static void sighandler_cancel ( int sig ) {
    enum ach_status r = ach_cancel(&channel, NULL);
    if( ACH_OK != r ) exit(EXIT_FAILURE);
}

...

void setup_sighandler_cancel ( int sig ) {
    struct sigaction act;
    memset(&act, 0, sizeof(act));
    act.sa_handler = &sighandler_cancel;
    if( sigaction(sig, &act, NULL) ) exit(EXIT_FAILURE);
}
      </programlisting>
      </example>

      <example><title>Cancel waits in another thread</title>
      <para>
        This will notify waits in another thread to cancel.
      </para>
      <programlisting language="C">
ach_channel_t channel;

...

ach_cancel_attr_t attr;
ach_cancel_attr_init(&attr);
attr.async_unsafe = 1; /* permit functions that are unsafe in signal handlers */
enum ach_status r = ach_cancel(&channel, &attr);
if( ACH_OK != r ) exit(EXIT_FAILURE);
      </programlisting>
      </example>

    </sect2>

    <sect2><title>Closing a channel</title>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_close</function></funcdef>
        <paramdef>ach_channel_t *<parameter>channel</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>
    </sect2>

    <sect2><title>Deleting a channel</title>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_unlink</function></funcdef>
        <paramdef>const char *<parameter>name</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>
    </sect2>

    <sect2><title>Channel Permissions</title>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_chmod</function></funcdef>
        <paramdef>ach_channel_t *<parameter>channel</parameter></paramdef>
        <paramdef>mode_t <parameter>mode</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>
    <para>Set the POSIX permission bits of the channel to
    <varname>mode</varname>.  Note that any channel access requires
    both the read (4) and write (2) bits to be set, because we must
    write in order to hold the mutex.  The executable bit (1) is
    irrelevant.</para>

    <example><title>Make channel user and group accessible</title>
    <para>It is best to specify <varname>mode</varname> as an octal
    number.  A value of 0660 will give the owning user and group
    access, and deny access to all others.</para>
    <programlisting language="C">
      enum ach_status r = ach_get( &channel, 0660 );
    </programlisting>
    </example>
    </sect2>

    <sect2><title>Ignore old messages</title>
    <funcsynopsis>
      <funcprototype>
        <funcdef>enum ach_status <function>ach_flush</function></funcdef>
        <paramdef>ach_channel_t *<parameter>channel</parameter></paramdef>
      </funcprototype>
    </funcsynopsis>
    <para>Updates channel counters to ignore all previously published
    messages.</para>
    </sect2>

  </sect1>

  <sect1>
    <title>Language Bindings</title>

    <sect2><title>Common Lisp</title>
    <para>
      Ach includes bindings for Common Lisp using CFFI.  They can be loaded via ASDF.
    </para>
    </sect2>

    <sect2><title>Python</title>
    <para>
      Ach includes bindings for Python using a C extension module and
      wrapper class.
    </para>

    <para>
      The module uses Python's <emphasis>Buffer Protocol</emphasis> to
      obtain a serialized representation for Python objects.  This
      will work with <classname>string</classname>,
      <classname>bytearray</classname>,
      <classname>ctypes.Structure</classname> and other types which
      implement the Buffer Protocol.
    </para>

    <example><title>Python Put</title>
    <programlisting language="python">
#!/usr/bin/env python
import ach
c = ach.Channel('foo')
c.put('bar')
c.close()
    </programlisting>
    </example>

    <example><title>Python Get</title>
    <programlisting language="python">
#!/usr/bin/env python
import ach
c = ach.Channel('foo')
c.flush()
b = bytearray(10)
[status, framesize] = c.get( b, wait=True, last=False )
if status == ach.ACH_OK or status == ach.ACH_MISSED_FRAME:
    print b
else:
    raise ach.AchException( c.result_string(status) )
c.close()
    </programlisting>
</example>
    </sect2>
  </sect1>

  <sect1 id="achtool">
    <title>ach Shell Tool</title>
      <para>The <command>ach</command> command allows creation and
      deletion of channels from the shell.
      </para>

      <cmdsynopsis>
        <command>ach</command>
        <group choice="req">
          <arg choice="req">mk</arg>
          <arg choice="req">rm</arg>
          <arg choice="req">chmod</arg>
          <arg choice="req">dump</arg>
          <arg choice="req">file</arg>
        </group>
        <arg><replaceable>octal_mode</replaceable></arg>
        <arg choice="req"><replaceable>chanel_name</replaceable></arg>
        <arg>-o <replaceable>octal_mode</replaceable></arg>
        <arg>-m <replaceable>frame_count</replaceable></arg>
        <arg>-n <replaceable>frame_size</replaceable></arg>
        <arg>-t</arg>
        <arg>-v</arg>
        <arg>-V</arg>
        <arg>-?</arg>
      </cmdsynopsis>

      <example><title>Create a channel</title>
      <para>Create channel named "my_channel" with slots for 10
      messages which have a nominal size of 64 bytes.</para>
      <cmdsynopsis>
        <command>ach</command>
         <arg choice="plain">mk</arg>
         <arg choice="plain"><replaceable>my_channel</replaceable></arg>
         <arg choice="plain">-m <replaceable>10</replaceable></arg>
         <arg choice="plain">-n <replaceable>64</replaceable></arg>
      </cmdsynopsis>
      </example>

      <example><title>Delete a channel</title>
     <para>Delete channel named "my_channel".</para>
      <cmdsynopsis>
        <command>ach</command>
         <arg choice="plain">rm</arg>
         <arg choice="plain"><replaceable>my_channel</replaceable></arg>
      </cmdsynopsis>
      </example>

      <example><title>Create a world-accessible channel</title>
      <para>Create channel named "my_channel" which is
      world-accessible (permisions 666).
      </para>
      <cmdsynopsis>
        <command>ach</command>
         <arg choice="plain">mk</arg>
         <arg choice="plain"><replaceable>my_channel</replaceable></arg>
         <arg choice="plain">-m <replaceable>10</replaceable></arg>
         <arg choice="plain">-n <replaceable>64</replaceable></arg>
         <arg choice="plain">-o <replaceable>666</replaceable></arg>
      </cmdsynopsis>
      </example>

      <example><title>Create a channel unless it already exists</title>
      <para>Create channel named "my_channel" unless it already
      exists, in which case do nothing.
      </para>
      <cmdsynopsis>
        <command>ach</command>
         <arg choice="plain">mk</arg>
         <arg choice="plain"><replaceable>my_channel</replaceable></arg>
         <arg choice="plain">-m <replaceable>10</replaceable></arg>
         <arg choice="plain">-n <replaceable>64</replaceable></arg>
         <arg choice="plain">-1</arg>
      </cmdsynopsis>
      </example>

      <example><title>Set channel permissions</title>
      <para>Make channel accessible only by user and group.</para>
      <cmdsynopsis>
        <command>ach</command>
         <arg choice="plain">chmod</arg>
         <arg choice="plain"><replaceable>660</replaceable></arg>
         <arg choice="plain"><replaceable>my_channel</replaceable></arg>
      </cmdsynopsis>
      </example>

  </sect1>

  <sect1>
    <title>Linux Kernel Module</title>
    <para>
      Ach includes a Linux kernel module which implements channels in
      kernel-space.  This allows library clients to wait for messages
      on multiple channels (and in combination with any other file
      descriptors) using the <function>poll()</function> or
      <function>select()</function> family of functions.
    </para>

    <sect2><title>Kernel Module Installation</title>

    <para>
      The most convenient way to build the kernel module will probably
      be with <ulink
      url="http://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support">DKMS</ulink>
      (Dynamic Kernel Module System).  DKMS enables building and
      rebuilding of out-of-tree kernel modules (such as ach, device
      drivers, etc.) when the kernel is upgraded.  On Debian and
      Ubuntu, the package manager automatically invokes DKMS to
      rebuild the registered modules.  To setup ach for DKMS, call
      configure with <command>./configure
      --enable-dkms-build</command>, and the module sources for will
      be installed and built when you call <command>make
      install</command>.
    </para>
    <example><title>Build kernel module with dkms</title>
    <programlisting>
./configure --enable-dkms-build
make
sudo make install
</programlisting>
    </example>
    <para>
      Alternatively, you can build the module in the ach source tree
      directly by calling <command>./configure --enable-kbuild
      --disable-dkms</command>.  If your kernel headers are not in the
      typical location and/or do not match the currently running
      kernel, you may to specify the kernel release and header
      directory.
    </para>
    <example><title>Manually specifying kernel information for kbuild</title>
    <programlisting>
./configure --enable-kbuild \
            --disable-dkms \
             KERNELRELEASE=3.13.0-44-lowlatency \
             KDIR=/usr/src/linux-headers-3.13.0-44-lowlatency
make
sudo make install
</programlisting>
    </example>

    <para>
      After the kernel module is build and installed, you need to link
      it into the kernel.  This is done with the
      <command>modprobe</command> command.  The kernel module can be
      removed with <command>rmmod</command>.  The module takes one
      optional parameter, <parameter>max_devices</parameter> which
      specifies the maximum number of kernel channels that can be
      created.
    </para>

    <example><title>Load kernel module</title>
    <programlisting>
sudo modprobe ach
    </programlisting>
    </example>

    <example><title>Load kernel module and specify max_devices</title>
    <programlisting>
sudo modprobe ach max_devices=1024
    </programlisting>
    </example>

    <example><title>Unload kernel module</title>
    <programlisting>
sudo rmmod ach
    </programlisting>
    </example>

    </sect2>

    <sect2><title>Creating Kernel Channels</title>

    <para>
      Kernel channels can be created using the <command>ach</command>
      command by passing the <parameter>-k</parameter> flag.
    </para>

    <example><title>Create kernel channel from shell</title>
    <programlisting>
ach mk -k my-channel
    </programlisting>
    </example>

    <example><title>Unlink kernel channel from shell</title>
    <programlisting>
ach rm my-channel
    </programlisting>
    </example>

    </sect2>

    <sect2><title>Kernel Channel Permissions</title>

    <para>
      By default, kernel channels can only be created and accessed by
      root.  However, you can use <ulink
      url="http://en.wikipedia.org/wiki/Udev">udev</ulink> to control
      kernel channel permissions.  The following example will set the
      group ownership of all ach devices to the <code>ach</code>
      group.  The file needs to be placed in
      <filename>/etc/udev/rules.d/</filename>.
    </para>

    <example><title>udev rules file to set channel group </title>
    <programlisting>
# /etc/udev/rules.d/10-ach.rules
# Control permissions for ach character devices

# Set the permissions of the ach control device
KERNEL=="achctrl", GROUP=="ach", MODE=="660"

# Set the permissions of ach channel devices
KERNEL=="ach-*", GROUP=="ach", MODE=="660"

# Create symbolic links for ach channel devices
KERNEL=="ach-*", SYMLINK+="ach/%k"
    </programlisting>
    </example>

    </sect2>

    <sect2><title>Waiting on Multiple Kernel Channels</title>
    <para>
      Kernel channels support waiting for multiple events using the
      <function>poll</function>, <function>select</function> and
      similar functions.  These functions require the file descriptor
      for the channel.
    </para>

    <example><title>Obtaining the channel file descriptor</title>
    <programlisting>
ach_channel_t channel;
int channel_fd;
enum ach_status r;

/* Open channel */
/* ... */

/* Get Channel File Descriptor */
r = ach_channel_fd( &channel, &channel_fd );
if( ACH_OK != r ) {
    fprintf(stderr, "could not get file descriptor for channel '%s': %s\n",
            names[i], ach_result_to_string(r));
    exit(EXIT_FAILURE);
}
    </programlisting>
    </example>

    <para>
      The channel file descriptor can then be passed to
      <function>poll</function>, <function>select</function>, etc. to
      determine when there is new data to read.  Note that channels
      are always ready to write.
    </para>

    <example><title>Complete poll() example</title>
    <programlisting>
#include <stdlib.h>
#include <pthread.h>
#include <inttypes.h>
#include <ach.h>

#include <stdio.h>
#include <poll.h>
#include <unistd.h>

int main(int argc, char **argv)
{
    const char *names[] = {"channel-0", "channel-1"};
    const size_t n = sizeof(names) / sizeof(names[0]);
    ach_channel_t channel[n];
    struct pollfd pfd[n];

    for( size_t i = 0; i < n; i ++ ) {
        /* Open Channel */
        enum ach_status r = ach_open( &channel[i], names[i], NULL );
        if( ACH_OK != r ) {
            fprintf(stderr, "could not open channel '%s': %s\n",
                    names[i], ach_result_to_string(r));
            exit(EXIT_FAILURE);
        }
        /* Get Channel File Descriptor */
        r = ach_channel_fd( &channel[i], &pfd[i].fd );
        if( ACH_OK != r ) {
            fprintf(stderr, "could not get file descriptor for channel '%s': %s\n",
                    names[i], ach_result_to_string(r));
            exit(EXIT_FAILURE);
        }
        /* Set events to poll for */
        pfd[i].events = POLLIN;
    }

    /* read forever */
    for(;;) {
        /* poll for new messages */
        int r_poll = poll( pfd, n, -1 );
        if( r_poll < 0 ) {
            perror("poll");
            exit(EXIT_FAILURE);
        }
        /* find channels with new data */
        for( size_t i = 0; i < n && r_poll > 0; i++ ) {
            if( (pfd[i].revents & POLLIN) ) {
                /* There's new data on this channel */
                char buf[512];
                size_t frame_size;
                enum ach_status r = ach_get( &channel[i], buf, sizeof(buf), &frame_size,
                                             NULL, ACH_O_NONBLOCK | ACH_O_FIRST );
                switch(r) {
                case ACH_OK:
                case ACH_MISSED_FRAME: {
                    /* this example just writes it to stdout */
                    ssize_t wr = write( STDOUT_FILENO, buf, frame_size );
                    if( wr < 0 ) {
                        perror("write");
                        exit(EXIT_FAILURE);
                    }
                    break;
                }
                default:
                    fprintf( stderr, "Error getting data from '%s': %s\n",
                             names[i], ach_result_to_string(r));
                    exit(EXIT_FAILURE);
                }
                r_poll--;
            }
        }
    }
    return 0;
}
    </programlisting>
    </example>

    <tip>
      <para>
        The <function>poll</function> function has an advantage over
        <function>select</function> in that the arguments to
        <function>poll</function> can be reused over multiple calls
        while the arguments to select must be recreated for each
        call. However <function>poll</function> provides only
        millisecond timing resolution.  Instead, the more recent
        <function>ppoll</function> supports nanosecond timing
        resolution.  For details on these functions, consult the man
        pages with <command>man -s 2 poll</command> and <command>man
        -s 2 ppoll</command>.
      </para>
    </tip>

    </sect2>

  </sect1>

  <sect1>
    <title>
      achd Network Daemon
    </title>
    <para> While the primary design goal of Ach is low-latency, single
    host IPC, the <command>achd</command> daemon additionally provides a way
    to relay messages over the network. </para>

    <sect2><title>Server</title>

    <cmdsynopsis>
      <command>achd</command>
      <arg choice="plain">serve</arg>
    </cmdsynopsis>

    <para>The achd server should be setup to run from the
    <command><ulink
    url="http://en.wikipedia.org/wiki/Inetd">inetd</ulink></command>
    super-server. Some distributions use the alternative
    <command><ulink
    url="http://www.xinetd.org/">xinetd</ulink></command>, which is
    configured differently.</para>

    <note>
      <para>
        Inetd and xinetd work by listening for incoming connections on
        all the ports specified in their configuration.  When (x)inetd
        recieves a new connection, it forks off the process as given
        in the configuration file, (in this case achd), and hooks the
        standard input and output of that process to the socket
        connection.
      </para>
    </note>

    <tip>
      <para>On Debian-based systems (including Ubuntu and Mint), you
      can use the <userinput>openbsd-inetd</userinput> package,
      installed with <userinput>apt-get install
      openbsd-inetd</userinput>.  Alternatively, you can install
      xinetd with with <userinput>apt-get install xinetd</userinput>.
      </para>
    </tip>

    <sect3><title>Traditional Inetd Configuration</title>

    <para>You should add the following line to
    <filename>/etc/inetd.conf</filename>, assuming you have installed
    ach under <filename>/usr/local/</filename>:</para>

    <screen>
#/etc/inetd.conf
8076  stream  tcp  nowait  nobody  /usr/local/bin/achd /usr/local/bin/achd serve
    </screen>

    <tip>
      <para>
        On Debian-based systems where you have installed ach via the
        package manger, you can enable achd in inetd by running
        <userinput>dpkg-reconfigure ach-utils</userinput>.
      </para>
    </tip>

    <para>
      You will probably need to restart inetd after editing the
      configuration file.  How to do this depends on the init system
      used by your operating system and the particular inetd you are
      using.  Here are some options which may work:
    </para>
    <variablelist>
      <varlistentry><term>SysV Init (general GNU/Linux)</term>
      <listitem><para><userinput>/etc/init.d/inetd
      restart</userinput></para></listitem>
      </varlistentry>
      <varlistentry><term>Debian (Ubuntu/Mint) with openbsd-inetd</term>
      <listitem><para><userinput>service openbsd-inetd
      restart</userinput></para></listitem>
      </varlistentry>
    </variablelist>
    </sect3>

    <sect3><title>Xinetd Configuration</title>
    <para>If xinetd is run with the with the inetd_compat flag, then
    you can use a similar configuration to the traditional inetd.
    </para>

    <screen>
#/etc/inetd.conf
achd  stream  tcp  nowait  nobody  /usr/local/bin/achd /usr/local/bin/achd serve
</screen>

    <para>
    Otherwise, you can add the following stanza to your xinetd
    configuration.  Typically, this would go in the per-service file
    <filename>/etc/xinetd.d/achd</filename>.
    </para>
    <screen>
#/etc/xinetd.d/achd
service achd
{
	flags           = REUSE
	socket_type     = stream
	protocol        = tcp
	port            = 8076
	wait            = no
	user            = nobody
	server          = /usr/local/bin/achd
	server_args     = serve
	disable         = no
}
    </screen>

    <para>In addition, you will need to list the service in
    <filename>/etc/services</filename>.
    </para>
    <screen>
#/etc/services
achd           8076/tcp
    </screen>

<para>Then, tell xinetd to reload its configuration</para>

    <variablelist>
      <varlistentry><term>SysV Init (general GNU/Linux)</term>
      <listitem><para><userinput>/etc/init.d/xinetd
      restart</userinput></para></listitem>
      </varlistentry>
      <varlistentry><term>Redhat/Debian/Ubuntu with xinetd</term>
      <listitem><para><userinput>service xinetd reload</userinput></para></listitem>
      </varlistentry>
    </variablelist>
    </sect3>
    <sect3><title>Testing Server Configuration</title>

    <para>Now, you can test this setup by telnetting to port 8076 on
    the server: <userinput>telnet SERVER 8076</userinput>.  If you
    type in some gibberish (e.g. <userinput>asdf</userinput>) followed
    by a newline, then you should see something like this:
    </para>
    <screen>
Escape character is '^]'.
asdf
status: 14 # ACH_BAD_HEADER
message: malformed header

.
Connection closed by foreign host.
    </screen>

    <para>Which will mean that achd is properly setup.</para>

    <tip>
      <para>
        If <command>achd</command> is not operating properly, check
        the log files on your machine for more information.  The log
        output will typically be stored in a file such as
        <filename>/var/log/daemon.log</filename>, though this depends
        on how <ulink
        url="http://en.wikipedia.org/wiki/Syslog">syslog</ulink> is
        configured on your system.  You can also add the
        <userinput>-vv</userinput> flags to both server
        <command>achd</command> (in <filename>inetd.conf</filename>)
        the client <command>achd</command> commands to enable
        debugging output.  Syslog may store the debug output in
        <filename>/var/log/debug.log</filename>.
      </para>
    </tip>

    <tip>
      <para>
        For the <command>achd</command> server to access any channel,
        it must be readable and writable by the user under which
        <command>achd</command> runs ("nobody" in the above example
        line for inetd.conf).  If you trust all users on the local
        machine, you can set the channel permissions to "666", but
        please note that this is not a secure configuration.  An
        alternative is to create a separate user account for
        <command>achd</command> and ensure that channels are readable
        and writable by that account or one of its groups.
      </para>
    </tip>
    <warning>
      <para>
        Currently, <command>achd</command> performs no authentication,
        access control, or encryption (this may be added in the
        future).  If security is a concern in your application, it
        must currently be addressed at the network level, e.g. by
        physical isolation, firewalling, a VPN such as IPSec, and/or
        SSH tunnelling.
      </para>
    </warning>
    </sect3>
    </sect2>

    <sect2><title>Client</title>
    <para>From the client machine, you can run achd to push or pull
    message to or from the server.</para>

    <cmdsynopsis>
      <command>achd</command>
      <group choice="req">
        <arg choice="req">push</arg>
        <arg choice="req">pull</arg>
      </group>
      <arg choice="req"><replaceable>hostname</replaceable></arg>
      <arg choice="req"><replaceable>chanel_name</replaceable></arg>
      <arg>-t <replaceable>tcp|udp</replaceable></arg>
      <arg>-p <replaceable>port</replaceable></arg>
      <arg>-z <replaceable>remote_channel_name</replaceable></arg>
      <arg>-u <replaceable>microseconds</replaceable></arg>
      <arg>-d</arg>
      <arg>-r</arg>
      <arg>-q</arg>
      <arg>-v</arg>
      <arg>-V</arg>
      <arg>-?</arg>
    </cmdsynopsis>

    <example><title>Pull channel from server</title>
    <cmdsynopsis>
      <command>achd</command>
      <arg choice="plain">pull</arg>
      <arg choice="plain"><replaceable>server_name</replaceable></arg>
      <arg choice="plain"><replaceable>channel_name</replaceable></arg>
    </cmdsynopsis>
    </example>

    <example><title>Push channel to server and retry dropped connections</title>
    <cmdsynopsis>
      <command>achd</command>
      <arg choice="plain">-r</arg>
      <arg choice="plain">pull</arg>
      <arg choice="plain"><replaceable>server_name</replaceable></arg>
      <arg choice="plain"><replaceable>channel_name</replaceable></arg>
    </cmdsynopsis>
    </example>

    <example><title>Pull channel from server via UDP</title>
    <cmdsynopsis>
      <command>achd</command>
      <arg choice="plain">-t udp</arg>
      <arg choice="plain">pull</arg>
      <arg choice="plain"><replaceable>server_name</replaceable></arg>
      <arg choice="plain"><replaceable>channel_name</replaceable></arg>
    </cmdsynopsis>
    </example>

    <example><title>Push channel to server, running the the background</title>
    <cmdsynopsis>
      <command>achd</command>
      <arg choice="plain">-d</arg>
      <arg choice="plain">push</arg>
      <arg choice="plain"><replaceable>server_name</replaceable></arg>
      <arg choice="plain"><replaceable>channel_name</replaceable></arg>
    </cmdsynopsis>
    </example>

    <example><title>Pull channel from server at 10Hz</title>
    <cmdsynopsis>
      <command>achd</command>
      <arg choice="plain">pull</arg>
      <arg choice="plain"><replaceable>server_name</replaceable></arg>
      <arg choice="plain"><replaceable>channel_name</replaceable></arg>
      <arg choice="plain">-u 100000</arg>
    </cmdsynopsis>
    </example>

    <para>
      To add both encryption and compression, you can also forward
      achd over SSH.  This will tunnel the achd TCP connection through
      the encrypted and compressed SSH connection.
    </para>

    <example><title>Pull channel over compressed SSH connection</title>
    <cmdsynopsis>
      <command>ssh</command>
      <arg choice="plain">-f</arg>
      <arg choice="plain">-C</arg>
      <arg choice="plain">-L1234:localhost:8076</arg>
      <arg choice="plain"><replaceable>server_name</replaceable></arg>
      <arg choice="plain">sleep 120</arg>
    </cmdsynopsis>
    <cmdsynopsis>
      <command>achd</command>
      <arg choice="plain">-p</arg>
      <arg choice="plain">1234</arg>
      <arg choice="plain">pull</arg>
      <arg choice="plain">localhost</arg>
      <arg choice="plain"><replaceable>channel_name</replaceable></arg>
    </cmdsynopsis>
    </example>

    </sect2>

    <sect2><title>Locating Channels via mDNS</title>
    <para>
      Multicast DNS (mDNS) can be used to browse the Ach channels on
      the local network and to lookup the hostname for an channel.
      mDNS names are generally in the <systemitem
      class="domainname">.local</systemitem> domain To advertise an
      Ach channel, it is registered as a service with the local mDNS
      server.
    </para>

    <example><title>Create a channel and register using the Avahi mDNS server</title>
    <cmdsynopsis>
      <command>ach </command>
      <arg choice="plain">mk</arg>
      <arg choice="plain">-a</arg>
      <arg choice="plain"><replaceable>channel_name</replaceable></arg>
    </cmdsynopsis>
    </example>

    <example><title>Browse local network channels using Avahi</title>
    <cmdsynopsis>
      <command>avahi-browse</command>
      <arg choice="plain">-t</arg>
      <arg choice="plain">_ach._tcp</arg>
    </cmdsynopsis>
    </example>

    <example><title>Lookup channel origin in achd client</title>
    <cmdsynopsis>
      <command>achd</command>
      <arg choice="plain">pull</arg>
      <arg choice="plain"><replaceable>channel_name</replaceable></arg>
    </cmdsynopsis>
    <para>
      This performs an mDNS query to determine the origin of the
      specified channel, then connects to that host.
    </para>
    </example>

    <note>
      <para>
        <ulink url="http://tools.ietf.org/html/rfc6762">mDNS</ulink>
        is a peer-to-peer variation on DNS that reduces the need for
        prior configuration by automatically discovering services on
        the local network.  Each peer runs an mDNS server, and
        services, e.g., file shares, printers, Ach channels, are
        located by sending a <ulink
        url="http://en.wikipedia.org/wiki/Multicast">multicast</ulink>
        query.  Popular implementations of mDNS are <ulink
        url="https://www.apple.com/support/bonjour/">Bonjour</ulink>
        and <ulink url="http://avahi.org/">Avahi</ulink>.
      </para>
      <para>Some communication systems, which were generally developed
      prior to mDNS, use specialized naming services.  For example,
      ONC RPC uses the port mapper and CORBA has its own naming
      service.</para>
    </note>

    </sect2>
  </sect1>

  <sect1>
    <title>
      achcop Watchdog Utility
    </title>
    <para>
      The <command>achcop</command> program can be used to start
      daemons, and restart them if they die.
    </para>

    <cmdsynopsis>
      <command>achcop</command>
      <arg>-P <replaceable>cop-pid-file</replaceable></arg>
      <arg>-p <replaceable>child-pid-file</replaceable></arg>
      <arg>-e <replaceable>stderr-file</replaceable></arg>
      <arg>-o <replaceable>stdout-file</replaceable></arg>
      <arg>-d</arg>
      <arg>-r</arg>
      <arg>-v</arg>
      <arg>-V</arg>
      <arg>-?</arg>
      <arg choice="plain">--</arg>
      <arg choice="req">child-program</arg>
      <arg choice="opt">child-arguments...</arg>
    </cmdsynopsis>
    <sect2><title>Features</title>
    <itemizedlist>
      <listitem>
        <para>
          Detach process from terminal and run in background
          (daemonize), <userinput>-d</userinput> flag.
        </para>
      </listitem>
      <listitem>
        <para>
          Lock PID files and write PIDs, <userinput>-P</userinput> and
          <userinput>-p</userinput> flags.
        </para>
      </listitem>
      <listitem>
        <para>
          Reconnect if connection is broken, <userinput>-r</userinput>
          flag.
        </para>
      </listitem>
      <listitem>
        <para>
          Redirect standard output and standard error to files,
          <userinput>-o</userinput> and <userinput>-e</userinput>
          flags.
        </para>
      </listitem>
      <listitem>
        <para>
          Restart process if <constant>SIGHUP</constant> is received
        </para>
      </listitem>
    </itemizedlist>
    </sect2>

    <sect2><title>Signals </title>

    <note>
      <para>
        <emphasis>Signals</emphasis> are how POSIX user programs are
        notified of asynchrous events. They are similar in spirit to
        low-level interrupts and interrupt service routines.  When a
        process receives a signal, its execution is interrupted, and
        it runs the specified handler for that signal.  For example,
        when you hit <userinput>Control-C</userinput> at a terminal,
        the process in the forground receives the interrupt signal,
        <constant>SIGINT</constant>.  The default handler for this
        signal will terminate the process.  For more details, see
        <command>man 7 signal</command>.
      </para>
    </note>

    <para><command>Achcop</command> will respond to the following
    signals.  You can send a signal with the <command>kill</command>
    shell command or the <function>kill</function> C function.</para>

    <variablelist>
      <varlistentry>
        <term><constant>SIGTERM</constant>, <constant>SIGINT</constant></term>
        <listitem>
          <para><command>Achcop</command> sends SIGTERM to the child,
          waits for child to exit, and return child's exit status</para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><constant>SIGHUP</constant></term>
        <listitem>
          <para><command>Achcop</command> sends SIGTERM to the child,
          waits for child to exit, and restarts the child</para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><constant>SIGUSR1</constant></term>
        <listitem>
          <para>Indicates that the child has started successfully.  If
          the child does not send SIGUSR1 and continues running past
          an initial timeout, <command>achcop</command> assumes it
          started successfully.</para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><constant>SIGUSR2</constant></term>
        <listitem>
          <para>Indicates that the child has failed to start properly.
          If the child fails on its first attempt and before a timeout
          has elapsed, it will not be restarted.  If your child does
          not send this signal, <command>achcop</command> may wait a
          few seconds to ensure that the child runs
          successfully.</para>
        </listitem>
      </varlistentry>
    </variablelist>
    </sect2>

    <sect2><title>Examples</title>

    <sect3><title>Shell Commands</title>
    <para>The following shell lines show how one might run <command>achcop</command>.</para>

    <example><title>Run achd in the background (-d), restarting (-r) if it fails</title>
    <cmdsynopsis>
      <command>achcop</command>
      <arg choice="plain">-d</arg>
      <arg choice="plain">-r</arg>
      <arg choice="plain">-P /var/run/achcop-foo.pid</arg>
      <arg choice="plain">-p /var/run/achd-foo.pid</arg>
      <arg choice="plain">--</arg>
      <arg choice="plain">achd</arg>
      <arg choice="plain">-s</arg>
      <arg choice="plain">-r</arg>
      <arg choice="plain">push</arg>
      <arg choice="plain"><replaceable>server</replaceable></arg>
      <arg choice="plain"><replaceable>channel_foo</replaceable></arg>
    </cmdsynopsis>
    </example>

    <example><title>Command achcop to restart achd</title>
    <cmdsynopsis>
      <command>kill</command>
      <arg choice="plain">-HUP</arg>
      <arg choice="plain">`cat /var/run/achcop-foo.pid`</arg>
    </cmdsynopsis>
    </example>

    <example><title>Command achcop to terminate achd</title>
    <cmdsynopsis>
      <command>kill</command>
      <arg choice="plain">`cat /var/run/achcop-foo.pid`</arg>
    </cmdsynopsis>
    </example>

    </sect3>

    <sect3><title>Signaling Achcop from Child</title>
    <para>
      The following C code will notify a parent
      <command>achcop</command> of your child's success or failure.
      This is an optional step. <command>Achcop</command> will operate
      properly without these signals; however, providing the signals
      means <command>achcop</command> does not need to wait for a
      brief timeout to ensure that your child continues operating.
    </para>

    <example><title>Signal achcop of child success or failure</title>
    <programlisting language="C">
/* only send signal if running under achcop */
if( my_parent_is_achcop ) {
    int sig;
    /* determine signal to send */
    if( initialized_correctly ) {
        sig = SIGUSR1; /* success */
    } else {
        sig = SIGUSR2; /* failure */
    }
    /* send the signal */
    int r = kill( getppid(), sig );
    /* check that we sent the signal */
    if( r ) {
        perror("Couldn't signal parent");
        exit(EXIT_FAILURE);
    }
}
    </programlisting>
    </example>
      <para>
        Note that the default signal handlers for
        <constant>SIGUSR1</constant> and <constant>SIGUSER2</constant>
        terminate the process; therefore, if you send these signals to
        your shell, your shell will terminate.
      </para>
    </sect3>
    </sect2>

  </sect1>

  <sect1>
    <title>achlog Logging Utility</title>

    <para>
      The <command>achlog</command> program can be used to log
      messages from an ach channel to disk.
    </para>

   <cmdsynopsis>
      <command>achlog</command>
      <arg>-z</arg>
      <arg>-V</arg>
      <arg>-?</arg>
      <arg choice="req">channels...</arg>
    </cmdsynopsis>

    <sect2><title>Log Format</title>
    <para>
      The Log format is similar to the TCP network protocol for
      achpipe.  The log files begins with a sequence of ASCII
      key-value headers, followed by a single "." character, followed
      by a sequence of messages.
    </para>

    <example><title>achlog headers</title>
    <screen>
ACHLOG
channel-name: foo
log-version: 0
log-time-ach: 6226133.801454417
log-time-real: 1371238788.956337009 # Fri Jun 14 15:39:48 2013
local-host: daneel
user: ntd # Neil T. Dantam
.
</screen>
    </example>
    <para>
      Messages are framed in the file with the following C struct:
    </para>
    <programlisting language="C">
struct {
    uint8_t reserved[8];   /* reserved for future use */
    uint8_t size_bytes[8]; /* size of data, stored little endian */
    uint8_t data[1];       /* flexible array containing size_bytes of data */
};
    </programlisting>
    </sect2>

    <sect2><title>Usage</title>
    <example><title>Log channel foo</title>
    <cmdsynopsis>
      <command>achlog</command>
      <arg choice="plain">foo</arg>
    </cmdsynopsis>
    </example>

    <example><title>Log channel foo and bar, compress with gzip</title>
    <cmdsynopsis>
      <command>achlog</command>
      <arg choice="plain">-z</arg>
      <arg choice="plain">foo</arg>
      <arg choice="plain">bar</arg>
    </cmdsynopsis>
    </example>

    <example><title>Log channel foo in the background</title>
    <cmdsynopsis>
      <command>achcop</command>
      <arg choice="plain">-d</arg>
      <arg choice="plain">-P /var/run/achcop-achlog-foo.pid</arg>
      <arg choice="plain">-p /var/run/achlog-foo.pid</arg>
      <arg choice="plain">--</arg>
      <arg choice="plain">achlog</arg>
      <arg choice="plain">foo</arg>
    </cmdsynopsis>
    </example>

    <example><title>Stop background logging</title>
    <cmdsynopsis>
      <command>kill</command>
      <arg choice="plain">`cat /var/run/achcop-achlog-foo.pid`</arg>
    </cmdsynopsis>
    </example>

    </sect2>

  </sect1>

  <sect1>
    <title>
      Performance Tuning
    </title>
    <para>The performance of your real-time application will be highly
    dependent on your underlying hardware and operating system.  It is
    quite likely than any off-the-shelf hardware or operating system
    you use will be tuned for something other than low-latency
    real-time operation, e.g. maximum thoughput or minimum power
    consumption.  However, there are several paremeters to adjust to
    improve your performance.</para>

    <sect2><title>Disable CPU frequency scaling</title>
    <para>
      Many current CPUs will lower their frequency when idle in order
      to reduce consumption.  This is great for extending your
      laptop's battery life but poor for minimizing latency.  On an
      x86/amd64 PC, you can likely disable frequency scaling from the
      BIOS, which will reduce your system's latency.  In the author's
      experience, simply changing to a "performance" CPU governer from
      within the operating system did NOT provide the same latency
      reduction; you need to disable freqency scaling from the BIOS.
    </para>
    </sect2>

    <sect2><title>Use a Real-Time Operating System</title>
    <para>
      General purpose operating systems (e.g. GNU/Linux, FreeBSD) are
      designed to maximize throughput and perhaps to reduce latency to
      "human-tolerable" (0.1s) levels.  The worst-case performance may
      be sacrificed in order to improve average-case performance.  For
      a real-time application, it is generally worst-case performance
      that matters rather than average case, and latency is more
      important than throughput.
    </para>

    <sect3><title>Linux PREEMPT_RT</title>
    <para>
      If using a Linux kernel, you can apply the PREEMPT_RT patch to
      allow a fully-preemptible kernel.  Without this, a low priority
      task in the middle of a system call may block a high priority
      task from running until the system call completes.  With a
      fully-preemptible kernel, the high priority task gets to run
      immediately.
    </para>
    </sect3>

    <sect3><title>Linux / Xenomai</title>
    <para>
      Xenomai is a dual-kernel real-time operating system.  A hard
      real-time kernel, Adeos, handles interrupts and a full Linux
      kernel runs along-side, permitting normal Linux processes to run
      directly.  Real-time processes must use a specific Xenomai API
      <emphasis>skin</emphasis>.  One of these skins is POSIX, and
      this POSIX interface is sufficient to use Ach for real-time
      Xenomai processes.
    </para>

    <warning>
      <para>
        There is one important caveat when using Ach on Xenomai.
        Because of the dual-kernel approach, Xenomai scheduling
        primitives -- mutexes and condition variables -- are different
        from the Linux primitives; a Linux process and a Xenomai
        process cannot synchronize on the same condition variable.
        Therefore, Ach compiled for Xenomai cannot directly share
        channels with Ach compiled for Linux.  If you want a Linux
        process and a Xenomai process on the same machine to
        communicate via Ach, you can use <command>achd</command> to
        forward the messages between the two sides.  This issue does
        not affect messages forwarded between different machines.
      </para>
    </warning>

    </sect3>

    <sect3><title>Other Real-Time Operating Systems</title>
    <para>
      While Ach development has focused on Linux, it should be
      portable to other operating systems which provide the POSIX
      real-time extensions.  In particular, Ach depends on POSIX
      shared memory and process-shared mutexes and condition
      variables.  Contributions to increase portability are extremely
      welcome.
    </para>
    </sect3>

    </sect2>

    <sect2><title>Benchmark your system</title>
    <para>
      You can use the included <command>achbench</command> command to
      test performance on your system while you try different
      configurations.  This program will <function>fork</function> a
      specified number of publishers and subscribers and measure the
      latency of message passing.
    </para>

    <example><title>Benchmarking Latency</title>
    <para>Measure latency with one publisher and two subscribers over ten seconds</para>

      <cmdsynopsis>
        <command>achbench</command>
         <arg choice="plain">-p <replaceable>1</replaceable></arg>
         <arg choice="plain">-r <replaceable>2</replaceable></arg>
         <arg choice="plain">-s <replaceable>10</replaceable></arg>
         <arg choice="plain"> > <replaceable>output_file</replaceable></arg>
      </cmdsynopsis>
    </example>
    </sect2>

  </sect1>

  <sect1>
    <title>
      Theory of Operation
    </title>

    <para>
      POSIX provides a rich variety of IPC mechanisms, but none of
      them are ideal for real-time control.  The fundamental
      difference is that as soon as a new sample of the signal (in the
      oscilloscope sense, not a POSIX signal) is produced, nearly
      everything no longer cares about older samples.  Thus, we want
      to always favor new data over old data whereas general-purpose
      POSIX IPC favors the old data. This problem is typically
      referred to as Head of Line (HOL) Blocking.  The exception to
      this is POSIX shared memory. However, synchronization of shared
      memory is a difficult programming problem, making the typical
      and direct use of POSIX shared memory unfavorable for developing
      robust systems.  Furthermore, some parts of the system, such as
      logging, may need to access older samples, so this also should
      be permitted at least on a best-effort basis.
    </para>

    <sect2><title>Data Structure</title>
    <para>
      The core data structure of an Ach channel is a pair of circular
      arrays: a data array and an index array.  The data array
      contains variable sized entries which store the actual message
      frames sent through the Ach channel.  The index array contains
      fixed size entries where each entry has both an offset into the
      data array and the length of that data entry.  A head offset
      into each array indicates both the place to insert the next data
      and the location of the most recent message frame.  This pair of
      circular arrays allows us to find the variable sized message
      frames by first looking at a known offset in the fixed-sized
      index array.
    </para>

    <para>
      Access to the channel is synchronized using a mutex and a queue
      of waiting readers.  This allows readers to either periodically
      poll the channel for new data or to wait on the condition
      variable until a writer has posted a new message.  Using a
      read/write lock instead would have allowed only polling.
      Additionally, synchronization using a mutex prevents starvation
      and enables proper priority inheritance between processes,
      important to maintaining real-time performance.
    </para>
    </sect2>

    <sect2><title>Core Procedures</title>
    <para>
      Two procedures compose the core of ach:
      <function>ach_put</function> and <function>ach_get</function>.
    </para>

    <sect3><title>ach_put</title>
    <para>
      The procedure <function>ach_put</function> inserts new messages
      into the channel.  Its function is analogous to
      <function>write</function>, <function>sendmsg</function>, and
      <function>mq_send</function>.  The procedure is given a pointer
      to the shared memory region for the channel and a byte array
      containing the message to post.  There are four broad steps to
      the procedure:
    </para>
    <orderedlist>
      <listitem><para> Get an index entry. If there is at least one
      free index entry, use it.  Otherwise, clear the oldest index
      entry and its corresponding message in the data array.
      </para></listitem>
      <listitem><para> Make room in the data array.  If there is
      enough room already, continue.  Otherwise, repeatedly free the
      oldest message until there is enough room.  </para></listitem>
      <listitem><para>Copy the message into data array.</para>
      </listitem>
      <listitem><para>Update the offset and free counts in the channel structure.
      </para></listitem>
    </orderedlist>
    </sect3>

    <sect3><title>ach_get</title>
    <para>
      The procedure <function>ach_get</function> receives a message
      from the channel.  Its function is analogous to
      <function>read</function>, <function>recvmsg</function>, and
      <function>mq_receive</function>.  The procedure takes a pointer
      to the shared memory region, a storage buffer to copy the
      message to, the last message sequence number received, the next
      index offset to check for a message, and option flags indicating
      whether to block waiting for a new message and whether to return
      the newest message bypassing any older unseen messages.
    </para>

    <orderedlist>
      <listitem><para> If we are to wait for a new message and there
      is no new message, then wait.  Otherwise, if there are no new
      messages, return a status code indicating this
      fact.</para></listitem>
      <listitem><para> Find the index entry to use.  If we are to
      return the newest message, use that entry.  Otherwise, if the
      next entry we expected to use contains the next sequence number
      we expect to see, use that entry.  Otherwise, use the oldest
      entry.</para></listitem>
      <listitem><para> According to the offset and size from the
      selected index entry, copy the message from the data array into
      the provided storage buffer.</para></listitem>
      <listitem><para> Update the sequence number count and next index
      entry offset for this receiver.</para></listitem>
    </orderedlist>
    </sect3>

    </sect2>

    <sect2><title>Channel Mappings</title>
    <para>
      Ach supports multiple back-end mappings for the channel data
      structure: (1) the process-local heap, (2) a shared-memory file,
      and (3) a kernel-space buffer on Linux.
    </para>

    <sect3><title>Anonymous Channels</title>
    <para>Process-local channel allocation stores the channel data
    structure in the heap (<constant>ACH_MAP_ANON</constant>).  Access
    to anonymous channels is synchronized using a mutex and condition
    variable.</para>
    </sect3>

    <sect3><title>User Channels</title>
    <para>Channels can also be allocated in POSIX shared memory
    (<constant>ACH_MAP_USER</constant>). Access to user channels is
    synchronized using a process-shared mutex and condition
    variable.</para>
    </sect3>

    <sect3><title>Kernel Channels</title>

    <para>
      Under Linux, channels can created in-kernel
      (<constant>ACH_MAP_KERNEL</constant>).  Access to kernel
      channels is synchronized using an in-kernel rt_mutex and wait_queue.
    </para>

    <para>
      Kernel channels are implemented as virtual devices.  There is a
      control device, <filename>/dev/achctrl</filename> used to create
      and unlink channels via <function>ioctl</function>s.  Each
      channel is exposed to userspace as a separate virtual device,
      <filename>/dev/ach-CHANNEL_NAME</filename>.  Putting and getting
      frames on the channel is done through <function>write</function>
      and <function>read</function> calls on the device. This is
      handled internally in the <function>ach_put</function> and
      <function>ach_get</function> functions, which also handle
      retrying transfers after signal interrupts.  Additional
      operations, e.g., canceling and flushing, are implented as
      <function>ioctl</function>s on the device file, also called
      internally in the Ach library.
    </para>

    <para>
      There are two advantages to creating channels in kernel-space.
      First, rogue processes cannot corrupt or deadlock kernel-space
      channels as they could with channels in shared memory.  Second,
      readers can wait for messages on multiple channels using the
      <function>poll</function>/<function>select</function> family of
      calls.
    </para>
    </sect3>

    </sect2>

    <sect2><title>Serialization</title>
    <para>
      Ach is orthogonal to the issue of serialization: it only handles
      raw byte arrays.  Thus, you can choose whatever type of
      serialization is most appropriate for you application.  For
      maximum performance, it may be appropriate to define message
      types as simple C structures.  If more flexibility is required,
      there are a variety of other options to choose from: <ulink
      url="http://tools.ietf.org/html/rfc1014">XDR</ulink>, <ulink
      url="http://www.itu.int/ITU-T/asn1/introduction/">ASN.1</ulink>,
      <ulink
      url="https://developers.google.com/protocol-buffers/">Protocol
      Buffers</ulink>, etc.
    </para>
    </sect2>

  </sect1>

  <colophon>
    <para>This manual is written in Docbook XML.</para>

     <!--<legalnotice>-->
       <para>
         Redistribution and use in source (XML DocBook) and 'compiled' forms
         (SGML, HTML, PDF, PostScript, RTF and so forth) with or without
         modification, are permitted provided that the following conditions are
         met:
       </para>
       <orderedlist>
         <listitem><para>
           Redistributions of source code (XML DocBook) must retain the
           above copyright notice, this list of conditions and the
           following disclaimer as the first lines of this file
           unmodified.
         </para></listitem>
         <listitem><para>
           Redistributions in compiled form (transformed to other DTDs,
           converted to PDF, PostScript, RTF and other formats) must
           reproduce the above copyright notice, this list of conditions
           and the following disclaimer in the documentation and/or other
           materials provided with the distribution.
         </para></listitem>
       </orderedlist>
       <para>
         THIS DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND
         CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
         INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
         MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
         DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR
         CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
         SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
         NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
         LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
         HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
         CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
         OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
         DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
         DAMAGE.
       </para>
     <!-- </legalnotice> -->
  </colophon>

</article>


text:
package org.broadinstitute.hellbender.tools.spark.sv.discovery.inference;

import htsjdk.samtools.SAMSequenceDictionary;
import htsjdk.variant.variantcontext.Allele;
import htsjdk.variant.variantcontext.VariantContext;
import htsjdk.variant.variantcontext.VariantContextBuilder;
import htsjdk.variant.vcf.VCFConstants;
import org.broadinstitute.hellbender.engine.datasources.ReferenceMultiSource;
import org.broadinstitute.hellbender.exceptions.GATKException;
import org.broadinstitute.hellbender.tools.spark.sv.discovery.BreakEndVariantType;
import org.broadinstitute.hellbender.tools.spark.sv.discovery.SimpleSVType;
import org.broadinstitute.hellbender.tools.spark.sv.discovery.SvType;
import org.broadinstitute.hellbender.tools.spark.sv.discovery.alignment.AlignmentInterval;
import org.broadinstitute.hellbender.utils.SimpleInterval;
import scala.Tuple2;

import java.util.*;

import static org.broadinstitute.hellbender.tools.spark.sv.utils.GATKSVVCFConstants.*;

/**
 * Holding utility methods used for constructing test data
 * as well as expected results.
 *
 * NO TESTS ARE RUN IN THIS PARTICULAR CLASS AND ITS CHILD CLASSES.
 */
public abstract class AssemblyBasedSVDiscoveryTestDataProvider {

    public abstract static class AssemblyBasedSVDiscoveryTestDataForSimpleChimera {
        // these are used for building
        public final AlignmentInterval firstAlignment;
        public final AlignmentInterval secondAlignment;
        public final String evidenceAssemblyContigName;
        public final byte[] evidenceContigSeq;

        // these are used for testing
        public final boolean expectedFirstContigRegionHasLaterReferenceMapping;
        public final SimpleChimera expectedSimpleChimera;
        public final NovelAdjacencyAndAltHaplotype expectedNovelAdjacencyAndAltSeq;

        public final List<SvType> expectedSvTypes;
        public final List<VariantContext> expectedVariantContexts;

        public final Class<? extends BreakpointsInference> expectedInferencerClass;

        public AssemblyBasedSVDiscoveryTestDataForSimpleChimera(final AlignmentInterval firstAlignment, final AlignmentInterval secondAlignment,
                                                                final String evidenceAssemblyContigName, final byte[] evidenceContigSeq,
                                                                final boolean expectedFirstContigRegionHasLaterReferenceMapping,
                                                                final SimpleChimera expectedSimpleChimera,
                                                                final NovelAdjacencyAndAltHaplotype expectedNovelAdjacencyAndAltSeq,
                                                                final List<SvType> expectedSvTypes,
                                                                final List<VariantContext> expectedVariantContexts,
                                                                final Class<? extends BreakpointsInference> expectedInferencerClass) {
            this.firstAlignment = firstAlignment;
            this.secondAlignment = secondAlignment;
            this.evidenceAssemblyContigName = evidenceAssemblyContigName;
            this.evidenceContigSeq = evidenceContigSeq;
            this.expectedFirstContigRegionHasLaterReferenceMapping = expectedFirstContigRegionHasLaterReferenceMapping;
            this.expectedSimpleChimera = expectedSimpleChimera;
            this.expectedNovelAdjacencyAndAltSeq = expectedNovelAdjacencyAndAltSeq;
            this.expectedSvTypes = expectedSvTypes;
            this.expectedVariantContexts = expectedVariantContexts;
            this.expectedInferencerClass = expectedInferencerClass;
        }

        public abstract SAMSequenceDictionary getAppropriateDictionary();

        public abstract ReferenceMultiSource getAppropriateRef();

        public abstract Class<? extends BreakpointsInference> getAppropriateBreakpointInferencer();
    }

    public abstract List<AssemblyBasedSVDiscoveryTestDataForSimpleChimera> getAllTestData();

    // same event, two representations from opposite strands
    public final List<Tuple2<AssemblyBasedSVDiscoveryTestDataForSimpleChimera, AssemblyBasedSVDiscoveryTestDataForSimpleChimera>> getAllTestDataPaired() {
        final List<AssemblyBasedSVDiscoveryTestDataForSimpleChimera> allTestData = getAllTestData();
        final List<Tuple2<AssemblyBasedSVDiscoveryTestDataForSimpleChimera, AssemblyBasedSVDiscoveryTestDataForSimpleChimera>> testDataForSimpleSVs
                = new ArrayList<>(allTestData.size()/2);
        for (int i = 0; i < allTestData.size() - 1; i += 2) {
            testDataForSimpleSVs.add(new Tuple2<>(allTestData.get(i), allTestData.get(i+1)));
        }
        return Collections.unmodifiableList(testDataForSimpleSVs);
    }

    // data block ======================================================================================================

    private static String makeID(final String prefix, final String chr1, final int start, final String chr2, final int stop,
                                 final String postfix) {
        return prefix + INTERVAL_VARIANT_ID_FIELD_SEPARATOR +
                SvType.makeLocationString(chr1, start, chr2, stop) +
                (postfix.isEmpty() ? "" : INTERVAL_VARIANT_ID_FIELD_SEPARATOR + postfix);
    }

    static final byte[] EMPTY_BYTE_ARRAY = new byte[]{};

    private static final Allele INV_SYMB_ALLELE = Allele.create(SimpleSVType.createBracketedSymbAlleleString(SimpleSVType.SupportedType.INV.name()), false);
    private static final Allele DEL_SYMB_ALLELE = Allele.create(SimpleSVType.createBracketedSymbAlleleString(SimpleSVType.SupportedType.DEL.name()), false);
    private static final Allele INS_SYMB_ALLELE = Allele.create(SimpleSVType.createBracketedSymbAlleleString(SimpleSVType.SupportedType.INS.name()), false);
    private static final Allele DUP_SYMB_ALLELE = Allele.create(SimpleSVType.createBracketedSymbAlleleString(SimpleSVType.SupportedType.DUP.name()), false);

    // utils block =====================================================================================================

    /**
     * Note that {@code delRange} is expected to be VCF spec compatible,
     * e.g. if chr1:101-200 is deleted, then {@code delRange} should be chr1:100-200
     */
    static final VariantContextBuilder makeDeletion(final SimpleInterval delRange, final Allele refAllele, final boolean isFromDupContraction) {

        return new VariantContextBuilder()
                .chr(delRange.getContig()).start(delRange.getStart()).stop(delRange.getEnd())
                .alleles(Arrays.asList(refAllele, DEL_SYMB_ALLELE))
                .id(makeID((isFromDupContraction? DUP_TAN_CONTRACTION_INTERNAL_ID_START_STRING : SimpleSVType.SupportedType.DEL.name()), delRange.getContig(), delRange.getStart(), delRange.getContig(), delRange.getEnd(), ""))
                .attribute(VCFConstants.END_KEY, delRange.getEnd())
                .attribute(SVLEN, - delRange.size() + 1)
                .attribute(SVTYPE, SimpleSVType.SupportedType.DEL.name());
    }

    static final SvType makeDeletionType(final SimpleInterval delRange, final Allele refAllele, final boolean isFromDupContraction) {
        return new SimpleSVType.Deletion(
                delRange.getContig(), delRange.getStart(), delRange.getEnd(),
                makeID((isFromDupContraction? DUP_TAN_CONTRACTION_INTERNAL_ID_START_STRING : SimpleSVType.SupportedType.DEL.name()),
                        delRange.getContig(), delRange.getStart(), delRange.getContig(), delRange.getEnd(), ""),
                refAllele,
                DEL_SYMB_ALLELE,
                - delRange.size()+ 1,
                isFromDupContraction ? Collections.singletonMap(DUP_TAN_CONTRACTION_STRING, true) :Collections.emptyMap());
    }

    static final VariantContextBuilder makeInversion(final SimpleInterval invertedRegion, final Allele refAllele) {
        return new VariantContextBuilder()
                .chr(invertedRegion.getContig()).start(invertedRegion.getStart() - 1).stop(invertedRegion.getEnd())     // TODO: 5/2/18 VCF spec doesn't require left shift by 1 for inversion POS
                .alleles(Arrays.asList(refAllele, INV_SYMB_ALLELE))
                .id(makeID(SimpleSVType.SupportedType.INV.name(), invertedRegion.getContig(), invertedRegion.getStart() - 1, invertedRegion.getContig(), invertedRegion.getEnd(), ""))
                .attribute(VCFConstants.END_KEY, invertedRegion.getEnd())
                .attribute(SVLEN, 0)                                                                 // TODO: 5/2/18 this is following VCF spec,
                .attribute(SVTYPE, SimpleSVType.SupportedType.INV.name());
    }

    static final SvType makeInversionType(final SimpleInterval invRange, final Allele refAllele, final boolean isInv55) {
        return new SimpleSVType.Inversion(
                invRange.getContig(),
                invRange.getStart() - 1,
                invRange.getEnd(),
                makeID((isInv55? INV55 : INV33),
                        invRange.getContig(), invRange.getStart() - 1, invRange.getContig(), invRange.getEnd(), ""),
                refAllele,
                INV_SYMB_ALLELE, invRange.size(),
                Collections.singletonMap((isInv55) ? INV55 : INV33, true));
    }

    static final VariantContextBuilder makeInsertion(final String chr, final int pos, final int end, final int svLen,
                                                     final Allele refAllele) {

        return new VariantContextBuilder().chr(chr).start(pos).stop(end)
                .alleles(Arrays.asList(refAllele, INS_SYMB_ALLELE))
                .id(makeID(SimpleSVType.SupportedType.INS.name(), chr, pos, chr, end, ""))
                .attribute(VCFConstants.END_KEY, end)
                .attribute(SVLEN, svLen)
                .attribute(SVTYPE, SimpleSVType.SupportedType.INS.name());
    }

    static final SvType makeInsertionType(final SimpleInterval insertionPos, final Allele refAllele, final int svLen) {
        return new SimpleSVType.Insertion(
                insertionPos.getContig(),
                insertionPos.getStart(),
                insertionPos.getEnd(),
                makeID(SimpleSVType.SupportedType.INS.name(),
                        insertionPos.getContig(), insertionPos.getStart(), insertionPos.getContig(), insertionPos.getEnd(), ""),
                refAllele,
                INS_SYMB_ALLELE, svLen,
                Collections.emptyMap());
    }

    static final VariantContextBuilder makeTandemDuplication(final SimpleInterval duplicatedRange, final Allele refAllele,
                                                             final int svLen) {
        return new VariantContextBuilder().chr(duplicatedRange.getContig()).start(duplicatedRange.getStart() - 1).stop(duplicatedRange.getStart() - 1) // TODO: 5/24/18 by left shifting 1, we are treating it as insertions
                .id(makeID(DUP_TAN_EXPANSION_INTERNAL_ID_START_STRING,
                        duplicatedRange.getContig(), duplicatedRange.getStart() , duplicatedRange.getContig(), duplicatedRange.getEnd(), ""))
                .alleles(Arrays.asList(refAllele, DUP_SYMB_ALLELE))
                .attribute(VCFConstants.END_KEY, duplicatedRange.getStart() - 1) // TODO: 5/24/18 see todo above
                .attribute(SVLEN, svLen)
                .attribute(SVTYPE, SimpleSVType.SupportedType.DUP.name());
    }

    static final SvType makeTandemDuplicationType(final SimpleInterval duplicatedRange, final Allele refAllele, final int svLen) {
        return new SimpleSVType.DuplicationTandem(
                duplicatedRange.getContig(),
                duplicatedRange.getStart() - 1,
                duplicatedRange.getStart() - 1,
                makeID(DUP_TAN_EXPANSION_INTERNAL_ID_START_STRING,
                        duplicatedRange.getContig(), duplicatedRange.getStart(), duplicatedRange.getContig(), duplicatedRange.getEnd(), ""),
                refAllele,
                DUP_SYMB_ALLELE, svLen,
                Collections.singletonMap(DUP_TAN_EXPANSION_STRING, true));
    }

    static final VariantContextBuilder makeBND(final SimpleInterval upstreamLoc, final SimpleInterval dnstreamLoc,
                                               final Allele refAllele, final String insertedSeq, final String bndSubtypeString,
                                               final boolean forUpstreamLoc, final boolean refBaseFirst, final boolean bracketPointsLeft) {

        final String upstreamAltString = upstreamLoc.getContig() + ":" + upstreamLoc.getStart();
        final String dnstreamAltString = dnstreamLoc.getContig() + ":" + dnstreamLoc.getEnd();

        final Allele altAllele;
        if (refBaseFirst) {
            if (bracketPointsLeft)
                altAllele = Allele.create(refAllele.getBaseString() + insertedSeq + "]" + (forUpstreamLoc ? dnstreamAltString : upstreamAltString) + "]");
            else
                altAllele = Allele.create(refAllele.getBaseString() + insertedSeq + "[" + (forUpstreamLoc ? dnstreamAltString : upstreamAltString) + "[");
        } else {
            if (bracketPointsLeft)
                altAllele = Allele.create("]" + (forUpstreamLoc ? dnstreamAltString : upstreamAltString) + "]" + insertedSeq + refAllele.getBaseString());
            else
                altAllele = Allele.create("[" + (forUpstreamLoc ? dnstreamAltString : upstreamAltString) + "[" + insertedSeq + refAllele.getBaseString());
        }
        if (forUpstreamLoc) {
            return new VariantContextBuilder().chr(upstreamLoc.getContig()).start(upstreamLoc.getStart()).stop(upstreamLoc.getEnd())
                    .alleles(Arrays.asList(refAllele, altAllele))
                    .id(makeID(BREAKEND_STR + (bndSubtypeString.isEmpty() ? "" : INTERVAL_VARIANT_ID_FIELD_SEPARATOR + bndSubtypeString),
                            upstreamLoc.getContig(), upstreamLoc.getStart(), dnstreamLoc.getContig(), dnstreamLoc.getEnd(), "1"))
                    .attribute(SVTYPE, BREAKEND_STR);
        } else {
            return new VariantContextBuilder().chr(dnstreamLoc.getContig()).start(dnstreamLoc.getStart()).stop(dnstreamLoc.getEnd())
                    .alleles(Arrays.asList(refAllele, altAllele))
                    .id(makeID(BREAKEND_STR + (bndSubtypeString.isEmpty() ? "" : INTERVAL_VARIANT_ID_FIELD_SEPARATOR + bndSubtypeString),
                            upstreamLoc.getContig(), upstreamLoc.getStart(), dnstreamLoc.getContig(), dnstreamLoc.getEnd(), "2"))
                    .attribute(SVTYPE, BREAKEND_STR);
        }
    }

    static final SvType makeBNDType(final String variantCHR, final int variantPOS, final String variantId,
                                    final Allele refAllele, final Allele altAllele, final Map<String, Object> extraAttributes,
                                    final boolean isTheUpstreamMate,
                                    final BreakEndVariantType.SupportedType type) {
        switch (type) {
            case INTRA_CHR_STRAND_SWITCH_55:
                return new BreakEndVariantType.IntraChromosomalStrandSwitch55BreakEnd(variantCHR, variantPOS, variantId, refAllele, altAllele, extraAttributes, isTheUpstreamMate);
            case INTRA_CHR_STRAND_SWITCH_33:
                return new BreakEndVariantType.IntraChromosomalStrandSwitch33BreakEnd(variantCHR, variantPOS, variantId, refAllele, altAllele, extraAttributes, isTheUpstreamMate);
            case INTRA_CHR_REF_ORDER_SWAP:
                return new BreakEndVariantType.IntraChromosomeRefOrderSwap(variantCHR, variantPOS, variantId, refAllele, altAllele, extraAttributes, isTheUpstreamMate);
            case INTER_CHR_STRAND_SWITCH_55:
            case INTER_CHR_STRAND_SWITCH_33:
            case INTER_CHR_NO_SS_WITH_LEFT_MATE_FIRST_IN_PARTNER:
            case INTER_CHR_NO_SS_WITH_LEFT_MATE_SECOND_IN_PARTNER:
                return new BreakEndVariantType.InterChromosomeBreakend(variantCHR, variantPOS, variantId, refAllele, altAllele, extraAttributes, isTheUpstreamMate);
            default:
                throw new GATKException("Unrecognized type: " + type.name());
        }
    }
}

redpajama_set_name: RedPajamaGithub


text:
#include "ThreadMultiPatch.h"
#include "InterpreterThread.h"

void wasmint::ThreadMultiPatch::update() {
    if (functionStatesOffset_ < thread_->functionStackSize()) {
    }
    if (instructionStatesOffset_ < thread_->instructionStackSize()) {

    }
}

void wasmint::ThreadMultiPatch::applyPatch() {

}


text:
namespace content {

bool WebUIController::OverrideHandleWebUIMessage(const GURL& source_url,
                                                 const std::string& message,
                                                 const base::ListValue& args) {
  return false;
}

WebUIController::Type WebUIController::GetType() {
  return nullptr;
}

}  // namespace content


text:
'use strict';

module.exports = {
  up: function up(queryInterface, Sequelize) {

    // add priority and minutes
    queryInterface.addColumn('Tasks', 'priority', {
      type: Sequelize.INTEGER
    });

    return queryInterface.addColumn('Tasks', 'minutes', {
      type: Sequelize.INTEGER
    });
  },

  down: function down(queryInterface, Sequelize) {

    queryInterface.removeColumn('Tasks', 'priority');

    return queryInterface.removeColumn('Tasks', 'minutes');
  }
};
//# sourceMappingURL=20160611235949-add-minutes-and-priority-to-tasks.js.map


text:
package com.google.android.launcher.layouts.shamu;

import android.content.BroadcastReceiver;
import android.content.Context;
import android.content.Intent;

public class PartnerCustomizationReceiver extends BroadcastReceiver {

    public PartnerCustomizationReceiver() {
    }

    public void onReceive(Context context, Intent intent) {
    }
}


text:
package eu.bitwalker.useragentutils;

import java.util.ArrayList;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public final class HtmlParserUtils {
	private static final Pattern patternHtmlTag = Pattern.compile("<[^>]*>");
	public static class Pair<A, B> {
		final A a;
		final B b;

		public static <X, Y> Pair<X, Y> create(X a, Y b) {
			return new Pair<X, Y>(a, b);
		}

		public Pair(A a, B b) {
			this.a = a;
			this.b = b;
		}

		public A getA() {
			return this.a;
		}

		public B getB() {
			return this.b;
		}

		@Override
		public String toString() {
			return (getA() == null ? "[null]" : getA().toString()) + " " + //
					(getB() == null ? "[null]" : getB().toString());
		}
	}

	public static String extractArea(String html, Pattern begin, Pattern end, String name) throws Exception {
		Matcher m = begin.matcher(html);
		if (!m.find())
			throw new Exception("Begin of " + name + " area is not found");
		html = html.substring(m.end());
		m = end.matcher(html);
		if (!m.find())
			throw new Exception("End of " + name + " area is not found");
		html = html.substring(0, m.start());
		return html;
	}

	// returns Pair<startPos, endPos>
	// returns null if tag is not found
	private static Pair<Integer, Integer> findTagInnerHtml(String html, String beginTag, String endTag, int pos) throws Exception {
		pos = html.indexOf(beginTag, pos);
		if (pos < 0)
			return null;
		pos = html.indexOf('>', (pos + beginTag.length()) - 1);
		if (pos < 0)
			throw new Exception("Could not find end of open tag " + beginTag);
		++pos;
		int pos2 = html.indexOf(endTag, pos);
		if (pos2 < 0)
			throw new Exception("Could not find close tag " + endTag);
		return new Pair<Integer, Integer>(pos, pos2);
	}

	public static List<String> parseTagsInnerHtml(String html, String beginTag, String endTag) throws Exception {
		List<String> res = new ArrayList<String>();
		int pos = 0;
		while (true) {
			Pair<Integer, Integer> poses = findTagInnerHtml(html, beginTag, endTag, pos);
			if (poses == null) {
				break;
			}
			String s = poses.getA().equals(poses.getB()) ? "" : html.substring(poses.getA(), poses.getB());
			res.add(s);
			pos = poses.getB() + endTag.length();
		}
		return res;
	}

	public static List<String> parseTagsInnerHtml(String html, String tag) throws Exception {
		final String beginTag = "<" + tag;
		final String endTag = "</" + tag + ">";
		return parseTagsInnerHtml(html, beginTag, endTag);
	}

	public static String getInnerText(String html) {
		Matcher m = patternHtmlTag.matcher(html);
		return m.replaceAll("");
	}
}


text:
title: Telerik Friends Sample App for NativeScript
slug: sample-backend-services-friends-nativescript
position: 1
publish: true
page_title: Telerik Friends Sample App for NativeScript | Telerik Platform
description:
tags: NativeScript, Native app, Sample App, Application, iOS, Android, JavaScript, BaaS, mBaaS, Social, Analytics, Social Login, AD FS
---

# Telerik Friends Sample App for NativeScript

* [Overview](#overview)
* [Requirements](#requirements)
* [Configuration](#configuration)
* [Running the Sample](#running-the-sample)

## Overview

This repository contains the Telerik Friends app for PhoneGap/Cordova. It is a sample mobile app demonstrating how to integrate a large gamut of Telerik Platform services into a native mobile application using the [NativeScript](http://github.com/nativescript/nativescript) framework.

The Telerik Friends sample app showcases these features and SDKs:

- Cloud data access (Telerik Backend Services)
- Working with files (Telerik Backend Services)
- User registration and authentication (Telerik Backend Services)
- Using custom user account fields (Telerik Backend Services)
- Basic app analytics (Telerik Analytics)
- Tracking feature usage (Telerik Analytics)

- Authentication with social login providers (Facebook, Google, etc.) (Telerik Backend Services)
- Authentication with AD FS (Telerik Backend Services)

To implement all the features listed above, the sample app utilizes the following Telerik products and SDKs:

- Telerik Backend Services - this is where all data, files, and user accounts are stored in the cloud
- Telerik Backend Services JavaScript SDK - to connect the app to Telerik Backend Services
- Telerik Analytics - used to store and monitor analytics data in the cloud
- Telerik Analytics JavaScript SDK - to connect the app with Telerik Analytics

## Requirements

Before you begin, you need to ensure that you have the following:

- **An active Telerik Platform account**
Ensure that you can log in to a Telerik Platform account. This can be a free trial account. Depending on your license you may not be able to use all app features. For more information on what is included in the different editions, check out the pricing page. All features included in the sample app work during the free trial period.

- **Telerik AppBuilder** The sample app requires Telerik AppBuilder to run. This can be the in-browser client, the desktop client or the extension for Visual Studio.

## Configuration

The Friends sample app comes fully functional, but to see it in action you must link it to your own Telerik Platform account.

1. Click the "Try in AppBuilder" button to clone the repository in AppBuilder.

	An app called "My App" is created for you with an AppBuilder project set up.
2. Click **My App** in the navigation bar at the top to go the app home.
3. Create a Backend Services project, selecting the **Start from the Friends app** option.
4. Once the Backend Services project is ready, go to **Overview > API Keys**.
5. Take note of your API Key.
6. Go back to the AppBuilder project and continue setting up as explained in the next sections.

### API Key for Telerik Backend Services

This is a unique string that links the sample mobile app to a project in Telerik Backend Services where all the data is read from/saved. When creating the project, you must base it on the Friends sample Backend Services project that has all the necessary data prepopulated.

You must use this project's API key. To set it in the app:

1. Open the `/app/app.js` file.
2. Replace `$EVERLIVE_API_KEY$` with the API Key of your Friends Backend Services project.

> If you happen to break the structure of the automatically generated Friends sample project, you can delete it and a fresh instance will be created again for you automatically. Alternatively, you could create a new project and choose to start from a Friends template, instead of starting from a blank project.

### (Optional) Project Key for Telerik Analytics

This is a unique string that links the sample mobile app to a Telerik Analytics project in your account. If you do not set this the sample will still work, but no analytics data will be collected.

1. Open the `/app/views/main-page.js` file.
2. Replace `$EQATEC_PROJECT_KEY$` with the Project Key of your Friends Analytics project.

### HTTPS Connections

By default the Friends Sample is configured to use HTTP. If you want to switch to HTTPS, update the `global.BS_SCHEME` setting in  `/app/app.js` to `https`.

## Running the Sample

Once the app is configured, you can run it either on a real device or a native emulator.

To run it, follow the steps in the product's documentation: [Running Apps on Devices](http://docs.telerik.com/platform/appbuilder/testing-your-app/running-on-devices/working-with-devices).

> Ensure that the emulator or the device that you are using has Internet connectivity when running the sample.

![](https://ga-beacon.appspot.com/UA-111455-24/nativescript/sample-friends?pixel)