\documentclass{amsart}
\input preamble
\usepackage{listings}
\usepackage{xcolor}
\usepackage{tikz}
\usetikzlibrary{shapes}
\tikzset{utriangle/.style={regular polygon, regular polygon sides=3,
    inner sep=0.5pt}}
\tikzset{btriangle/.style={regular polygon, regular polygon sides=3,
    shape border rotate=180, inner sep=0.5pt}}

\definecolor{backcolor}{rgb}{0.95,0.95,0.92}

\lstdefinestyle{mystyle}{
    basicstyle=\ttfamily,
    columns=fullflexible,
    keepspaces=true,
}
\lstset{style=mystyle}
\newcommand{\vlength}{0.2cm}

\begin{document}
\title{\texttt{tikzsd} version 1.0.0 user manual}
\maketitle
\tableofcontents
\section{Introduction}
    This is a manual for version 1.0.0 of \texttt{tikzsd}, a Haskell program
        which generates code for inclusion into a \LaTeX \ file.
    The generated code is a \texttt{tikzpicture} environment.

    The program works by defining categorical objects, and then specifying
        a natural transformation using those categorical objects, with notation
        for how it is to be drawn.

    For example,
    \lstinputlisting{adjunction-definitions.tzsd}
    will define the categorical objects needed to specify an adjunction.
    The notation for these statements is explained in section .

    If one then writes
    \lstinputlisting{adjunction-draw.tzsd}
    the program will write code to the file \texttt{adjunction-eq-1.tex}.
    This code will be a \texttt{tikzpicture} environment,
        which can be included in \LaTeX\ code.
    For example, if one writes

\vspace{\vlength}
\begin{verbatim}
\input adjunction-eq-1
\end{verbatim}
\vspace{\vlength}
in a \LaTeX \ file which contains the lines

\vspace{\vlength}
\begin{verbatim}
\usepackage{tikz}
\usetikzlibrary{shapes}
\tikzset{utriangle/.style={regular polygon, regular polygon sides=3,
    inner sep=0.5pt}}
\tikzset{btriangle/.style={regular polygon, regular polygon sides=3,
    shape border rotate=180, inner sep=0.5pt}}
\end{verbatim}
\vspace{\vlength}
    in the preamble
    and is in the same folder as \texttt{adjunction-eq-1.tex},
    then the following string diagram
\[
    \input adjunction-eq-1
\]
will be created at the corresponding place in the document.
\section{Usage}
Once installed, one can invoke the program by typing in a terminal

\vspace{\vlength}
\texttt{\$ tikzsd file1 ... filen}
\vspace{\vlength}

\noindent where \texttt{file1 ... filen} is a list of one or more text files
    containing code to be processed by the program.
The program processes the commands in these files in order.

In our examples, we have named our text files with code for processing
    by \texttt{tikzsd} with extension \texttt{.tzsd}, i.e.
    \texttt{file1.tzsd}.
This is not necessary, and just our convention.
In the rest of this document, by a \texttt{.tzsd} file we mean a text file
    with code to be processed by \texttt{tikzsd}.

One of the commands which can be given in a \texttt{.tzsd} file
    is a draw command telling the program to output TikZ code to a file
    (see section \ref{draw-ref}).
This file can then be \texttt{\textbackslash input} into a \texttt{.tex}
    file where you want the corresponding diagram.

We recommend looking at the example in \texttt{/doc/examples/benabou-roubaud/}
    of this repository for an example usage of this program.

\section{Defining Categorical Objects}
\subsection{Defining categories}
One defines a category using the following notation:

\vspace{\vlength}
\begin{verbatim}
define category <id> "<latex-string>"
\end{verbatim}
\vspace{\vlength}
where
\begin{itemize}
    \item \texttt{<id>} is a string of characters which does not contain a space character,
        or one of the characters \texttt{\&}, \texttt{\char`\\}, \texttt{[}, or \texttt{]}.
    This \texttt{<id>} is used to refer to the category in the rest of the \texttt{.tzsd} file.
    \item \texttt{<latex-string>} is a string of characters which does not contain the character
        \texttt{"}.
    This string contains the \LaTeX \ code used to label the category.
\end{itemize}
The spaces in the above code may be replaced with any nonempty sequence of space characters,
    such as spaces, tabs or new lines.

\begin{example}
The code
\vspace{\vlength}
\begin{verbatim}
define category C "$C$"
\end{verbatim}
\vspace{\vlength}
\noindent will define a category, which is referred to by the id \texttt{C} in the rest of the
\texttt{.tzsd} file, and which will be labeled using \texttt{\$C\$}.
\end{example}
\begin{remark}
In version 1.0.0 of \texttt{tikzsd}, the \texttt{<latex-string>} is never used.
There is at least one planned future feature which will use it.
This \texttt{<latex-string>} will likely be made optional in a future release.
\end{remark}
\subsection{Defining functors}
One defines a functor using the following notation:
\vspace{\vlength}
\begin{verbatim}
define functor <id> "<latex-string>" source:<s-id> target:<t-id>
\end{verbatim}
\vspace{\vlength}
where
\begin{itemize}
    \item \texttt{<id>} is a string of characters which does not contain a space character,
        or one of the characters \texttt{\&}, \texttt{\char`\\}, \texttt{[}, or \texttt{]}.
        This \texttt{<id>} is used to refer to the functor in the rest of the \texttt{.tzsd}
            file.
    \item \texttt{<latex-string>} is a string of characters which does not contain the character
            \texttt{"}.
        This string contains the \LaTeX \ code used to label the functor.
    \item \texttt{<s-id>} is the id of the category which is the source of the functor.
    \item \texttt{<t-id>} is the id of the category which is the target of the functor.
\end{itemize}
The spaces in the above code may be replaced by any nonempty sequence of space characters, such as
    spaces, tabs or new lines.
One may also insert space characters after \texttt{source} or \texttt{target} before the \texttt{:}
    character, or after the \texttt{:} character before the \texttt{<s-id>} or \texttt{<t-id>}.
\begin{example}
The code
\vspace{\vlength}
\begin{verbatim}
define functor F "$F$"
    source : C
    target : D
\end{verbatim}
\vspace{\vlength}
will define a functor, which is referred to by the id \texttt{F} in the rest of the \texttt{.tzsd} file,
and which will be labelled using \texttt{\$F\$}.
This functor has source the category \texttt{C} and target the category \texttt{D}.
The program will print an error if the category \texttt{C} or the category \texttt{D} have not been
    defined.

Due to how spacing works, one can also put all of the above code on one line.
So
\vspace{\vlength}
\begin{verbatim}
define functor F "$F$" source:C target:D
\end{verbatim}
\vspace{\vlength}
is equivalent to the code at the beginning of this example.
\end{example}
\subsection{Specifying compositions of functors (and spacing)} \label{specifying-func-comp}
A composition of functors is specified by a \texttt{\&} separated list of elements
    which is terminated by the two characters \texttt{\char`\\}\texttt{\char`\\}, as below:
\vspace{\vlength}
\begin{verbatim}
<elem-1> & <elem-2> & ... & <elem-n> \\
\end{verbatim}
\vspace{\vlength}
Here, \texttt{<elem-i>} corresponds to the $i$th column.
Each \texttt{<elem-i>} is either
\begin{enumerate}
    \item The id of a functor.
    \item The id of a category, corresponding to the identity functor of that category.
    \item An empty string.
        This is a placeholder used for spacing purposes.
\end{enumerate}
At least one of the elements is not of type (3).
The spaces in the above code may be replaced with zero or more space characters,
    such as spaces, tabs and new lines.

In a composition of functors, we shall call elements of the first type \emph{basic functors}.
These will correspond to strings in a string diagram.
The length of the composite will be the number of basic functors in the composite.

As is conventional for how string diagrams are drawn, the composition is from left to right.
So to specify a composition $F\circ G$, one might write
\vspace{\vlength}
\begin{verbatim}
G & F \\
\end{verbatim}
\vspace{\vlength}
which corresponds to a string for $G$ in the first column and a string for $F$ in the second column,
or one might write
\vspace{\vlength}
\begin{verbatim}
 & G & & F \\
\end{verbatim}
\vspace{\vlength}
which corresponds to a string for $G$ in the second column, and a string for $F$ in the fourth
column.
\begin{remark}
    Since identity functors are not drawn by the program, one can often omit elements of type (2)
        and use empty elements (3) instead.

    For example, if $F:C\to D$ and $G:D\to C$ are functors, then
\vspace{\vlength}
\begin{verbatim}
F & D & G \\
\end{verbatim}
\vspace{\vlength}
    and
\vspace{\vlength}
\begin{verbatim}
F & & G \\
\end{verbatim}
\vspace{\vlength}
    both specify the same composite functor $G\circ F$ and the same spacing for the
    basic functors $F$ and $G$.

The one exception to this is when specifying an identity functor by itself:
\vspace{\vlength}
\begin{verbatim}
C \\
\end{verbatim}
\vspace{\vlength}
cannot be changed to
\vspace{\vlength}
\begin{verbatim}
 \\
\end{verbatim}
\vspace{\vlength}
because an empty composition is not well-defined, so the meaning of a composition
with only empty elements cannot be imputed.
\end{remark}
\subsection{Defining natural transformations}
One defines a natural transformation using the following notation:
\vspace{\vlength}
\begin{verbatim}
define natural transformation <id> "<latex-string>"
    source : <source-composition>
    target : <target-composition>
    shape : <shape>
\end{verbatim}
\vspace\vlength
where
\begin{enumerate}
    \item \texttt{<id>} is a string of characters which does not contain a space character,
        or one of the characters \texttt{\&}, \texttt{\char`\\}, \texttt{[}, or \texttt{]}.
        This \texttt{<id>} is used to refer to the natural transformation in the rest of the
        \texttt{.tzsd} file.
    \item \texttt{<latex-string>} is a string of characters which does not contain the character
        \texttt{"}.
        This string contains the \LaTeX\ code used to label the natural transformation.
    \item \texttt{<source-composition>} is code used to specify a composite functor which is
        the source of the natural transformation.
        Notation for this code is described in \ref{specifying-func-comp}.
    \item \texttt{<target-composition>} is code used to specify a composite functor which is
        the target of the natural transformation.
        Notation for this code is described in \ref{specifying-func-comp}.
    \item The line \texttt{shape:<shape>} is optional.
        If it exists, \texttt{<shape>} is a string of alphanumeric characters
            (no spaces or punctuation),
            and is \LaTeX\ code used to specify the shape of the
            node representing the natural
            transformation.
        If it doesn't exist, no \LaTeX code specifying the shape is passed to the
            node, and the shape of the node will be the default shape
            in your \texttt{tikzpicture} environment.
\end{enumerate}
In the current version of the program, the two composite functors specified by \texttt{<source-composition>}
    and \texttt{<target-composition>} cannot both be the identity functor of a category $C$.
Conventionally, string diagrams do not draw identity functors, so the node
    corresponding to the natural transformation would be a very
    ambiguous node with no in or out strings.
\begin{remark}
    In \texttt{<source-composition>} and \texttt{<target-composition>},
        only the composite functor is desired, so the particular spacing given in these strings is
        ignored.
    In other words, for the purposes of specifying the source or target of a natural transformation,
\vspace{\vlength}
\begin{verbatim}
    F & G \\
\end{verbatim}
\vspace{\vlength}
and
\vspace{\vlength}
\begin{verbatim}
    F & & G \\
\end{verbatim}
\vspace{\vlength}
    are equivalent.
\end{remark}
\begin{remark}
    The string \texttt{<shape>} is just treated as \LaTeX\ code which is passed to
        every node for the natural transformation.
    In particular, you may need to define the shape in your \LaTeX\ file
        if it is not a shape given by the package \texttt{tikz}.
\end{remark}
\begin{example}
    The code
\vspace\vlength
\begin{verbatim}
define natural transformation unit "$\eta$"
    source : C \\
    target : F & G \\
    shape : utriangle
\end{verbatim}
\vspace\vlength
defines a natural transformation, referred to by the id \texttt{unit} in the rest
    of the \texttt{.tzsd} file, and which will be labeled by using \texttt{\$\char`\\eta\$}.
When represented by a tikz node, the program will pass the string
    \texttt{utriangle} as an option.
If \texttt{C} is the id of a category $C$, and \texttt{F} and \texttt{G}
    are the ids of functors $C\to D$ and $D\to C$,
    the source of our defined natural transformation is the identity functor of $C$,
    and the target is the composition $G\circ F$.
\end{example}
\section{Drawing String Diagrams} \label{draw-ref}
Once the categorical objects are defined, the program is ready to draw string diagrams.
For us, a string diagram is a way of visualizing a vertical composition
    of horizontal compositions of natural transformations.

\subsection{Draw a natural transformation}
The notation for drawing a string diagram is as follows:
\vspace\vlength
\begin{verbatim}
draw <output-file> [<options>]
    <line-1>
    <line-2>
    ...
    <line-n>
\end{verbatim}
\vspace\vlength
where
\begin{enumerate}
    \item \texttt{<output-file>} is a string of characters which does not include
        any space character.
        This is the name of the file the program will write the \LaTeX\ code
        for a \texttt{tikzpicture} environment drawing the string diagram.
        The program will overwrite the contents of the file if it exists.
    \item \texttt{[<options>]} is optional.
        If it exists, \texttt{<options>} is a string of characters which does not
        include the characters \texttt{[} or \texttt{]}.
        This string is passed as options to the \texttt{tikzpicture}
        written to the output file.
        For example, one can write
        \texttt{[x=0.5cm,y=0.5cm]} to change the horizontal and vertical spacing.
    \item \texttt{<line-1>}, \texttt{<line-2>}, ..., \texttt{<line-n>}
        are lines used to specify the string diagram to be drawn.
        Each line is either a \emph{functor line}, which starts with \texttt{f:},
        or a \emph{natural transformation line}, which starts with \texttt{n:}.
        Roughly speaking, each natural transformation line specifies a horizontal
            composition of defined natural transformations, and the
            functor lines specify the spacing of the sources and targets of these
            natural transformations.
\end{enumerate}
We rewrite the example from the introduction here for reference before describing the
    notation for functor lines and natural transformation lines.
\vspace\vlength
\lstinputlisting{adjunction-draw.tzsd}
\vspace\vlength

\subsection{Functor lines} A functor line is of the form
\vspace\vlength
\begin{verbatim}
f : <composition>
\end{verbatim}
\vspace\vlength
    where \texttt{<composition>} is code for specifying a composition of functors,
    with spacing, as described in \ref{specifying-func-comp}.

    For example,
\vspace\vlength
\begin{verbatim}
f: F & G & F \\
\end{verbatim}
\vspace\vlength
    specifies the composition $F\circ G\circ F$, and says that the strings for
    these functors should be placed in the first, second and third columns.
\subsection{Natural transformation lines}
    Natural transformation lines are used to specify horizontal compositions of natural transformations,
        with empty spaces imputed to be the identity natural transformation.

    A natural transformation line is of the form
\vspace\vlength
\begin{verbatim}
n : <elem-1> & <elem-2> & ... & <elem-n> \\
\end{verbatim}
\vspace\vlength
    where each \texttt{<elem-i>} is either
    \begin{enumerate}
        \item The id of a natural transformation.
        \item The id of a functor, representing the identity natural transformation of that functor.
        \item The id of a category, representing the identity functor of the identity natural
            transformation of that category.
        \item An empty string. This tells the program use the expected source of the natural
        transformation specfied by this line of natural transformations
        (which comes from the preceding lines) to impute an identity functor at at this position.
    \end{enumerate}
\vspace\vlength
The horizontal composition is from left to right, following the conventional layout of a string diagram
    in category theory.
\subsection{Default spacing}
One can specify the string diagram to be drawn by alternating functor and natural transformation
lines, starting and ending with a functor line.
The functor lines specify the spacing of the strings for the functors in horizontal cross-sections
of the string diagram, while the natural
transformation lines specify the natural transformations between these functor lines.

\begin{example}
For example, we have seen
\vspace\vlength
\begin{verbatim}
f : & & F \\
n : unit & \\
f : F & G & F \\
n : & counit \\
f : F & & \\
\end{verbatim}
\vspace\vlength
    which alternates between functor lines and natural transformation lines.

If one omits a functor line, the program can figure out which strings are expected at that cross
section from the natural transformation lines, but it cannot figure out the spacing.
If a functor line is omitted, the default spacing is used, where the leftmost string gets put in the
first column, the second leftmost string in the second columns, ..., until the $n$th string
gets put in the $n$th column.
So to be less verbose, the above example is equivalent to
\vspace\vlength
\begin{verbatim}
f : & & F \\
n : unit & \\
n : & counit \\
\end{verbatim}
\vspace\vlength
Here, we've omitted the second and third functor lines and let the program use the default spacing
    for these cross sections.
\end{example}
\subsection{Identity imputation}
If a natural transformation line contains a empty element,
    the program looks at the previous lines and imputes an identity natural transformation to put
    in the corresponding place.
\begin{example}
In the example from the introduction of this manual,
\vspace\vlength
\begin{verbatim}
f : & & F \\
n : unit & \\
\end{verbatim}
\vspace\vlength
    after processing the first line, the program sees that the current horizontal cross-section has one string,
    corresponding to the functor with id \texttt{F}.
When processing the second line, the program sees that the natural transformation with
    id \texttt{unit} has $0$ strings coming in and $2$ strings coming out.
It then imputes that the blank space should be the identity of the first string of the current
cross-section.
Thus, the program imputes \texttt{F} at the empty space, and processes the above as if it were
\vspace\vlength
\begin{verbatim}
f : & & F \\
n : unit & F \\
\end{verbatim}
\vspace\vlength
Similarly,
\vspace\vlength
\begin{verbatim}
f : F & G & F \\
n : & counit \\
\end{verbatim}
\vspace\vlength
is processed as if it were
\vspace\vlength
\begin{verbatim}
f : F & G & F \\
n : F & counit \\
\end{verbatim}
\vspace\vlength
\end{example}
\section{Future Roadmap}
This is the user manual for version 1.0.0 of the \texttt{tikzsd} program.
Here is a list of some features planned for future releases.
\begin{enumerate}
\item Give a way to pass options to the labels of the strings.
    For example passing the option \texttt{swap} should put the label on the opposite side.
\item Give a way to change the positioning of the  the labels of the strings/
        have multiple labels per string/change the default behavior in other ways.

    By default, the program attempts to put the label at approximately the vertical midpoint
        of the string.
    It should be possible to have a way of specifying something like \texttt{pos=20},
        \texttt{pos=above} or \texttt{pos=below}.
    Since each string is actually several Bezier curves spliced together,
        one cannot just pass this as an option, and the program needs to compute the positioning
        itself.
\item Give alternate, less verbose methods of defining functors and natural transformations.
    Instead of writing \texttt{source} and \texttt{target} every time, maybe just
        use an arrow \texttt{->} or \texttt{=>}.
\item Give a way to define a functor as a composition of other functors.
    Then when drawing, give an option to either use one string to represent this functor
        or to \texttt{expand} the functor into its components.
    So a functor defined to be the composition of two other functors will be drawn
        as two strings when expanded, and just one string when not expanded.
    There needs a way to associate a default spacing when expanding this functor.

    One can view identity functors of categories as expanded versions of compositions
        of no functors.
\item When drawing natural transformations, for functor lines, allow the specification
        of spacing without rewriting the names of the functors.
\item Allow other ways of representing natural transformations.
    Currently, all natural transformations are represented by a TikZ node.
    We should be able to allow crossing over, crossing under, etc.
\item Give notation to define common categorical constructs.
    For example, there can be simplified notation for defining adjunctions,
        monads, comonads, algebras for a monad, etc.
\item Give a way to write comments.
\item Give a way to change the base directory where the program writes its output.
\end{enumerate}
\end{document}


\section{Model-based Attack Generation}

One of the main challenges to solving this problem is how to generate effective test cases (\ie, attacks)
since there is an exponential number of possible test cases.
To overcome this challenge, we devise a model-guided attack generation technique
that focuses on testing only important aspects of the congestion control technique.
The key insights behind our approach are: (i) the model of a congestion control technique
essentially describes how it transitions through different states and
takes appropriate actions to deal with the current network condition
and (ii) the attacker induces the victim to take some actions that are not
appropriate in the current context but favorable for the attacker.
Depending on the attacker's goal (\eg, increase or decrease throughput),
we utilize the user-provided model to find the paths containing
the vulnerable actions that the attacker wants the victim to perform.
These paths provide us some potential abstract test cases, which
in turn need to be converted to concrete ones before conducting the actual
test using our platform.

\subsection*{Algorithm}

\paragraph{Problem definition} Given a directed \textbf{cyclic} graph (\ie, the model), we need to
find all paths containing specific edge labels (\ie, actions) starting from the root node.

\paragraph{Background}
%The characteristics of
The graph ($G = \langle V, E \rangle$):
\begin{itemize}
  \item
  is a directed graph
  \item
  Contains cycles (makes the problem challenging)
  \item
  By specific edge label, we mean only the action part of an edge label.
  %In other words, a specific edge label is vulnerable action
  \item
  One vulnerable action can appear on multiple edges.
  Formally, a vulnerable action $A$ can appear $(x, y) \in E$ for any $x, y \in V$
\end{itemize}

\paragraph{Requirements}
\begin{itemize}
  \item
  A desired goal of the attacker may require a combination of one or more vulnerable actions to take place.
  For now, we assume a goal maps to only one vulnerable action.
  \item
  The desired path must contain an edge with the vulnerable action
  \item
  To obtain a path that can generate a test case with repeated attacks,
  the path must contain a vulnerable action edge as a part of a cycle.
  The cycle dictates the repetition of the attack.
\end{itemize}

Our algorithm is shown in Algorithm~\ref{algorithm:modelsearch}.

\begin{algorithm} %[!t]
\DontPrintSemicolon
\SetKwProg{Fn}{Function}{}{}
\SetKwFunction{FMain}{Main}
\SetKwFunction{FScan}{Scan}

%\small
 \tcc{{Objective}: Find all paths containing a specific action in a given graph (model)}
 \KwIn{Graph $G = (V,E)$ and action $A$}
 \KwOut{All paths containing $A$ }

 \BlankLine
 \BlankLine

 \tcc{
 $adj: (V \times E) \mapsto V$. For example,
 if there is an edge $e$ between nodes $u$ and $v$, then $adj(u, e) = v$.}
 \tcc{
 Global variable \pred{ResultPaths} contains all the resulted paths
 }

 \BlankLine
 \BlankLine

 \Fn(\tcc* {program starts here}){\FMain {$G, A$}}{
	$root$ = getRoot($G$)\;
	Mark $root$ as \pred{visited} \;
	\ForEach{edge $e$ starting at $root$}{
		Create a new path $P$ and add $(root, e)$ to $P$ \;
		$v$ = $adj(root, e)$ \;
		\FScan{$v, P, A$} \;
	}
	\If{\pred{ResultPaths} is not empty}{
		\KwRet \pred{ResultPaths}
	}
 }

 \BlankLine
  \BlankLine

 \Fn(\tcc*{search continue from $v$}){\FScan {$v, P, A$}}{
 	$base\_case$ = \pred{false} \;
 	\If(\tcc*[h]{reached a cycle}){$v$ is already \pred{visited}}{
 		$base\_case$ = \pred{true} \;
 	}
 	\ElseIf(\tcc*[h]{has no child}){$v$ is a leaf node}{
 		$base\_case$ = \pred{true} \;
 		Add $(v, \bot)$ to $P$ \;
 	}
 	\If{$base\_case$ is \pred{true}}{
 		\If{$P$ contains any edge with $A$}{
 			Add $P$ to \pred{ResultPaths} \;
 		}
 	}
 	\Else{
 		Mark $v$ as \pred{visited} \;
 		\ForEach{edge $e$ starting at $v$}{
 			$v'$ = $adj(v, e)$ \;
 			$P'$ = $copy(P)$ \;
 			Add $(v', e)$ to $P'$ \;
 			\FScan{$v', P', A$}
 		}
 		Mark $v$ as \pred{unvisited} \;
 	}
 	\KwRet \tcc*{void}
 }

% \Fn(\tcc*[h]{search continue from $v$}){\FScan {$v, P, A$}}{
% Queue $W_e \gets \emptyset$; FSM $M \gets \emptyset$\;
% Create initial program state $e_0$\;
%% \tcp{$q_0$ is the initial protocol FSM state}
% $q_0\gets\pred{ExtractFsmState}(e_0);$~~~$//q_0:$ initial protocol state\\
% $W_e.\pred{enqueue}(e_0);$\\
% $M.Q\gets \{q_0\};$ ~~~$//M.Q:$ set of states\\
% $M.q_I\gets q_0;$ ~~~~~~$//M.q_I:$ initial state\\
% $M.R\gets \emptyset;$ ~~~~~~~~$//M.R:$ set of transitions\\
% Mark state $q_0$ as old\;
% \While{$W_e \neq\emptyset$}{
%  $e_i \gets W_e.\pred{dequeue}()$\;
%  $q_i \gets \pred{ExtractFsmState}(e_i)$\;
%  \ForEach{Event $\tau\in \pred{EventList}$}{
%    $S_q\gets \pred{SymbolicExecution}(S, C, N, e_i,\tau)$\;
%    \ForEach{$\langle e_j, c, a\rangle\in S_q$}{
%        $q_j\gets \pred{ExtractFsmState}(e_j)$\;
%        \If{$q_j$ is not old}{
%         $W_e.\pred{enqueue}(e_j)$\;
%         $M.Q\gets M.Q\cup \{q_j\}$\;
%         Mark state $q_j$ as old \;
%        }
%        \If{Transition $\langle q_i, \tau, c, a, q_j\rangle$ is not old}{
%           $M.R\gets M.R\cup \{\langle q_i, \tau, c, a, q_j\rangle\}$\;
%           Mark transition $\langle q_i, \tau, c, a, q_j\rangle$ as old \;
%        }
%    }
%  }
% }
% \Return $M$
% }
 \caption{Find Vulnerable Paths\label{algorithm:modelsearch}}
\end{algorithm}


\documentclass{article}

\usepackage{biblatex}
\bibliography{refs.bib}

\usepackage[letterpaper]{geometry}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc}    % use 8-bit T1 fonts

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{hyperref}       % hyperlinks
\usepackage{cleveref}
\usepackage{url}            % simple URL typesetting
\usepackage{booktabs}       % professional-quality tables
\usepackage{amsfonts}       % blackboard math symbols
\usepackage{nicefrac}       % compact symbols for 1/2, etc.
\usepackage{microtype}      % microtypography

\usepackage{algorithmic}
\usepackage{algorithm}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}

\newcommand{\DD}{\mathbb{D}}
\newcommand{\cD}{\mathcal{D}}
\newcommand{\ex}{\mathbb{E}}
\newcommand{\cH}{\mathcal{H}}
\newcommand{\cS}{\mathcal{S}}
\newcommand{\tp}{\intercal}

\title{Posterior sampling over partitioned data with Stein variational gradient descent.}
\author{Feynman Liang, Qiang Liu, Michael Mahoney}
\date{\today}

\begin{document}

\maketitle

\section{Introduction}

Drawing samples from a posterior distribution $P(\theta \mid \mathcal{D})$ is a fundamental
tool powering many applications of Bayesian statistics.
Stein variational gradient descent (SVGD) \cite{liu2016stein} is a recently developed
particle-based algorithm for performing posterior sampling from arbitrary distributions.

Modern datasets are increasingly large, and in many problems of interest the data $X$
does not fit on a single machine. In such environments, development of distributed algorithms
which require only subsets of the data at each iteration are of particular interest. Furthermore,
the increased communication costs incurred from network I/O between machines motivates
the need for communication efficient algorithms.

In this work, we propose a new extension to SVGD which can be applied when data
and/or particles are partitioned across multiple machines. Our work closes the scalability
gap for SVGD and enables it to scale to big datasets and a large number of potentially high
dimensional particles.

\section{Background}
\label{sec:background}

Given some probability measure $\nu$ with density $p$ (e.g.\ the posterior distribution $P(\theta\ mid \cD)$),
we want to find a particle approximation $\{\theta_j\}_{j=1}^n$ whose empirical
measure $\hat\mu_n = \frac{1}{n}\sum^{n}_{j=1} \delta_{\theta_j}$ converges
weakly
\begin{align}
    \mu_n \Rightarrow \nu
\end{align}

SVGD does this by initialiing $\theta_j \overset{\sim}{\text{iid}} \mu$ and performs iterative updates
\begin{align}
    T_{\epsilon, \phi}(\theta) = \theta + \epsilon \phi(x)
\end{align}
The choice of $\phi$ should make the approximation better, and is formalized by the functional optimization
\begin{align}
    \max_{\phi \in \cH} \left\{
        - \frac{d}{d\epsilon} KL(T \mu \mid \nu ) \mid_{\epsilon = 0} : \|\phi\|_\cH \leq 1
    \right\}
\end{align}
where $\cH$ is a Banach space.

\cite{liu2016stein} made the connection between this objective and the Stein operator from statistics
\begin{align}
    - \frac{d}{d\epsilon} KL(T \mu \mid \nu) \mid_{\epsilon = 0}
    = \ex_\mu \cS_p \phi
    \quad\text{where}\quad
    \cS_p \phi(x) = \nabla \log p(x)^\tp \phi(x) + \nabla \cdot \phi(x)
\end{align}

The optimization thus is equivalent to
\begin{align}
    \DD(\mu \mid \nu) = \max_{\phi \in \cH} \left\{ \ex_\mu \cS_p \phi : \|\phi\|_\cH \leq 1 \right\}
\end{align}
which is called the \emph{Stein discrepancy}. Restricting $\cH$ to a RKHS renames this quantity
to the \emph{Kernelized Stein discrepancy}.

\cite{liu2016kernelized} showed that the optimal solution
\begin{align}
    \phi_{\mu,p}^*(\cdot) \propto \ex_{x \sim \mu} \left[
        \nabla \log p(x) k(x, \cdot) + \nabla_x k(x, \cdot)
    \right]
\end{align}
suggesting an iterative algorith given in \cref{alg:svgd}.

\begin{algorithm}
    \caption{The SVGD algorithm}
    \label{alg:svgd}
    \begin{algorithmic}
        \REQUIRE{Likelihood $p(\cD \mid \theta)$, prior $p(\theta)$, and initial particles $\{\theta_i^0\}_{i=1}^n$}
        \ENSURE{Particles $\{\theta_i\}_{i=1}^n$ approximating the posterior $p(\theta \mid \cD)$}
        \FOR{iteration $\ell$}
            \FOR{particle~$i=1$ \TO $n$} \STATE{
                $\theta_i^{\ell+1} \leftarrow \theta_i^\ell + \epsilon_\ell \hat\phi^*(\theta_i^\ell)$
                ~where~
                $\hat\phi^*(\theta) = \frac{1}{n} \sum_{j=1}^n \left[
                    k(\theta_j^\ell, \theta) \nabla_{\theta_j^\ell} [\log p(\cD \mid \theta_j^\ell) + \log p(\theta_j^\ell)]
                    + \nabla_{\theta_j^\ell} k(\theta_j^\ell, \theta)
                \right]$
            }
            \ENDFOR
        \ENDFOR
    \end{algorithmic}
\end{algorithm}

Notice that:
\begin{enumerate}
    \item The iteration over particles $i$ is embaressingly parallel
    \item Each of the $n^2$ updates requires computing a score $\log p(\cD \mid \theta)$ over
        the full dataset $\cD$
\end{enumerate}

\section{Distributed SVGD}
\label{sec:distributed_svgd}

The second observation motivates our first proposed method, which extends SVGD
to the setting where $\cD = \sqcup_{s=1}^S \cD_s$ is partitioned across $s$ different shards.

We make the following key assumption:
\begin{enumerate}
    \item Data is iid, so we can decompose $\log p(\cD \mid \theta) = \sum_{s=1}^S p(\cD_s \mid \theta)$
    \item The set of particles $\{\theta_i\}_{i=1}^n$ can fit on a single machine
\end{enumerate}

Assumption 1 enables us to rewrite the computation for $\hat\phi^*(\theta)$ as
\begin{align}
    \hat\phi^*(\theta)
    &= \frac{1}{n} \sum_{j=1}^n \left[
        k(\theta_j^\ell, \theta) \left[
            \sum_{s=1}^S \nabla_{\theta_j^\ell} \log p(\cD_s \mid \theta_j^\ell) + \nabla_{\theta_j^\ell} \log p(\theta_j^\ell)
        \right]
        + \nabla_{\theta_j^\ell} k(\theta_j^\ell, \theta)
    \right]
\end{align}

Assumption 2 permits broadcasting of the set of particles $\theta_j$ across to each of the $s$ workers, leading
to a map-reduce style distributed algorithm
\begin{enumerate}
    \item Master broadcast $\{\theta_j\}_{j=1}^n$ to all of the $s$ workers
    \item Each worker computes a local score $\nabla_{\theta_j^\ell} \log p(\cD_s \mid \theta_j^\ell)$ (same dimensionality as $\{\theta_j\}_{j=1}^n$)
    \item Results are grouped by particle $\theta_j$ and reduced (by summation) onto the master node, which performs the particle update
\end{enumerate}

\subsection{Analysis of communication complexity}
\label{sub:analysis_of_communication_complexity}

We assume that communication occurs point-to-point, nodes have their own
upstream and downstream links, and that a barrier synchronization occurs at the
end of each iteration. Such assumptions are realistic in distributed computing
settings such as cloud environments where issues with large datasets are typically
encountered.

The communication complexity between two synchronization barriers is determined
by the slowest node, so we focus attention on the bottleneck nodes.
If $\theta$ is $p$-dimensional, the communication complexity of each iteration is
\begin{align}
    p n \times S + p n \times S
\end{align}
where the first term corresponds to the sequential broadcast of all particles through
the master's upstream link and the latter to the reduction of all local scores through
the master's downstream link.

A $k$-wise tree-broadcast and reducecan be applied to lower communication costs
to $2 p n k \log_k S$, but incurs an additional constant message
serialization/deserialization overhead.

\subsection{Partitioning particles to minimize communication bottleneck}
\label{sub:partitioning_particles_to_minimize_communication_bottleneck}

The previous algorithm exhibits a communication bottleneck on the master node,
resulting in a higher communication cost per iteration. To avoid this
bottleneck, we can exploit our first observation that the iteration over particles
is trivially parallelizable. Rather than designating a master node,
partition the particles equally across all nodes.
In doing so, the broadcast is converted into an \texttt{all\_gather} and
the reduce into an \texttt{all\_reduce}.

By doing so, we avoid hot-spotting on the master and get communication cost
\begin{align}
    p (n/S) (S-1) + p (n/S) (S-1) + p n k \log_k S
    \approx 2 p n + p n k \log_k S
\end{align}

\subsection{Stochastic approximations}%
\label{sub:stochastic_approxmations}

Treat partitions as subsampled minibatches (in reality we don't resample the minibatch)
\begin{align}
    \nabla_\theta \log p(\theta \mid \cD)
    &= \nabla_\theta \log p(\theta) + \nabla_\theta \log p(\cD \mid \theta) \\
    &\approx \nabla_\theta \log p(\theta) + \frac{\lvert \cD \rvert}{\lvert \cD_s \rvert} \nabla_\theta \log p(\cD_s \mid \theta)
\end{align}

Subsample the particle interactions using only local particles
\begin{align}
    \frac{1}{n} \sum^{n}_{j=1} \left[
        k(x_j, x) \nabla_{x_j} \log p(x_j) + \nabla_{x_j} k(x_j, x)
    \right]
    \approx \frac{1}{\Omega} \sum_{j \in \Omega}  \left[
        k(x_j, x) \nabla_{x_j} \log p(x_j) + \nabla_{x_j} k(x_j, x)
    \right]
\end{align}

To prove: both LHS and RHS are empirical averages, concentration about mean.

If $\Omega \subset [n]$ is sampled iid with replacement, then both are iid
samples of the data $X$. If $f \leq B$, then Hoeffding's inequality gives
\begin{align}
    P(\sum_1^n f(X_i) - \ex f(X) \geq \alpha) \leq \exp \left( - \frac{\alpha^2}{2 n B^2}\right)
\end{align}

\section{Experiments}
\label{sec:experiments}

\printbibliography

\end{document}


    \filetitle{windex}{Simple weighted or Divisia index}{tseries/windex}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
Y = windex(X,W,Range)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{X} {[} tseries {]} - Input times series.
\item
  \texttt{W} {[} tseries \textbar{} numeric {]} - Fixed or time-varying
  weights on the input time series.
\item
  \texttt{Range} {[} numeric {]} - Range on which the Divisia index is
  computed.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{Y} {[} tseries {]} - Weighted index based on \texttt{X}.
\end{itemize}

\paragraph{Options}\label{options}

\begin{itemize}
\item
  \texttt{'method='} {[} `divisia' \textbar{} \emph{`simple'} {]} -
  Weighting method.
\item
  \texttt{'log='} {[} \texttt{true} \textbar{} \emph{\texttt{false}} {]}
  - Logarithmise the input data before computing the index,
  delogarithmise the output data.
\end{itemize}

\paragraph{Description}\label{description}

\paragraph{Example}\label{example}


\section{Hyperbolic Functions Revisited}\label{sec:inv hyperbolic}\label{sec:hyperbolic revisited}

In Section \ref{sec:hyperbolic functions} we introduced the hyperbolic functions.  From Definition \ref{def:hyperbolic_functions} we recall their respective definitions.

\begin{enumerate}
\item		$\ds \cosh x = \frac{e^x+e^{-x}}2$
\item		$\ds \sinh x = \frac{e^x-e^{-x}}2$
\item		$\ds \tanh x = \frac{\sinh x}{\cosh x}$
\item		$\ds \sech x = \frac{1}{\cosh x}$
\item		$\ds \csch x = \frac{1}{\sinh x}$
\item		$\ds \coth x = \frac{\cosh x}{\sinh x}$
\end{enumerate}

\begin{example}{Exploring properties of hyperbolic functions}{ex_hf1}
{
Use the definitions of the hyperbolic functions to rewrite the following expressions.
\begin{enumerate}
\item		$\frac{d}{dx}\big(\cosh x\big)$
\item		$\frac{d}{dx}\big(\sinh x\big)$
\item		$\frac{d}{dx}\big(\tanh x\big)$
\end{enumerate}
}
\end{example}

\begin{solution}
{\begin{enumerate}
\item  \hfill$\begin{aligned}[t]
	\frac{d}{dx}\big(\cosh x\big) &= \frac{d}{dx}\left(\frac{e^x+e^{-x}}2\right) \\
					&= \frac{e^x-e^{-x}}2\\
					&= \sinh x.
	\end{aligned}\hfill$

So $\frac{d}{dx}\big(\cosh x\big) = \sinh x.$

\item  \hfill$\begin{aligned}[t]
	\frac{d}{dx}\big(\sinh x\big) &= \frac{d}{dx}\left(\frac{e^x-e^{-x}}2\right) \\
					&= \frac{e^x+e^{-x}}2\\
					&= \cosh x.
	\end{aligned}\hfill$

So $\frac{d}{dx}\big(\sinh x\big) = \cosh x.$

\item  \hfill$\begin{aligned}[t]
	\frac{d}{dx}\big(\tanh x\big) &= \frac{d}{dx}\left(\frac{\sinh x}{\cosh x}\right) \\
					&= \frac{\cosh x \cosh x - \sinh x \sinh x}{\cosh^2 x}\\
					&= \frac{1}{\cosh^2 x}\\
					&= \sech^2 x.
	\end{aligned}\hfill$

So $\frac{d}{dx}\big(\tanh x\big) = \sech^2 x.$
\end{enumerate}
\vskip-\baselineskip
}
\end{solution}

The following Key Idea summarizes many of the important identities relating to hyperbolic functions. Each can be verified by referring back to Definition \ref{def:hyperbolic_functions}.

\begin{formulabox}[Useful Hyperbolic Function Properties]
\label{idea:hyperbolic_identities}
\footnotesize{\begin{minipage}[t]{.33\textwidth}
\textbf{Basic Identities}\par
\begin{enumerate}
\item $\cosh^2x-\sinh^2x=1$%
\index{hyperbolic function!identities}\index{hyperbolic function!derivatives}\index{hyperbolic function!integrals}\index{derivative!hyperbolic funct.}\index{integration!hyperbolic funct.}%
\item	$\tanh^2x+\sech^2x=1$
\item	$\coth^2x-\csch^2x = 1$
\item	$\cosh 2x=\cosh^2x+\sinh^2x$
\item	$\sinh 2x = 2\sinh x\cosh x$
\item	$\ds\cosh^2x = \frac{\cosh 2x+1}{2}$
\item $\ds \sinh^2x=\frac{\cosh 2x-1}{2}$
\end{enumerate}
\end{minipage}
\begin{minipage}[t]{.33\textwidth}
\textbf{Derivatives}
\begin{enumerate}
\item $\frac{d}{dx}\big(\cosh x\big) = \sinh x$
\item $\frac{d}{dx}\big(\sinh x\big) = \cosh x$
\item $\frac{d}{dx}\big(\tanh x\big) = \sech^2 x$
\item $\frac{d}{dx}\big(\sech x\big) = -\sech x\tanh x$
\item $\frac{d}{dx}\big(\csch x\big) = -\csch x\coth x$
\item $\frac{d}{dx}\big(\coth x\big) = -\csch^2x$
\end{enumerate}
\end{minipage}
\begin{minipage}[t]{.33\textwidth}
\textbf{Integrals}
\begin{enumerate}
\item $\ds\int \cosh x\ dx = \sinh x+C$
\item $\ds\int \sinh x\ dx = \cosh x+C$
\item $\ds\int \tanh x\ dx = \ln(\cosh x) +C$
\item $\ds\int \coth x\ dx = \ln|\sinh x\,|+C$
\end{enumerate}
\end{minipage}
}
\end{formulabox}

Next, we practice using these properties.\\

\begin{example}{Derivatives and integrals of hyperbolic functions}{ex_hf2}
{
Evaluate the following derivatives and integrals.

\begin{minipage}[t]{.5\linewidth}
\begin{enumerate}
\item		$\ds\frac{d}{dx}\big(\cosh 2x\big)$
\item		$\ds\int \sech^2(7t-3)\ dt$
\end{enumerate}
\end{minipage}
\begin{minipage}[t]{.5\linewidth}
\begin{enumerate}\addtocounter{enumi}{2}
\item		$\ds \int_0^{\ln 2} \cosh x\ dx$
\end{enumerate}
\end{minipage}
}
\end{example}

\begin{solution}
{\begin{enumerate}
\item		Using the Chain Rule directly, we have $\frac{d}{dx} \big(\cosh 2x\big) = 2\sinh 2x$.

Just to demonstrate that it works, let's also use the Basic Identity found in Key Idea \ref{idea:hyperbolic_identities}: $\cosh 2x = \cosh^2x+\sinh^2x$.
\begin{align*}\frac{d}{dx}\big(\cosh 2x\big) = \frac{d}{dx}\big(\cosh^2x+\sinh^2x\big) &= 2\cosh x\sinh x+ 2\sinh x\cosh x\\ &= 4\cosh x\sinh x.
\end{align*}
Using another Basic Identity, we can see that $4\cosh x\sinh x = 2\sinh 2x$. We get the same answer either way.

\item	  We employ substitution, with $u = 7t-3$ and $du = 7dt$ to get:
$$ \int \sech^2 (7t-3)\ dt =\int \frac{1}{7}\sech^2 (u)\ du= \frac17\tanh (u) + C = \frac17\tanh (7t-3) + C.$$

\item		$$\int_0^{\ln 2} \cosh x\ dx = \sinh x\Big|_0^{\ln 2} = \sinh (\ln 2) - \sinh 0 = \sinh(\ln 2).$$
We can simplify this last expression as $\sinh x$ is based on exponentials:
$$\sinh(\ln 2) = \frac{e^{\ln 2}-e^{-\ln 2}}2 = \frac{2-1/2}{2} = \frac34.$$
\end{enumerate}
\vskip-1.5\baselineskip
}
\end{solution}

\noindent\textbf{\large Inverse Hyperbolic Functions}\\

Just as the inverse trigonometric functions are useful in certain integrations, the inverse hyperbolic functions are useful with others. Figure \ref{fig:hfinverse2} shows the restrictions on the domains to make each function one-to-one and the resulting domains and ranges of their inverse functions. Their graphs are shown in Figure \ref{fig:hfinverse1}.\index{hyperbolic function!inverse}

Because the hyperbolic functions are defined in terms of exponential functions, their inverses can be expressed in terms of logarithms as shown in Definition \ref{def:idea:hyperbolic_log}. It is often more convenient to refer to $\sinh^{-1}x$ than to $\ln\big(x+\sqrt{x^2+1}\big)$, especially when one is working on theory and does not need to compute actual values. On the other hand, when computations are needed, technology is often helpful but many hand-held calculators lack a \textit{convenient} $\sinh^{-1}x$ button. (Often it can be accessed under a menu system, but not conveniently.) In such a situation, the logarithmic representation is useful. The reader is not encouraged to memorize these, but rather know they exist and know how to use them when needed.

%\clearpage

\mtable{.7}{Graphs of $\cosh x$, $\sinh x$ and their inverses.}{fig:hfinverse3}{\begin{tikzpicture}
\begin{axis}[width=.4\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
%			ytick={-.002,.002,.004},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-.9,ymax=10.9,%
			xmin=-.9,xmax=10.9,%
			scaled ticks=false
]

\addplot [{\colorone},thick,smooth] coordinates {(0,1.)(0.1,1.01) (0.2,1.02) (0.3,1.05) (0.4,1.08) (0.5,1.13) (0.6,1.19)(0.7,1.26) (0.8,1.34) (0.9,1.43) (1.,1.54) (1.1,1.67) (1.2,1.81) (1.3,1.97) (1.4,2.15) (1.5,2.35) (1.6,2.58) (1.7,2.83) (1.8,3.11)(1.9,3.42) (2.,3.76) (2.1,4.14) (2.2,4.57) (2.3,5.04) (2.4,5.56)
(2.5,6.13) (2.6,6.77) (2.7,7.47) (2.8,8.25) (2.9,9.11) (3.,10.1)};

\draw (axis cs:8,4) node {\scriptsize $y=\cosh^{-1} x$};
\draw (axis cs:5.6,10) node {\scriptsize $y=\cosh x$};

\addplot [{\colortwo},smooth,thick] coordinates {(1.,0)(1.1,0.444)(1.2,0.622)(1.3,0.756)(1.4,0.867)(1.5,0.962)(1.6,1.05)(1.7,1.12)(1.8,1.19)(1.9,1.26)(2.,1.32)(2.1,1.37)(2.2,1.43)(2.3,1.48)(2.4,1.52)(2.5,1.57)(2.6,1.61)(2.7,1.65)(2.8,1.69)(2.9,1.73)(3.,1.76)(3.1,1.8)(3.2,1.83)(3.3,1.86)(3.4,1.89)(3.5,1.92)(3.6,1.95)(3.7,1.98)(3.8,2.01)(3.9,2.04)(4.,2.06)(4.1,2.09)(4.2,2.11)(4.3,2.14)(4.4,2.16)(4.5,2.18)(4.6,2.21)(4.7,2.23)(4.8,2.25)(4.9,2.27)(5.,2.29)(5.1,2.31)(5.2,2.33)(5.3,2.35)(5.4,2.37)(5.5,2.39)(5.6,2.41)(5.7,2.43)(5.8,2.44)(5.9,2.46)(6.,2.48)(6.1,2.49)(6.2,2.51)(6.3,2.53)(6.4,2.54)(6.5,2.56)(6.6,2.57)(6.7,2.59)(6.8,2.6)(6.9,2.62)(7.,2.63)(7.1,2.65)(7.2,2.66)(7.3,2.68)(7.4,2.69)(7.5,2.7)(7.6,2.72)(7.7,2.73)(7.8,2.74)(7.9,2.76)(8.,2.77)(8.1,2.78)(8.2,2.79)(8.3,2.81)(8.4,2.82)(8.5,2.83)(8.6,2.84)(8.7,2.85)(8.8,2.86)(8.9,2.88)(9.,2.89)(9.1,2.9)(9.2,2.91)(9.3,2.92)(9.4,2.93)(9.5,2.94)(9.6,2.95)(9.7,2.96)(9.8,2.97)(9.9,2.98)(10.,2.99)};

\addplot [dashed,domain=-.5:10] {x};

%\draw (axis cs:2.4,-0.002) node {\scriptsize $f(x)$};
\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture} \hspace*{10pt}\begin{tikzpicture}
\begin{axis}[width=.4\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
%			ytick={-.002,.002,.004},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-10.9,ymax=10.9,%
			xmin=-10.9,xmax=10.9,%
			scaled ticks=false
]

\addplot [{\colorone},thick,smooth] coordinates {(-3.,-10.) (-2.9,-9.06) (-2.8,-8.19) (-2.7,-7.41) (-2.6,-6.69)(-2.5,-6.05) (-2.4,-5.47) (-2.3,-4.94) (-2.2,-4.46) (-2.1,-4.02)(-2.,-3.63) (-1.9,-3.27) (-1.8,-2.94) (-1.7,-2.65) (-1.6,-2.38)(-1.5,-2.13) (-1.4,-1.9) (-1.3,-1.7) (-1.2,-1.51) (-1.1,-1.34)(-1.,-1.18) (-0.9,-1.03) (-0.8,-0.888) (-0.7,-0.759) (-0.6,-0.637)(-0.5,-0.521) (-0.4,-0.411) (-0.3,-0.305) (-0.2,-0.201) (-0.1,-0.1)(0,0) (0.1,0.1) (0.2,0.201) (0.3,0.305) (0.4,0.411) (0.5,0.521)(0.6,0.637) (0.7,0.759) (0.8,0.888) (0.9,1.03) (1.,1.18) (1.1,1.34)(1.2,1.51) (1.3,1.7) (1.4,1.9) (1.5,2.13) (1.6,2.38) (1.7,2.65)(1.8,2.94) (1.9,3.27) (2.,3.63) (2.1,4.02) (2.2,4.46) (2.3,4.94)(2.4,5.47) (2.5,6.05) (2.6,6.69) (2.7,7.41) (2.8,8.19) (2.9,9.06)(3.,10.) };

\draw (axis cs:-6,7) node {\scriptsize $y=\sinh x$};
\draw (axis cs:6,-5) node {\scriptsize $y=\sinh^{-1} x$};
\draw[->,>=stealth] (axis cs:-1,7) -- (axis cs:2,7);
\draw[->,>=stealth] (axis cs:7,-3) -- (axis cs:7,2);

\addplot [{\colortwo},smooth,thick] coordinates {(-10.,-3.)(-9.6,-2.96)(-9.2,-2.92)(-8.8,-2.87)(-8.4,-2.82)(-8.,-2.78)(-7.6,-2.73)(-7.2,-2.67)(-6.8,-2.62)(-6.4,-2.56)(-6.,-2.49)(-5.6,-2.42)(-5.2,-2.35)(-4.8,-2.27)(-4.4,-2.19)(-4.,-2.09)(-3.6,-1.99)(-3.2,-1.88)(-2.8,-1.75)(-2.4,-1.61)(-2.,-1.44)(-1.6,-1.25)(-1.2,-1.02)(-0.8,-0.733)(-0.4,-0.39)(0,0)(0.4,0.39)(0.8,0.733)(1.2,1.02)(1.6,1.25)(2.,1.44)(2.4,1.61)(2.8,1.75)(3.2,1.88)(3.6,1.99)(4.,2.09)(4.4,2.19)(4.8,2.27)(5.2,2.35)(5.6,2.42)(6.,2.49)(6.4,2.56)(6.8,2.62)(7.2,2.67)(7.6,2.73)(8.,2.78)(8.4,2.82)(8.8,2.87)(9.2,2.92)(9.6,2.96)(10.,3.)};

\addplot [dashed,domain=-10:10] {x};

%\draw (axis cs:2.4,-0.002) node {\scriptsize $f(x)$};
\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}}

\mtable{.3}{Graphs of $\tanh^{-1} x$, $\coth^{-1} x$, $\sech^{-1} x$ and $\csch^{-1} x$.}{fig:hfinverse4}{\begin{tikzpicture}
\begin{axis}[width=.4\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
			xtick={-2,2},%
%			xticklabels={$-1\qquad $,$\quad 1$},
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
			ytick={-2,2},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-3.2,ymax=3.2,%
			xmin=-3.2,xmax=3.2,%
			scaled ticks=false
]

%\addplot [{\colorone},thick,smooth] coordinates {(-3.,-0.995)(-2.9,-0.994)(-2.8,-0.993)(-2.7,-0.991)(-2.6,-0.989)(-2.5,-0.987)(-2.4,-0.984)(-2.3,-0.98)(-2.2,-0.976)(-2.1,-0.97)(-2.,-0.964)(-1.9,-0.956)(-1.8,-0.947)(-1.7,-0.935)(-1.6,-0.922)(-1.5,-0.905)(-1.4,-0.885)(-1.3,-0.862)(-1.2,-0.834)(-1.1,-0.8)(-1.,-0.762)(-0.9,-0.716)(-0.8,-0.664)(-0.7,-0.604)(-0.6,-0.537)(-0.5,-0.462)(-0.4,-0.38)(-0.3,-0.291)(-0.2,-0.197)(-0.1,-0.0997)(0,0)(0.1,0.0997)(0.2,0.197)(0.3,0.291)(0.4,0.38)(0.5,0.462)(0.6,0.537)(0.7,0.604)(0.8,0.664)(0.9,0.716)(1.,0.762)(1.1,0.8)(1.2,0.834)(1.3,0.862)(1.4,0.885)(1.5,0.905)(1.6,0.922)(1.7,0.935)(1.8,0.947)(1.9,0.956)(2.,0.964)(2.1,0.97)(2.2,0.976)(2.3,0.98)(2.4,0.984)(2.5,0.987)(2.6,0.989)(2.7,0.991)(2.8,0.993)(2.9,0.994)(3.,0.995)};

\addplot [{\colortwo},thick,smooth] coordinates {(1.005,2.997)(1.01,2.65)(1.02,2.31)(1.03,2.11)(1.04,1.97)(1.05,1.86)(1.06,1.77)(1.07,1.69)(1.08,1.63)(1.09,1.57)(1.1,1.52)(1.2,1.2)(1.3,1.02)(1.4,0.896)(1.5,0.805)(1.6,0.733)(1.7,0.675)(1.8,0.626)(1.9,0.585)(2.,0.549)(2.1,0.518)(2.2,0.49)(2.3,0.466)(2.4,0.444)(2.5,0.424)(2.6,0.405)(2.7,0.389)(2.8,0.374)(2.9,0.36)(3.,0.347)};

\addplot [{\colortwo},thick,smooth] coordinates {(-3.,-0.347)(-2.9,-0.36)(-2.8,-0.374)(-2.7,-0.389)(-2.6,-0.405)(-2.5,-0.424)(-2.4,-0.444)(-2.3,-0.466)(-2.2,-0.49)(-2.1,-0.518)(-2.,-0.549)(-1.9,-0.585)(-1.8,-0.626)(-1.7,-0.675)(-1.6,-0.733)(-1.5,-0.805)(-1.4,-0.896)(-1.3,-1.02)(-1.2,-1.2)(-1.1,-1.52)(-1.09,-1.57)(-1.08,-1.63)(-1.07,-1.69)(-1.06,-1.77)(-1.05,-1.86)(-1.04,-1.97)(-1.03,-2.11)(-1.02,-2.31)(-1.01,-2.65)(-1.005,-2.997)};

\draw (axis cs:2.2,1.5) node {\tiny $y=\coth^{-1} x$};
\draw (axis cs:2.2,-1.5) node {\tiny $y=\tanh^{-1} x$};
\draw [->,>=stealth] (axis cs:1.8,-1.3) -- (axis cs:.2,-.2);
%\draw [->,>=stealth] (axis cs:-.1,2.5) -- (axis cs:.7,2.5);
\draw [loosely dashed] (axis cs:-1,-3)--(axis cs:-1,3);
\draw [loosely dashed] (axis cs:1,-3)--(axis cs:1,3);

\addplot [{\colorone},smooth,thick] coordinates {(-0.999,-3.8)(-0.99,-2.65)(-0.945,-1.78)(-0.9,-1.47)(-0.855,-1.27)(-0.81,-1.13)(-0.765,-1.01)(-0.72,-0.908)(-0.675,-0.82)(-0.63,-0.741)(-0.585,-0.67)(-0.54,-0.604)(-0.495,-0.543)(-0.45,-0.485)(-0.405,-0.43)(-0.36,-0.377)(-0.315,-0.326)(-0.27,-0.277)(-0.225,-0.229)(-0.18,-0.182)(-0.135,-0.136)(-0.09,-0.0902)(-0.045,-0.045)(0,0)(0.045,0.045)(0.09,0.0902)(0.135,0.136)(0.18,0.182)(0.225,0.229)(0.27,0.277)(0.315,0.326)(0.36,0.377)(0.405,0.43)(0.45,0.485)(0.495,0.543)(0.54,0.604)(0.585,0.67)(0.63,0.741)(0.675,0.82)(0.72,0.908)(0.765,1.01)(0.81,1.13)(0.855,1.27)(0.9,1.47)(0.945,1.78)(0.99,2.65)(0.994,2.95)(0.999,3.8)};

\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}\hspace*{10 pt}\begin{tikzpicture}
\begin{axis}[width=.4\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
			ytick={-3,-2,-1,1,2,3},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-3.5,ymax=3.5,%
			xmin=-3.5,xmax=3.5,%
			scaled ticks=false
]

\draw (axis cs:1.5,-1.5) node {\tiny $y=\sech^{-1} x$};
\draw (axis cs:-2.2,-1.6) node {\tiny $y=\csch^{-1} x$};
\draw [->,>=latex] (axis cs:1,-1.2) -- (axis cs:1,-.2);

\addplot [{\colortwo},smooth,thick] coordinates {(-3.,-0.3275)(-2.9,-0.3383)(-2.8,-0.35)(-2.7,-0.3624)(-2.6,-0.3757)(-2.5,-0.39)(-2.4,-0.4055)(-2.3,-0.4221)(-2.2,-0.4402)(-2.1,-0.4598)(-2.,-0.4812)(-1.9,-0.5046)(-1.8,-0.5303)(-1.7,-0.5587)(-1.6,-0.5901)(-1.5,-0.6251)(-1.4,-0.6643)(-1.3,-0.7085)(-1.2,-0.7585)(-1.1,-0.8156)(-1.,-0.8814)(-0.9,-0.9578)(-0.8,-1.048)(-0.7,-1.154)(-0.6,-1.284)(-0.5,-1.444)(-0.4,-1.647)(-0.3,-1.919)(-0.2,-2.312)(-0.1,-2.998)};

\addplot [{\colortwo},smooth,thick] coordinates {(0.1,2.998)(0.2,2.312)(0.3,1.919)(0.4,1.647)(0.5,1.444)(0.6,1.284)(0.7,1.154)(0.8,1.048)(0.9,0.9578)(1.,0.8814)(1.1,0.8156)(1.2,0.7585)(1.3,0.7085)(1.4,0.6643)(1.5,0.6251)(1.6,0.5901)(1.7,0.5587)(1.8,0.5303)(1.9,0.5046)(2.,0.4812)(2.1,0.4598)(2.2,0.4402)(2.3,0.4221)(2.4,0.4055)(2.5,0.39)(2.6,0.3757)(2.7,0.3624)(2.8,0.35)(2.9,0.3383)(3.,0.3275)};

\addplot [{\colorone},thick,smooth] coordinates {(0.1,2.993)(0.2,2.292)(0.3,1.874)(0.4,1.567)(0.5,1.317)
(0.55,1.205)(0.6,1.099)(0.65,0.9961)(0.7,0.8956)(0.75,0.7954)(0.8,0.6931)(0.85,0.5857)(0.9,0.4671)(0.95,0.323)(1.,0)};
\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}}

%\clearpage

\noindent\begin{minipage}{\textwidth}
%\centering
\small
\begin{tabular}{ccc}
Function & Domain & Range\\ \hline
$\cosh x$ & $[0,\infty)$ & $[1,\infty)$\\
$\sinh x$ & $(-\infty,\infty)$ & $(-\infty,\infty)$\\
$\tanh x$ & $(-\infty,\infty)$ & $(-1,1)$\\
$\sech x$ & $[0,\infty)$ & $(0,1]$ \\
$\csch x$ & $(-\infty,0) \cup (0,\infty)$ & $(-\infty,0) \cup (0,\infty)$\\
$\coth x$ & $(-\infty,0) \cup (0,\infty)$ & $(-\infty,-1) \cup (1,\infty)$
\end{tabular}
\hskip 40pt
%\end{minipage}\hskip 40pt
%\begin{minipage}{.7\textwidth}
%\centering\small
\begin{tabular}{ccc}
Function & Domain & Range\\ \hline
\rule{0pt}{10pt}$\cosh^{-1} x$ & $[1,\infty)$ & $[0,\infty)$ \\
$\sinh^{-1} x$ & $(-\infty,\infty)$ & $(-\infty,\infty)$\\
$\tanh^{-1} x$ & $(-1,1)$ & $(-\infty,\infty)$\\
$\sech^{-1} x$ & $(0,1]$ & $[0,\infty)$ \\
$\csch^{-1} x$ & $(-\infty,0) \cup (0,\infty)$ & $(-\infty,0) \cup (0,\infty)$\\
$\coth^{-1} x$ & $(-\infty,-1) \cup (1,\infty)$ & $(-\infty,0) \cup (0,\infty)$
\end{tabular}
%\captionsetup{type=figure}%
%\caption{Restricted domains and ranges of the hyperbolic functions.}\label{fig:hfinverse1}
\captionsetup{type=figure}%
\caption{Domains and ranges of the hyperbolic and inverse hyperbolic functions.}\label{fig:hfinverse2}
\end{minipage}
\enlargethispage{3\baselineskip}

\noindent
\begin{minipage}{\textwidth}
\begin{tabular}{ccc}
%\myincludegraphics[scale=0.95]{figures/fighfarccosh}
\begin{tikzpicture}
\begin{axis}[width=.45\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
%			ytick={-.002,.002,.004},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-.9,ymax=10.9,%
			xmin=-.9,xmax=10.9,%
			scaled ticks=false
]

\addplot [{\colorone},thick,smooth] coordinates {(0,1.)(0.1,1.01) (0.2,1.02) (0.3,1.05) (0.4,1.08) (0.5,1.13) (0.6,1.19)(0.7,1.26) (0.8,1.34) (0.9,1.43) (1.,1.54) (1.1,1.67) (1.2,1.81) (1.3,1.97) (1.4,2.15) (1.5,2.35) (1.6,2.58) (1.7,2.83) (1.8,3.11)(1.9,3.42) (2.,3.76) (2.1,4.14) (2.2,4.57) (2.3,5.04) (2.4,5.56)
(2.5,6.13) (2.6,6.77) (2.7,7.47) (2.8,8.25) (2.9,9.11) (3.,10.1)};

\draw (axis cs:8,4) node {\scriptsize $y=\cosh^{-1} x$};
\draw (axis cs:5.6,10) node {\scriptsize $y=\cosh x$};

\addplot [{\colortwo},smooth,thick] coordinates {(1.,0)(1.1,0.444)(1.2,0.622)(1.3,0.756)(1.4,0.867)(1.5,0.962)(1.6,1.05)(1.7,1.12)(1.8,1.19)(1.9,1.26)(2.,1.32)(2.1,1.37)(2.2,1.43)(2.3,1.48)(2.4,1.52)(2.5,1.57)(2.6,1.61)(2.7,1.65)(2.8,1.69)(2.9,1.73)(3.,1.76)(3.1,1.8)(3.2,1.83)(3.3,1.86)(3.4,1.89)(3.5,1.92)(3.6,1.95)(3.7,1.98)(3.8,2.01)(3.9,2.04)(4.,2.06)(4.1,2.09)(4.2,2.11)(4.3,2.14)(4.4,2.16)(4.5,2.18)(4.6,2.21)(4.7,2.23)(4.8,2.25)(4.9,2.27)(5.,2.29)(5.1,2.31)(5.2,2.33)(5.3,2.35)(5.4,2.37)(5.5,2.39)(5.6,2.41)(5.7,2.43)(5.8,2.44)(5.9,2.46)(6.,2.48)(6.1,2.49)(6.2,2.51)(6.3,2.53)(6.4,2.54)(6.5,2.56)(6.6,2.57)(6.7,2.59)(6.8,2.6)(6.9,2.62)(7.,2.63)(7.1,2.65)(7.2,2.66)(7.3,2.68)(7.4,2.69)(7.5,2.7)(7.6,2.72)(7.7,2.73)(7.8,2.74)(7.9,2.76)(8.,2.77)(8.1,2.78)(8.2,2.79)(8.3,2.81)(8.4,2.82)(8.5,2.83)(8.6,2.84)(8.7,2.85)(8.8,2.86)(8.9,2.88)(9.,2.89)(9.1,2.9)(9.2,2.91)(9.3,2.92)(9.4,2.93)(9.5,2.94)(9.6,2.95)(9.7,2.96)(9.8,2.97)(9.9,2.98)(10.,2.99)};

\addplot [dashed,domain=-.5:10] {x};

%\draw (axis cs:2.4,-0.002) node {\scriptsize $f(x)$};
\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture} & \ \hskip 15pt\ &
%\myincludegraphics[scale=0.95]{figures/fighfarcsinh}
\begin{tikzpicture}
\begin{axis}[width=.45\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
%			ytick={-.002,.002,.004},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-10.9,ymax=10.9,%
			xmin=-10.9,xmax=10.9,%
			scaled ticks=false
]

\addplot [{\colorone},thick,smooth] coordinates {(-3.,-10.) (-2.9,-9.06) (-2.8,-8.19) (-2.7,-7.41) (-2.6,-6.69)(-2.5,-6.05) (-2.4,-5.47) (-2.3,-4.94) (-2.2,-4.46) (-2.1,-4.02)(-2.,-3.63) (-1.9,-3.27) (-1.8,-2.94) (-1.7,-2.65) (-1.6,-2.38)(-1.5,-2.13) (-1.4,-1.9) (-1.3,-1.7) (-1.2,-1.51) (-1.1,-1.34)(-1.,-1.18) (-0.9,-1.03) (-0.8,-0.888) (-0.7,-0.759) (-0.6,-0.637)(-0.5,-0.521) (-0.4,-0.411) (-0.3,-0.305) (-0.2,-0.201) (-0.1,-0.1)(0,0) (0.1,0.1) (0.2,0.201) (0.3,0.305) (0.4,0.411) (0.5,0.521)(0.6,0.637) (0.7,0.759) (0.8,0.888) (0.9,1.03) (1.,1.18) (1.1,1.34)(1.2,1.51) (1.3,1.7) (1.4,1.9) (1.5,2.13) (1.6,2.38) (1.7,2.65)(1.8,2.94) (1.9,3.27) (2.,3.63) (2.1,4.02) (2.2,4.46) (2.3,4.94)(2.4,5.47) (2.5,6.05) (2.6,6.69) (2.7,7.41) (2.8,8.19) (2.9,9.06)(3.,10.) };

\draw (axis cs:-6,7) node {\scriptsize $y=\sinh x$};
\draw (axis cs:6,-5) node {\scriptsize $y=\sinh^{-1} x$};
\draw[->,>=stealth] (axis cs:-1,7) -- (axis cs:2,7);
\draw[->,>=stealth] (axis cs:7,-3) -- (axis cs:7,2);

\addplot [{\colortwo},smooth,thick] coordinates {(-10.,-3.)(-9.6,-2.96)(-9.2,-2.92)(-8.8,-2.87)(-8.4,-2.82)(-8.,-2.78)(-7.6,-2.73)(-7.2,-2.67)(-6.8,-2.62)(-6.4,-2.56)(-6.,-2.49)(-5.6,-2.42)(-5.2,-2.35)(-4.8,-2.27)(-4.4,-2.19)(-4.,-2.09)(-3.6,-1.99)(-3.2,-1.88)(-2.8,-1.75)(-2.4,-1.61)(-2.,-1.44)(-1.6,-1.25)(-1.2,-1.02)(-0.8,-0.733)(-0.4,-0.39)(0,0)(0.4,0.39)(0.8,0.733)(1.2,1.02)(1.6,1.25)(2.,1.44)(2.4,1.61)(2.8,1.75)(3.2,1.88)(3.6,1.99)(4.,2.09)(4.4,2.19)(4.8,2.27)(5.2,2.35)(5.6,2.42)(6.,2.49)(6.4,2.56)(6.8,2.62)(7.2,2.67)(7.6,2.73)(8.,2.78)(8.4,2.82)(8.8,2.87)(9.2,2.92)(9.6,2.96)(10.,3.)};

\addplot [dashed,domain=-10:10] {x};

%\draw (axis cs:2.4,-0.002) node {\scriptsize $f(x)$};
\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}\\[15pt]
%\myincludegraphics[scale=0.95]{figures/fighfarctanharccoth}
\begin{tikzpicture}
\begin{axis}[width=.45\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
			xtick={-2,2},%
%			xticklabels={$-1\qquad $,$\quad 1$},
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
			ytick={-2,2},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-3.2,ymax=3.2,%
			xmin=-3.2,xmax=3.2,%
			scaled ticks=false
]

%\addplot [{\colorone},thick,smooth] coordinates {(-3.,-0.995)(-2.9,-0.994)(-2.8,-0.993)(-2.7,-0.991)(-2.6,-0.989)(-2.5,-0.987)(-2.4,-0.984)(-2.3,-0.98)(-2.2,-0.976)(-2.1,-0.97)(-2.,-0.964)(-1.9,-0.956)(-1.8,-0.947)(-1.7,-0.935)(-1.6,-0.922)(-1.5,-0.905)(-1.4,-0.885)(-1.3,-0.862)(-1.2,-0.834)(-1.1,-0.8)(-1.,-0.762)(-0.9,-0.716)(-0.8,-0.664)(-0.7,-0.604)(-0.6,-0.537)(-0.5,-0.462)(-0.4,-0.38)(-0.3,-0.291)(-0.2,-0.197)(-0.1,-0.0997)(0,0)(0.1,0.0997)(0.2,0.197)(0.3,0.291)(0.4,0.38)(0.5,0.462)(0.6,0.537)(0.7,0.604)(0.8,0.664)(0.9,0.716)(1.,0.762)(1.1,0.8)(1.2,0.834)(1.3,0.862)(1.4,0.885)(1.5,0.905)(1.6,0.922)(1.7,0.935)(1.8,0.947)(1.9,0.956)(2.,0.964)(2.1,0.97)(2.2,0.976)(2.3,0.98)(2.4,0.984)(2.5,0.987)(2.6,0.989)(2.7,0.991)(2.8,0.993)(2.9,0.994)(3.,0.995)};

\addplot [{\colortwo},thick,smooth] coordinates {(1.005,2.997)(1.01,2.65)(1.02,2.31)(1.03,2.11)(1.04,1.97)(1.05,1.86)(1.06,1.77)(1.07,1.69)(1.08,1.63)(1.09,1.57)(1.1,1.52)(1.2,1.2)(1.3,1.02)(1.4,0.896)(1.5,0.805)(1.6,0.733)(1.7,0.675)(1.8,0.626)(1.9,0.585)(2.,0.549)(2.1,0.518)(2.2,0.49)(2.3,0.466)(2.4,0.444)(2.5,0.424)(2.6,0.405)(2.7,0.389)(2.8,0.374)(2.9,0.36)(3.,0.347)};

\addplot [{\colortwo},thick,smooth] coordinates {(-3.,-0.347)(-2.9,-0.36)(-2.8,-0.374)(-2.7,-0.389)(-2.6,-0.405)(-2.5,-0.424)(-2.4,-0.444)(-2.3,-0.466)(-2.2,-0.49)(-2.1,-0.518)(-2.,-0.549)(-1.9,-0.585)(-1.8,-0.626)(-1.7,-0.675)(-1.6,-0.733)(-1.5,-0.805)(-1.4,-0.896)(-1.3,-1.02)(-1.2,-1.2)(-1.1,-1.52)(-1.09,-1.57)(-1.08,-1.63)(-1.07,-1.69)(-1.06,-1.77)(-1.05,-1.86)(-1.04,-1.97)(-1.03,-2.11)(-1.02,-2.31)(-1.01,-2.65)(-1.005,-2.997)};

\draw (axis cs:2.2,1.5) node {\tiny $y=\coth^{-1} x$};
\draw (axis cs:2.2,-1.5) node {\tiny $y=\tanh^{-1} x$};
\draw [->,>=stealth] (axis cs:1.8,-1.3) -- (axis cs:.2,-.2);
%\draw [->,>=stealth] (axis cs:-.1,2.5) -- (axis cs:.7,2.5);
\draw [loosely dashed] (axis cs:-1,-3)--(axis cs:-1,3);
\draw [loosely dashed] (axis cs:1,-3)--(axis cs:1,3);

\addplot [{\colorone},smooth,thick] coordinates {(-0.999,-3.8)(-0.99,-2.65)(-0.945,-1.78)(-0.9,-1.47)(-0.855,-1.27)(-0.81,-1.13)(-0.765,-1.01)(-0.72,-0.908)(-0.675,-0.82)(-0.63,-0.741)(-0.585,-0.67)(-0.54,-0.604)(-0.495,-0.543)(-0.45,-0.485)(-0.405,-0.43)(-0.36,-0.377)(-0.315,-0.326)(-0.27,-0.277)(-0.225,-0.229)(-0.18,-0.182)(-0.135,-0.136)(-0.09,-0.0902)(-0.045,-0.045)(0,0)(0.045,0.045)(0.09,0.0902)(0.135,0.136)(0.18,0.182)(0.225,0.229)(0.27,0.277)(0.315,0.326)(0.36,0.377)(0.405,0.43)(0.45,0.485)(0.495,0.543)(0.54,0.604)(0.585,0.67)(0.63,0.741)(0.675,0.82)(0.72,0.908)(0.765,1.01)(0.81,1.13)(0.855,1.27)(0.9,1.47)(0.945,1.78)(0.99,2.65)(0.994,2.95)(0.999,3.8)};

\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}
& & %\myincludegraphics[scale=0.95]{figures/fighfarcsecharccsch}
\begin{tikzpicture}
\begin{axis}[width=.45\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
			ytick={-3,-2,-1,1,2,3},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-3.5,ymax=3.5,%
			xmin=-3.5,xmax=3.5,%
			scaled ticks=false
]

\draw (axis cs:1.5,-1.5) node {\tiny $y=\sech^{-1} x$};
\draw (axis cs:-2.2,-1.6) node {\tiny $y=\csch^{-1} x$};
\draw [->,>=latex] (axis cs:1,-1.2) -- (axis cs:1,-.2);

\addplot [{\colortwo},smooth,thick] coordinates {(-3.,-0.3275)(-2.9,-0.3383)(-2.8,-0.35)(-2.7,-0.3624)(-2.6,-0.3757)(-2.5,-0.39)(-2.4,-0.4055)(-2.3,-0.4221)(-2.2,-0.4402)(-2.1,-0.4598)(-2.,-0.4812)(-1.9,-0.5046)(-1.8,-0.5303)(-1.7,-0.5587)(-1.6,-0.5901)(-1.5,-0.6251)(-1.4,-0.6643)(-1.3,-0.7085)(-1.2,-0.7585)(-1.1,-0.8156)(-1.,-0.8814)(-0.9,-0.9578)(-0.8,-1.048)(-0.7,-1.154)(-0.6,-1.284)(-0.5,-1.444)(-0.4,-1.647)(-0.3,-1.919)(-0.2,-2.312)(-0.1,-2.998)};

\addplot [{\colortwo},smooth,thick] coordinates {(0.1,2.998)(0.2,2.312)(0.3,1.919)(0.4,1.647)(0.5,1.444)(0.6,1.284)(0.7,1.154)(0.8,1.048)(0.9,0.9578)(1.,0.8814)(1.1,0.8156)(1.2,0.7585)(1.3,0.7085)(1.4,0.6643)(1.5,0.6251)(1.6,0.5901)(1.7,0.5587)(1.8,0.5303)(1.9,0.5046)(2.,0.4812)(2.1,0.4598)(2.2,0.4402)(2.3,0.4221)(2.4,0.4055)(2.5,0.39)(2.6,0.3757)(2.7,0.3624)(2.8,0.35)(2.9,0.3383)(3.,0.3275)};

\addplot [{\colorone},thick,smooth] coordinates {(0.1,2.993)(0.2,2.292)(0.3,1.874)(0.4,1.567)(0.5,1.317)
(0.55,1.205)(0.6,1.099)(0.65,0.9961)(0.7,0.8956)(0.75,0.7954)(0.8,0.6931)(0.85,0.5857)(0.9,0.4671)(0.95,0.323)(1.,0)};
\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}
\end{tabular}
\captionsetup{type=figure}%
\caption{Graphs of the hyperbolic functions and their inverses.}\label{fig:hfinverse1}
\end{minipage}

\begin{definition}{Logarithmic definitions of Inverse Hyperbolic Functions}{idea:hyperbolic_log}
{\noindent%
\begin{minipage}[t]{.55\textwidth}
\begin{enumerate}
\item $\ds\cosh^{-1}x=\ln\big(x+\sqrt{x^2-1}\big);\ x\geq1$\index{hyperbolic function!inverse!logarithmic def.}\rule[-10pt]{0pt}{20pt}
\item $\ds\tanh^{-1}x = \frac12\ln\left(\frac{1+x}{1-x}\right);\ |x|<1$\rule[-10pt]{0pt}{20pt}
\item $\ds \sech^{-1}x = \ln\left(\frac{1+\sqrt{1-x^2}}x\right);\ 0<x\leq1$\rule[-10pt]{0pt}{20pt}
\end{enumerate}
\end{minipage}
\begin{minipage}[t]{.5\textwidth}
\begin{enumerate}\addtocounter{enumi}{3}
\item $\ds\sinh^{-1}x = \ln\big(x+\sqrt{x^2+1}\big)$\rule[-10pt]{0pt}{20pt}
\item	 $\ds\coth^{-1}x = \frac12\ln\left(\frac{x+1}{x-1}\right);\ |x|>1$\rule[-10pt]{0pt}{20pt}
\item $\ds\csch^{-1}x = \ln\left(\frac1x+\frac{\sqrt{1+x^2}}{|x|}\right);\ x\neq0$\rule[-10pt]{0pt}{20pt}
\end{enumerate}
\end{minipage}
}
\end{definition}

The following gives the derivatives and integrals relating to the inverse hyperbolic functions. Both the inverse hyperbolic and logarithmic function representations of the antiderivative are given, based on Definition \ref{def:idea:hyperbolic_log}. Again, these latter functions are often more useful than the former. Note how inverse hyperbolic functions can be used to solve integrals we used Trigonometric Substitution to solve in Section \ref{sec:trig_sub}.

%\mtable{.62}{Logarithmic definitions of the inverse hyperbolic functions.}{fig:hfinverse5}{%
%\begin{align*}
%\cosh^{-1}x&=\ln\big(x+\sqrt{x^2-1}\big);\ x\geq1\\
%\sinh^{-1}x &= \ln\big(x+\sqrt{x^2+1}\big)\\
%\tanh^{-1}x &= \frac12\ln\left(\frac{1+x}{1-x}\right);\ |x|<1\\
%\sech^{-1}x &= \ln\left(\frac{1+\sqrt{1-x^2}}x\right);\ 0<x\leq1\\
%\csch^{-1}x &= \ln\left(\frac1x+\frac{\sqrt{1+x^2}}{|x|}\right);\ x\neq0\\
%\coth^{-1}x &= \frac12\ln\left(\frac{x+1}{x-1}\right);\ |x|>1
%\end{align*}
%}

\begin{formulabox}[{Derivatives Involving Inverse Hyperbolic Functions}]
\label{idea:hyperbolic_inverse_derivatives}
{%
\begin{minipage}[t]{.45\textwidth}
\begin{enumerate}
\item $\ds\frac{d}{dx}\big(\cosh^{-1} x\big) = \frac{1}{\sqrt{x^2-1}};\ x>1$\index{derivative!inverse hyper.}\index{hyperbolic function!inverse!derivative}
\item $\ds\frac{d}{dx}\big(\sinh^{-1} x\big) = \frac{1}{\sqrt{x^2+1}}$
\item $\ds\frac{d}{dx}\big(\tanh^{-1} x\big) = \frac{1}{1-x^2};\ |x|<1$
\end{enumerate}
\end{minipage}
\begin{minipage}[t]{.55\textwidth}
\begin{enumerate}\addtocounter{enumi}{3}
\item $\ds\frac{d}{dx}\big(\sech^{-1} x\big) = \frac{-1}{x\sqrt{1-x^2}}; 0<x<1$
\item $\ds\frac{d}{dx}\big(\csch^{-1} x\big) = \frac{-1}{|x|\sqrt{1+x^2}};\ x\neq0$
\item $\ds\frac{d}{dx}\big(\coth^{-1} x\big) = \frac{1}{1-x^2};\ |x|>1$
\end{enumerate}
\end{minipage}
}
\end{formulabox}

\begin{formulabox}[{Integrals Involving Inverse Hyperbolic Functions}]
\label{idea:hyperbolic_inverse_integrals}
{%
\begin{enumerate}
\item \parbox{70pt}{$\ds\int \frac{1}{\sqrt{x^2-a^2}}\ dx$} \parbox{180pt}{$\ds=\qquad \cosh^{-1}\left(\frac xa\right)+C;\ 0<a<x$} $\ds=\ln\Big|x+\sqrt{x^2-a^2}\Big|+C$\index{integration!inverse hyper.}\index{hyperbolic function!inverse!integration}

\item \parbox{70pt}{$\ds\int \frac{1}{\sqrt{x^2+a^2}}\ dx$} \parbox{180pt}{$\ds=\qquad \sinh^{-1}\left(\frac xa\right)+C;\ a>0$} $\ds=\ln\Big|x+\sqrt{x^2+a^2}\Big|+C$

\item \parbox{70pt}{$\ds\int \frac{1}{a^2-x^2}\ dx$} \parbox{180pt}{$\ds=\qquad \left\{\begin{array}{ccc} \frac1a\tanh^{-1}\left(\frac xa\right)+C & & x^2<a^2 \\ \\
\frac1a\coth^{-1}\left(\frac xa\right)+C & & a^2<x^2 \end{array}\right.$} $\ds=\frac12\ln\left|\frac{a+x}{a-x}\right|+C$

\item \parbox{70pt}{$\ds\int \frac{1}{x\sqrt{a^2-x^2}}\ dx $} \parbox{180pt}{$\ds=\qquad -\frac1a\sech^{-1}\left(\frac xa\right)+C;\ 0<x<a$} $\ds= \frac1a \ln\left(\frac{x}{a+\sqrt{a^2-x^2}}\right)+C $

\item	\parbox{70pt}{$\ds\int \frac{1}{x\sqrt{x^2+a^2}}\ dx $} \parbox{180pt}{$\ds=\qquad -\frac1a\csch^{-1}\left|\frac xa\right| + C;\ x\neq 0,\ a>0$}$\ds= \frac1a \ln\left|\frac{x}{a+\sqrt{a^2+x^2}}\right|+C $
\end{enumerate}
%\end{minipage}
}
\end{formulabox}

We practice using the derivative and integral formulas in the following example.\\

\begin{example}{Derivatives and integrals involving inverse hyperbolic functions}{ex_hf3}
{Evaluate the following.

\noindent%
\begin{minipage}[t]{.5\textwidth}
\begin{enumerate}
\item	$\ds \frac{d}{dx}\left[\cosh^{-1}\left(\frac{3x-2}{5}\right)\right]$
\item	$\ds \int\frac{1}{x^2-1}\ dx$
\end{enumerate}
\end{minipage}
\begin{minipage}[t]{.5\textwidth}
\begin{enumerate}\addtocounter{enumi}{2}
\item	$\ds \int \frac{1}{\sqrt{9x^2+10}}\ dx$
\end{enumerate}
\end{minipage}
}
\end{example}

\begin{solution}
{\begin{enumerate}
\item	Applying the Chain Rule gives:
		$$\frac{d}{dx}\left[\cosh^{-1}\left(\frac{3x-2}5\right)\right] = \frac{1}{\sqrt{\left(\frac{3x-2}5\right)^2-1}}\cdot\frac35.$$

\item		Multiplying the numerator and denominator by $(-1)$ gives: $\ds \int \frac{1}{x^2-1}\ dx = \int \frac{-1}{1-x^2}\ dx$. The second integral can be solved with a direct application of item \#3 from Integrals Involving Inverse Hyperbolic Functions, with $a=1$. Thus
\begin{align}
\int \frac{1}{x^2-1}\ dx &= -\int \frac{1}{1-x^2}\ dx \notag \\
		&= \left\{\begin{array}{ccc} -\tanh^{-1}\left(x\right)+C & & x^2<1 \\ \\
-\coth^{-1}\left(x\right)+C & & 1<x^2 \end{array}\right. \notag\\
     &=-\frac12\ln\left|\frac{x+1}{x-1}\right|+C\notag\\
     &=\frac12\ln\left|\frac{x-1}{x+1}\right|+C.\label{eq:hf3}
     \end{align}

We should note that this exact problem was solved at the beginning of Section \ref{sec:partial_fraction}. In that example the answer was given as $\frac12\ln|x-1|-\frac12\ln|x+1|+C.$ Note that this is equivalent to the answer given in Equation \ref{eq:hf3}, as $\ln(a/b) = \ln a - \ln b$.

%The key to linking the two seemingly different answers together is Figure \ref{fig:hfinverse5}, where the logarithmic definitions of the inverse hyperbolic functions are given. Note that the definitions of $\tanh^{-1}x$ and $\coth^{-1}x$ are very similar; the conditions placed on $|x|$ ensure that the argument of $\ln$ is always positive. Thus one could say
%$$\frac12\ln\left|\frac{x+1}{x-1}\right| = \left\{\begin{array}{ccc} \tanh^{-1}x+C & & |x|<1 \\ \\
%\coth^{-1}x+C & & |x|>1 \end{array}\right..$$
%
%We reconcile the two answers by returning to Equation \ref{eq:hf3} and continuing:
%\begin{align*}
%\int \frac{1}{x^2-1}\ dx &= \int \frac{-1}{1-x^2}\ dx \\
%			&= \left\{\begin{array}{ccc} -\frac1a\tanh^{-1}\left(\frac xa\right)+C & & x^2<a^2 \\ \\
%-\frac1a\coth^{-1}\left(\frac xa\right)+C & & a^2<x^2 \end{array}\right. \\
%			&= -\frac12\ln\left|\frac{x+1}{x-1}\right|+C \\
%			&= -\frac12\ln|x+1| + \frac12\ln|x-1| +C,
%\end{align*}
%matching the answer previously obtained.

\item		This requires a substitution, then item \#2 of Integrals Involving Inverse Hyperbolic Functions can be applied.

Let $u = 3x$, hence $du = 3dx$. We have
\begin{align*}
\int \frac{1}{\sqrt{9x^2+10}}\ dx &= \frac13\int\frac{1}{\sqrt{u^2+10}}\ du. \\
		\intertext{Note $a^2=10$, hence $a = \sqrt{10}.$ Now apply the integral rule.}\\
		 &= \frac13 \sinh^{-1}\left(\frac{3x}{\sqrt{10}}\right) + C \\
		 &= \frac13 \ln \Big|3x+\sqrt{9x^2+10}\Big|+C.
\end{align*}
\end{enumerate}
\vskip-\baselineskip
}
\end{solution}

This section covers a lot of ground. New functions were introduced, along with some of their fundamental identities, their derivatives and antiderivatives, their inverses, and the derivatives and antiderivatives of these inverses.
%Four Key Ideas were presented, each including quite a bit of information.

Do not view this section as containing a source of information to be memorized, but rather as a reference for future problem solving.

%Key Idea \ref{idea:hyperbolic_inverse_integrals} contains perhaps the most useful information. Know the integration forms it helps evaluate and understand how to use the inverse hyperbolic answer and the logarithmic answer.
%
%The next section takes a brief break from demonstrating new integration techniques. It instead demonstrates a technique of evaluating limits that return indeterminate forms. This technique will be useful in Section \ref{sec:improper_integration}, where limits will arise in the evaluation of certain definite integrals.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for Section \ref{sec:hyperbolic revisited}}

\begin{enumialphparenastyle}

\begin{ex}
Verify the given identity.
\begin{enumerate}
\item {$\ds\frac{d}{dx}\left[\sech x\right] = -\sech x\tanh x$}

\item {$\ds\frac{d}{dx}\left[\coth x\right] = -\csch^2x$}
\item {$\ds \int \tanh x\ dx  = \ln (\cosh x) + C$}

\item {$\ds \int \coth x\ dx  = \ln |\sinh x| + C$}

\end{enumerate}
	\begin{sol}
		\begin{enumerate}
		\item {\hfill$\begin{aligned}[t]
				\frac{d}{dx}\left[\sech x\right] &= \frac{d}{dx}\left[\frac{2}{e^x+e^{-x}}\right]  \\
													&= \frac{-2(e^x-e^{-x})}{(e^x+e^{-x})^2} \\
													&= -\frac{2(e^x-e^{-x})}{(e^x+e^{-x})(e^x+e^{-x})} \\
													&= -\frac{2}{e^x+e^{-x}}\cdot \frac{e^x-e^{-x}}{e^x+e^{-x}}\\
													&= -\sech x\tanh x
		\end{aligned}$\hfill\null}
		\item
		{\hfill$\begin{aligned}[t]
				\frac{d}{dx}\left[\coth x\right] &= \frac{d}{dx}\left[\frac{e^x+e^{-x}}{e^x-e^{-x}}\right]  \\
													&= \frac{(e^x-e^{-x})(e^x-e^{-x})-(e^x+e^{-x})(e^x+e^{-x})}{(e^x-e^{-x})^2}\\
													&= \frac{e^{2x}+e^{-2x} - 2 - (e^{2x}+e^{-2x}+2)}{(e^x-e^{-x})^2}\\
													&= -\frac{4}{(e^x-e^{-x})^2}\\
													&=-\csch^2x
		\end{aligned}$\hfill\null}
		\item {$\ds	\int \tanh x\ dx = \int \frac{\sinh x}{\cosh x}\ dx$\\
		Let $u = \cosh x$; $du = (\sinh x) dx$\\
		\hfill$\begin{aligned}[t]
													&= \int \frac{1}{u}\ du \\
													&= \ln |u| + C \\
													&= \ln (\cosh x) + C.
		\end{aligned}$\hfill\null}
		\item {\hfill$\begin{aligned}[t]
				\int \coth x\ dx &= \int \frac{\cosh x}{\sinh x}\ dx\\
													\text{Let $u = \sinh x$; $du = (\cosh x) dx$} 	\\
													&= \int \frac{1}{u}\ du \\
													&= \ln |u| + C \\
													&= \ln |\sinh x| + C.
		\end{aligned}$\hfill\null}
		\end{enumerate}
	\end{sol}
\end{ex}

\begin{ex}
Find the derivative of the given function.
\begin{enumerate}
\item {$f(x) = \cosh 2x$}

\item {$f(x) = \tanh (x^2)$}

\item {$f(x) = \ln(\sinh x)$}

\item {$f(x) = \sinh x\cosh x$}

\item {$f(x) = x\sinh x-\cosh x$}

\item {$f(x) = \sech^{-1} (x^2)$}

\item {$f(x) = \sinh^{-1} (3x)$}

\item {$f(x) = \cosh^{-1} (2x^2)$}

\item {$f(x) = \tanh^{-1} (x+5)$}

\item {$f(x) = \tanh^{-1} (\cos x)$}

\item {$f(x) = \cosh^{-1} (\sec x)$}

\end{enumerate}
	\begin{sol}
	\begin{enumerate}
	\item {$2\sinh 2x$}
	\item {$2x\sec^2(x^2)$}
	\item {$\coth x$}
	\item {$\sinh^2x+\cosh^2x$}
	\item {$x\cosh x$}
	\item {$\frac{-2x}{(x^2)\sqrt{1-x^4}}$}
	\item {$\frac{3}{\sqrt{9x^2+1}}$}
	\item {$\frac{4x}{\sqrt{4x^4-1}}$}
	\item {$\frac{1}{1-(x+5)^2}$}
	\item {$-\csc x$}
	\item {$\sec x$}
	\end{enumerate}
	\end{sol}
\end{ex}

\begin{ex}
Find the equation of the line tangent to the function at the given $x$-value.
\begin{enumerate}
\item {$f(x) = \sinh x$ at $x=0$}

\item {$f(x) = \cosh x$ at $x=\ln 2$}

\item {$f(x) = \sech^2 x$ at $x=\ln 3$}

\item {$f(x) = \sinh^{-1} x$ at $x=0$}

\item {$f(x) = \cosh^{-1} x$ at $x=\sqrt 2$}

\end{enumerate}
	\begin{sol}
	\begin{enumerate}
	\item {$y=x$}
	\item {$y=3/4(x-\ln 2)+5/4$}
	\item {$y=-72/125(x-\ln 3)+9/25$}
	\item {$y=x$}
	\item  {$y=(x-\sqrt{2})+\cosh^{-1}(\sqrt{2}) \approx (x-1.414)+0.881$}
	\end{enumerate}
	\end{sol}
\end{ex}

\begin{ex}
Evaluate the given indefinite integral.
\begin{enumerate}
\item {$\ds \int \tanh (2x)\ dx$}

\item {$\ds \int \cosh (3x-7)\ dx$}

\item {$\ds \int \sinh x\cosh x\ dx$}

\item {$\ds \int x\cosh x\ dx$}

\item {$\ds \int x\sinh x\ dx$}

\item {$\ds \int \frac{1}{9-x^2}\ dx$}

\item {$\ds \int \frac{2x}{\sqrt{x^4-4}}\ dx$}

\item {$\ds \int \frac{\sqrt{x}}{\sqrt{1+x^3}}\ dx$}

\item {$\ds \int \frac{1}{x^4-16}\ dx$}

\item {$\ds \int \frac{1}{x^2+x}\ dx$}

\item {$\ds \int \frac{e^x}{e^{2x}+1}\ dx$}

\item {$\ds \int \sinh^{-1}x \ dx$}

\item {$\ds \int \tanh^{-1}x \ dx$}

\item {$\ds \int \sech x \ dx$ \quad(Hint: mutiply by $\frac{\cosh x}{\cosh x}$; set $u = \sinh x$.)}

\end{enumerate}
	\begin{sol}
	\begin{enumerate}
	\item {$1/2\ln (\cosh(2x))+C$}
	\item {$1/3\sinh(3x-7)+C$}
	\item {$1/2\sinh^2x+C$ or $1/2\cosh^2x+C$}
	\item {$x \sinh (x)-\cosh (x)+C$
	}
	\item {$x \cosh (x)-\sinh (x)+C$
	}
	\item {$\left\{\begin{array}{ccc} \frac13\tanh^{-1}\left(\frac x3\right)+C & & x^2<9 \\ \\
	\frac13\coth^{-1}\left(\frac x3\right)+C & & 9<x^2 \end{array}\right. = \frac12\ln |x+1| - \frac12\ln |x-1|+C$}
	\item  {$\cosh^{-1} (x^2/2) + C = \ln (x^2+\sqrt{x^4-4})+C$}
	\item {$2/3\sinh^{-1} x^{3/2} + C = 2/3\ln (x^{3/2}+\sqrt{x^3+1})+C$}
	\item {$\frac{1}{16}\tan^{-1}(x/2)+\frac{1}{32}\ln |x-2|+\frac1{32}\ln|x+2|+C$}
	\item {$\ln x- \ln|x+1|+C$}
	\item {$\tan^{-1}(e^x)+C$}
	\item {$x\sinh^{-1}x-\sqrt{x^2+1}+C$}
	\item {$x\tanh^{-1}x+1/2\ln|x^2-1|+C$}
	\item  {$\tan^{-1}(\sinh x)+C$}
	\end{enumerate}
	\end{sol}
\end{ex}

\begin{ex}
Evaluate the given definite integral.
\begin{enumerate}
\item {$\ds \int_{-1}^1 \sinh x \ dx$}

\item {$\ds \int_{-\ln 2}^{\ln 2} \cosh x \ dx$}
\item {$\ds \int_{0}^{1} \tanh^{-1} x \ dx$}
\end{enumerate}
	\begin{sol}
	\begin{enumerate}
	\item {$0$}
	\item {$3/2$}

	\item {$2$}
	\end{enumerate}
	\end{sol}
\end{ex}

\end{enumialphparenastyle}


\documentclass[12pt,letterpaper,oneside,reqno]{amsart}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{float}
\usepackage{mathrsfs}
\usepackage{colonequals}
\usepackage[font=small,labelfont=bf]{caption}
\usepackage[left=1in,right=1in,bottom=1in,top=1in]{geometry}
\usepackage[pdfpagelabels,hyperindex,colorlinks=true,linkcolor=blue,urlcolor=magenta,citecolor=green]{hyperref}
\usepackage{setspace}
\onehalfspacing
\emergencystretch=1em

\newcommand \convPower [3]{(#1 \sp{#2} \ast #1 \sp{#2}) [#3]}
\newcommand \bernoulli [2][B] {{#1}\sb{#2}}
\newcommand \multifoldSum [2][x]{{#1}\sb{1} + {#1}\sb{2} + \cdots + {#1}\sb{#2}}
\newcommand \iversonBracket [1][s]{[#1 \; \mathrm{is} \; \mathrm{even}]}
\newcommand \floor [1]{\lfloor #1 \rfloor}
\newcommand \coeffA [3][A] {{\mathbf{#1}} \sb{#2,#3}}
\newcommand \coeffH [4][H] {{\mathbf{#1}} \sb{#2,#3} (#4)}
\newcommand \polynomialX [4][X] {{\mathbf{#1}}\sb{#2,#3} (#4)}
\newcommand \polynomialP [4][P]{{\mathbf{#1}}\sp{#2} \sb{#3}(#4)}
\newcommand \polynomialL [4][L]{{\mathbf{#1}}\sb{#2}(#3,#4)}
\newcommand \polynomialT [3][T]{{\mathbf{#1}}\sp{#2} (#3)}

\newtheorem{thm}{Theorem}[section]
\newtheorem{cor}[thm]{Corollary}
\newtheorem{prop}[thm]{Proposition}
\newtheorem{lem}[thm]{Lemma}
\newtheorem{ppty}[thm]{Property}
\newtheorem{quest}[thm]{Question}
\numberwithin{equation}{section}

%--------Meta Data: Fill in your info------
\title[On the link between binomial theorem and discrete convolution]
{On the link between binomial theorem and discrete convolution}
\author[Petro Kolosov]{Petro Kolosov}
\email{kolosovp94@gmail.com}
\keywords{Binomial theorem, Convolution, Discrete convolution, Polynomials}
\urladdr{https://kolosovpetro.github.io}
\subjclass[2010]{44A35 (primary), 11C08 (secondary)}
\date{\today}
\hypersetup{
    pdftitle={On the link between binomial theorem and discrete convolution},
    pdfsubject={Discrete Mathematics, Number Theory, Combinatorics},
    pdfauthor={Petro Kolosov},
    pdfkeywords={Binomial theorem, Discrete convolution, Power function, Polynomials, Convolution,
    Multinomial theorem, Binomial coefficient, Bernoulli number, Pascal's triangle, Faulhaber's formula,
    Power sum, Worpitzky identity, Binomial expansion}
}
\begin{document}
    \begin{abstract}
        \input{sections/abstract.tex}
    \end{abstract}
    \maketitle
    \tableofcontents

    \section{Definitions, notations and conventions} \label{sec:definitions-notations-and-conventions}
    \input{sections/definitions-notations-conventions.tex}

    \section{Introduction and main results} \label{sec:introduction}
    \input{sections/introduction.tex}

    \section{Polynomial \texorpdfstring{$\polynomialP{m}{b}{x}$}{P[m,b,x]} and its properties}
    \label{sec:polynomial-p-and-its-properties}
    \input{sections/polynomial-p-and-its-properties.tex}

    \section{Polynomial \texorpdfstring{$\polynomialP{m}{b}{x}$}{P[m,b,x]} in terms of Binomial theorem}
    \label{sec:odd-binomial-expansion-as-partial-case-of-polynomial-p}
    \input{sections/odd-binomial-theorem-as-partial-case-of-p.tex}

    \section{Polynomial \texorpdfstring{$\polynomialP{m}{b}{x}$}{P[m,b,x]} in terms of Discrete convolution}
    \label{sec:relation-between-p-and-convolution-of-polynomials}
    \input{sections/relation-between-p-and-convolution-of-polynomials.tex}

    \section{Relation between Binomial theorem and Discrete convolution}
    \label{sec:relation-between-binomial-theorem-and-discrete-convolution}
    \input{sections/relation-between-binomial-theorem-and-discrete-convolution.tex}

    \section{Derivation of coefficient \texorpdfstring{$\coeffA{m}{r}$}{A[m,r]}}
    \label{sec:derivation-of-coefficients-a}
    \input{sections/derivation-of-coefficients-a.tex}

    \section{Verification of the results and examples}
    \label{sec:verification-of-the-results-and-examples}
    \input{sections/verification-of-results.tex}

    \section{Acknowledgements}
    \label{sec:acknowledgements}
    \input{sections/acknowledgements.tex}

    \section{Conclusion}
    \label{sec:conclusion}
    \input{sections/conclusion.tex}

    \bibliographystyle{alpha}
    \bibliography{OnTheBinomialTheoremAndDiscreteConvolutionReferences}
\end{document}


\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{algpseudocode}
\usepackage[margin=1.0in]{geometry}
\title{The APF Library}
\author{Dan Ibanez}
\date{}
\begin{document}
\maketitle

\section{Overview}
The Attached Parallel Field library is a C++ library that supports
the manipulation of finite element fields.
It supports a wide variety of finite element basis functions and
numerical integration rules.
Field storage is provided for multiple mesh databases, including
support for frequent migration and modification of the mesh.
Some common finite element field operations are implemented
in the core library, while large or user-specific libraries
are included, called modules.

\section{Notation}
\begin{tabular}{ll}
$f^e(\xi)$ & The value of a field $f$ in mesh element $e$ at local
coordinates $\xi$\\
$W(e)$ & The set of multipliers associated with mesh entity $e$\\
$W(\bar{e})$ & The set of multipliers affecting element $e$\\
$w_a$ & The distribution function of multiplier $a$\\
$f_a$ & The value of field $f$ at multiplier $a$\\
$L^p_t$ & The set of fields distributing $t$-order tensors
using Lagrange polynomials of order $p$\\
$S^d_t$ & The set of fields distributing $t$-order tensors
step-wise over elements of dimension $d$\\
\end{tabular}

\section{Field}
Each field represents a distribution of a tensor value over a domain.
A field is defined by a independent tensor type, distribution, and domain.

\section{Tensor}
Field values are by definition tensors, and any tensor types
required by finite element applications are potentially within
the scope of APF.
At the moment three types of tensor are supported and distinguished since
they are those most used in target applications:

\begin{center}
\begin{tabular}{lcl}
name & order & space \\ \hline
scalar & 0 & $\mathbb{R}$ \\
vector & 1 & $\mathbb{R}^3$ \\
matrix & 2 & $\mathbb{R}^3\times \mathbb{R}^3$ \\
\end{tabular}
\end{center}

\section{Distribution}
There are many kinds of distributions available just within the context of
finite element meshes.
APF's support is initially restricted to the class of weighted-sum
element-local distributions.
Specifically, APF supports fields such that
\[f^e(\xi) = \sum_{a\in W(\bar{e})} w_a(\xi)\cdot f_a\]
Where the affecting multipliers $W(\bar{e})$ are those associated with
entities on the closure of $e$, Thus:
\[W(\bar{e})=\cup_{c\in \bar{e}} W(c)\]
Each multiplier is associated with one mesh entity, and can affect
multiple elements.
While the distribution function of a multiplier depends on the element,
its reference value $f_a$ does not.

\subsection{Lagrange}
The first distributions supported by APF are first and second
order Lagrangian shape function over simplex elements.
These distributions are well described in other literature.
This document offers a notation for a field which distributes
a $t$-order tensor using $p$-order Lagrange polynomials:
\[f\in L^p_t\]

\subsection{Step-wise}
When dealing with derivative quantities of fields, it becomes
convenient to define fields with less-than-linear polynomial order.
Such fields have $C^{-1}$ continuity between elements and are constant
within an element.
The element distribution function, in this case, is the higher-dimensional
analogue of a step function.
Technically these fields fit the description for an APF distribution:
\[f^e(\xi)=f_0=\sum_{a\in W(\bar{e})} w_a(\xi)\cdot f_a\]
\[|W(\bar{e})|=1, w_0(\xi)=1\]
A Lagrange polynomial distribution over a face is also well-defined
over its edges, but step-wise distributions are valid only within
mesh elements and discontinuous at all adjacent entities of lower
dimension.
Thus the defining parameter in a step-wise distribution is the
dimension $d$ of elements.
From this the notation for a field distributing a $t$-order
tensor over elements of dimension $d$ could be written as follows:
\[f\in S^d_t\]

\section{The Coordinate Field}
Coordinates are treated
as a separate field $x^e(\xi)$, whose evaluation defines a mapping and whose
local gradient defines the Jacobian.
\[J^e(\xi) = x_{,\xi}^e(\xi) = \sum_{a\in W(\bar{e})} w_{a,\xi}(\xi) \otimes x_a\]
The coordinate field has several conditions that make it special:
\begin{enumerate}
\item Each mesh has one and only one coordinate field
\item All other fields query the coordinate field
\item Coordinates are stored implicitly in the mesh database
\end{enumerate}

The first condition requires a tight coupling between the mesh and
the coordinate field: that field must be retrievable given the mesh,
and irreplaceable over the mesh lifetime.
If the coordinate field were treated like all other fields,
the second condition would cause a problematic infinite loop of queries.
Lastly, the final condition makes implementation troublesome since
the mesh database is not purely
topological and coordinates cannot be handled entirely by the field
library.
For practical reasons, this will likely remain the case for some time.

\section{Operators}

The core of APF provides a set of general operators for the different
supported fields.
Operators which are more application-specific will be organized into
modules that make use of the core.
The following core operators are available in the first release:

\subsection{Gradient}
The gradient of a field $f$ at a point $\xi$ is a tensor
describing the partial derivatives of the field along each spatial dimension:

\[ \text{grad}(f) = \nabla \otimes f = f_{,x} \]

The formula in terms of distribution functions for computing a scalar gradient is:

\[(\nabla \otimes f)^e(\xi) = \sum_{a\in W(\bar{e})} w_{a,x}(\xi) \otimes f_a\]

Where $w_{a,x}$ is the gradient of multiplier $a$'s distribution function with respect
to global spatial coordinates.
These, in turn, are computed from the local gradients of $f$ and the Jacobian
given by the coordinate field:

\[w_{a,x} = J^{-1}w_{a,\xi}\]

\subsection{Divergence}

Given a vector field, the divergence describes how much outward
flow occurs from each point in the field:

\[\text{div}(f) = \nabla \cdot f =
\frac{\partial f_x}{\partial x} +
\frac{\partial f_y}{\partial y} +
\frac{\partial f_z}{\partial z}\]

The divergence can be computed from multipliers as follows:

\[(\nabla \cdot f)^e(\xi) = \sum_{a\in W(\bar{e}} w_{a,x}(\xi) \cdot f_a\]

\subsection{Curl}

The curl of a vector field describes the twist or spin of the field around
a particular point.
Curl is a vector along the axis of spin whose magnitude indicates the
amount of spin.

\[(\nabla \times f)^e(\xi) =
\sum_{a\in W(\bar{e})} w_{a,x}(\xi) \times f_a\]

\section{Implementation}

\subsection{Linear Algebra}

APF provides classes representing vectors and matrices.
A study of other libraries shows that many of them implemented similar classes
out of a universal need for linear algebra in scientific computing.
APF's classes aim to be good enough that any package can use them,
and are provided publicly along with the API.

\subsection{Mesh}

APF maintains an abstract mesh interface in the form of a purely virtual base class.
This can be implemented separately for each mesh database.
All reference values are associated with mesh entities as attached data.
The tensor data is packed as efficiently as possible within the constraints of the
mesh API and the requirements for adaptivity and field operations.
Modules around APF for high-level operations such as error estimation
can use the mesh interface and gain portability across mesh databases.

The APF mesh object does not have to represent the entire mesh; it could provide
a clean solution to fields defined over subsets of the mesh.
Once this extension becomes necessary, we may rename these objects
Domains...

\subsection{Mesh Element}

The APF Mesh Element object represents the coordinate
field over an element.
These objects exist to cache coordinate multiplier data in the case
of multiple queries.
A Mesh Element provides the following information:
\begin{enumerate}
\item Local to global coordinate mapping
\item Jacobian at a point
\item Differential volume (Jacobian determinant) at a point
\item Integration points given an order of accuracy
\end{enumerate}
An APF Mesh constructs a Mesh Element from a mesh entity
at the user's request.

\subsection{Field}

APF's field objects are created given an APF mesh and tensor type.
Different creation functions exist for each family of distribution functions
due to the variation in parameters.
Upon creation, a field defines itself over the entire given mesh.

\subsection{Field Element}

A Field Element represents its parent field over a particular element.
Once created, the Field Element provides the following:
\begin{enumerate}
\item Field evaluation at a point
\item Gradient evaluation at a point
\item Operator evaluation at a point (curl, div, etc.)
\end{enumerate}
Field Elements are created by a field given a Mesh Element.
This allows multiple Field Elements (representing different fields
defined over the same mesh entity) to share one Mesh Element
object for their coordinate field queries.

\section{Upcoming Considerations}

\begin{enumerate}
\item geometric classification and attributes
\end{enumerate}

\end{document}