\chapter{Generic Dataflow Analysis Framework}

This chapter summarizes the basic considerations behind the ROSE dataflow framework as well as how its API can be used to implement inter- and intra-procedural dataflow analyses. It is oriented towards potential users of the framework.

\section{Basics of DataFlowAnalysis}
\label{basics_dfa}
Dataflow analysis is a technique for determining an application’s possible states at various points in a program. It works as a fixed-point iteration over a space of possible facts about each node in the application’s Control Flow Graph (CFG). The algorithm starts with no information about each node and iterates by accumulating all the constraints on the application's state at each node until it reaches a fixed point where no additional constraints can be discovered. The designer of a given dataflow analysis must specify an abstract representation of the set of all possible application states that maintains the relevant details of the state (e.g. whether a variable has a constant value or the linear relationships between variable pairs), while igoring the rest. For example, a state abstraction for the constant propagation analysis may have three different values: the special symbol $\bot$ if the variable is uninitialized, a numeric value if this is the only value the variable may hold at a given CFG node or $\top$ if it may have more than one value. More sophisticated abstractions represent application state using polyhedral constraints or predicate logic.  Further, the designer must specify a ``transfer'' function that maps the abstract state before any application operation to the state after it. For example, if before statement $i++$ it is known that $i==n$ then after the statement it is known that $i-1==n$.  To deal with control flow the designer also specifies a meet function that conjoins the abstract states along multiple control paths. For example, if at the end of the {\scriptsize if} branch of a conditional it is known that i=5 and at the end of the {\scriptsize else} branch i\textless10, the strongest fact that is true immediately after both branches of the conditional is 5 $\le$ i \textless 10.
~\\
The set of possible abstract states must form a lattice, which is a partial order where for any pair of elements there exists a unique least upper bound. Intuitively, states that are lower in the partial order represent fewer constraints on the application state and higher states represent more constraints. The special state $\bot$ corresponds to the least state in the partial order, where the application has done nothing to constrain its state (e.g. all variables are uninitialized).
The meet function must guarantee the uniqueness of the least upper bound and the transfer function must be monotonic (if A$\le$B then transfer(A)$\le$transfer(B)). The dataflow fixed-point iteration ensures that the abstract state of every CFG node rises monotonically as it incorporates information about more possible application behaviors. When the analysis reaches a fixed point, the abstract state at each CFG node corresponds to the tightest set of constraints that are can be specified by the abstraction about the application state at that location.

\begin{figure}
\includegraphics[scale=0.7]{\TutorialExampleDirectory/dfa_cpa}
\caption{Example of a constant propagation analysis.}
\label{Tutorial:dfa_cpa}
\end{figure}

For an intuition about how dataflow analyses work, ~\ref{Tutorial:dfa_cpa} presents an example of a constant propagation analysis. The CFG is on the left and the table on the right shows the fixed-point solution of the abstract state immediately after each node. At each node the abstract application state records for each variable one of the following values: (i) $\bot$, which indicates that the variable is uninitialized, (ii) a specific constant value if the variable may only have this value at node n or (iii) $\top$ which indicates that the variable may have more than one value (i.e., is not representable as a single constant).It shows that immediately after node A it is known that i=0 and similarly after node B, i=0 and j=5. The same is true after node C since it has no side-effects and after the assignment in node D, the state changes to i=2, j=5.  When the two conditional branches meet, the abstract state is the union of the states on both branches: the strongest assertions that are true of both states. Since j has the same value and i has two different values, the abstract state after node E is i=$\top$, j=5.

\begin{figure}
\includegraphics[scale=0.7]{\TutorialExampleDirectory/dfa_ex}
\caption{Example of a dataflow analysis with abstraction of affine constraints.}
\label{Tutorial:dfa_ex}\end{figure}

~\ref{dfa_ex} presents an example with a more complex abstraction: conjunction of linear relationships between variables. At node B the dataflow analysis computes that i=0 and j=5. When this state is propagated through the loop, the analysis discovers that after the first iteration i=4 and j=5. It then computes the meet of i=0 $\wedge$ j=5 and i=4 $\wedge$ j=5, the facts along both paths. Since this abstraction represents linear relationships, the union finds the tightest linear relationships that are true of both input states. It thus infers that i=0 (mod 4), i=0 (mod 5) (i is divisible by 4 and j by 5) and that 5i=4j-5. When this state is propagated again through the body of the loop, these assertions are discovered to be the invariants of this loop and become the fixed-point solution after node C. If they were not invariants, the algorithm would iterate until invariants were found or it reached the abstract state $\top$ which means that no linear constraints are known. Further, since the conditional j\textless100 is also linear, j\textless100 is recorded in the states of the nodes inside the loop and j $\ge$ 100 is recorded at node F after the loop.

\section{ROSE Dataflow Framework}
\label{ROSE_dfa}
ROSE provides a framework for implementing dataflow analyses. It allows users to specify their dataflow analysis by implement the standard dataflow components: (i) an abstraction of the application’s state, (ii) a transfer function that specifies the effects of code on the application state and (iii) a meet operator that combines multiple possible abstract states into one. These are implemented by extending base classes provided by the framework and implementing key virtual methods that correspond to the above functionality. The framework then solves the dataflow equations using the user-provided classes and saves the results at each CFG node. This section describes the functionality provided by the framework and how it can be used to implement analyses. ~\ref{Func_DFA} summarizes the functionality provided by the framework.

\begin{table}
  \centering
  %\tbl{The Functionality of the Dataflow Interface}{
  \scriptsize
  \begin{tabular}{ | p{2cm} | p{2cm} | p{3.5cm} | p{3.5cm} |}
    \hline
    Class & Purpose & Interface & User Responsibilities  \\ \hline
    {\bf Analysis} & Implement Simple CFG passes & Classes {\bf IntraProceduralAnalysis InterProceduralAnalysis} & Extend the classes and implement their {\bf runAnalysis} and {\bf transfer} methods \\ \hline
    {\bf Intra Procedural Dataflow} & Implement intraprocedural dataflow iteration & Classes {\bf IntraFWDataflow} {\bf IntraBWDataflow} & Extend classes and implement {\bf genInitState} and {\bf transfer} methods\\ \hline
    {\bf Inter Procedural Dataflow} & Implement the inter-procedural dataflow & Classes - {\bf Context Insensitive InterProcedural Dataflow} & Execute on a given instance of {\bf IntraDataflow} \\ \hline
    {\bf Lattice} & Generic interface for abstractions of the application's state & Methods {\bf initialize copy meetUpdate operator== str} & Extend the Lattice class and implement interface methods \\ \hline
    {\bf Nodestate} & Stores dataflow information of each CFG node & Methods {\bf setLatticeAbove getLatticeAbove deleteLatticeAbove setFact getFact deleteFacts} & Call methods to access dataflow information \\ \hline
    {\bf AstInterface} & Transforms the CFG & Methods {\bf insertBeforeUsing CommaOp, insertAfterUsing CommaOp, replaceWithPattern } & Call methods \\
  \hline
  \end{tabular}
%}
  \caption{The Functionality of the Dataflow Interface}
\label{Func_DFA}
\end{table}

\subsection{Call and Control-Flow Graphs}
The ROSE dataflow analysis framework operates on top of the ROSE Call Graph (CG) and Virtual Control-Flow Graph (VCFG). The CG documents the caller/callee relationships between application functions. The VCFG connects SgNodes in the application’s AST to identify the possible execution orders between them. The VCFG is dynamic in that instead of being computing once for the entire application, it computes the outgoing and incoming edges of a given SgNode fresh every time this information is needed. This makes the VCFG very flexible because it automatically responds to changes in the AST with no need for complex adjustments to the graph.

\subsection{Analyses}
ROSE supports both inter-and intra-procedural analyses. Users implement basic, non-dataflow analyses by extending the {\scriptsize IntraProceduralAnalysis} and {\scriptsize InterProceduralAnalysis} classes. Intra analyses iterate over the CFG of each function, and inter analyses apply intra analyses to individual functions. To implement an analysis an application developer must derive a class from the {\scriptsize IntraProceduralAnalysis} and/or {\scriptsize InterProceduralAnalysis} classes and implement the {\scriptsize runAnalysis} method. Classes {\scriptsize UnstructuredPassInterAnalysis} and {\scriptsize UnstructuredPassIntraAnalysis} {Figure ~\ref{unstruct-ex} provide examples of simple analyses. {\scriptsize UnstructuredPassInterAnalysis} takes as an argument a reference to an {\scriptsize InterProceduralAnalysis} and iterates once through all functions. It applies the {\scriptsize runAnalysis} method of the intra analysis to each function. {\scriptsize UnstructuredPassIntraAnalysis} iterates once through all the CFG nodes in the given function, applying its {\scriptsize visit} method to each node.

\begin{figure}
\centering
\begin{lstlisting}
class UnstructuredPassInterAnalysis : virtual public InterProceduralAnalysis {
  UnstructuredPassInterAnalysis(IntraProceduralAnalysis& intraAnalysis)
  void runAnalysis();
};

class UnstructuredPassIntraAnalysis : virtual public IntraProceduralAnalysis {
   bool runAnalysis(const Function& func, NodeState* state);
   virtual void visit(const Function& func, const DataflowNode& n,
       NodeState& state)=0;
};
\end{lstlisting}%
\caption{Example of simple analyses}
\label{unstruct-ex}
\end{figure}

These analyses can be used to implement simple passes through the application’s CFG and serve as the foundation of the dataflow analysis framework. For example, {\scriptsize src/simpleAnalyses/saveDotAnalysis.C} and {\scriptsize src/simpleAnalyses/printAnalysisStates.C} are examples of simple one-pass analyses. {\scriptsize saveDotAnalysis} prints the application’s CFG as a DOT file and {\scriptsize printAnalysisStates} prints the dataflow states of all CFG nodes in the application, which is useful for debugging.

\subsection{Dataflow}
To implement a dataflow analysis in ROSE users must first extend the {\scriptsize Lattice} class to create an abstraction of the application’s state that will be used by the analysis. {\scriptsize Lattices} implement methods such as meet, equality, ordering and operators that allow the {\scriptsize Lattice} to be moved from one lexical scope to another (e.g. from a caller function to the callee). Users then create an intra-procedural analysis by extending the {\scriptsize IntraFWDataflow} to create a forward analysis and from {\scriptsize IntraBWDataflow} to create a backward analysis. Within this class they must implement a function that returns the default abstract state of any given node at the start of the analysis. Further, they implement a {\scriptsize transfer} function that maps the application’s abstract state from before a given CFG node to the state that results from the execution of the node’s expression or statement. Finally, users combine the intra-procedural analysis that they have developed with an inter-procedural analysis of their choice. This analysis will apply the intra-procedural analysis the user has implemented to the application’s functions and resolve the effects of function calls on the application’s abstract state, utilizing the user’s own state abstraction.

For a concrete example, consider how the classical constant-propagation analysis is implemented using ROSE. This analysis uses a simple abstraction of application state, where the abstract state of each variable may be a value the lattice shown in ~\ref{lattice_cpa}

\begin{figure}
\includegraphics[scale=0.7]{\TutorialExampleDirectory/lattice_cpa}
\caption{Each variable's lattice for constant-propagation analysis}
\label{lattice_cpa}
\end{figure}

The code in below shows a class that implements this lattice. This class derives from the {\scriptsize FiniteLattice} class because the distance between the smallest and largest value in the lattice is finite. Similar functionality is provided for infinite lattices. Its state (lines 4-15) consists of its current level in the lattice as well as its value if the level is {\scriptsize valKnown}. Since the type of {\scriptsize value} is {\scriptsize long}, this abstraction can only represent integral constants. Further, the class has a special {\scriptsize uninitialized} level that means that the object has not yet been used as part of a dataflow analysis. This class implements methods {\scriptsize meetUpdate} (lines 42-65) and the equality operator (lines 68-74) to provide the basic semantics of a lattice. {\scriptsize meetUpdate} computes least upper bound of the constraints in this lattice object and another one, storing the results in this object.If both lattices have the same state, the meet is equal to this state and if they have different states, the meet is $\top$ since the variable represented by the lattice may be set to multiple values on different execution paths. The equality operator determines whether two lattice objects have the same information content. Further, the class implements utility methods that help the dataflow framework manipulate it. The {\scriptsize initialize} method (lines 24-27) ensures the object is ready to be used. Further, two {\scriptsize copy} methods (lines 30-38) make it easy to clone lattice objects. Finally, an {\scriptsize str} method (lines 82-88) simplifies analysis debugging by printing the abstract states at CFG nodes.

%\begin{latexonly}
%   \lstinputlisting{\TutorialExampleDirectory/codesnip_CPA_Lattice.C}
%\end{latexonly}

\begin{frame}
\centering
\begin{lstlisting}
class constPropLat : public FiniteLattice
{
  // The different levels of this object’s lattice
  typedef enum {
    uninitialized=0, // This object is uninitialized
    bottom=1, // No constrains on this object’s value are known
    valKnown=2, // The value of the variable is known (one assignment seen)
    top=3 // This variable may have more than one value
  } latticeLevels;

  // The level of this object within its lattice
  latticeLevels level;

  // The value of the variable (if level == valKnown)
  long value;

  nodeConstLattice() { level=uninitialized; }

  nodeConstLattice(const nodeConstLattice& that) :
                              value(that.value), level(that.level) {}

  // Initializes this Lattice to its default state,
  // if it is not already initialized
  void initialize() {
    if(level == uninitialized)
      level=bottom;
  }

  // Returns a copy of this lattice
  Lattice* copy() const { return new nodeConstLattice(*this); }

  // Overwrites the state of this Lattice with that of that Lattice
  void copy(Lattice* that) {
    nodeConstLattice* that = dynamic_cast<nodeConstLattice*>(that_arg);

    value = that->value;
    level = that->level;
  }

  // Computes the meet of this and that and saves the result in this
  // returns true if this causes this to change and false otherwise
  bool meetUpdate(Lattice* that) {
    // Record this object’s original state to enable change detection
    unsigned long  oldValue = value;
    latticeLevels oldLevel = level;

    // Cast that into a nodeConstLattice and abort if this is not possible
    nodeConstLattice* that = dynamic_cast<nodeConstLattice*>(that_arg);
    ROSE_ASSERT(that);

    // If that is at a higher lattice level than this, the variable must have
    // multiple possible value on different execution paths
    if(that->level > level) level = top;
    // If both are at the same level
    else if(that->level == level) {
      // If lattices correspond to different values of the variable
      if(level == valKnown && value != that->value)
        level = top; // The union of both these facts is top
    }
    // Otherwise, this lattice doesn’t change

    // Return whether this object was modified
    return (oldValID != valID) ||
           (oldLevel != level);
  }

  // Equality Operator
  bool operator==(Lattice* that_arg) {
    // Cast that into a nodeConstLattice and abort if this is not possible
    nodeConstLattice* that = dynamic_cast<nodeConstLattice*>(that_arg);
    ROSE_ASSERT(that);

    return level==that->level && (level!=valKnown || value==that->value);
  }

  // Returns a string representation of this object (this function is
  // required to simplify debugging)
  string str(string indent="") { … }

  // Sets the state of this lattice to the given value. Returns true if this
  // causes the lattice's state to change, false otherwise
  bool set(long value)
  {
    bool modified = this->level != valKnown || this->value != value;
    this-> value = value;
    level = valKnown;
    return modified;
  }
};
\end{lstlisting}%
\label{unstruct-ex}
\end{frame}

The second step in implementing constant propagation is to provide a class that implements the dataflow analysis itself. This is done by extending the {\scriptsize IntraFWDataflow} class, which implements forward intra-procedural analyses and implementing the {\scriptsize genInitState} and {\scriptsize transfer} methods, described below.

\begin{frame}
\centering
\begin{lstlisting}
class constPropAnalysis : public IntraFWDataflow
{
  constPropAnalysis (): IntraFWDataflow() { }

  // Generates the initial lattice state for the given dataflow node, in the
  // given function, with the given NodeState
  void genInitState(const Function& func, const DataflowNode& n,
                    const NodeState& state, vector<Lattice*>& initLattices,
                    vector<NodeFact*>& initFacts);

  // The transfer function that is applied to every node in the CFG
  // n - The dataflow node that is being processed
  // state - The NodeState object that describes the state of the node, as
  //         established by earlier analysis passes
  // dfInfo - The Lattices that this transfer function operates on. The
  //          function takes these lattices as input and overwrites them with
  //          the result of the transfer.
  // Returns true if any of the input lattices changed as a result of the
  //    transfer function and false otherwise.
  bool transfer(const Function& func, const DataflowNode& n,
              NodeState& state, const vector<Lattice*>& dfInfo);
}
\end{lstlisting}%
\end{frame}

The {\scriptsize constPropAnalysis} implementation of method {\scriptsize genInitState} creates a lattice (lines 5-7) that maintains the initial abstract state of the application at CFG node {\scriptsize n}. This lattice is an instance of the utility class {\scriptsize FiniteVarsExprsProductLattice}, which creates one copy of {\scriptsize constPropLat} for each variable that is live at node {\scriptsize n}. Since it is a product of lattices, this class is also a lattice with well-defined meet and equality operators based on the operators of its constituent lattices. The dataflow framework provides an identical class for infinite lattices as well as a generic ProductLattice class for arbitrary products of lattices. The function then adds (line 4) the lattice to vector initLattices, which is read by the dataflow framework. This function can also specify one or more facts that the framework will maintain at each node. These facts are not subject to dataflow iteration and can be used to maintain information that is useful independently of the current dataflow state.

\begin{frame}
\centering
\begin{lstlisting}
void constPropAnalysis::genInitState(const Function& func,
              const DataflowNode& n, const NodeState& state,
              vector<Lattice*>& initLattices, vector<NodeFact*>& initFacts) {
  initLattices.push_back(
          new FiniteVarsExprsProductLattice(true, false,
                                            new constPropLat(),
                                            NULL, func, n, state));
}
\end{lstlisting}%
\end{frame}

The {\scriptsize transfer} method maps the abstract state before the CFG node n to the state that results from its execution. It begins by accessing the application’s abstract state above node n from the {\scriptsize dfInfo} argument (lines 6-7) . This is the vector of lattices created by {\scriptsize genInitState} for node {\scriptsize n}. It can also be obtained from the {\scriptsize state} object, which maintains the state of the lattices both below and above each node, as well as the facts at each node. The function then initializes all the {\scriptsize constPropLats} in the product lattice (10-13) and advances to analyze the effects of the current node on the abstract state.

Lines 16-127 show how the transfer function operates on different types of {\scriptsize SgNodes}. This code leverages a key feature of how ROSE represents the application’s structure. Since ROSE focuses on source-to-source transformations that minimize the changes in the application’s source code, all analyses must work on the original AST and cannot perform large normalization passes such as transforming the application into SSA form. Since it is difficult to implement complex analyses on top of the AST, we have developed an ``on-demand'' normalization that significantly simplifies the analysis development without changing the AST. Working with AST is difficult because AST sub-trees that describe the structure of expressions are complex and difficult to parse (e.g. consider analyzing all the side-effects of a=b=foo(c=d)). As such, our framework treats every SgExpression that does not correspond to an actual memory object as if it produces a temporary object that is read by its parent {\scriptsize SgExpression}. For example, in the {\scriptsize SgExpression} a=(b=c*5+d), {\scriptsize SgIntVal} 5 produces a temporary variable that is consumed by {\scriptsize SgMultiplyOp c*5. SgVarRefExp c} produces a real application variable, which is also consumed by the {\scriptsize SgMultiplyOp}. The SgMultiplyOp in turn produces a temporary variable that is consumed by {\scriptsize SgAddOp c*5+d}, which produces a temporary variable that is consumed by {\scriptsize SgAssignOp b=c*5+d}, the result of which is consumed by {\scriptsize SgAssignOp a=(b=c*5+d)}. The use of these temporary variables makes it possible for user analyses to focus on just the effects of individual AST nodes without having to analyze sub-trees of the AST. Section~\ref{transfer_analyses} discusses how this on-demand normalization in maintained when updating to the AST.

The effects of integral constants (e.g. {\scriptsize SgIntVal} or {\scriptsize SgLongLongIntVal}) are transferred on lines 16-31. On line 20 the analysis calls function SgExpr2Var to convert the {\scriptsize SgExpression} into a {\scriptsize varID}, which is an abstract representation of the memory object (either a real or temporary variable) denoted by the {\scriptsize SgExpression}. On line 21 it queries the {\scriptsize FiniteVarsExprsProductLattice} prod with this {\scriptsize varID} to get the {\scriptsize constPropLat} associated with this memory object. If this variable is live (a non-NULL lattice is returned), on lines 26-28 it sets the lattice object to be at level {\scriptsize valKnown} and sets the value to be equal to the constant represented by the {\scriptsize SgExpression}.

 The same logic is used for non-integral constants on lines 33-40. However, since our abstraction cannot represent such constants, their lattices are set to $\top$. Lines 44-60 manage assignments and the lattices of the left-hand-side expression and the assignment {\scriptsize SgAssignOp} itself are set to be equal to the lattice of the right-hand-side expression. The code for variable declaration (lines 63-81) and initialization (lines 84-98) are similar in that the lattice of the right-hand-side is copied to the lattice of the initialized variable. Finally, lines 101-127 focus on arithmetic operations. If the lattices of the left- and right-hand-side expressions are both at levels {\scriptsize valKnown},the operation is performed immediately by the analysis on their statically known values and the result is stored in the lattices of the left-hand-side expression and the {\scriptsize SgExpression} itself. Finally, on line 129 the function returns the {\scriptsize modified} variable, which keeps track of whether the state of the downstream lattices has changed. Since these lattices are inputs to other {\scriptsize SgExpressions}, this informs the dataflow framework whether it needs to analyze how these lattices are transferred by those expressions.

\begin{frame}
\centering
\begin{lstlisting}
bool constPropAnalysis::transfer(const Function& func,
                       const DataflowNode& n,
                       NodeState& state, const vector<Lattice*>& dfInfo) {
  bool modified=false;
  // Get the lattice object
  FiniteVarsExprsProductLattice* prodLat =
            dynamic_cast<FiniteVarsExprsProductLattice*>(*(dfInfo.begin()));

  // Make sure that all the non-constant Lattices are initialized
  const vector<Lattice*>& lattices = prodLat->getLattices();
  for(vector<Lattice*>::const_iterator it = lattices.begin();
      it!=lattices.end(); it++)
    (dynamic_cast<nodeConstLattice*>(*it))->initialize();

  // Integral Numeric Constants
  if(isSgLongLongIntVal(n.getNode())  ||
    // Other types of integral constants
    ...) {
    // Memory object and lattice of the expression’s result
    varID res = SgExpr2Var(isSgExpression(n.getNode()));
    constPropLat* resLat = dynamic_cast<constPropLat*>(
                                                prodLat->getVarLattice(res));

    // If the result expression is live
    if(resLat) {
      if(isSgLongLongIntVal(n.getNode()))
        modified = resLat->set(isSgLongLongIntVal(n.getNode())->get_value())
                   || modified;
      // Same for other types of integral constants
      ...
    }
  // Non-integral Constants
  } else if(isSgValueExp(n.getNode())) {
    // Memory object and lattice of the expression’s result
    varID res = SgExpr2Var(isSgExpression(n.getNode()));
    constPropLat* resLat = dynamic_cast<constPropLat*>(
                                                prodLat->getVarLattice(res));
    // If the result expression is live, set it to top since we only work
    // with integral constants
    if(resLat) modified = resLat->setTop() || modified;

  // Plain assignment: lhs = rhs
  } else if(isSgAssignOp(n.getNode())) {
    // Memory objects denoted by the expression’s left- and right-hand
    // sides as well as the SgAssignOp itself
    varID lhs = SgExpr2Var(isSgAssignOp(n.getNode())->get_lhs_operand());
    varID rhs = SgExpr2Var(isSgAssignOp(n.getNode())->get_rhs_operand());
    varID res = SgExpr2Var(isSgExpression(n.getNode()));

    // The lattices associated the three memory objects
    constPropLat* resLat =
            dynamic_cast<constPropLat*>(prodLat->getVarLattice(res));
    constPropLat* lhsLat =
            dynamic_cast<constPropLat*>(prodLat->getVarLattice(lhs));
    constPropLat* rhsLat =
            dynamic_cast<constPropLat*>(prodLat->getVarLattice(rhs));

    // If the lhs and/or the SgAssignOp are live, copy lattice from the rhs
    if(lhsLat){ lhsLat->copy(rhsLat); modified = true; }
    if(resLat){ resLat->copy(rhsLat); modified = true; }

  // Variable Declaration
  } else if(isSgInitializedName(n.getNode())) {
    varID var(isSgInitializedName(n.getNode()));
    constPropLat* varLat = dynamic_cast<constPropLat*>(
                                               prodLat->getVarLattice(var));

    // If this variable is live
    if(varLat) {
      // If there was no initializer, initialize its lattice to Bottom
      if(initName->get_initializer()==NULL)
        modified = varLat->setBot() || modified;
      // Otherwise, copy the lattice of the initializer to the variable
      else {
        varID init = SgExpr2Var(
                        isSgInitializedName(n.getNode())->get_initializer());
        ConstPropLat* initLat = dynamic_cast<ConstPropLat*>(
                                               prodLat->getVarLattice(init));
        if(initLat) { varLat->copy(initLat); modified = true; }
      }
    }

  // Initializer for a variable
  } else if(isSgAssignInitializer(n.getNode())) {
    // Memory objects of the initialized variable and the
    // initialization expression
    varID res = SgExpr2Var(isSgAssignInitializer(n.getNode()));
    varID asgn = SgExpr2Var(isSgAssignInitializer(
                                         n.getNode())->get_operand());

    // The lattices associated both memory objects
    constPropLat* resLat =
            dynamic_cast<constPropLat*>(prodLat->getVarLattice(res));
    constPropLat* asgnLat =
            dynamic_cast<constPropLat*>(prodLat->getVarLattice(asgn));

    // If the variable is live, copy lattice from the assignment
    if(resLat){ resLat->copy(asgnLat); modified = true; }

  // += Arithmetic Operation
  } else if(isSgPlusAssignOp(n.getNode())) {
    // Memory objects denoted by the expression’s left- and right-hand
    // sides as well as the SgAssignOp itself
    varID lhs = SgExpr2Var(isSgAssignOp(n.getNode())->get_lhs_operand());
    varID rhs = SgExpr2Var(isSgAssignOp(n.getNode())->get_rhs_operand());
    varID res = SgExpr2Var(isSgExpression(n.getNode()));

    // The lattices associated the three memory objects
    constPropLat* resLat =
            dynamic_cast<constPropLat*>(prodLat->getVarLattice(res));
    constPropLat* lhsLat =
            dynamic_cast<constPropLat*>(prodLat->getVarLattice(lhs));
    constPropLat* rhsLat =
            dynamic_cast<constPropLat*>(prodLat->getVarLattice(rhs));

    // If the lhs and/or the SgAssignOp are live and we know both their
    // values of the value of the rhs expression, set their lattice to be the
    // sum of the two.
    if(lhsLat && lhsLat->level==constPropLat::valKnown &&
       rhsLat->level==constPropLat::valKnown)
    { modified = lhsLat->set(lhsLat->value + rhsLat->value) || modified; }
    if(resLat && resLat->level==constPropLat::valKnown &&
       rhsLat->level==constPropLat::valKnown)
    { modified = resLat->set(resLat->value + rhsLat->value) || modified; }
  }
  // Same for other arithmetic operations
  ...

  return modified;
}
\end{lstlisting}%
\end{frame}
Once the intra-procedural analysis has been implemented, it can be executed on the application by combining it with an inter-procedural analysis. Currently two such analyses are implemented. {\scriptsize ContextInsensitiveInterProceduralDataflow} implements a basic context-insensitive analysis that propagates abstract state from callers to callees but does not differentiate between different call sites of the same function. As such, it is sensitive to inter-procedural data flows but can be imprecise because it takes into account control flows that are actually impossible, such as entering a function from one call site but returning to another. The code below provides an example of how this analysis is used to create an inter-procedural constant propagation analysis. The dataflow framework is initialized on line 6 and the application’s call graph is built on lines 9-11. The intra-procedural analysis object is created on line 17 and the context-insensitive inter-procedural analysis is created on line 20. The user passes into its constructor references to their intra-procedural analysis and the call graph. Finally, on line 23 the user applies the full inter-procedural analysis to the entire application.

\begin{frame}
\centering
\begin{lstlisting}
int main( int argc, char * argv[] )  {
  // Build the AST used by ROSE
  SgProject* project = frontend(argc,argv);

  // Initialize the ROSE dataflow framework
  initAnalysis(project);

  // Build the call graph
  CallGraphBuilder cgb(project);
  cgb.buildCallGraph();
  SgIncidenceDirectedGraph* graph = cgb.getGraph();

  // Set the debug level to print the progress of the dataflow analysis
  analysisDebugLevel = 1;

  // Create the intra-procedural constant propagation analysis
  constPropAnalysis cp(project);

  // Create the inter-procedural analysis for intra-analysis cp
  ContextInsensitiveInterProceduralDataflow inter_cp(&cp, graph);

  // Run inter-procedural constant propagation on the entire application
  inter_cp.runAnalysis();
}
\end{lstlisting}
\end{frame}

To simplify debugging the framework also provides the {\scriptsize UnstructuredPassInterDataflow} analysis, which simply applies the user’s intra-procedural analysis on each function within the application. While this produces globally incorrect results, it simplifies debugging analyses on individual functions.

\subsection{Transferring Information Between Analyses}
\label{transfer_analyses}
Since in practice users need to implement multiple analyses where one depends on the results of another, the ROSE dataflow framework maintains the results of all analyses at each CFG nodes and makes it easy for analyses to access this data. The lattices and facts of a given CFG node are stored in its associated {\scriptsize NodeState} object. The data produced by an analysis can be retrieved by using its pointer, as shown in the example below.

This code shows analysis {\scriptsize exAnalysis}, which takes in its constructor a pointer to the {\scriptsize constPropAnalysis} described above (lines 4-5). Inside its transfer function this analysis calls the {\scriptsize getLatticeBelow} method of its argument state (instance of the {\scriptsize NodeState} class) to get the lattice associated with {\scriptsize constPropAnalysis} that has index 0 (lines 11-13). It then gets the {\scriptsize constPropLat} of any variable it cares about and make analysis decisions based on what is statically known about its state.

\begin{frame}
\centering
\begin{lstlisting}
class exAnalysis {
  // Class maintains a pointer to the constant propagation analysis to make
  // it possible to access its results
  constPropAnalysis& cpAnalysis;
  exAnalysis(constPropAnalysis* cpAnalysis) : cpAnalysis (cpAnalysis) {}

  bool transfer(const Function& func, const DataflowNode& n,
                NodeState& state, const vector<Lattice*>& dfInfo) {
    // Get the Lattices computed by the constant propagation analysis for the
    // current CFG node
    FiniteVarsExprsProductLattice* prodLat =
               dynamic_cast<FiniteVarsExprsProductLattice*>(
                                      state->getLatticeBelow(cpAnalysis, 0));

    // Some application variable of interest
    varID var = ...;

    // The constPropLat of this variable
    constPropLat varCPLat = dynamic_cast<constPropLat*>(
                                                prodLat->getVarLattice(res));

    // Analyze differently depending on what is known about the
    // variable’s value
    if(varCPLat)
      if(varCPLat->level == constPropLat::bottom) {
        ...
      } else if(varCPLat->level == constPropLat::valKnown) {
        ...
      } else if(varCPLat->level == constPropLat::top) {
        ...
      }
  }
  ...
};
\end{lstlisting}
\end{frame}

The code below shows the full functionality of the {\scriptsize NodeState} class. . Lines 5-16 show the functions to set, get and delete the lattices above and below the associated CFG node.  Lines 20-26 provide the same functionality for facts. The {\scriptsize str} method on line 20 returns a string representation of the lattices and facts associated with the CFG node, which is very useful for debugging. Lines 37-50 show the object’s static methods. The {\scriptsize getNodeState} method on line 37 returns the {\scriptsize NodeState} object of a given CFG node. Since the ROSE virtual CFG can have multiple CFG nodes for the same AST node, this method requires an additional index argument to identify the node in question. Finally, method copyLattices\_aEQa and related methods (lines 39-50) copy lattice information from above a CFG node to below it and vice versa, from one node to another or from one analysis at a given node to another analysis at the same node.

\begin{frame}
\centering
\begin{lstlisting}
class NodeState
{
  // Sets the lattices above/below this node for the given analysis to the
  // given lattice vector
  void setLatticeAbove(const Analysis* analysis, vector<Lattice*>& lattices);
  void setLatticeBelow(const Analysis* analysis, vector<Lattice*>& lattices);

  // Returns the lattice latticeName generated by the given analysis from
  // above/below the node
  Lattice* getLatticeAbove(const Analysis* analysis, int latticeName) const;
  Lattice* getLatticeBelow(const Analysis* analysis, int latticeName) const;

  // Deletes all lattices above/below this node that are associated with the
  // given analysis
  void deleteLatticeAbove(const Analysis* analysis);
  void deleteLatticeBelow(const Analysis* analysis);

  // Sets the facts at this node for the given analysis to the given
  // fact vector
  void setFacts(const Analysis* analysis, const vector<NodeFact*>& newFacts);

  // Returns the given fact, owned by the given analysis
  NodeFact* getFact(const Analysis* analysis, int factName) const ;

  // Deletes all facts at this node associated with the given analysis
  void deleteFacts(const Analysis* analysis);

  // Returns a string representation of all the lattices and facts
  // associated with the CFG node
  string str(Analysis* analysis, string indent) const;

  // --- Static Methods ---
  // Returns the NodeState object associated with the given dataflow node
  // index is used when multiple NodeState objects are associated with a
  // given node
  // (ex: SgFunctionCallExp has 3 NodeStates: entry, function body, exit)
  static NodeState* getNodeState(const DataflowNode& n, int index=0);

  // Copies from's above lattices for the given analysis to to's above
  // lattices for the same analysis
  static void copyLattices_aEQa(Analysis* analysis, NodeState& to, const
                                NodeState& from);

  // Copies from's above lattices for analysisA to to's above lattices for
  // analysisB
  static void copyLattices_aEQa(Analysis* analysisA, NodeState& to,
                                Analysis* analysisB, const NodeState& from);

  // Similar methods for copying in different permutations
  ...
};
\end{lstlisting}
\end{frame}

\subsection{CFG Transformations}
ROSE makes it easy to modify the application’s AST as a result of dataflow analyses. The dataflow framework maintains an on-demand normal form, where analyses can focus on the actions of individual SgNodes and ignore how they are arranged within the AST. ROSE maintains this abstraction by providing an API that inserts new SgExpressions into the application’s CFG, making all the needed changes in the AST to make sure that the correct control flow is maintained.

\begin{figure}
\includegraphics[scale=0.7]{\TutorialExampleDirectory/transformation_cfg.pdf}
\caption{Example of Transformation on the CFG}
\label{tr_cfg}
\end{figure}

To get the intuition of this functionality consider the expression {\scriptsize foo()=(bar()+baz())}. Suppose the user has decided based on the dataflow state before the {\scriptsize SgAddOp +} that it they want to add a call to function {\scriptsize print} immediately before it. From the perspective of the CFG, this is a simple and well-defined operation, as shown at the top of Figure 4. The side-effects of the calls to {\scriptsize bar} and {\scriptsize baz} must complete before the call to print and the side-effects of print must complete before the execution of the + operation. The call to {\scriptsize foo} is not well-ordered relative print or the other operations by the structure of the CFG.

Unfortunately, it is difficult to implement these semantics in the context of the AST because (i) there is no way to add a function call to an {\scriptsize SgAddOp} and (ii) because in C++ the sequence points required by the above semantics (some side-effects much complete before others) are provided by a few specific constructs such as statement boundaries and the comma operator. As such, the transformation requires the complex set of AST changes shown in the Figure 5. We must create temporary variables to hold the results of the calls to bar and baz. We then transform the original {\scriptsize SgAddOp} into a longer {\scriptsize SgCommaOpExp}, where we first call {\scriptsize bar} and {\scriptsize baz}, saving their results into the temporary variables, then call print and finally perform the addition. The result of the addition is the result of the entire comma expression, so this transformation correctly enforces the semantics of the desired transformation.

\begin{figure}
\includegraphics[scale=0.7]{\TutorialExampleDirectory/transformation_srcode.pdf}
\caption{Example of the Transformation on the Source Code}
\label{tr_srcode}
\end{figure}

ROSE provides the three functions to make it easy to insert expressions into the CFG. Functions {\scriptsize insertBeforeUsingCommaOp} and {\scriptsize insertAfterUsingCommaOp} insert {\scriptsize SgExpressions} before or after existing {\scriptsize SgExpressions} using a generalization of the transformation described in Figure ~\ref{tr_srcode}.

\begin{frame}
\centering
\begin{lstlisting}
// Insert an expression (new_exp) before another expression (anchor_exp) has
// possible side effects, without changing the original semantics. This is
// achieved by using a comma operator: (new_exp, anchor_exp). The comma
// operator is returned.
SgCommaOpExp *insertBeforeUsingCommaOp(SgExpression* new_exp,
                                       SgExpression* anchor_exp);
// Insert an expression (new_exp) after another expression (anchor_exp) has
// possible side effects, without changing the original semantics. This is
// done by using two comma operators:
//     type T1; ... ((T1 = anchor_exp, new_exp),T1) )... ,
// where T1 is a temp variable saving the possible side effect of anchor_exp.
// The top level comma op exp is returned. The reference to T1 in T1 =
// anchor_exp is saved in temp_ref.
SgCommaOpExp *insertAfterUsingCommaOp(SgExpression* new_exp,
                    SgExpression* anchor_exp, SgStatement** temp_decl = NULL,
                    SgVarRefExp** temp_ref = NULL);
\end{lstlisting}%
\label{insert-ex}
\end{frame}

Function replaceWithPattern (Figure~\ref{rep}) replaces one SgExpression with another. However, since the original expression may still be valuable, it allows the original expression to be included at one or more locations inside the new expression that contain nodes of type SgVariantExpression.

\begin{frame}
\centering
\begin{lstlisting}
// Replace an anchor node with a specified pattern subtree with optional
// SgVariantExpression. All SgVariantExpression in the pattern will be
// replaced with copies of the anchor node.
SgNode* replaceWithPattern (SgNode * anchor, SgNode* new_pattern);
\end{lstlisting}%
\label{rep}
\end{frame}

An example of this transformation is shown Figure~\ref{cfg-srcode}, where the original code is the same as in the example above and the {\scriptsize new\_pattern} expression is a single {\scriptsize SgMultOp} where the arguments are both {\scriptsize SgVariantExpressions}. The result of the transformation is that the original {\scriptsize SgAddOp} is replaced with a multiplication the arguments of which are copies of the {\scriptsize SgAddOp: (bar()+baz())*(bar()+baz())}.

\begin{figure}
\includegraphics[scale=0.7]{\TutorialExampleDirectory/code_replacement_transformation.pdf}
\caption{Code Replacement Transformation}
\label{cfg-srcode}
\end{figure}


\hypertarget{dcel---doubly-connected-edge-list}{%
\section{DCEL - Doubly Connected Edge
List}\label{dcel---doubly-connected-edge-list}}

\hypertarget{introduction}{%
\subsection{Introduction}\label{introduction}}

Doubly connected edge list (DCEL), which is also known as halfedge data
structure, is used to efficiently represent planar graphs. Using a DCEL
structure, we can easily manipulate the topological information of the
planar graph such as vertices, faces and edges.

In general, a DCEL conatins a book-keeping record of all edges, vertices
and faces of the graph. Each record individually can contain information
of all it's incident geometric objects. We will look at how a DCEL
stores this information in
\protect\hyperlink{data-structure-approach}{Data Structure Approach}
section.

\hypertarget{how-to-run}{%
\subsection{How to Run}\label{how-to-run}}

The src folder contains the source code for the DCEL data structure.
\texttt{g++} from the GNU compiler suite is required to compile the
program to a executable.

Steps to Compile:

\begin{enumerate}
\def\labelenumi{\arabic{enumi})}
\tightlist
\item
  \texttt{cd} into the src directory
\item
  Run \texttt{g++\ main.cpp} which generates an executable called
  \texttt{a.out} in the same directory
\item
  Run the executable using \texttt{./a.out} (on linux)

  \begin{enumerate}
  \def\labelenumii{\arabic{enumii})}
  \tightlist
  \item
    The executable takes a dataset from command line argument. For
    example, to use an existing dataset, run
    \texttt{./a.out\ ../datasets/1sq.txt}
  \item
    If no command-line argument is given, it takes input from the shell
    directly (stdin)
  \end{enumerate}
\end{enumerate}

\hypertarget{input}{%
\subsection{Input}\label{input}}

The required input format for the algorithm to work correctly is:

\begin{itemize}
\tightlist
\item
  Input can be given both from file (via command-line args) or stdin
\item
  First line must contain the no of Edges to be taken as input by the
  program.
\item
  Each of next line must contain 4 integers, space seperated denoting
  the (x1, y1), (x2, y2) coordinates of endpoints of each edge.
\item
  Each coordinate must be of integer type in the range -10\^{}8 to
  10\^{}8.
\item
  Number of coordinates must be less than 1 Billion.
\end{itemize}

\hypertarget{output}{%
\subsection{Output}\label{output}}

Output of the algorithm prints all the HalfEdges incident to each face
by traversing its representative half edges. All the faces outputed by
the program can also be verified by running it through the plotting
diagram.\\
The last two lines of output show the time taken to take input in
microseconds and the time taken by the algorithm to compute DCEL (also
in microseconds).

An example of output of dataset
\href{./datasets/random4.txt}{random4.txt}:

\includegraphics[width=8cm,height=8cm]{img/DCELrandom.png}\\

\hypertarget{documentation-and-report}{%
\subsection{Documentation and Report}\label{documentation-and-report}}

Documentation of this algorithm, functions and classes can be found in
the \texttt{docs} folder in the current directory. Open the
\href{../DCEL/docs/html/index.html}{index.html} file from the docs
directory with your preferred browser to go through the documentation

\hypertarget{performance-analysis}{%
\subsection{Performance Analysis}\label{performance-analysis}}

This analysis does not include the time taken to show output on
terminal. Analysis is performed with a system running:

\begin{itemize}
\tightlist
\item
  OS: Arch Linux (64Bit) running Linux Kernel version 5.7.2
\item
  Processor: Intel Core i7 7700HQ
\item
  RAM: 8GB
\item
  Compiler: GNU G++ (GCC) 10.1.0
\end{itemize}

\textbf{The following observations are recorded:}

Manual generated test files:

\begin{longtable}[]{@{}lccccc@{}}
\toprule
\begin{minipage}[b]{0.12\columnwidth}\raggedright
Filename\strut
\end{minipage} & \begin{minipage}[b]{0.15\columnwidth}\centering
Input Vertices\strut
\end{minipage} & \begin{minipage}[b]{0.12\columnwidth}\centering
Input Edges\strut
\end{minipage} & \begin{minipage}[b]{0.12\columnwidth}\centering
Input Faces\strut
\end{minipage} & \begin{minipage}[b]{0.15\columnwidth}\centering
File read time\strut
\end{minipage} & \begin{minipage}[b]{0.17\columnwidth}\centering
DCEL Creation Time\strut
\end{minipage}\tabularnewline
\midrule
\endhead
\begin{minipage}[t]{0.12\columnwidth}\raggedright
1sq.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
4\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
4\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
2\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
505 microsec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
27 microsec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
4sq.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
9\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
12\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
5\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
396 microsec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
46 microsec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
random1.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
10\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
12\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
4\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
257 microsec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
52 microsec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
random3.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
23\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
26\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
5\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
175 microsec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
84 microsec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
random2.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
23\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
37\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
16\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
192 microsec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
102 microsec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
a.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
31\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
33\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
4\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
150 microsec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
109 microsec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
random4.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
44\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
48\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
6\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
364 microsec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
165 microsec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
house.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
71\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
81\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
12\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
250 microsec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
300 microsec\strut
\end{minipage}\tabularnewline
\bottomrule
\end{longtable}

Using real world datasets:

\begin{itemize}
\tightlist
\item
  Road network of hyderabad, source: GPS coordinates filtered from
  osm(openstreetmap) data on hyderabad

  \begin{itemize}
  \tightlist
  \item
    hyd\_1.txt: contains road network having tag of \emph{highway} set
    as `trunk'
  \item
    hyd\_2.txt: contains road network having tag of \emph{highway} set
    as `trunk' and `primary'
  \item
    hyd\_2.txt: contains road network having tag of \emph{highway} set
    as `trunk', `primary' and `secondary'
  \item
    hyd\_2.txt: contains road network having tag of \emph{highway} set
    as `trunk', `primary', `secondary', and `motorway'
  \end{itemize}
\end{itemize}

\begin{longtable}[]{@{}lccccc@{}}
\toprule
\begin{minipage}[b]{0.12\columnwidth}\raggedright
Filename\strut
\end{minipage} & \begin{minipage}[b]{0.15\columnwidth}\centering
Input Vertices\strut
\end{minipage} & \begin{minipage}[b]{0.12\columnwidth}\centering
Input Edges\strut
\end{minipage} & \begin{minipage}[b]{0.12\columnwidth}\centering
Input Faces\strut
\end{minipage} & \begin{minipage}[b]{0.15\columnwidth}\centering
File read time\strut
\end{minipage} & \begin{minipage}[b]{0.17\columnwidth}\centering
Algorithm runtime\strut
\end{minipage}\tabularnewline
\midrule
\endhead
\begin{minipage}[t]{0.12\columnwidth}\raggedright
hyd\_1.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
685\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
733\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
58\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
2.4 millisec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
1.68 millisec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
hyd\_2.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
1787\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
1912\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
229\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
3.5 millisec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
107.5 millisec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
hyd\_3.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
2959\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
3198\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
519\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
5.4 millisec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
276.9 millisec\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright
hyd\_4.txt\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
3612\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
3850\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\centering
532\strut
\end{minipage} & \begin{minipage}[t]{0.15\columnwidth}\centering
24.3 millisec\strut
\end{minipage} & \begin{minipage}[t]{0.17\columnwidth}\centering
337.8 millisec\strut
\end{minipage}\tabularnewline
\bottomrule
\end{longtable}

\hypertarget{data-structure-approach}{%
\subsection{Data Structure Approach}\label{data-structure-approach}}

The following geometric objects are used in DCEL:

\begin{itemize}
\tightlist
\item
  \textbf{Vertex}: Object to store vertex coordinates and any one of its
  incident halfedges as its representative.
\item
  \textbf{Edge}: Object to store its vertex endpoints.
\item
  \textbf{Face}: Object to store a unique face-id and one of its
  incident halfedges as its representative
\item
  \textbf{HalfEdge}: Object to store a directed halfedge which has
  pointers to all its incident geometric objects (Vertex, Face, next and
  previous halfedges and its twin halfedge)
\end{itemize}

This implementation uses a Hash Map to store list of all incident
halfedges for each vertex. This Hash Map uses vertex pointer as key and
stores a list of halfedges as value. Our DCEL also has a list of
Vertices and Faces which can be used to iterate through all Vertices and
Faces quickly.

A half-edge has a pointer to the next half-edge and previous half-edge
of the same face. Each half-edge has a single face as its representative
Face (which is present towards left side of it). All half-edges
associated with a face are counter-clockwise which can be obtained by
traversing its representative Halfedge. To reach the other face, we can
go to the twin of the half-edge and then traverse the other face. Each
half-edge also has a pointer to its origin vertex which can be used to
get all the Vertex points of the corresponding face.

\hypertarget{results}{%
\subsection{Results}\label{results}}

This algorithm to construct a DCEL takes time complexity of O(V+E). From
the analysis of example dataset runtimes, we can easily see that
increase in number of vertices and edges directly leads to increase in
DCEL construction time.

\includegraphics[width=8cm,height=8cm]{img/DCELhyd4.png}\\

The output of the algorithm is direct traversal of representative edges
of all faces present in the input. In the above example of Hyderabad
road network dataset, it shows all the faces bounded by the roads. All
faces might not be visible directly as the road system branches without
forming many faces. But as we zoom, the faces can be spotted between
roads.

\hypertarget{conclusion}{%
\subsection{Conclusion}\label{conclusion}}

The time complexity O(V+E) cannot be reduced further because to store
the graph, it is required to traverse through all Vertices and Edges
present in the graph. Although the current implementation only supports
connected graphs, it can also be extended to store disconnected
components using techniques like hidden vertices.

From the above results, we can be sure that DCEL can be a good candidate
data structure for easy representation of planar graphs. But as every
other data structure, even DCEL has it downfalls. The main one being the
amount of Storage required to store. The storage requirement can be
further reduced if we only need to store Vertices or Edges, but this
does not give huge improvements. Also, as the size of DCEL increases,
the time to add an extra edge increases in a linear fashion which might
be bad based on the application.DCEL is a simple data structure which
helps to easily manupulate the graph structre, but it might not be an
appropriate choice for low memory applications such as in embedded
systems.


\subsubsection{\tt ipdd}
\label{benchmarkipdd}

{\tt ipdd} is a synthetic AKS benchmark that performs
a double-double ({\apl D\qplus\qdot\qtimes\qtran\0D}) matrix product on
arrays of shape {\tt [500, 500]}. This is the classic
{\em matmul} of linear algebra.

We expect that \wlf and \awlf should exhibit similar performance
behavior on this benchmark.


% !TEX root = as_grf_sopt.tex

% \begin{appendices}

\section{Predictive Covariance Matrix}
\label{sec:appendix:cov}

%%%%%%%%%%%%
\begin{lemma}
\label{lemma:nnC}
  For augmented graph Laplacian, %$\bC=(\cLap+\omega_0\bI)$,
  the posterior covariance matrix, $C_t(v,v')\geq0, \forall v, v'$.
\end{lemma}%
\begin{proof}
  Let $h_k = \sum_{\tau=1}^t e_{v_\tau}(v_k)$ to be the count of queries on node $k$; further define its diagonal matrix, $\bH = \diag(h_1,\dotsc,h_n)$.
  We rewrite \eqref{eq:post_prec} as, %The conjugate inference model in term of the precision matrix is,
\begin{equation*}
    (\bC_t)^{-1} = (\bC_0)^{-1} + \sigma_n^{-2}\bH
    = \bD - \bA + \omega_0\bI + \sigma_n^{-2}\bH
  \end{equation*}
  Define $\bD_t = \bD + \omega_0\bI + \sigma_n^{-2}\bH$,
  we have
  \begin{align*}
    \bC_t = (\bD_t - \bA)^{-1}
    =
    \bD_t^{-\frac{1}{2}}
    \left(\sum_{k=0}^\infty \left(\bD_t^{-\frac{1}{2}}\bA\bD_t^{-\frac{1}{2}}\right)^k \right)
    \bD_t^{-\frac{1}{2}},
  \end{align*}
where the right hand side is always nonnegative.

%%%%%%%%%%%%%
The convergence of $\|\bD_t^{-\frac{1}{2}} \bA \bD_t^{-\frac{1}{2}}\|_2 < 1$ is as follows.
%Assume $\sigma_n^2>0$, show  $\|\bD_t^{-\frac{1}{2}} \bA \bD_t^{-\frac{1}{2}}\|_2 < 1$.

Define the components for the posterior as $\bD_t = \diag(d_1^{(t)}, \dotsc, d_n^{(t)}$ with $d^{(t)} = \sum_{i=1}^n d_i^{(t)}$. Also, define for the prior model $\bD = \diag(d_1^{(0)}, \dotsc, d_n^{(0)}$ with $d^{(0)} = \sum_{i=1}^n d_i^{(0)}$.

The following holds for any $\bv\in\mathbb{R}^n$,
\begin{align*}
	&\quad \bv^\top \bD_t^{-\frac{1}{2}} \bA \bD_t^{-\frac{1}{2}} \bv
	= \sum_{ij}\frac{v_i v_j a_{ij}}{\sqrt{d^{(t)}_i}\sqrt{d^{(t)}_j}}
	\\
	&\leq
	\sqrt{
		\left(  \sum_{ij} \frac{v_i^2 a_{ij}}{d^{(t)}_i}  \right)
		\left(  \sum_{ij} \frac{v_j^2 a_{ij}}{d^{(t)}_j}  \right)
	}
	=\sum_i v_i^2 \frac{d_i}{d^{(t)}_i}
	\leq \|\bv\|_2^2.
\end{align*}

Further, both equalities cannot hold simultaneously, because for the first equality to hold, it is required that $\frac{v_i^2a_{ij}}{d_i^{(t)}} \propto \frac{v_j^2a_{ij}}{d_j^{(t)}}$, i.e., $v_j^2 \propto d_j^{(t)}, \forall j$ in the same connected component, which then dictates that,
\begin{align*}
	\sum_i v_i^2 \frac{d_i}{d^{(t)}_i}
	= \sum_i \left( \frac{d_i^{(t)}}{d^{(t)}} \|\bv\|_2^2\right) \frac{d_i}{d^{(t)}_i}
	= \frac{ d^{(0)} }{ d^{(t)} }\|\bv\|_2^2
	<\|\bv\|_2^2.
\end{align*}
\end{proof}

%
%%%%%%%%%%%%%
%\begin{remark}
% Notice that Lemma~\ref{lemma:nnC} also applies to normalized graph Laplacians, when because the counterpart of \eqref{eq:precision} allows a similar derivation,
% \begin{align*}
%   \widetilde{\bC}_t
%   &=
%   \left( \widetilde{\cLap} +
%   \widetilde{\bOmega} + \sigma_n^{-2}\bH
%   \right)^{-1}
%   \numberthis
%   \label{eq:nnC_normalized}
%   \\
%%    &=
%%    \bD^{\frac{1}{2}}
%%    \left( \bD - \bA
%%    + \widetilde{\bOmega}\bD + \sigma_n^{-2}\bH\bD
%%    \right)^{-1}
%%    \bD^{\frac{1}{2}},
%%    \\
%   &=
%   \bD^{\frac{1}{2}}
%   \widetilde{\bD}^{-\frac{1}{2}}
%   \left( \sum_{k=0}^\infty \left(
%     \widetilde{\bD}_t^{-\frac{1}{2}}
%     \bA
%     \widetilde{\bD}_t^{-\frac{1}{2}}
%   \right)^k
%   \right)^{-1}
%   \widetilde{\bD}^{-\frac{1}{2}}
%   \bD^{\frac{1}{2}},
% \end{align*}
% when the counterpart of $\bD_t$ is defined as,
% \begin{equation}
%   \widetilde{\bD}_t = \bD + \widetilde{\bOmega}\bD + \sigma_n^{-2}\bH\bD.
% \end{equation}
% Generally we require $\sigma_n > 0$, but it is not necessary for submatrix of covariance matrix on unqueried nodes (more in Appendix~\ref{app:sec:nnC}).
%\end{remark}
%

%%%%%%%%%%%%%

%\begin{lemma}
%If  prior is defined with respect to an unnormalized graph Laplacian, we have for any $t$ and $(v,v')$ pair,
%\begin{equation}
%  C_t(v,v) \geq C_t(v,v'),
%\end{equation}
%which essentially means $\sigma_t(v) \geq \rho_t(v,v')\sigma_t(v'), \forall v'$.
%\end{lemma}%
%%%%%%%%%%%%%%%%
%
%We can use spring analogy in spectral graph models, which assumes that the nodes are physical particles that can move vertically. Two particles $v,v'$, are connected with springs of strength $A(v,v')$, if the weight is nonnegative. Every node is also connected to a reference point (assuming that to be of level zero) with spring of strength $\omega_0+\frac{h_v}{\sigma_n^2}$.
%
%  In this analogy, the displacement of a node $v$ in height by applying unit form is proportional to $C(v,v)$, whereas the vertical displacement of another node $v'$  by applying unit force on node $v$ is $C(v,v')$. For a trivial example, imagine $A=0$ and displacement, inversely related to the spring strength, is exactly $(\omega_0+\frac{h_v}{\sigma_n^2})^{-1}$. From physical models, we can observe that the displacement of any other node is always smaller than self-displacement, when force is applied.
%\end{proof}

\begin{lemma}
\label{lemma:nnC_sub}
The diagonal elements in $\bC_t$ is always no smaller than the off-diagonal elements, i.e., $\sigma_t(v)^2 = C_t(v,v)\geq C_t(v,v'), \forall v,v'$.
% Let $C$ denote the posterior covariance conditioned on a set $S$ of nodes. For any $U \subset V \setminus S$, we have
% that $C_{U,U}$ is non-negative elementwise.
\end{lemma}

\newcommand{\bbv}{\bar{\bv}}
\newcommand{\bell}{\boldsymbol{\ell}}

\begin{proof}
	Without loss of generality, let $v$ be the last index of $\bC_t=(\widetilde{\cLap}_0 + \sigma_n^{-2}\bH)^{-1}$. For simplicity, let $\widetilde{\cLap}_t = \widetilde{\cLap}_0 + \sigma_n^{-2}\bH$ and it has the following matrix partition,
	\begin{equation*}
		\widetilde{\cLap}_t
		= \begin{pmatrix}
			\widetilde{\cLap}_{\bbv \bbv} & \tilde{\bell}_{\bbv v}
			\\
			\tilde{\bell}_{\bbv v}^\top & \tilde{\ell}_{vv}
		\end{pmatrix},
	\end{equation*}
	where $\bbv$ is the complement of $v$.
	From Woodbury matrix inversion lemma, we have
	\begin{equation}
		\bC_t = \widetilde{\cLap}_t^{-1}
		= \begin{pmatrix}
			\mathbf{M}
			 &
			-\frac{1}{m}
			\widetilde{\cLap}_{\bbv\bbv}^{-1} \tilde{\bell}_{\bbv v}
			\\
			-\frac{1}{m}
			\tilde{\bell}_{\bbv v}^\top \widetilde{\cLap}_{\bbv\bbv}^{-1}
			&
			\frac{1}{m}
		\end{pmatrix},
	\end{equation}
	where $m=\tilde{\ell}_{vv} - \tilde{\bell}_{\bbv v}^\top \widetilde{\cLap}_{\bbv\bbv}^{-1} \tilde{\bell}_{\bbv v} $
	and $\mathbf{M}=
			 \widetilde{\cLap}_{\bbv\bbv}^{-1}+\frac{1}{m}
			 \widetilde{\cLap}_{\bbv\bbv}^{-1} \tilde{\bell}_{\bbv v}
			 \tilde{\bell}_{\bbv v}^\top \widetilde{\cLap}_{\bbv\bbv}^{-1}
	$.
	To show that $C_t(v,v)\geq C_t(v,v')$, we need to verify that
$
	(			-\widetilde{\cLap}_{\bbv\bbv}^{-1} \tilde{\bell}_{\bbv v}
)_{v'}\leq1.
$

	In fact, since $\widetilde{\cLap}_t
	%=\widetilde{\cLap}_0+\sigma^{-2}\bH
	$ is diagonally dominant, we have
	$\widetilde{\cLap}_t \bone_n\geq0.
	$
	Take its first $n-1$ rows to get
	$
		\widetilde{\cLap}_{\bbv \bbv}\cdot \bone_{n-1} +
		\tilde{\bell}_{\bbv v} \geq 0.
	$
	Notice $\widetilde{\cLap}_{\bbv \bbv}$ is also a valid augmented graph Laplacian. By Lemma \ref{lemma:nnC}, we could left multiply the element-wise nonnegative matrix $\widetilde{\cLap}_{\bbv \bbv}^{-1}$ to both sides to obtain,
$		\bone_{n-1} +
		\widetilde{\cLap}_{\bbv \bbv}^{-1}\tilde{\bell}_{\bbv v} \geq 0,
$
	which completes our proof for any $v'\in \bbv$.
\end{proof}

% \begin{proof}
% 	Let $\tilde{L} := L + \omega_0 I$.
% Suppose the prior covariance matrix decomposes into,
% \begin{equation}
% 	K \;=\;
% 	\begin{bmatrix}
% 		\tilde{L}_{S,S}  & \tilde{L}_{S,U} \\
% 		\tilde{L}_{U,S} & \tilde{L}_{U,U}
% 	\end{bmatrix}^{-1}.
% \end{equation}
% Schur complement then gives
% \begin{align}
% 	K_{S,S} &= (\tilde{L}_{S,S} - \tilde{L}_{S,U} (\tilde{L}_{U,U})^{-1}\tilde{L}_{U,S})^{-1},\\
% 	K_{U,U} &= (\tilde{L}_{U,U})^{-1}( I + \tilde{L}_{U,S} K_{S,S} \tilde{L}_{S,U} (\tilde{L}_{U,U})^{-1}),\\
% 	K_{S,U} &= -K_{S,S} \tilde{L}_{S,U} (\tilde{L}_{U,U})^{-1}.
% \end{align}
% By conjugate inference formula, we have
% \begin{align}
% 	C_{S,S} &= K_{S,S} - K_{S,S}(K_{S,S}+\sigma^2 I)^{-1}K_{S,S}\\
%                 &= (\sigma^{-2}I + K_{S,S}^{-1})^{-1}\\
% 			      & = (\tilde{L}_{S,S} + \sigma^{-2} I- \tilde{L}_{S,U} (\tilde{L}_{U,U})^{-1}\tilde{L}_{U,S})^{-1}, \label{eq:C_SS}\\
% 	C_{U,U}
% 	&= K_{U,U} - K_{U,S}(K_{S,S} + \sigma^2 I)^{-1} K_{S,U}\\
% 	&= (\tilde{L}_{U,U})^{-1}(I + \tilde{L}_{U,S}C_{S,S}\tilde{L}_{S,U}(\tilde{L}_{U,U})^{-1}).
% \end{align}
% Plugging \eqref{eq:C_SS} into \eqref{eq:C_UU} and applying the matrix inversion lemma, we get
% \begin{equation}
% 	C_{U,U} = (\tilde{L}_{U,U} - \tilde{L}_{U,S}(\tilde{L}_{S,S} + \sigma^{-2}I)^{-1}\tilde{L}_{S,U})^{-1}.
% \label{eq:C_UU}
% \end{equation}
% Again by Schur complement, we get
% \begin{equation}
% 	C_{U,U} = (\bar{L}^{-1})_{U,U},
% \end{equation}
% where
% \begin{equation}
% 	\bar{L} :=
% 	\begin{bmatrix}
% 		\tilde{L}_{S,S} + \sigma^{-2}I & \tilde{L}_{S,U} \\
% 		\tilde{L}_{U,S} & \tilde{L}_{U,U}
% 	\end{bmatrix}.
% \end{equation}
% Because $\bar{L}$ satisfies Proposition 6 of \cite{ma_2013}, $\bar{L}^{-1}$ is non-negative elementwise, and so is $C_{U,U}$.
% \end{proof}

% In fact, this Lemma can be applied to also labeled nodes by constructing an auxiliary ``labeled node'' that connects to every other node with weight equal to the regularization parameter of that node.

% \end{appendices}


%===============================================================================
\section*{Introduction}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Extension, torsion and bending of rods/beams}

  \begin{tabularx}{\linewidth}{XX}
    {
    \begin{alignat*}{1}
      F &= E \, A \, \epsilon \\
        %&= E \, A \, \frac{\Delta l}{l}
    \end{alignat*}
    } & {
    \null
    } \\
    %\hline

    {
    \begin{alignat*}{1}
      M_t = G \, J \, \frac{\Theta}{L} \\
    \end{alignat*}
    } & {
    \null
    } \\
    %\hline

    {
    \begin{alignat*}{1}
      M = E \, I \, \kappa \\
    \end{alignat*}
    } & {
    \null
    }
  \end{tabularx}
  % TODO revisit this/these motivational slide/slides
  % put in three pictures
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{3D elasticity (recap of some basics)}

  \textbf{deformation map} $\underline{x} = \underline{f}(\underline{X})$
  \begin{itemize}
    \item takes a (Langrangian/material) vector/point $\underline{X}$ in the undeformed configuration
    \item returns a (Eulerian/spatial) vector/point $\underline{x}$ in the deformed configuration
  \end{itemize}
  \vspace{1em}

  \textbf{deformation gradient} $\doubleunderline{F}(\underline{X}) = \nabla_{\underline{X}} \,\underline{f}(\underline{X})$
  \begin{itemize}
    \item takes a (Langrangian/material) vector/point $\underline{X}$ in the undeformed configuration \newline
      $\rightarrow$ $\underline{X}$ is material point where the gradient is evaluated
    \item returns a second order (two-point) tensor $\doubleunderline{F}$ that depends on $\underline{X}$ (in the general case)
    \item $\doubleunderline{F}$ contains (first order accurate) information about how line elements in the material configuration are transformed into deformed (stretched and rotated) line elements in the spatial configuration by $\underline{f}(.)$ \newline
      $\rightarrow$ $\dif \underline{x} = \doubleunderline{F}(\underline{X}) \cdot \dif \underline{X}= \nabla_{\underline{X}} \,\underline{f}(\underline{X}) \cdot \dif \underline{X}$
  \end{itemize}
  \vspace{1em}

  \textbf{equations of equilibrium}
  \begin{itemize}
    \item material description: $\nabla_{\underline{X}} \cdot \doubleunderline{P} + \underline{B} = \rho_0 \cdot \underline{A}$
    \item spatial description: $\nabla_{\underline{x}} \cdot \doubleunderline{\sigma} + \underline{b} = \rho \cdot \underline{a}$
  \end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{1D elasticity}

  we talk about \textbf{slender bodies} ($\frac{\mathrm{Length}}{\mathrm{Diameter}} > 10$)
  \vspace{1em}

  we only have \textbf{one material coordinate} $s$ along the axis of the beam
  \begin{itemize}
    \item $s$ uniquely identifies a particular cross section along the beam axis
    \item model equations will be a system of ODEs in $s$ (instead of PDE)
    \item equations are obtained by averaging a PDE in 3 dimensions over the cross section
  \end{itemize}
  \vspace{1em}

  kinematic quantities in the model
  \begin{itemize}
    \item \textbf{position of centerline} $\underline{r}(s)$
    \item \textbf{orientation/rotation of cross section} $\doubleunderline{R}(s)$
    \item more accuracy $\rightarrow$ more unknowns
  \end{itemize}
\end{frame}


\section{Method}
\label{sec:method}

Written words are more concrete than ideas in your head.
Math equations and pseudo-codes are even more concrete than words, and can help, even though not essential, for exposition.
Programs that produce the actual results are the most concrete, but the amount and level of information is not suitable for a research paper.
I like to write down math and pseudo-code first as a way to design the algorithms before actual implementation.

\subsection{Math}
\label{sec:math}

\begin{align}
\energy = \mass \lightspeed^2
\label{eqn:emc2}
\end{align}

Always use defined instead of naked math symbols for clarify and consistency.
For example, in \Cref{eqn:emc2} I define all symbols inside \filename{symbols.tex} so later if I need to change $\energy$ from $E$ to $e$ I just need to change one line in \filename{symbols.tex} instead of chasing $E$ everywhere.
This can save you a lot of time and sanity if you have many math symbols.

\subsection{Code}
\label{sec:code}

\input{dart_throw_alg}

Pseudo-code can help summarize and explain the algorithms, but only if presented with sufficient clarity and simplicity.
\Cref{alg:dart_throw} is an example.
All algorithms should be understandable from the main texts without looking at the pseudo-codes, which are usually more suitable for summarization than explanation.


\chapter{Classification of Spectra}
\label{ch:spectra_classification}

\begin{wrapfigure}{o}{0.82\textwidth}
    \centering
    \vspace{-3.4cm}
    \includegraphics[width=0.95\textwidth]{sp_classification-fig1.png}
    \label{fig:spectra_classification-fig1}
\end{wrapfigure}

Let’s open the collagen data set again and see how well can logistic regression predict its four classes. Straightforward, right? Connect \widget{Datasets}, \widget{Logistic Regression}, \widget{Predictions}, \widget{Confusion Matrix} and that's it. We would also like to do some spectral processing (we will only keep the columns for wavenumbers between 1500\wn and 1800\wn).

\begin{figure}[h]
\hspace{-1cm}\stackinset{r}{-0.7\linewidth}{t}{+0.25\linewidth}
  {\includegraphics[scale=0.35]{sp_classification-fig2b.png}}
  {\includegraphics[scale=0.45]{sp_classification-fig2a_.png}}
  \caption{The \widget{Spectra} widget shows wrong predictions for the DNA class.}
  \label{fig:spectra_classification-fig2}
\end{figure}

Let’s not forget that it is pointless to predict for the same data as we used for learning. We could either  use a \widget{Data Sampler} and connect its Sample output to \widget{Preprocess Spectra} and Remaining output to \widget{Predictions}, or obtain predictions from the \widget{Test and Score} widget.
\widget{Confusion Matrix} now shows the mistakes of the model (scored with cross-validation). We can select them and inspect them further in a \widget{Spectra} widget. Here we colored them by the predicted class (see the Menu).

\begin{wrapfigure}{o}{1.1\textwidth}
  \centering
  \includegraphics[width=1.1\textwidth]{sp_classification-fig3.png}%
  \label{fig:spectra_classification-fig3}
\end{wrapfigure}
But how does the model make its decisions? We already inspected a different model, classification tree, where each node represents a decision on a value of a column.  \widget{Logistic regression} works differently. On the training data it computes weights for all columns (wavelengths), which are then used for prediction, where values are multiplied with weights. To see the weights, connect \widget{Logistic Regression} to a \widget{Data Table}.

\begin{wrapfigure}{o}{0.9\textwidth}
%  \centering
  \vspace{-0.7cm}
  \includegraphics[width=0.9\textwidth]{sp_classification-fig4.png}
  \label{fig:spectra_classification-fig4}
\end{wrapfigure}
We get a table that is hard to understand. What if we visualize it? First, \widget{Transpose} the data. Then, use \widget{Select Columns} to make the visualization prettier: in the widget remove the intercept.

Now, open \widget{Logistic Regression} and try changing its parameters. Observe the effect on the weights.

\begin{figure}[h]
\hspace{-1cm}\stackinset{r}{0\linewidth}{t}{-0.22\linewidth}
  {\includegraphics[scale=0.35]{sp_classification-fig5a.png}}
  {\includegraphics[scale=0.35]{sp_classification-fig5b.png}}
  \label{fig:spectra_classification-fig5}
\end{figure}


\input{../header.tex}
\title{\vspace{-2cm}INF3490/INF4490 Exercises - Advanced Neural Networks}
\author{Eivind Samuelsen\input{../author_footnote.tex}}
\date{}

% Removing paragraph indents is sometimes useful:
\setlength\parindent{0pt}
% ==============================================================================

% ================================= DOCUMENT ===================================
\begin{document}
    \renewcommand\marginsymbol[1][0pt]{%
  \tabto*{0cm}\makebox[-1cm][c]{$\mathbb{P}$}\tabto*{\TabPrevPos}}

\maketitle
\input{../intro.tex}

\section{Perceptron activation functions}
Last week we used the activation function
\[
g(h) =
\begin{cases}
      1 & h > 0 \\
      0 & h \leq 0
   \end{cases}
\]
Why is this not used with backpropagation?

\section{Hidden layers}
What is the minimum number of hidden neuron layers needed in order to approximate an arbitrary continuous function, and why?

\section{Validation}
Why do we use a validation set?
Describe how the three different cross-validation methods presented in the lecture slides work, and what their advantages and disadvantages are.

\section{Multi Layer Perceptron \marginsymbol}
Implement the MLP shown below, and train it to correctly perform the XOR function.

\begin{figure}[H]
\begin{center}
\includegraphics[width=0.4\textwidth]{mlp.png}
\caption{MLP with one hidden layer and two hidden nodes.}
\label{fig:mlp}
\end{center}
\end{figure}

\section{Delta(error) function}
In the lecture slides the backpropagation deltas are first presented as
\[
\delta_k = (y_k - t_k)y_k(1-y_k)
\]
What does this tell us about the activation function in use?

\section{Natural language MLP}
You are to design an MLP that would learn to hyphenate words correctly.
You would have a dictionary that shows correct hyphenation examples for lots of words.
Think about the following:
What should the input to the neural net be?
\begin{itemize}
    \item How should this input be encoded to work well with the classifier?
    \item How is should the output be encoded?
    \item How many layers do you need?
    \item How many neurons should there be in each layer?
\end{itemize}
\input{../contact.tex}
\end{document}
% ==============================================================================


\subsection{Gridification, Time Tables, and Time Series Generation}
\label{grid}

The platform's tabular order data are sliced with respect to both location and
    time and then aggregated into time series where an observation tells
    the number of orders in an area for a time step/interval.
Figure \ref{f:grid} shows how the orders' delivery locations are each
    matched to a square-shaped cell, referred to as a pixel, on a grid
    covering the entire service area within a city.
This gridification step is also applied to the pickup locations separately.
The lower-left corner is chosen at random.
Applications of this gridification idea to model location-routing problems
    can be viewed, for example, in \cite{winkenbach2015}, \cite{bergmann2020},
    \cite{janjevic2019}, \cite{snoeck2020}, and \cite{janjevic2020}
    while \cite{singleton2017} portray it as a standard method in the field of
    urban analytics.
With increasing pixel sizes, the time series exhibit more order aggregation
    with a possibly stronger demand pattern.
On the other hand, the larger the pixels, the less valuable become the
    generated forecasts as, for example, a courier sent to a pixel
    preemptively then faces a longer average distance to a restaurant in the
    pixel.

\begin{center}
\captionof{figure}{Gridification for delivery locations in Paris with a pixel
                   size of $1~\text{km}^2$}
\label{f:grid}
\includegraphics[width=.8\linewidth]{static/gridification_for_paris_gray.png}
\end{center}

After gridification, the ad-hoc orders within a pixel are aggregated by their
    placement timestamps into sub-daily time steps of pre-defined lengths
    to obtain a time table as exemplified in Figure \ref{f:timetable} with
    one-hour intervals.

\begin{center}
\captionof{figure}{Aggregation into a time table with hourly time steps}
\label{f:timetable}
\begin{tabular}{|c||*{9}{c|}}
    \hline
    \backslashbox{Time}{Day} & \makebox[2em]{\ldots}
        & \makebox[3em]{Mon} & \makebox[3em]{Tue}
        & \makebox[3em]{Wed} & \makebox[3em]{Thu}
        & \makebox[3em]{Fri} & \makebox[3em]{Sat}
        & \makebox[3em]{Sun} & \makebox[2em]{\ldots} \\
    \hline
    \hline
    11:00 & \ldots & $y_{11,Mon}$ & $y_{11,Tue}$ & $y_{11,Wed}$ & $y_{11,Thu}$
                   & $y_{11,Fri}$ & $y_{11,Sat}$ & $y_{11,Sun}$ & \ldots \\
    \hline
    12:00 & \ldots & $y_{12,Mon}$ & $y_{12,Tue}$ & $y_{12,Wed}$ & $y_{12,Thu}$
                   & $y_{12,Fri}$ & $y_{12,Sat}$ & $y_{12,Sun}$ & \ldots \\
    \hline
    \ldots & \ldots & \ldots & \ldots & \ldots
           & \ldots & \ldots & \ldots & \ldots & \ldots \\
    \hline
    20:00 & \ldots & $y_{20,Mon}$ & $y_{20,Tue}$ & $y_{20,Wed}$ & $y_{20,Thu}$
                   & $y_{20,Fri}$ & $y_{20,Sat}$ & $y_{20,Sun}$ & \ldots \\
    \hline
    21:00 & \ldots & $y_{21,Mon}$ & $y_{21,Tue}$ & $y_{21,Wed}$ & $y_{21,Thu}$
                   & $y_{21,Fri}$ & $y_{21,Sat}$ & $y_{21,Sun}$ & \ldots \\
    \hline
    \ldots & \ldots & \ldots & \ldots & \ldots
           & \ldots & \ldots & \ldots & \ldots & \ldots \\
    \hline
\end{tabular}
\end{center}
\

Consequently, each $y_{t,d}$ in Figure \ref{f:timetable} is the number of
    all orders within the pixel for the time of day $t$ and day of week
    $d$ ($y_t$ and $y_{t,d}$ are the same but differ in that the latter
    acknowledges a 2D view).
The same trade-off as with gridification applies:
The shorter the interval, the weaker is the demand pattern to be expected in
    the time series due to less aggregation while longer intervals lead to
    less usable forecasts.
We refer to time steps by their start time, and their number per day, $H$,
    is constant.
Given a time table as in Figure \ref{f:timetable} there are two ways to
    generate a time series by slicing:
\begin{enumerate}
    \item \textbf{Horizontal View}:
    Take only the order counts for a given time of the day
    \item \textbf{Vertical View}:
    Take all order counts and remove the double-seasonal pattern induced
    by the weekday and time of the day with decomposition
\end{enumerate}
Distinct time series are retrieved by iterating through the time tables either
    horizontally or vertically in increments of a single time step.
Another property of a generated time series is its length, which, following
    the next sub-section, can be interpreted as the sum of the production
    training set and the test day.
In summary, a distinct time series is generated from the tabular order data
    based on a configuration of parameters for the dimensions pixel size,
    number of daily time steps $H$, shape (horizontal vs. vertical), length,
    and the time step to be predicted.


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
% \usepackage{graphicx}
%     \DeclareGraphicsExtensions{.png, .jpeg}
% \usepackage{caption}
\usepackage[top=1in, bottom=1in, left=1in, right=1in]{geometry}

\title{STAT 775: Machine Learning \\ HW 05}
\author{Terence Henriod}
\date{\today}

\begin{document}

\clearpage            % All
\maketitle            % this,
\thispagestyle{empty} % removes the page number from the title page

\begin{abstract}
In this assignment, logistic regression and neural networks of single neurons are used for simple digit image classification.
\end{abstract}

\newpage
\section{Exercise 01}
\subsection{Problem Statement}
Using the zip code digit data from the ESL website, use Logistic Regression to perform binary classification between 2s and 3s, 4s and 5s, and 7s and 9s.

\subsection{Results}
The classification results are as follows:\\
%
\newline
\begin{tabular}{| c || c | c | c |}
  \hline
                                  & Classified as 2 & Classified as 3 & \% Correct \\
  \hline
  Actual class 2                  & 188             & 10              & 94.9 \\
  \hline
  Actual class 3                  & 6               & 160             & 96.4 \\
  \hline
  \hline
  Overall Correct Prediction Rate &                 &                 & 95.6 \\
  \hline
\end{tabular}

\hfill\\%
\newline
\begin{tabular}{| c || c | c | c |}
  \hline
                                  & Classified as 4 & Classified as 5 & \% Correct \\
  \hline
  Actual class 4                  & 198             & 2               & 99.0 \\
  \hline
  Actual class 5                  & 6               & 154             & 96.3 \\
  \hline
  \hline
  Overall Correct Prediction Rate &                 &                 & 97.8 \\
  \hline
\end{tabular}

\hfill\\%
\newline
\begin{tabular}{| c || c | c | c |}
  \hline
                                  & Classified as 7 & Classified as 9 & \% Correct \\
  \hline
  Actual class 7                  & 139             & 8               & 94.6 \\
  \hline
  Actual class 9                  & 4               & 173             & 97.7 \\
  \hline
  \hline
  Overall Correct Prediction Rate &                 &                 & 96.3 \\
  \hline
\end{tabular}

%
% \newpage
\subsection{Code}
The following R code was used to perform the LDA and classification:
\begin{verbatim}
#
# Initial Setup
#
setwd("C:/Users/Terence/Documents/GitHub/STAT775/HW05")

#
# Data Cleaning
#
DATA.PATH <- "../DataSets/zip.data/"
ZIP.TRAIN.FILE.NAME <- paste0(DATA.PATH, "zip.train")
ZIP.TEST.FILE.NAME <- paste0(DATA.PATH, "zip.test")

create.data.tuple <- function(file.path.name, class0, class1) {
  data <- read.table(file.path.name)
  data <- subset(data, V1 == class0 | V1 == class1)

  labels <- matrix(0, nrow(data), 1)
  for (i in 1:nrow(data)) {
    if (data[i, 1] == class1) {
      labels[i, 1] <- 1.0
    }
  }

  data <- data[, -1]

  data.tuple <- list(
    labels = data.matrix(labels),
    observations = data.matrix(data)
  )
  return(data.tuple)
}

train.data <- create.data.tuple(ZIP.TRAIN.FILE.NAME, 4, 5)
test.data <- create.data.tuple(ZIP.TEST.FILE.NAME, 4, 5)

#
# Logistic Regression Pieces
#
logistic.predict <- function(x, b) {
  #
  # Args:
  #   x: a p x 1 column vector of observation values
  #   b: a p X 1 column vector of model weights (betas)

  e <- exp(t(b) %*% rbind(1.0, x))
  return(e / (1.0 + e))
}

newton.raphson.iteration <- function(X, y, old.betas, step.size = 0.5) {
  #
  # Args:
  #   X: N x p matrix of observations
  #   y: column vector of N target labels
  #   old.betas: column vector of (p + 1) model parameters
  #   step.size: a rate controlling parameter

  p <- matrix(0, nrow = nrow(y), ncol = 1)
  for (i in 1:nrow(y)) {
    p[i, 1] <- logistic.predict(x = data.matrix(X[i, ]), b = old.betas)
  }

  W <- diag(1, nrow(y), nrow(y))
  for (i in 1:nrow(y)) {
    W[i, i] <- p[i, 1] * (1.0 - p[i, 1])
  }

  X.b <- cbind(1.0, X)  # X with 1s prepended for bias term

  z <- (X.b %*% old.betas) +
       (solve(W + diag(0.0001, nrow(W), ncol(W))) %*% (y - p))

  step <- solve((t(X.b) %*% W %*% X.b) + diag(0.0001, ncol(X.b), ncol(X.b))) %*%
          t(X.b) %*% (y - p)
  step <- step * step.size

  return(old.betas + step)
}

train.logistic.model <- function(X, y, n.iterations = 75, step.size = 0.5) {
  #
  # Args:
  #   X: N x p matrix of observations, observations are rows
  #   y: N x 1 column vector of class labels {0, 1}
  #   n.iterations: the number of approximation iterations to perform
  #   step.size: used to slow the rate of convergence to prevent overshooting

  # start with model parameters at 0 - arbitrary starting point
  betas <- matrix(0, nrow = ncol(X) + 1, 1)

  # iterate until parameter updates become arbitrarily small
  old.betas <- betas + 1.0
  i <- 0
  while (i < n.iterations) {
    betas <- newton.raphson.iteration(
      X = X,
      y = y,
      old.betas = betas,
      step.size = step.size
    )

    # TODO: use method for finding convergence, not just arbitrary iterations

    i <- i + 1
  }

  return(betas)
}

#
# Testing
#
test.cases <- list(
  list(2, 3),
  list(4, 5),
  list(7, 9)
)

for (case in test.cases) {
  train <- create.data.tuple(ZIP.TRAIN.FILE.NAME, case[[1]], case[[2]])
  model.betas <- train.logistic.model(train$observations, train$labels)

  test <- create.data.tuple(ZIP.TEST.FILE.NAME, case[[1]], case[[2]])
  confusion.matrix <- matrix(0, 2, 2 + 1)
  rownames(confusion.matrix) <- c(case[[1]], case[[2]])
  colnames(confusion.matrix) <- c(case[[1]], case[[2]], '%.Correct')
  correct <- 0

  for (i in 1:nrow(test$labels)) {
    prediction <- logistic.predict(
      x = data.matrix(test$observations[i, ]),
      b = model.betas
    )
    prediction <- round(prediction, 0)

    if (test$labels[[i]] == 0) {
      if (prediction == 0) {
        confusion.matrix[1, 1] <- confusion.matrix[1, 1] + 1
      } else {
        confusion.matrix[1, 2] <- confusion.matrix[1, 2] + 1
      }
    } else {
      if (prediction == 0) {
        confusion.matrix[2, 1] <- confusion.matrix[2, 1] + 1
      } else {
        confusion.matrix[2, 2] <- confusion.matrix[2, 2] + 1
      }
    }

    if (prediction == test$labels[[i]]) {
      correct <- correct + 1
    }
  }
  print (correct / nrow(test$labels))
  confusion.matrix[1, 3] <- confusion.matrix[1, 1] /
                            (confusion.matrix[1, 1] + confusion.matrix[1, 2])
  confusion.matrix[2, 3] <- confusion.matrix[2, 2] /
                            (confusion.matrix[2, 1] + confusion.matrix[2, 2])
  print(confusion.matrix)
}
\end{verbatim}

%
\newpage
\section{Exercise 02}
\subsection{Problem Statement}
Using the zip code digit data from the ESL website, use Single Neuron Neural Networks to perform binary classification between 2s and 3s, 4s and 5s, and 7s and 9s.

\subsection{Results}
The classification results (with 1000 training iterations and a step size of 0.1) are as follows:\\
\begin{tabular}{| c || c | c | c |}
  \hline
                                  & Classified as 2 & Classified as 3 & \% Correct \\
  \hline
  Actual class 2                  & 191             & 7               & 96.5 \\
  \hline
  Actual class 3                  & 4               & 162             & 97.6 \\
  \hline
  \hline
  Overall Correct Prediction Rate &                 &                 & 97.0 \\
  \hline
\end{tabular}

\hfill\\%
\newline
\begin{tabular}{| c || c | c | c |}
  \hline
                                  & Classified as 4 & Classified as 5 & \% Correct \\
  \hline
  Actual class 4                  & 194             & 6               & 97.0 \\
  \hline
  Actual class 5                  & 4               & 156             & 97.5 \\
  \hline
  \hline
  Overall Correct Prediction Rate &                 &                 & 97.2 \\
  \hline
\end{tabular}

\hfill\\%
\newline
\begin{tabular}{| c || c | c | c |}
  \hline
                                  & Classified as 7 & Classified as 9 & \% Correct \\
  \hline
  Actual class 7                  & 138             & 9               & 93.9 \\
  \hline
  Actual class 9                  & 4               & 173             & 97.7 \\
  \hline
  \hline
  Overall Correct Prediction Rate &                 &                 & 96.0 \\
  \hline
\end{tabular}

%
% \newpage
\subsection{Code}
The following R code was used to perform the LDA and classification:
\begin{verbatim}
#
# Initial Setup
#
setwd("C:/Users/Terence/Documents/GitHub/STAT775/HW05")

#
# Data Cleaning
#
DATA.PATH <- "../DataSets/zip.data/"
ZIP.TRAIN.FILE.NAME <- paste0(DATA.PATH, "zip.train")
ZIP.TEST.FILE.NAME <- paste0(DATA.PATH, "zip.test")

create.data.tuple <- function(file.path.name, class0, class1) {
  data <- read.table(file.path.name)
  data <- subset(data, V1 == class0 | V1 == class1)

  labels <- matrix(0, nrow(data), 1)
  for (i in 1:nrow(data)) {
    if (data[i, 1] == class1) {
      labels[i, 1] <- 1.0
    }
  }

  data <- data[, -1]

  data.tuple <- list(
    labels = data.matrix(labels),
    observations = data.matrix(data)
  )
  return(data.tuple)
}

#
# Neuron Structure
#
create.neuron <- function(dim, activation.f, activation.f.derivative) {
#
# Args:
#   dim: an integer dimensionality of the inputs (do not include bias)
#   activation.f: the activation function of the neuron
#   activation.f.derivative: a the function that is the derivative of activation.f

  w <- matrix(rand(), dim + 1, 1)

  return(list(
    w = w,
    f = activation.f,
    f.prime = activation.f.derivative,
    output = 0,
    error.prime = 9999
  ))
}

sigmoid.f <- function(x) {
  return(1.0 / (1.0 + exp(-x)))
}

sigmoid.f.prime <- function(x) {
  return(sigmoid.f(x) * (1.0 - sigmoid.f(x)))
}

weighted.sum.inputs <- function(b, x) {
#
# Args:
#   x: a p x 1 column vector of feature values
#   b: a (p + 1) x 1 column vector of input weights
  return((t(b[-1, ]) %*% x) - b[1,1])
}

#
#
#

test.cases <- list(
  list(2, 3),
  list(4, 5),
  list(7, 9)
)

case <- test.cases[[1]]
i <- 1
learn.rate <- 0.1

for (case in test.cases) {
  train.data <- create.data.tuple(ZIP.TRAIN.FILE.NAME, case[[1]], case[[2]])
  W <- matrix(runif(ncol(train.data$observations) + 1))

  k <- 1000
  while (k > 0) {
    for (i in 1:nrow(train.data$labels)) {
      x <- data.matrix(train.data$observations[i, ])
      y <- train.data$labels[[i]]
      input <- weighted.sum.inputs(x = x, b = W)
      o <- sigmoid.f(input)
      e <- y - o
      W <- W + learn.rate * (rep(((o * (1.0 - o)) * e), 257) * rbind(1.0, x))

#       D <- sigmoid.f.prime(input) * e
#       W <- W - rep((learn.rate * o * D), 257)
    }

    k <- k - 1
  }

  test.data <- create.data.tuple(ZIP.TEST.FILE.NAME, case[[1]], case[[2]])
  confusion.matrix <- matrix(0, 2, 2 + 1)
  rownames(confusion.matrix) <- c(case[[1]], case[[2]])
  colnames(confusion.matrix) <- c(case[[1]], case[[2]], '%.Correct')
  correct <- 0

  for (i in 1:nrow(test.data$labels)) {
    x <- data.matrix(test.data$observations[i, ])
    prediction <- round(sigmoid.f(weighted.sum.inputs(x = x, b = W)))

    if (test.data$labels[[i]] == 0) {
      if (prediction == 0) {
        confusion.matrix[1, 1] <- confusion.matrix[1, 1] + 1
      } else {
        confusion.matrix[1, 2] <- confusion.matrix[1, 2] + 1
      }
    } else {
      if (prediction == 0) {
        confusion.matrix[2, 1] <- confusion.matrix[2, 1] + 1
      } else {
        confusion.matrix[2, 2] <- confusion.matrix[2, 2] + 1
      }
    }

    if (prediction == test.data$labels[[i]]) {
      correct <- correct + 1
    }
  }
  print(correct / nrow(test.data$labels))
  confusion.matrix[1, 3] <- confusion.matrix[1, 1] /
                            (confusion.matrix[1, 1] + confusion.matrix[1, 2])
  confusion.matrix[2, 3] <- confusion.matrix[2, 2] /
                            (confusion.matrix[2, 1] + confusion.matrix[2, 2])
  print(confusion.matrix)
}
\end{verbatim}

\end{document}