\chapter{Introduction}
\label{chapter:intro}
\section{Problem Statement and its Relevance}

Previously to our work, there was a considerable gap in methods and tools between the scientific side of physical wireless communications (the perspective of physics, mathematics, etc.) and the engineering perspective (telecommunications). We bridged this gap by taking a three-dimensional approach to our research: models, algorithms and implementations.

We provided models for wireless communications based on techniques of high performance scientific computing (more precisely the Wave
Equation). We took into consideration signal physical propagation, reflections, damping and modulation.
The models are non limitative and can be equally applied to earthquake studies, radar, wireless communications, quantum computing or econophysics. In fact, our approach can be used as a guideline to deriving algorithms and implementations from other scientific models, in unrelated ares.

We then took our research one step further and designed algorithms based on these models and the most usual paradigms of mathematical computing: numerical methods, automatic differentiation and symbolic computation.

The final step was to implement the algorithm using various parallel, hardware description, and mathematical computing approaches. We compared them and have shown the highlights and drawbacks.

\section{Work We Built Upon}
I presented a significant part of this thesis at the SYNASC 2020 conference, under the guidance of Emil Slusanschi, and together with him and Filip Manole wrote a research paper indexed ~\cite{synasc} by IEEE, of which I am first author.
Therefore, some sections of this thesis are originating in that paper, and are marked as such.

More than this, some data for comparisons were taken, as indicated, from Filip's thesis ~\cite{filip}.

It is also worth mentioning that the initial experiments for acoustic wave propagation were provided by Mihai Trascau in his diploma project ~\cite{Trascau}.

\chapter{Models}
\section{The Wave Equation}

We start developing our model for wireless signal propagation, from which we derive the algorithms and implementations, from the wave equation, similar to the work of \cite{Trascau}.

Using notations specific to mechanical engineering, the wave equation \cite{wave-eq} is explained as in the first form. Its meaning is easier to see by expanding the Laplacian ($\nabla^2$) and using Leibniz's notation, to obtain the second form in equation \ref{eq:wave}:

\begin{equation}
\ddot{A}=c^{2}\nabla^{2}A \Leftrightarrow
\frac{\partial ^{2} A}{\partial x^{2}} + \frac{\partial ^{2} A}{\partial y^{2}} + \frac{\partial ^{2} A}{\partial z^{2}} = \frac{1}{c^{2}} \frac{\partial ^{2} A}{\partial t^{2}} \label{eq:wave}
\end{equation}

From the second form, we make a number of observations.

Firstly, the meaning of the function $A$ is not fixed in the equation. Indeed, this is a mathematical construct first and foremost, and the meaning of the function is given by the physical interpretation. Physical phenomenons that obey the wave equation are mechanical waves (through the amplitude displacement quantity, with applications such as sound engineering ~\cite{sound}, sonar ~\cite{sonar}, earthquakes research ~\cite{earthquake}), electromagnetic waves (through the electric intensity and the magnetic induction qunatities, with applications such as wireless communications ~\cite{synasc}, radar ~\cite{radar}, spectral analysis) and, with some variations, even quantum phenomenons (the quantum wave ecuation, with applications in quantum computing ~\cite{quantum}).

Secondly, it relates the space coordinates ($x$, $y$, $z$) to the time coordinate ($t$).

Thirdly, the proportionality factor of the equation is square of the speed of propagation of the wave, which is the speed of light in the case of wireless communications.

Fourthly, the equation represents a partial differential equation (PDE), and in mathematics not all PDEs have an analitical solution (or their analytical form is known) ~\cite{pde}, one famous example of such an equation being the Navier-Stokes ~\cite{navier-stokes} equation.

A solution of equation \ref{eq:wave} is a function that verifies it and is of the form

\begin{equation}
A = f(x, y, z, t), \forall (x,y,z,t) \in D^3 \times T \label{eq:4D}
\end{equation}

and the wave equation is a particular case of PDE which can be analytically solved (called a hyperbolic partial differential equation). An example of such an analytical solution is given in section \ref{sec:anal-sol}.

For the purpose of physical wireless communications, we are intersted in studing, simulating and augmenting the equation of wave with a view to telecommunications. For this reason, we adopt the following 2 physical interpretations of the wave equation.

The first case we are interested in is the one in which the function is actually the intensity of the electric field ($E$) in wireless communications.

\begin{equation}
\frac{\partial ^{2} E}{\partial x^{2}} + \frac{\partial ^{2} E}{\partial y^{2}} + \frac{\partial ^{2} E}{\partial z^{2}} = \frac{1}{c^{2}} \frac{\partial ^{2} E}{\partial t^{2}} \label{eq:wave-electric}
\end{equation}

The second case is the one where the solution of the PDE is the magnetic induction quantity ($B$).

\begin{equation}
\frac{\partial ^{2} B}{\partial x^{2}} + \frac{\partial ^{2} B}{\partial y^{2}} + \frac{\partial ^{2} B}{\partial z^{2}} = \frac{1}{c^{2}} \frac{\partial ^{2} B}{\partial t^{2}} \label{eq:wave-magnetic}
\end{equation}

Since for both the electric field intensity and for the magnetic induction the equation is the same, the models, algorithms and implementations we develop can be equally understood to apply to both of the implementations. Likewise, the results we derive from our experiments apply to both of them.

\section{Analytical Models}

\subsection{Standing Wave Solution to the Wave Equation}
\label{sec:anal-sol}

We start our analytical model from the 1D case of the wave equation (\ref{1d-wave-eq}) and progressively extend it to the 2D case.

\begin{equation}
\label{1d-wave-eq}
\frac{\partial ^{2} \psi}{\partial x^{2}} = \frac{1}{c^{2}} \frac{\partial ^{2} \psi}{\partial t^{2}}
\end{equation}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.42]{src/img/pic-standing-wave}
\caption{A Standing Wave \footnote{https://spacelights.tumblr.com/post/37429141540/infinity-imagined-standing-waves-a-standing-wave}}
\label{pic-standing-wave}
\end{center}
\end{figure}

Starting from the 1D wave equation (formula \ref{1d-wave-eq}) one can show that function in expression \ref{1d-solution} is a solution for this PDE, where $k$ is defined in relationship to the wave length $\lambda$ or the frequency $f$ and the propagation speed $c$, as shown in formula \ref{k-vector-module}.

This solution is the so called standing wave ~\cite{standing-wave} and assumes that the wave is already propagated in the whole environment, as depicted in figure \ref{pic-standing-wave} \footnote{https://spacelights.tumblr.com/post/37429141540/infinity-imagined-standing-waves-a-standing-wave}.

The proof is simply plugging the solution into the PDE and noticing that it holds.

Similary, since vectors benefit from the same algebric and calculus properties as scalars, one can replace the scalar $x$ in equation \ref{2d-wave-eq} with a displacement vector $\vec{r}$ in a bidimensional space to obtain a bidimensional solution to the wave equation.

Finally, formula \ref{2d-wave-eq} which is in a vectorial form, can be expressed in a scalar form for a planar standing wave by decomposing the vector $\vec{r}$.

\begin{equation}
\label{1d-solution}
\psi(x, t) = A_{max} sin(kx - \omega t) \in Sol \biggr{\{}
\frac{\partial ^{2} \psi}{\partial x^{2}} = \frac{1}{c^{2}} \frac{\partial ^{2} \psi}{\partial t^{2}} \biggr{\}}
, \forall x \in D, \forall t \in T
\end{equation}

\begin{equation}
\label{2d-wave-eq}
\Rightarrow
\psi(\vec{r}, t) = A_{max} sin(\vec{k}\vec{r} - \omega t) \in Sol \biggr{\{}
\frac{\partial ^{2} \psi}{\partial \vec{r} ^{\,\,2}} = \frac{1}{c^{2}} \frac{\partial ^{2} \psi}{\partial t^{2}} \biggr{\}}
, \forall \vec{r} \in D^2, \forall t \in T
\end{equation}

\begin{equation}
\label{2d-wave-eq-scalar}
\Rightarrow
\psi(x, y, t) = A_{max} sin(k_{x} x + k_{y} y - \omega t) \in Sol \biggr{\{}
\frac{\partial ^{2} \psi}{\partial x^{2}} + \frac{\partial ^{2} \psi}{\partial y^{2}}  = \frac{1}{c^{2}} \frac{\partial ^{2} \psi}{\partial t^{2}} \biggr{\}}
, \forall x, y \in D, \forall t \in T
\end{equation}

In expressions \ref{1d-solution}, \ref{2d-wave-eq}, $\vec{k}$ is the so called wave vector ~\cite{standing-wave} which has the direction and the orientation of the wave propagation, while the module is the same as in the 1D case, namely

\begin{equation}
\label{k-vector-module}
|\vec{k}| = k = \frac{2 \pi}{\lambda} = \frac{2 \pi f}{c}
\end{equation}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.45]{src/img/kdesc}
\caption{Vectorial Decomposition of the k Vector}
\label{decomposition-pic}
\end{center}
\end{figure}

For expressions \ref{2d-wave-eq-scalar}, $k_{x}$ and $k_{y}$ can be determined from the definition of trigonometric functions in the right triangle, since they represent the orthogonal decomposition of $\vec{k}$. That is, according to figure \ref{decomposition-pic}:

\begin{equation}
\label{k-decomposition}
\vec{k} = k_x \vec{i} + k_y \vec{j}, \text{ where }
\end{equation}

\begin{equation}
\label{angle-expr-1}
k_{x} = |\vec{k}| cos \alpha
\stackrel{Fig \ref{decomposition-pic}}{=}
|\vec{k}| \frac{|x - x_{0}|} {\sqrt{{(x - x_{0})}^2 + {(y - y_{0})}^2}} \stackrel{\ref{k-vector-module}}{=}
 \frac{2 \pi f}{c} {\frac{|x - x_{0}|} {\sqrt{{(x - x_{0})}^2 + {(y - y_{0})}^2}}}
\end{equation}

\begin{equation}
\label{angle-expr-2}
k_{y} = |\vec{k}| sin \alpha
\stackrel{Fig \ref{decomposition-pic}}{=}
|\vec{k}| \frac{|y - y_{0}|} {\sqrt{{(x - x_{0})}^2 + {(y - y_{0})}^2}}
\stackrel{\ref{k-vector-module}}{=}
\frac{2 \pi f}{c}  {\frac{|y - y_{0}|} {\sqrt{{(x - x_{0})}^2 + {(y - y_{0})}^2}}}
\end{equation}

Furthermore, the frequency of the signal is related to the pulsation $\omega$ via the formula \ref{2d-solution-scalar} \cite{standing-wave}, so the same solution can be expressed in terms of the frequency of the signal as:

\begin{equation}
\label{2d-solution-scalar}
\omega = 2 \pi f
\stackrel{\ref{2d-wave-eq-scalar}}{\Rightarrow}
\psi(x, y, t) = A_{max} sin(k_{x} x + k_{y} y - 2 \pi f t)
\end{equation}

\subsection{Moving Gaussians}

Nonetheless, formula \ref{2d-solution-scalar} has a major defficiency, in the sense that the the respective solution to the PDE is the solution for when the wave is already propagated in the entire points and all the points of the plane can oscilate at any given moment of time (that is, it is a standing wave).

While this is a good approximation, an even more accurate one is to consider a travelling wave in the bidimensional space, starting from the source. This can be mathematically expressed by multiplying (filtering) formula \ref{2d-solution-scalar} with a Gaussian function \cite{gauss} (\ref{2d-gaussian}) in order to keep the behaviour of the wave on a specific zone of the plane only. A graphical representation of such a 2D Gaussian is given in Figure \ref{pic-gaussian} \cite{gauss}.

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.32]{src/img/gaussian-2d}
\caption{A 2D Gaussian ~\cite{gauss}}
\label{pic-gaussian}
\end{center}
\end{figure}
% http://cs229.stanford.edu/section/gaussians.pdf

\begin{equation}
\label{2d-gaussian}
\Gamma(x, y)
\stackrel{\Delta}{=}
A exp\left[-\left(\frac {{(x - x_{0})}^2} {2 \sigma_{x}^{2}}
    + \frac {{(y - y_{0})}^2} {2 \sigma_{y}^{2}}
\right)\right]
\end{equation}

With a notation and fixing the value of variance, formula \ref{2d-gaussian} can be further simplified to:

\begin{equation}
\label{static-gaussian}
\left\{\begin{matrix}
x - x_0
\stackrel{not}{=}
\Delta x
\\
y - y_0
\stackrel{not}{=}
\Delta y
\end{matrix}\right.
\stackrel{\ref{2d-gaussian}}\Rightarrow
\Gamma(x, y) \biggr\rvert_{\biggr{{\sigma_{x}^{2} = \sigma_{y}^{2} = \frac{1}{2}}}} = A exp\left[-\left(\Delta^2 x + \Delta^2 y)
\right)\right]
\end{equation}

To introduce evolution in time, we need to consider that the center of the gaussian, $(x_0, y_0) $, is changing in time (or the mean value of the 2D Gaussian distribution), according to the laws of kinematics ~\cite{kinematics}. More precisely, considering the moving speed $v$, the law of movement with constat velocity gives the position of the point through time and space. This allows us to enhance expression \ref{static-gaussian} with time dependence according to these laws, as shown in \ref{moving-gaussian}.

\begin{equation}
\label{kinematics-formulae}
\left\{\begin{matrix}
x(t) = x_0 + v_x t
\\
y(t) = y_0 + v_y t
\end{matrix}\right.
\stackrel{\ref{static-gaussian}}\Rightarrow
\left\{\begin{matrix}
\Delta x(t) = x - x(t)
\\
\Delta y(t) = y - y(t)
\end{matrix}\right.
\end{equation}

\begin{equation}
\label{moving-gaussian}
\stackrel{\ref{static-gaussian}}\Rightarrow
\Gamma(x, y, t) \biggr\rvert_{\biggr{{\sigma_{x}^{2} = \sigma_{y}^{2} = \frac{1}{2}}}} = A exp\left[-(\Delta^2 x(t) + \Delta^2 y(t))\right]
\end{equation}

Note that in order for our model to behave as observed in experiments, we consider the initial perturbation of the wave to take place at the point $(x_0, y_0)$.

Because the wave is moving with constant velocity $c$, the vectorial velocity components $\vec{v_{x}}$ and $\vec{v_{y}}$ must add up to the wave velocity, namely $\vec{v_{x}} + \vec{v_{y}} = \vec{c}$. The coressponding scalar expression is $v_{x}^2 + v_{y}^2 = c^2$, as deduced using Pithagora's Theorem from Figure \ref{velocity-decomposition}.

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.45]{src/img/vdesc}
\caption{Velocity Vectorial Decomposition}
\label{velocity-decomposition}
\end{center}
\end{figure}

In a similar manner to equations \ref{angle-expr-1} and \ref{angle-expr-2}, and according to figure \ref{velocity-decomposition} the magnitude of the velocity can be computed as in formula \ref{velocity_x} and \ref{velocity_y}.

\begin{equation}
\label{velocity_x}
v_{x} = c \frac{|x - x_{0}|} {\sqrt{{(x - x_{0})}^2 + {(y - y_{0})}^2}}
\end{equation}

\begin{equation}
\label{velocity_y}
v_{y} = c \frac{|y - y_{0}|} {\sqrt{{(x - x_{0})}^2 + {(y - y_{0})}^2}}
\end{equation}

\subsection{Model of Induced Motion for Standing Waves}

Starting from the analytical solution presented in \ref{2d-solution-scalar}, it can be multiplied with the moving filter (equation \ref{moving-gaussian}) to obtain the final form of our propagating, nonmodulated, wireless signal wave.

\begin{equation}
\label{travelling-wave}
\Psi(x, y, t) \biggr\rvert_{\biggr{{\sigma_{x}^{2} = \sigma_{y}^{2} = \frac{1}{2}}}} = \psi(x, y, t)
\Gamma(x, y, t) \biggr\rvert_{\biggr{{\sigma_{x}^{2} = \sigma_{y}^{2} = \frac{1}{2}}}} =
\end{equation}

\begin{equation}
\label{travelling-wave2}
= A exp\left[-((\Delta^2 x(t) + \Delta^2 y(t))\right] sin(k_{x} x + k_{y} y - 2 \pi f t)
\end{equation}

\section{Signal Reflections According to the Law of Reflection and Boundary Conditions for the PDE \cite{synasc}}
\label{sec:dirichlet}

\begin{figure}[h]
\begin{center}
% \includegraphics [scale=0.65]{ref-law2}
\includegraphics [scale=0.42]{src/img/angles2}
\caption{Law of Reflection Graphically Explained}
\label{law-of-reflection}
\end{center}
\end{figure}

In order for the model to be more accurate we also use the Law of Reflection which is outlined in Figure~\ref{law-of-reflection} and obtained from the Dirichlet conditions of the PDE, as explained in \cite{boundary-conditions} and section \ref{sec:dirichlet}. To put it simply, the Law of Reflection states that the incidence angle has the same value as the reflection angle.

\begin{equation}
\sphericalangle \theta_{i} \equiv \sphericalangle \theta_{r}
\end{equation}

In order to integrate reflections of wireless signal into our model we need additional information, apart from expression \ref{eq:wave} and its initial conditions (generation of signal by a source point). Namely, we need to add information about the reflectors on the mesh. In the theory of PDEs this is achieved by fixing the so-called boundary conditions. Out of the many possible behaviours that can be exhibited at boundary, producing precisely the effect of reflection is achieved by using the so called Dirichlet boundary condition ~\cite{boundary-conditions}.

Let $D^3$ be the definition domain for the $(x, y, z)$ tuples in expression \ref{eq:wave} and $\partial D^3 \subset D^3$ its boundary (the points on the edge of the reflectors), then the Dirichlet boundary conditions can be stated as:

\begin{equation}
A(x,y,z,t) = 0, \forall (x,y,z) \in \partial D^3, \forall t \in T
\end{equation}

It is important to take into consideration that the actual Dirichlet boundary conditions produce a mirroring effect - the wave reflected by a reflector is returning with the same ampltiude, while a more accurate model would be the one where the wave is returning with negative amplitude (and same absolute value). This more realistic model is given by the Neumann boundary conditions.

\section{Signal Damping ~\cite{synasc}}

\subsection{Power-Loss Formula and Relationship to Magnitude}

For a wireless signal this simple description proves to be not enough, because it doesn't take into account the attenuation of the signal. In developing the damping model for the amplitude we started from the free path loss model for power attenuation of signal~\cite{rf}.

\begin{equation}
P_{r} = \frac{P_{t}G^{2}\sigma \lambda^{2}}{(4\pi)^3 R^4 L}
\label{Power-Loss Formula}
\end{equation}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.6]{src/img/power-dumping}
\caption{Log-Representation of Power Loss with Space}
\label{power-damp}
\end{center}
\end{figure}

Expression~\ref{Power-Loss Formula} relates the received power $P_{r}$, at a given point in space to the power transmitted $P_{t}$, by the signal source, based on the distance $R$ between them. A graphical power variation with space is represented in Figure~\ref{power-damp}.

In order to relate the amplitude to the power of the wave, the following formula \cite{rf} proved to be useful:

\begin{equation}
P(x, y, t) = \frac{1}{2}\mu \omega ^{2} v A ^ {2} (x, y, t),  \forall (x, y, t) \in D^2 \times T
\label{power-to-amp}
\end{equation}

\subsection{Travelling Waves with Damped Amplitudes from Wireless Signal Strength in a 2D-Space}

A common challenge for wireless communications is that the radio signal quickly vanishes into background noise even for relatively short propagation distances. For this reason, the effect can not be neglected by our model and our simulation.

While the wave equation offers information about the amplitude of the wave, the free path power loss formula offers information about the power of the signal. In order to obtain a model for the amplitude of the damped propagating wave, one needs to relate the two formulae by transforming the power loss into amplitude loss over space.

Simplifing formula ~\ref{power-to-amp} using a notation leads us to a time and space dependent expression for the power of the signal.

Next, let the coordinates be $(x_s, y_s, z_s)$, for the signal source, and $(x_r, y_r, z_r)$ for a receiver on the grid.

In order to maintain the generality of our model, we need to eliminate the proportionality factor which particularizes the model for a given frequency and propagation velocity. We do this by dividing the signal power at transmission against the power at reception. The environment is considered to be isotropic.

This allows us to relate the amplitude of the wave at transmission source against the amplitude at receiver, using a proportionality factor $\chi$, specific to each space coordinate and time moment.

\begin{equation}
\label{power-space-time}
\frac{1}{2}\mu \omega ^{2} v
\stackrel{not}{=}
\zeta
\stackrel{\ref{power-to-amp}}\Rightarrow
P(x,y,z,t) = \zeta A^{2}(x,y,z,t), \forall (x, y, z, t) \in D^3 \times T
\Rightarrow
\end{equation}

\begin{equation}
\label{A-square-expression}
\Rightarrow
\frac{P(x_s,y_s,z_s,t)}{P(x_r,y_r,z_r,t)}=\frac{\zeta A^2(x_s,y_s,z_s,t)}{\zeta A^2(x_r,y_r,z_r,t)}=
\frac{A^2(x_s,y_s,z_s,t)}{A^2(x_r,y_r,z_r,t)}, \forall t \in T
\Rightarrow
\end{equation}

\begin{equation}
\label{get-chi}
\Rightarrow
\frac{A(x_s,y_s,z_s,t)}{A(x_r,y_r,z_r,t)} = \sqrt{\frac{P(x_s,y_s,z_s,t)}{P(x_r,y_r,z_r,t)}}
\stackrel{not}{=}
\frac {1} {\chi(x_r,y_r,z_r,t)}, \forall t \in T
\end{equation}

\begin{equation}
\label{chi-correction}
\Leftrightarrow
A(x_r,y_r,z_r,t) = {A(x_s,y_s,z_s,t)}{\chi(x_r,y_r,z_r,t)}, \forall t \in T
\end{equation}

In other words, we use the free path loss formula for power to obtain the proportionality factor at any space coordinates, and considering the initial amplitude of the wave we can use expression \ref{chi-correction} to compute the amplitude of the damped wave anywhere in space and at any moment of time.

Finally, the damping factor can be integrated in the travelling wave formula.

\begin{equation}
\label{damped-travelling-wave}
\hat\Psi(x, y, t) \biggr\rvert_{\biggr{{\sigma_{x}^{2} = \sigma_{y}^{2} = \frac{1}{2}}}}= \psi(x, y, t)
\chi (x, y, t)
\Gamma(x, y, t) \biggr\rvert_{\biggr{{\sigma_{x}^{2} = \sigma_{y}^{2} = \frac{1}{2}}}}=
\end{equation}

\begin{equation}
\label{damped-travelling-wave2}
= A exp\left[-((\Delta^2 x(t) + \Delta^2 y(t))\right]
\chi (x, y, t)
sin(k_{x} x + k_{y} y - 2 \pi f t)
\end{equation}

\section{Model of Induced Motion for Standing Waves with Modulation}

In order to transmit information through wireless communication, we consider that it is represented by bits.
Each bits needs to be recognised in the form of the wave, and that means that equation \ref{damped-travelling-wave} needs to have 2 different forms, each of them a solution still to the wave equation (so that the laws of physics allows its propagation).

In other words, equation \ref{damped-travelling-wave} is modified such that:

\begin{equation}
\label{damped-modulated-travelling-wave}
\widetilde\Psi(x, y, t)
\biggr\rvert_{\biggr{{\sigma_{x}^{2} = \sigma_{y}^{2} = \frac{1}{2}}}}
= \widetilde\psi(x, y, t)
\chi (x, y, t)
\Gamma(x, y, t) \biggr\rvert_{\biggr{{\sigma_{x}^{2} = \sigma_{y}^{2} = \frac{1}{2}}}}
\end{equation}

The $\widetilde\psi$ function depends on the modulation scheme and is explained in section \ref{sec:psk-fsk}.

\subsection{Particular Cases: Phase and Frequency Modulation}
\label{sec:psk-fsk}

The information which is to be transmitted can be encoded into formula \ref{damped-modulated-travelling-wave} by either embedding it into small variations of the frequency of the signal or into small variations of the phase, among other ways.

For both modulation schemes, let's consider a sequence of bits of length $N$.

In order to introduce the bits into the time dependent solution of the wave equation, we need to map them to a value over a time interval. Let's consider a time domain such that each moment $t \in [0, \tau]$ and consider that a bit is transmitted for $T$ units of time, $0 < T \leq \tau$.

This value of the bit, mapped in ${\{\pm 1\}}$, is then multiplied by a sensitivity factor $\delta > 1$ to obtain a variation of the property of the wave (phase or frequency in our case).

\begin{equation}
\{b_n\}_{N \geq n \geq 1} \\, b_i \in \{0, 1\}
\end{equation}

\begin{equation}
\text{Let } m:  [0, \tau) \xrightarrow{} \{\pm 1\}, m(t) = \left\{\begin{matrix}
+1 \hspace{1cm} , if \hspace{2mm} b_{t \bmod T} = 1
\\
-1 \hspace{1cm} , if \hspace{2mm} b_{t \bmod T} = 0
\end{matrix}\right.
\end{equation}

In the case of Phase Key Modulation (PSK, Fig. \ref{pic-psk} ~\cite{tutorials-point}), the modulation factor is $\phi (t)$,
and the resulting traveling damped wave, modulated with the string of bits $\{b_i\}$ is obtained by adjusting the phase in equation \ref{damped-modulated-travelling-wave} (namely, the argument of the harmonic function) ~\cite{tutorials-point}:

\begin{equation}
\phi (t) = \delta_{p} m(t)
\end{equation}

\begin{equation}
\widetilde \psi(x, y, t)  =
sin(k_{x} x + k_{y} y - 2 \pi f t + \phi (t))
\end{equation}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=1]{src/img/psk}
\caption{Modulating a Carrier Signal Using Phase Key Modulation ~\cite{tutorials-point}}
\label{pic-psk}
\end{center}
\end{figure}
%https://www.tutorialspoint.com/digital_communication/digital_communication_phase_shift_keying.htm

For Frequency Key Modulation (FSK, Fig. \ref{pic-fsk} ~\cite{tutorials-point}), the modulation factor is $\Phi (t)$.
The idea behind FSK is to vary frequency with time, according to the corresponding modulation factor at the given time moment ~\cite{tutorials-point}.
It follows that:

\begin{equation}
\Phi (t) = \delta_{f} m(t)
\end{equation}

\begin{equation}
\label{modified-freq}
\hat f(t) = f + \Phi (t)
\end{equation}

\begin{equation}
\omega (t)
\stackrel{\Delta}{=}
\frac {d \varphi(t)} {dt}
\stackrel {\ref{2d-solution-scalar}} {\Longleftrightarrow}
2 \pi \hat f(t) = \frac {d \varphi (t)}
{dt} \Leftrightarrow
\end{equation}

\begin{equation}
\Leftrightarrow
\varphi (t) = 2 \pi \int \hat f(t) dt
\stackrel{\ref{modified-freq}}{=}
2 \pi \int ((f + \Phi(t)) dt
= 2 \pi f t + 2 \pi \int \Phi (t) dt
\end{equation}

where $\varphi (t)$ is the phase of the carrier signal (the harmonic argument of the nonmodulated signal in  \ref{damped-travelling-wave}).

The harmonic component in \ref{damped-modulated-travelling-wave} becomes:

\begin{equation}
\widetilde \psi(x, y, t)  =
A sin(k_{x} x + k_{y} y - \varphi (t))
\end{equation}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=1]{src/img/fsk}
\caption{Modulating a Carrier Signal Using Frequency Key Modulation ~\cite{tutorials-point}}
\label{pic-fsk}
\end{center}
\end{figure}
%https://www.tutorialspoint.com/digital_communication/digital_communication_frequency_shift_keying.htm

\chapter{Algorithms Based on Models}
\section{Numerical Methods}
\subsection{Role of the Mesh and the Nyquist-Shannon Limit}

In order to use our model in an algorithm, we need to discretize it.
To this aim, the bi-dimensional continuous space becomes a bi-dimensional grid by considering a finitely small space discretization step \(\delta_x\). We also use a smallest time duration possible \(\delta_t\) (the discretization step in time). The space discretization is depicted in Figure \ref{2d-mesh}.

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.38]{src/img/mesh}
\caption{Discretization of the 2D Space}
\label{2d-mesh}
\end{center}
\end{figure}

It is important to remember that wireless communications can be understood as signals in Signal Processing. This affects the choice of discretization in our model in the sense that we need to avoid the effect of aliasing. This can be achieved by taking Shannon-Nyquist Theorem ~\cite{shannon} into consideration. What this means is that sampling frequncy has to be at least twice the frequency of the carrier wave of the wireless signal. Since the maximum possible frequency is the modulated frequency for bit 1, we consider it to be the reference frequency from which we derive the sampling rate. Furthermore, the solution of the wave equation is a periodic function both in space and in time. For this reason, apart from the sampling rate limit, we also need to consider the Nyquist-Shannon limit for space periodicty.

\begin{equation}
 \xLongrightarrow[\text{Theorem}]{\text{Nyquist-Shannon}}
\left\{\begin{matrix}
\delta_t^{-1} \geq 2(f + m(t))
\\
2\delta_x \leq \lambda
\end{matrix}
\right.
\end{equation}

\subsection{Recursive Formula for Wave Discretization ~\cite{synasc}}

One simple way to numerically solve the the PDE equation of waves is to replace derivatives with finite diferences. An example of such a precomputed discretization is provided in~\cite{Trascau}.

The general form of the amplitude gradients on the grid is:

\begin{equation}
{\frac{\partial \psi }{\partial x}}_{i+\frac{1}{2}, j} \approx \frac{\psi_{i+1,j} - \psi_{i, j}}{\delta_x}
\label{first-derivative1}
\end{equation}

\begin{equation}
{\frac{\partial \psi }{\partial x}}_{i-\frac{1}{2}, j} \approx \frac{\psi_{i,j} - \psi_{i-1, j}}{\delta_x}
\end{equation}

\begin{equation}
{\frac{\partial \psi}{\partial x}}_{i, j+\frac{1}{2}} \approx \frac{\psi_{i,j+1} - \psi_{i, j}}{\delta_x}
\end{equation}

\begin{equation}
{\frac{\partial \psi}{\partial x}}_{i, j-\frac{1}{2}} \approx \frac{\psi_{i,j} - \psi_{i, j-1}}{\delta_x}
\label{first-derivative2}
\end{equation}

By differentiating an expression similar to expressions ~\ref{first-derivative1}--\ref{first-derivative2} one more time, we obtain the discrete expression \ref{second-derivative} for the second derivative with respect to time, in which $\psi^t_{i,j}$ denotes the amplitude at point $(i,j)$ of the mesh and at moment $t$.

\begin{equation}
\frac{\partial^{2} \psi }{\partial t^{2}} \approx \frac{\psi^{t+1}_{i, j} - 2 \psi^{t}_{i, j} + \psi^{t-1}_{i, j}}{\delta_t^{2}}
\label{second-derivative}
\end{equation}

Finally, by replacing the formulae in the wave equation we obtain the discrete form, as follows:

\begin{equation}
\frac{\psi^{t+1}_{i,j} - 2 \psi^{t}_{i,j} + \psi^{t-1}_{i,j}}{\delta_t^2} =
\frac{\psi^{t}_{i+1,j} + 2\psi^{t}_{i,j} + \psi^{t}_{i-1,j}}{\delta_x^2} \newline
+ \frac{\psi^{t}_{i, j+1} - 2 \psi^{t}_{i,j} \psi^{t}_{i, j-1}}{\delta_x^2}
\label{eq:to-stabilize}
\end{equation}

\begin{equation}
\stackrel{\ref{stability}}{\Rightarrow}
\psi^{t+1}_{i,j} = 2\psi^{t}_{i,j} - \psi^{t -1}_{i,j} +
\frac{1}{2} ( \psi^{t}_{i+1,j} - 2 \psi^{t}_{i,j} + \psi^{t}_{i-1, j} + \psi^{t}_{i, j + 1} - 2 \psi^{t}_{i,j} + \psi^{t}_{i, j - 1})
\label{eq:recursive}
\end{equation}

We notice that this is a formula which computes the amplitude at any given point in space (discretized in a bidimensional mesh) based on the amplitude of the neighbouring points at two previous time steps. In fact, the formula can be interpreted as a stencil operation which combines points on the mesh from 2 different time stamps to obtain the mesh at a new time moment. This implies that the points at the same time step can be computed in parallel, but have to be serial as time increases. In other words, the stencil kernel can be applied in parallel to generate the new mesh values, but it can't generate multiple new meshes at the same time - this has to happen serially. This influences the design of our algorithms, as it is presented in sections \ref{sec:serial-alg} and \ref{sec:parallel-alg}.

In \cite{Trascau} it is shown that this recursive numerical solution is stable for

\begin{equation}
\label{stability}
{\biggr{(}\frac{\delta_x}{\delta_t}\biggr{)}}^2 \leq \frac{1}{4}
\Rightarrow
\biggr{(}\frac{\delta_x}{\delta_y}\biggr{)} \leq \frac {1}{2}
\end{equation}

\subsection{Discretization of Boundary Conditions \cite{synasc}}

What the Dirichlet boundary ~\cite{boundary-conditions} conditions mean for our numerical solution of the PDE is that expression \ref{eq:recursive} is modified such that in computing the value of $\psi^{t+1}_{i,j}$ any terms for which points $(i,j)$ fall inside the boundary of the reflectors are replaced with 0. Figure \ref{boundary stencil} graphically explains the stencil operation. The reflector is colored in green. For a fixed point (in black) on the boundary, the squares in black, blue and red are the points involved in the stencil operation. The blue points are the points outside of the reflector and the red ones are inside the reflector. Therefore, the red points overwrite the corresponding terms of expression \ref{eq:recursive} with 0 while the blue points don't. The black point is also overwriting the amplitude value with 0.

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.38]{src/img/boundary}
\caption{Boundary Reflections using Modified Stencil Kernel}
\label{boundary stencil}
\end{center}
\end{figure}

For example, for Figure \ref{boundary stencil} the resulting stencil operation is reduced to:

\begin{equation}
\label{reflex-eq}
    \widehat\psi^{t+1}_{i,j} = \frac{1}{2} (\psi^{t}_{i-1,j} + \psi^{t}_{i, j-1})
\end{equation}

\subsection{Serial Algorithm ~\cite{synasc}}
\label{sec:serial-alg}

In order to integrate our propagation model in a simulation, we have first developed a serial algorithm. The serial algorithm serves as a base for both the software parallel implementation, as introduced in the next subsection, and the hardware description, as explained in section \ref{hardware-implementation}.

Before diving into the parallel algorithm, we first explain how we deduced a serial algorithm for simulating the propagation model. To this aim, we iterate over each discrete time step in the time interval during which the wave propagates, and for each such time step, at every point of the discrete space (the grid) we compute the amplitude of the signal as a numerical solution to the wave equation, i.e. by using the Formula~\ref{eq:recursive}.

Since equation~\ref{eq:wave} doesn't offer information about reflection, we use the boundary conditions to adjust the amplitude of the point where appropriate. Up to now, the algorithm simulates the propagation of the signal considering reflections, but not damping. In order to obtain the damping effect, we compute the \(\chi\) proportionality factor and we use it to normalize the previously obtained amplitude of the signal, as shown in Algorithm~\ref{alg:serial}.

The resulting algorithm is part of the class of structured grid algorithms in high performance scientific computing and is graphically explained in figure \ref{pic-alg-ser}.

\begin{algorithm}
\caption{Electromagnetic Wave Propagation with Damping - Serial}
\begin{algorithmic}
\STATE INPUT: $x_s, y_s, A_t, \#\tau, \#l, \#c$
\FOR{$\tau$ in $\#\tau$}
\STATE {$P_t$ \leftarrow $P(x \leftarrow  x_s, y \leftarrow  y_s, t \leftarrow  \tau, A_t)$ (expr. \ref{power-to-amp})}

\FOR{$l$ in $\# l$}
\FOR{$c$ in $\# c$}
\STATE {$A$ \leftarrow $\psi (x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau)$ (expr. \ref{eq:recursive})}
\STATE {$A$ \leftarrow $A$ $\circ \wide\psi (x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau)$ (expr. \ref{reflex-eq})}
\STATE {$P_r$ \leftarrow P_r $(x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau, A)$ (expr. \ref{power-to-amp})}
\STATE {$\chi$ \leftarrow $\chi (x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau, P_r, P_t)$ (expr. \ref{get-chi})}
\STATE {$A$ \leftarrow $\chi$ $(x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau, A_t)$ (expr. \ref{chi-correction})}
\ENDFOR
\ENDFOR
\ENDFOR
\end{algorithmic}
\label{alg:serial}
\end{algorithm}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.38]{src/img/alg}
\caption{Algorithm for Electromagnetic Wave Propagation with Damping - Serial}
\label{pic-alg-ser}
\end{center}
\end{figure}

\subsection{Parallel Algorithm ~\cite{synasc}}
\label{sec:parallel-alg}
In order to parallelize this algorithm, we notice that, for a given jiffy $\tau$, each point of the grid can be computed in parallel, being independent from one another, as they are based only on the past values of the amplitude, according to formula~\ref{eq:recursive}. What this means is that considering a fixed jiffy, the computation of every point of the mesh is independent of the value of the other points at the same time moment.

On the other hand, we need to run our algorithm in a serial manner throughout each individual time step. That happens because formula~\ref{eq:recursive} shows that the amplitudes at each jiffy depend on the amplitudes at the previously two jiffies. This results in a simulation which runs in parallel in space, but serially in time. This leads to the parallel algorithm outlined in Algorithm~\ref{alg:parallel} and graphically depicted in Figure \ref{pic-alg-par}.

We notice that this algorithm is data parallel, since the processing on the whole mesh is similar to the processing on a smaller portion of the mesh and the complete solution is composed of similar partial solutions. From this we can conclude that time dependant PDEs become stencil like operations when serially simulated and, furthermore, data parallel problems when parallelized.

One last aspect we wish to mention is that the computation of transmitted power is not computed in parallel, but serial for each moment of time, since it the same for any $x, y \in D^2$ for the same jiffy $\tau$.

\begin{algorithm}
\caption{Electromagnetic Wave Propagation with Damping - Parallel}
\begin{algorithmic}
\STATE INPUT: $x_s, y_s, A_t, \#\tau, \#l, \#c$
\FOR{$\tau$ in $\#\tau$}
\STATE \textbf{\emph{Execute the loop serially}}
\STATE {$P_t$ \leftarrow $P(x \leftarrow  x_s, y \leftarrow  y_s, t \leftarrow  \tau, A_t)$ (expr. \ref{power-to-amp})}

\FOR{$l$ in $\# l$}
\STATE \textbf{\emph{Execute the loop in parallel}}
\FOR{$c$ in $\# c$}
\STATE \textbf{\emph{Execute the loop in parallel}}
\STATE {$A$ \leftarrow $\psi (x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau)$ (expr. \ref{eq:recursive})}
\STATE {$A$ \leftarrow $A$ $\circ \wide\psi (x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau)$ (expr. \ref{reflex-eq})}
\STATE {$P_r$ \leftarrow P_r $(x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau, A)$ (expr. \ref{power-to-amp})}
\STATE {$\chi$ \leftarrow $\chi (x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau, P_r, P_t)$ (expr. \ref{get-chi})}
\STATE {$A$ \leftarrow $\chi$ $(x \leftarrow  l, y \leftarrow  c, t \leftarrow  \tau,A_t)$ (expr. \ref{chi-correction})}
\ENDFOR
\ENDFOR
\ENDFOR
\end{algorithmic}
\label{alg:parallel}
\end{algorithm}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.38]{src/img/alg-par}
\caption{Algorihm for Electromagnetic Wave Propagation with Damping - Parallel}
\label{pic-alg-par}
\end{center}
\end{figure}

\section{Automatic Differentiation}
\subsection{Theory of Automatic Differentiation} %TODO: FW. REVERSE

Once we have an analytical model for the propagation of wireless signal in 2D, the next step is to evaluate the values that appear in the wave PDE in order to provide better visualization and imaging to the researcher or the developer in the area of wireless networks.

More precisely, we wish to compute the following gradients and plot them on a 2D mesh.

\begin{equation}
\label{gradients}
\frac{\partial ^{2} \widetilde\Psi}{\partial x^{2}} \,,\, \frac{\partial ^{2} \widetilde\Psi}{\partial y^{2}}\,,\, \frac{\partial ^{2} \widetilde\Psi}{\partial t^{2}}
\end{equation}

 The key idea in automatic differentiation is that there are similarities between how differentiation of composed function is performed (using the chain rule, expression \ref{chain-rule}) and how assembly code is generated in compilers ~\cite{ad-explained}.

\begin{equation}
\label{chain-rule}
y : \mathbb{R}^m \to \mathbb{R}^n, u : \mathbb{R}^p \to \mathbb{R}^m, x \in \mathbb{R}^{p}
 \xLongrightarrow[\text{Theorem}]{\text{Chain}}
\frac{\partial(y_{1}, y_{2} ... y_{n})}{\partial x_{i}} = \frac{\partial(y_{1}, y_{2} ... y_{n})}{\partial(u_{1}, u_{2} ... u_{m})} \frac{\partial(u_{1}, u_{2} ... u_{m})}{\partial x_{i}}
\end{equation}

For a composition involving a larger function chain and for functions acting on and returning vectors, applying the chain rule results in a tree-like structure, which is similar to the abstract syntax tree used in compilers, as showin in figure \ref{pic-chain-rule} \footnote{https://towardsdatascience.com/automatic-differentiation-explained-b4ba8e60c2ad}.

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.68]{src/img/ad-explained}
\caption{Tree Like Structure arising from Chain Rule}
\label{pic-chain-rule}
\end{center}
\end{figure}
% https://towardsdatascience.com/automatic-differentiation-explained-b4ba8e60c2ad

As a result, one can write a function in a programming language which returns the output of a complex mathematical function and receives, as paramters, the inputs of the same complex mathematical function. This input source code can then be passed through a special compiler (a generic architecture is available in Fig. \ref{gen-arch} \cite{curs-hpc}) which generates a similar programming language function, but which computes its derivatives instead of its values.

How this happens is that the front-end of the compiler internally generates an abstract syntax tree and then reccursively traverses it. For each non-leaf node, the chain rule is used to assemble the derivative result from the children of that node. In case of a leaf node, the equivalent mathematical expression is a fundamental function with a well known derivative and it is readily computed as such.

The chain rule is associative, meaning that when computing the derivatives of multicomposed functions, the chain rule can either be applied with left grouping or with right grouping, and the result is the same. What this means is that for the code generation from the abstract syntax tree there will be 2 possible composition rules: the forward mode of automatic differentiation and the reverse mode of automatic differentiation. The associativity of the chain rule in presented below.

\begin{equation}
\label{asoc-chain}
\frac
    {\partial(f \circ g \circ h)(x)}
    {\partial x}
=
\frac
    {\partial((f \circ g) (h))(x)}
    {\partial x}
\frac
    {\partial h(x)}
    {\partial x}
=
\frac
    {\partial (f (g \circ h))(x)}
    {\partial x}
\frac
    {\partial (g \circ h)(x)}
    {\partial x}
\end{equation}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.50]{src/img/adcompiler}
\caption{Automatic Differentiation Generic Architecture for Compilation Process}
\label{gen-arch}
\end{center}
\end{figure}
%slideurile profului

\subsection{Advantages Compared to Other Methods}
There are a number of approaches that can be used in order to do this:

\begin{itemize}
  \item Numerical Differentiation
  \item Symbolic Differentiation
  \item Automatic Differentiation
\end{itemize}

Numerical Differentiation implies the use of finite diferences, namely in the definition of the derivative, one approximates it using a minimum step for the limit.

\begin{equation}
\label{finite-dif}
\frac{df(x)}{dx}=\lim_{\epsilon \to 0} \frac{f(x + \epsilon) - f(x)} {\epsilon} \sim \frac {f(x + h) - f(x)} {h}, h \in \mathbb{R}
\end{equation}

This results in a double numerical error: first of one because of the maximum massion precision (the choice of $h$) and the other one because of the division of a finite approximation of real numbers.

On the other hand, symbolic differentiation avoids these errors by using symbolic values instead of actual numeric values. Nonetheless, this approach has the disadvantages that it is inefficient from both the point of view of execution time and of memory usage.

Automatic Differentiation is an approach that draw from the benefits of both numerical and symbolic differenitation while inheriting none of the drawbacks.

\subsection{Forward Mode Algorithm}

If the number of input vector is very small compared to the number of elements in the output vector, one should use the forward mode, meaning right to left association in the chain theorem:

\begin{equation}
\label{asoc-chain-forward}
\frac
    {\partial(f \circ g \circ h)(x)}
    {\partial x}
=
\frac
    {\partial (f (g \circ h))(x)}
    {\partial x}
\frac
    {\partial (g \circ h)(x)}
    {\partial x}
\end{equation}

A visual representation of the forward mode for computing derivative of a function composed of elementary (atomic) functions is shown in Fig. \ref{autodiff-forward} \footnote{https://www.dealii.org/current/doxygen/deal.II/group__auto__symb__diff.html} and the derived algorithm is presented in \ref{alg:fw}.

Since for our model the dimension of the input vector is $3$ and the dimension of the output is $1$, it is not obvious if this rule holds for it.

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.25]{src/img/forward}
\caption{Forward Mode for Automatic Differentiation}
\label{autodiff-forward}
\end{center}
\end{figure}
%https://www.dealii.org/current/doxygen/deal.II/group__auto__symb__diff.html

\begin{algorithm}
\caption{2nd Derivatives for Wave Equation with Damping and Modulation using Automatic Differentiation in Forward Mode}
\begin{algorithmic}
\STATE INPUT: $l, c, \widetilde\Psi (\text{expr.} \ref{damped-modulated-travelling-wave}), \#\tau, \#l, \#c$
\FOR{$\tau$ in $\#\tau$}
\FOR{$l$ in $\# l$}
\FOR{$c$ in $\# c$}
\STATE {$\partial^2 \widetilde\Psi x \leftarrow \text{fw\_diff}(\widetilde\Psi, wrt2(l), at(l,c, \tau))$}
\STATE {$\partial^2 \widetilde\Psi y \leftarrow \text{fw\_diff}(\widetilde\Psi, wrt2(c), at(l,c, \tau))$}
\STATE {$\partial^2 \widetilde\Psi$$\tau$ $\leftarrow \text{fw\_diff}(\widetilde\Psi, wrt2($$\tau$$), at(l,c, \tau))$}

\ENDFOR
\ENDFOR
\ENDFOR
\end{algorithmic}
\label{alg:fw}
\end{algorithm}

\subsection{Reverse Mode Algorithm}

If the number of input vector is very large compared to the number of elements in the output vector, one should use the reverse mode, meaning left to right association in the chain theorem.

A visual representation of the reverse mode for computing derivative of a function composed of elementary (atomic) functions is shown in Fig. \ref{autodiff-reverse} \footnote{https://www.dealii.org/current/doxygen/deal.II/group__auto__symb__diff.html} and the derived algorithm is presented in \ref{alg:rev}.

Again, this rule is difficult to apply to our case since the difference in the dimensions of the input vector and that of the output vector is only $2$ for our model (it is a scalar function of 3 inputs).

\begin{equation}
\label{asoc-chain}
\frac
    {\partial(f \circ g \circ h)(x)}
    {\partial x}
=
\frac
    {\partial((f \circ g) (h))(x)}
    {\partial x}
\frac
    {\partial h(x)}
    {\partial x}
\end{equation}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.25]{src/img/reverse}
\caption{Reverse Mode for Automatic Differentiation}
\label{autodiff-reverse}
\end{center}
\end{figure}
%https://www.dealii.org/current/doxygen/deal.II/group__auto__symb__diff.html

\begin{algorithm}
\caption{2nd Derivatives for Wave Equation with Damping and Modulation using Automatic Differentiation in Reverse Mode}
\begin{algorithmic}
\STATE INPUT: $l, c, \widetilde\Psi (\text{expr.} \ref{damped-modulated-travelling-wave}), \#\tau, \#l, \#c$
\FOR{$\tau$ in $\#\tau$}
\FOR{$l$ in $\# l$}
\FOR{$c$ in $\# c$}

\STATE {[$\partial \widetilde\Psi x,  \partial \widetilde\Psi y, \partial \widetilde\Psi t]\leftarrow \text{fw\_rev}(\widetilde\Psi, at(l,c, \tau))$}
\STATE {[$\partial^2 \widetilde\Psi x,  \partial^2 \widetilde\Psi xy, \partial^2 \widetilde\Psi xt]\leftarrow \text{fw\_rev}(\partial \widetilde\Psi x, at(l,c, \tau))$}
\STATE {[$\partial^2 \widetilde\Psi yx,  \partial^2 \widetilde\Psi y, \partial^2 \widetilde\Psi yt]\leftarrow \text{fw\_rev}(\partial \widetilde\Psi y, at(l,c, \tau))$}
\STATE {[$\partial^2 \widetilde\Psi tx,  \partial^2 \widetilde\Psi ty, \partial^2 \widetilde\Psi t]\leftarrow \text{fw\_rev}(\partial \widetilde\Psi t, at(l,c, \tau))$}

\ENDFOR
\ENDFOR
\ENDFOR
\end{algorithmic}
\label{alg:rev}
\end{algorithm}

\section{Symbolic Computation}

Symbolic computation, also known as Computational Algebra ~\cite{comp-alg}, is a domain found at the intersection between compiler engineering, mathematics and theoretical computer science, which developed in the Golden Age of Artificial Intelligence from the need and the desire to show that machines can perform complex mathematical computations just as humans do.

In order to compute a mathematical expression (reach a desired literal result), the expression is seen as a string of symbols with specific pattern-matching and rewriting rules (the rules involved in computations by hand). The rules are mechanically applied until a desired end condition is met (such as there are no more derivatives, or there are no more algebraic simplifications possible) ~\cite{comp-alg}.

The greatest surprise, in the infancy of this technical specialization, was that many of the algorithms and computation rules used by hand, were non-tractable or nondecidable on computers, and hence a new class of specific computational techniques for symbolic computation had to be developed, in other words new mathematics ~\cite{comp-alg}.

This technique is similar to automatic differentiation, in the sense that an internal abstract representation is necessaire for the expressions and there are composition rules (the mathematical properties and theorems) ~\cite{comp-alg}. Furthermore, the data needs to be parsed simillarly to the front-end of a compiler.

%https://courses.corelab.ntua.gr/pluginfile.php/320/mod_resource/content/0/AlgComp_presentation.pdf

\subsection{Proof of Decidability using Richardson's Theorem}

One important thing to mention about Symbolic Computation is that there are mathematical expressions for which it is proved that there are no algorithms which can decide \cite{Turing} weather that expression evaluates to another (applying rewriting rules and/or heuristics).

More to the point, Richardson's Theorem \cite{Richardson} states that only expressions formed by specific types of atoms can be decided using computational algebra, and luckily computing the 2nd order derivative for our model (expr. \ref{damped-modulated-travelling-wave}) is one of them.

\begin{equation}
\mathbb{E}( \mathbb{Q}, \pi, ln2, x \text{ (a variable)}, +, -, *, \circ, sin, exp, abs) \subset \mathcal{D}
\end{equation}

where $\mathbb{E}(e)$ is the set of problems of symbolically computing expressions formed with only those elements $e$ $\mathcal{D}$ is the set of decidable problems in Turing sense.

\subsection{The Symbolic Algorithm}

Algorithm \ref{alg:sym} presents our approach for symbolic computing of 2nd order derivatives which appear in the wave equation.

The type system which is utilised in this pseudocode is as follows:
\begin{itemize}
    \item $\dagger x$ specify that $x$ is a symbolic form
    \item $\ddagger x$ specify that $x$ is a numeric form
    \item $x\dagger$ specify that $x$ is evaluating to a symbolic form
    \item $x\ddagger$ specify that $x$ is evaluating to a numeric form
\end{itemize}

These imply that $diff\dagger$ is used to compute the symbolic derivative of n-th order for a function and $eval\ddagger$ is used to evaluate a symbolic expression to a specific instance of the symbols it contains.

\begin{algorithm}
\caption{2nd Derivatives for Wave Equation with Damping and Modulation using Symbolic Computation}
\begin{algorithmic}
\STATE INPUT: $\ddagger A_t, \ddagger \sigma_x, \ddagger \sigma_y, \ddagger f, \ddagger c, \ddagger x_s, \ddagger y_s, \ddagger \tau, \ddagger \#l, \ddagger \#c $
\STATE $\dagger \widetilde\Psi \leftarrow \widetilde\Psi\dagger(\dagger x, \dagger y, \dagger t, \dagger A_t, \dagger \sigma_x, \dagger \sigma_y, \dagger f, \dagger c, \dagger x_s, \dagger y_s, \dagger \pi)$ (expr. \ref{damped-modulated-travelling-wave})
\STATE $\dagger \partial^2 \widetilde\Psi x \leftarrow diff\dagger(\dagger \widetilde\Psi, \dagger x, \ddagger2)$
\STATE $\dagger \partial^2 \widetilde\Psi y \leftarrow diff\dagger(\dagger \widetilde\Psi, \dagger y, \ddagger2)$
\STATE $\dagger \partial^2 \widetilde\Psi t \leftarrow diff\dagger(\dagger \widetilde\Psi, \dagger t, \ddagger2)$

\FOR{$\ddagger \tau$ in $\ddagger\#\tau$}

\FOR{$\ddagger l$ in $\ddagger\# l$}
\FOR{$\ddagger c$ in $\ddagger\# c$}
\STATE {$\ddagger \partial^2 \widetilde\Psi x \leftarrow eval\ddagger(\dagger \partial^2 \widetilde\Psi x,
\ddagger A_t, \ddagger \sigma_x, \ddagger \sigma_y, \ddagger f, \ddagger c, \ddagger x_s, \ddagger y_s, \ddagger \pi
)$}
\STATE {$\ddagger \partial^2 \widetilde\Psi y \leftarrow eval\ddagger(\dagger \partial^2 \widetilde\Psi y,
\ddagger A_t, \ddagger \sigma_x, \ddagger \sigma_y, \ddagger f, \ddagger c, \ddagger x_s, \ddagger y_s, \ddagger \pi
)$}
\STATE {$\ddagger \partial^2 \widetilde\Psi t \leftarrow eval\ddagger(\dagger \partial^2 \widetilde\Psi t,
\ddagger A_t, \ddagger \sigma_x, \ddagger \sigma_y, \ddagger f, \ddagger c, \ddagger x_s, \ddagger y_s, \ddagger \pi
)$}

\ENDFOR
\ENDFOR
\ENDFOR
\end{algorithmic}
\label{alg:sym}
\end{algorithm}

\chapter{Implementations (\& Results) Based on Algorithms}

The code for our experiments is available in \href{https://github.com/dorin-ionita/wireless-models-experiments}{our git repository}.

\section{Visual Result of the Model ~\cite{synasc}}

With respect to the model of wave propagation, a graphical representation without considering damping is shown in Figure~\ref{propag-no-dump} ~\cite{synasc}. The graphical results of the simulation which takes into account the damping of the wave are shown in the Figure~\ref{propag-damp} ~\cite{synasc}. Each simulation was run for $30$ seconds physical time, with a time-step of $0.05$ seconds, totaling $600$ iterations.

The input of our experimental programs were custom-format files, containing size of the mesh, discretisation step in space and discretization step in time, position and shape of reflectors.

The output consisted of Paraview-specific format files which were then intrepreted by Paraview ~\cite{paraview}.

\begin{figure}[h]
\begin{center}
\includegraphics [scale=1]{src/img/no-dumping}
\caption{Wave Propagation with no Amplitude Damping}
\label{propag-no-dump}
\end{center}
\end{figure}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=1]{src/img/dumping}
\caption{Wave Propagation with Amplitude Damping}
\label{propag-damp}
\end{center}
\end{figure}

\section{Experimental Environment \cite{synasc}}

The used experimental setup included two execution queues on the University Politehnica of Bucharest's cluster. The model without damping was run on a queue with 12 Intel Xeon cores running 24 threads at 2.67GHz and 32G RAM, while the model which includes the damping factor was run on a queue with 20 Intel Xeon cores with 40 threads running at 2.5GHz, 62G RAM. The CUDA version was tested on a Tesla K40M NVidia GPU, using API version $9$.

For our embedded FPGA experiments we used Zedboard from Xilinx, which incorporate a Zynq 7000 SoC . As part of the SoC, an ARM A9 CPU runs the bare metal drivers for the FPGA design, starting the execution and receiving the results. The base clock of the FPGA runs at 100MHz, but we had to lower it to 95MHz in order to achieve timing closure. Worth noting is that the FPGA of Zynq 7000 benefits from 36Kb internal BRAM.

For time measurements we have used the command time command in Linux and Valgrind ~\cite{valgrind} with the tool massif mode.

\begin{lstlisting}
time ./<compiled_program>
valgrind --tool=massif ./<compiled_program>
ms_print massif* | head
\end{lstlisting}

The propagation model without damping was implemented in OpenMP, TBB and Rust, while the model including damping was implemented in TBB, CUDA and HLS. The model including propagation, damping and modulation of bits was implemented using symboling computation (the GiNaC symbolic computing engine for C++) and automatic differentiation (the autodiff library for C++).

The measurement of power consumption is actually an estimation provided by Vivado.
And time measurement in FPGA were obtained using hardware timers as Intellectual Property (IPs) modules in Vivado.

For profiling and optimization we used Vivado HLS, gdb, Sun Studio, perf and nvprof.

\section{Parallel Implementations (\& Results) of the Numerical Algorithm ~\cite{synasc} ~\cite{filip} ~\cite{Trascau}}

With respect to parallel implementations and comparisons, we implemented the previous parallel algorithm in a number of different parallel programming paradigms, namely OpenMP, TBB, Rust and NVidia CUDA. The first technology which has been employed was OpenMP due to its increased productivity obtained by abstracting many low-level details from the programmer.

TBB was chosen to allow for more control than OpenMP. During our experiments we noticed that the productivity overhead of TBB is rather small when compared to the productivity of OpenMP. More to the point, about the same amount of time was invested in the TBB and OpenMP implementations.

Our third choice was Rust, as it emerged as a new programming language with the promise of easy and safe parallelization of code. Rust offers two paradigms for parallelization: one for shared memory and one for message passing, based on channels. The channels connect threads on the same machine so no special packing is required and therefore no additional overhead is incurred.

Finally, GP-GPU architectures have established themselves as one of the most important technologies for data parallel applications. Since NVidia CUDA is one of the most efficient and wide-spread libraries, we chose to port our code to this technology as well.

Figure~\ref{exec-mix} \cite{synasc} \cite{filip} presents the execution time for implementations which do not consider the damping of the wave. When it comes to the parallelization, TBB produces the binary with the shortest execution time, Rust and OpenMP being almost twice as slow. We noticed that while Rust seems slower than OpenMP and TBB in the serial versions, the difference between Rust and OpenMP shrinks significantly when multiple threads are used. Combining this with the deadlock and race conditions protection provided by Rust's type system, we consider that for data parallel problems one might easily choose Rust over OpenMP. This is particularly relevant when requiring better safety guarantees. TBB provides a significant improvement over OpenMP in terms of execution time. However, this incurs a small productivity penalty.

\begin{figure}
\centering
\begin{tikzpicture}
\begin{axis}[
	align =center,
    xlabel={Number of Threads},
    ylabel={Time (s)},
    xmin=1, xmax=16,
    ymin=0, ymax=150,
    xtick={1,2,4,6,8,10,12,14,16},
    ytick={0,25,50,75,100,125,150},
    legend pos=north est,
    ymajorgrids=true,
    grid style=dashed,
]

\addplot[
    color=orange,
    mark=square,
    ]
    coordinates {
    (1,130.625)(2,68.5)(4,37.75)(8,26.625)(10,26.25)(12,24.25)(14,23.75)(16,23.125)
    };
    \addlegendentry{Rust}

\addplot[
    color=blue,
    mark=diamond,
    ]
    coordinates {
    (1,73.308)(2,41.145)(4,25.254)(8,20.966)(10,20.135)(12,18.944)(14,17.735)(16,16.752)
    };
    \addlegendentry{OpenMP}

\addplot[
    color=red,
    mark=*,
    ]
    coordinates {
    (1,40.134)(2,20.304)(11.108)(8,6.69)(10,5.596)(12,5.105)(14,5.039)(16,5.032)
    };
    \addlegendentry{TBB}

\end{axis}
\end{tikzpicture}

\caption{Rust vs. OpenMP vs. TBB Execution Time without Damping}
\label{exec-mix}
\end{figure}

Figure~\ref{exec-tbb} presents the execution time for the TBB implementation when amplitude damping is taken intro consideration. The dimension of the grid in this case is $2048x2048$, the same as in the CUDA case. The execution speed for $24$ threads is $8.418s$.

\begin{figure}
\centering
\begin{tikzpicture}
\begin{axis}[
	align =center,
    % title={TBB Execution Time\\with Damping},
    xlabel={Number of Threads},
    ylabel={Time (s)},
    xmin=1, xmax=40,
    ymin=0, ymax=200,
    xtick={1,4,8,12,16,20,24,28,32,36,40},
    ytick={8.418,25,50,75,100,125,150,175,200},
    legend pos=north est,
    ymajorgrids=true,
    grid style=dashed,
]

\addplot[
    color=red,
    mark=*,
    ]
    coordinates {
    (1,157.849)(2,75.493)(4,38.243)(8,20.768)(10,18.903)(12,15.290)(14,13.702)(16,13.405)(18,11.579)(20,11.196)(22,10.641)(24,10.640)(26,9.999)(28,9.592)(30,9.571)(32,9.375)(34,9.205)(36,9.216)(38,8.620)(40,8.418)
    };
    \addlegendentry{TBB}

\end{axis}
\end{tikzpicture}

\caption{TBB Execution Time with Damping}
\label{exec-tbb}

% \end{center}
\end{figure}

Table~\ref{Table-I} shows the execution time for the CUDA implementation, when the input is the same as for Figure~\ref{exec-tbb} ~\cite{synasc} ~\cite{filip}. The performance of CUDA seems to vary greatly with the dimension of the kernel, with the best choice for dimension not being straightforward. In fact, what we did was to iterate through the space of hyper-parameters (possible kernel dimensions) to find the best fit for our simulation. All the kernels use bi-dimensional square grids and bi-dimensional square blocks. We can see that the best execution time is is $7.44s$, so it is about $11$\% better than TBB when it comes to execution speed. Also, Table~\ref{Table-I} outlines that symmetric sizes of kernels (i.e. obtained by swapping the grid size with the block size) exhibit similar execution times.

\begin{table}
\normalsize
\centering
\begin{tabular}{|l||*{3}{c|}}\hline
\backslashbox{Grid}{Block}
&\makebox[3em]{4}&\makebox[3em]{8}&\makebox[3em]{16}\\\hline\hline\xrowht{10pt}
\hfil4 & \hfil\ 103.12 & \hfil 7.79 & \hfil 41.14\\\hline\xrowht{10pt}
\hfil8 & \hfil\ 7.44 & \hfil 31.45 & \hfil 135.84\\\hline\xrowht{10pt}
\hfil16 & \hfil\ 27.21 & \hfil 149.22 & \hfil 592.19\\\hline
\end{tabular}
\caption{\label{tab:table-name}CUDA Execution Time with Damping (s)}
\label{Table-I}
\end{table}

In the case of the model without amplitude damping, the speed-up outlined in Figure~\ref{mix-speed-up} ~\cite{synasc} ~\cite{filip} shows that TBB is better than both OpenMP and Rust, with Rust offering better speed-up than OpenMP. One should note that for all considered programming paradigms the speed-up is flat-lining at $12$ threads, which is to be expected on a system with $12$ CPU physical cores running a CPU-intensive numerical simulation. Furthermore, Figure~\ref{speed-up-tbb} ~\cite{synasc} ~\cite{filip} and Table~\ref{cuda-speed-up} outline that the maximum speed-up achieved using a K40m with $2880$ NVidia CUDA cores is only slightly better than using TBB on a $20$ core Intel Xeon machine, i.e. $19.90$ vs. $18.31$.

\begin{figure}
\centering
\begin{tikzpicture}
\begin{axis}[
	align =center,
    % title={Speed-up of Rust, TBB and OpenMP\\without Damping},
    xlabel={Number of Threads},
    ylabel={Speed-up},
    xmin=1, xmax=16,
    ymin=1, ymax=8,
    xtick={1,2,4,6,8,10,12,14,16},
    ytick={1,2,3,4,5,6,7,8},
    legend pos=north west,
    ymajorgrids=true,
    grid style=dashed,
]

\addplot[
    color=red,
    mark=*,
    ]
    coordinates {
    (1,1)(2,1.97)(4,3.61)(8,5.99)(10,7.17)(12,7.86)(14,7.96)(16,7.97)
    };
    \addlegendentry{TBB}

\addplot[
    color=orange,
    mark=square,
    ]
    coordinates {
    (1,1)(2,1.90)(4,3.46)(8,4.90)(10,4.97)(12,5.38)(14,5.5)(16,5.64)
    };
    \addlegendentry{Rust}

\addplot[
    color=blue,
    mark=diamond,
    ]
    coordinates {
    (1,1)(2,1.78)(4,2.90)(8,3.49)(10,3.64)(12,3.86)(14,4.13)(16,4.37)
    };
    \addlegendentry{OpenMP}

\end{axis}
\end{tikzpicture}

\caption{Rust vs. OpenMP vs. TBB Speed-up without Damping}
\label{mix-speed-up}
\end{figure}

\begin{figure}
\centering
\begin{tikzpicture}
\begin{axis}[
	align =center,
    % title={Speedup TBB\\with Damping},
    xlabel={Number of Threads},
    ylabel={Speed-up},
    xmin=1, xmax=40,
    ymin=1, ymax=20,
    xtick={1,4,8,12,16,20,24,28,32,36,40},
    ytick={1,2,4,6,8,10,12,14,16,18,20},
    legend pos=north west,
    ymajorgrids=true,
    grid style=dashed,
]

\addplot[
    color=red,
    mark=*,
    ]
    coordinates {
    (1,1)(2,2.09)(4,4.12)(8,7.60)(10,8.35)(12,10.32)(14,11.52)(16,11.77)(18,13.63)(20,14.09)(22,14.834)(24,14.835)(26,15.786)(28,16.456)(30,16.492)(32,16.837)(34,17.148)(36,17.128)(38,18.312)(40,17.854)
    };
    \addlegendentry{TBB}

\end{axis}
\end{tikzpicture}

\caption{Speed-up TBB with Damping}
\label{speed-up-tbb}

\end{figure}

\begin{table}
\normalsize
\centering
\begin{tabular}{|l||*{3}{c|}}\hline
\backslashbox{Grid}{Block}
&\makebox[3em]{4}&\makebox[3em]{8}&\makebox[3em]{16}\\\hline\hline\xrowht{10pt}
\hfil4 & \hfil\ 1.43 & \hfil 19.01 & \hfil 4.10\\\hline\xrowht{10pt}
\hfil8 & \hfil\ 19.90 & \hfil 4.70 & \hfil 1.09\\\hline\xrowht{10pt}
\hfil16 & \hfil\ 5.44 & \hfil 0.99 & \hfil 0.25\\\hline
\end{tabular}
\caption{\label{tab:table-name}CUDA Speed-up}
\label{cuda-speed-up}
\end{table}

When it comes to computing efficiency, Figure~\ref{mix-ef} ~\cite{synasc} ~\cite{filip} shows the TBB performs best, with Rust and OpenMP performing significantly worse, in this domain.

\begin{figure}
\centering
\begin{tikzpicture}
\begin{axis}[
	align =center,
    % title={Efficiency of Rust, TBB and OpenMP\\without Damping},
    xlabel={Number of Threads},
    ylabel={Efficiency},
    xmin=1, xmax=16,
    ymin=0, ymax=1,
    xtick={1,2,4,6,8,10,12,14,16},
    ytick={0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1},
    legend pos=south west,
    ymajorgrids=true,
    grid style=dashed,
]

\addplot[
    color=red,
    mark=*,
    ]
    coordinates {
    (1,1)(2,0.98)(4,0.90)(8,0.74)(10,0.71)(12,0.65)(14,0.56)(16,0.49)
    };
    \addlegendentry{TBB}

\addplot[
    color=orange,
    mark=square,
    ]
    coordinates {
    (1,1)(2,0.95)(4,0.86)(8,0.61)(10,0.49)(12,0.44)(14,0.39)(16,0.35)
    };
    \addlegendentry{Rust}

\addplot[
    color=blue,
    mark=diamond,
    ]
    coordinates {
    (1,1)(2,0.89)(4,0.72)(8,0.43)(10,0.36)(12,0.32)(14,0.29)(16,0.27)
    };
    \addlegendentry{OpenMP}

\end{axis}
\end{tikzpicture}

\caption{Efficiency of Rust, TBB and OpenMP without Damping}
\label{mix-ef}

\end{figure}

\section{Hardware Implementation (\& Results) of the Numerical Algorithm ~\cite{synasc}}
\label{hardware-implementation}

\begin{figure}[h]
\begin{center}
\includegraphics [scale=0.7]{src/img/design9}
\caption{FPGA - CPU Integration}
\label{fpga-cpu-integration}
\end{center}
\end{figure}

Apart from using parallel programming paradigms to implement our algorithm, we also obtained a hardware description using the HLS mechanism which we then used to program our FPGA. The motivation behind this choice is that data parallel problems, such as the structured grid problem we used as example, are known to be well-suited for running on GPUs, but on the other hand FPGAs allow for close coupling of the hardware with the problem it solves. As a result of this, the abstractions stack on top of an FPGA are more shallow than on general computing devices such as CPUs and GPUs, resulting in less overhead. Therefore, both GPUs and FPGAs promise good result, but more clarity was needed in order to decide the trade-offs between these hardware architectures for the structured grid pattern.

In order to describe the hardware that implements our model, the three nested loops in Algorithm~\ref{alg:serial} were implemented in the C allowing for the translation of Vivado HLS to VHDL . The pipelining and hardware loop unrolling behaviour is emplying default settings, as we provided no manual directives to the compiler. The I/O ports allowing intercommunication between the IP implemented in FPGA and the CPU inside the SoC (System on Chip) were specified. The resulting IP was integrated with the CPU inside the SoC using an approach specific to AMBA, as shown in Fig. \ref{fpga-cpu-integration}.

The Processing System and the Programmable Logic are connected via a double master-slave interface using AXI Interconnect. Namely, the result port of the IP (Intellectual Property) plays the role of a slave to the processor, which, in this case, is the master. On the other hand, the control bus is a master to the Processing System. Furthermore, the interrupt line of the generated IP is connected to the interrupt line of the CPU.

For reset-control capabilities, a Processor System Reset IP from Vivado is used, all the components being connected in a single-clock domain infrastructure, provided by the Processing System and set to a frequency that allows for timing closure.

\begin{table}
\normalsize
\centering
\begin{tabular}{|l||*{3}{c|}}\hline
%\backslashbox{Grid}{Block}
\makebox{Resource}
&\makebox[5em]{Utilization}&\makebox[4em]{Available}&\makebox[5em]{Utilization \%}\\\hline\hline\xrowht{10pt}
\hfil LUT & \hfil\ 21710 & \hfil 53200 & \hfil 40.81\\\hline\xrowht{10pt}
\hfil LUTRAM &\hfil\ 764 & \hfil 17400 & \hfil 4.39\\\hline\xrowht{10pt}
\hfil FF &\hfil\ 16457 & \hfil 106400 & \hfil 15.47\\\hline\xrowht{10pt}
\hfil BRAM &\hfil\ 99 & \hfil 140 & \hfil 70.71\\\hline\xrowht{10pt}
\hfil DSP &\hfil\ 70 & \hfil 220 & \hfil 31.82\\\hline\xrowht{10pt}
\hfil BUFG &\hfil\ 1 & \hfil 32 & \hfil 3.13\\\hline
\end{tabular}
\caption{\label{tab:table-name}FPGA Consumed Resources}
\label{fpga-resources}
\end{table}

With respect to results, first thing we noticed is that memorising three bi-dimensional meshes inside the FPGA is the main bottleneck. This happens because the three matrices, representing the meshes, are bound to the BRAM cells. Since the BRAM size is limited, it is quickly exhausted. This means that we had to reduce the meshes to $100x100$. Consequently, required hardware resources are given in Table~\ref{fpga-resources}.

In fact, we consider that the tiny size of the internal memory of the FPGA is the main limitation against efficiently implementing our algorithm on this platform, since in order to execute the algorithm on larger meshes we need to run data through the FPGA multiple times, in a serial fashion. We consider that this is also the main advantage of the GPU, when dealing with data parallel problems: the fact that they have a large amount of internal memory. For example, the used GPU has $12GB$.

When comparing to the C serial version (which was modified to work with HLS), we notice that for the software implementation the execution time is $1.9ms$, while for the hardware one it is $466.71\mu s$. This translates to a speed-up of $4.07$.

With respect to power consumption, we notice Xilinx Vivado estimates it to be $2.269W$, which is only 0.96\% of the $235W$ consumed by Nvidia Kepler K40M, considering that our design runs at 12\% the frequency of the GPU ($95MHz$ instead of $745MHz$).

\section{Implementations (\& Results) of the Symbolic and Automatic Differentiation Algorithms}

We implemented our algorithms in C++, using the GINASC ~\cite{ginasc} symbolic computing engine and the autodiff ~\cite{autodiff-tool} automatic differentiation library for C++. The results in table \ref{ad-vs-sym} clearly show that automatic differentiation can be more efficient than symbolic computation , both in terms of processing power and memory usage, but the performance margin is rather more difficult to control than that of symbolic execution, and can easily turn worse than it. The impelementations were only serial and included wave propagation, reflections, damping and modulation of information.

On one hand, the disadvantage of automatic differentiation seems to be that choosing the right mode of execution and the right implementation of the technique is not trivial, as shown by the large differences between the performance of the forward mode and that of the reverse mode.

On the other hand, the disadvantage of symbolic computation is that due to the heuristic nature of symbolic computation (as it was initially a tool used only for hand computation), the expressions can easily become overly complicated and feel unnatural to the human eye. This is a known phenomenon and is named "expression swallowing" in literature. One such example for the ${\partial^2 \widehat \Psi} / {\partial t^2}$ derivative can be found in the listing below.

\begin{lstlisting}
-4*sin(2*_pi_*v*x*(x-x0)*((x-x0)^2+(y-y0)^2)^(-1/2)*f+2*_pi_*v*(x-x0)*((x-x0)^2+(y-y0)^2)^(-1/2)*y*f-2*_pi_*f*t)*_pi_^2*A*exp(-_sigma_^(-2)*((x-v*(x-x0)*((x-x0)^2+(y-y0)^2)^(-1/2))^2+(v*((x-x0)^2+(y-y0)^2)^(-1/2)*(y-y0)-y)^2))*f^2
\end{lstlisting}

\begin{table}
\normalsize
\centering
\begin{tabular}{|l||*{2}{c|}}\hline
%\backslashbox{Grid}{Block}
\makebox{Computation Technique}&\makebox{Memory (KB)}&\makebox{CPU time (s)}&\hline
\hfil Automatic Differentiation - Forward &\hfil\ 71 & \hfil 0.05 &\hline
\hfil Automatic Differentiation - Reverse &\hfil\ 143 000 & \hfil 9.5 &\hline
\hfil Symbolic Computation &\hfil\ 360 & \hfil 2.56 &\hline
\end{tabular}
\caption{\label{tab:table-name}Performance of Automatic Differentiation Compared to Symbolic Computation}
\label{ad-vs-sym}
\end{table}

\chapter{Insights \& Conclusion}

To conclude, we provide a model for the physical level of wireless communication, which bridges the gap between the traditional, telecommunications perspective over wireless communications, and the perspective of physics and scientific computing.

Since other models, even in other areas of engineering and science can be designed similarly to ours, starting from the underlying partial differential equation, we considered our model an instance of such a class of models, and therefore compared the performance of various algorithms derived from the model, implemented using multiple computing techniques (numeric, symbolic, automatic differentiation) and various programming and hardware description paradigms, on heteregeneous architectures.

More to the point, we've shown that when comparing parallel  implementations  in  OpenMP,  Rust, CUDA,  and  TBB,  OpenMP  and  Rust  exhibit similar  performance  and  that  the  best  performance  overall  is achieved by CUDA. We also compared the performance of a hardware implementation using HLS against the  serial  version  in  C  and  against  CUDA,  thus  outlining the  trade-offs  between  the  GPU  and  FPGA  platforms  for this use-case. Each tested paradigm has certain particularities which makes them better suited for particular use-cases, such as  safety  concerns,  rapid  development  needs,  performance, power efficiency, etc ~\cite{synasc}. On the computational side, automatic differentiation proves to have better performance than symbolic computation, but is more difficult to tune. We've achieved the aim of our project - to provide insight to users and application developers, helping them decide which parallel and mathematical computation paradigm to use, having specific performance or resource requirements.

% \textbf{This is just a demo file. It should not be used as a sample for a thesis.}\\
% \todo{Remove this line (this is a TODO)}

% \section{Project Description}
% \label{sec:proj}

% \subsection{Project Scope}
% \label{sub-sec:proj-scope}

% This thesis presents the \textbf{\project}.

% This is an example of a footnote \footnote{\url{www.google.com}}. You can see here a reference to \labelindexref{Section}{sub-sec:proj-objectives}.

% Here we have defined the CS abbreviation.\abbrev{CS}{Computer Science} and the UPB abbreviation.\abbrev{UPB}{University Politehnica of Bucharest}

% The main scope of this project is to qualify xLuna for use in critical systems.

% Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean aliquam lectus vel orci malesuada accumsan. Sed lacinia egestas tortor, eget tristiqu dolor congue sit amet. Curabitur ut nisl a nisi consequat mollis sit amet quis nisl. Vestibulum hendrerit velit at odio sodales pretium. Nam quis tortor sed ante varius sodales. Etiam lacus arcu, placerat sed laoreet a, facilisis sed nunc. Nam gravida fringilla ligula, eu congue lorem feugiat eu.

% Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean aliquam lectus vel orci malesuada accumsan. Sed lacinia egestas tortor, eget tristiqu dolor congue sit amet. Curabitur ut nisl a nisi consequat mollis sit amet quis nisl. Vestibulum hendrerit velit at odio sodales pretium. Nam quis tortor sed ante varius sodales. Etiam lacus arcu, placerat sed laoreet a, facilisis sed nunc. Nam gravida fringilla ligula, eu congue lorem feugiat eu.

% \subsection{Project Objectives}
% \label{sub-sec:proj-objectives}

% We have now included \labelindexref{Figure}{img:report-framework}.

% \fig[scale=0.5]{src/img/reporting-framework.pdf}{img:report-framework}{Reporting Framework}

% Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean aliquam lectus vel orci malesuada accumsan. Sed lacinia egestas tortor, eget tristiqu dolor congue sit amet. Curabitur ut nisl a nisi consequat mollis sit amet quis nisl. Vestibulum hendrerit velit at odio sodales pretium. Nam quis tortor sed ante varius sodales. Etiam lacus arcu, placerat sed laoreet a, facilisis sed nunc. Nam gravida fringilla ligula, eu congue lorem feugiat eu.

% We can also have citations like \cite{iso-odf}.

% \subsection{Related Work}

% Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean aliquam lectus vel orci malesuada accumsan. Sed lacinia egestas tortor, eget tristiqu dolor congue sit amet. Curabitur ut nisl a nisi consequat mollis sit amet quis nisl. Vestibulum hendrerit velit at odio sodales pretium. Nam quis tortor sed ante varius sodales. Etiam lacus arcu, placerat sed laoreet a, facilisis sed nunc. Nam gravida fringilla ligula, eu congue lorem feugiat eu.

% Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean aliquam lectus vel orci malesuada accumsan. Sed lacinia egestas tortor, eget tristiqu dolor congue sit amet. Curabitur ut nisl a nisi consequat mollis sit amet quis nisl. Vestibulum hendrerit velit at odio sodales pretium. Nam quis tortor sed ante varius sodales. Etiam lacus arcu, placerat sed laoreet a, facilisis sed nunc. Nam gravida fringilla ligula, eu congue lorem feugiat eu.

% Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean aliquam lectus vel orci malesuada accumsan. Sed lacinia egestas tortor, eget tristiqu dolor congue sit amet. Curabitur ut nisl a nisi consequat mollis sit amet quis nisl. Vestibulum hendrerit velit at odio sodales pretium. Nam quis tortor sed ante varius sodales. Etiam lacus arcu, placerat sed laoreet a, facilisis sed nunc. Nam gravida fringilla ligula, eu congue lorem feugiat eu.

% We are now discussing the \textbf{Ultimate answer to all knowledge}.
% This line is particularly important it also adds an index entry for \textit{Ultimate answer to all knowledge}.\index{Ultimate answer to all knowledge}

% \subsection{Demo listings}

% We can also include listings like the following:

% % Inline Listing example
% \lstset{language=make,caption=Application Makefile,label=lst:app-make}
% \begin{lstlisting}
% CSRCS = app.c
% SRC_DIR =..
% include $(SRC_DIR)/config/application.cfg
% \end{lstlisting}

% Listings can also be referenced. References don't have to include chapter/table/figure numbers... so we can have hyperlinks \labelref{like this}{lst:makefile-test}.

% \subsection{Tables}

% We can also have tables... like \labelindexref{Table}{table:reports}.

% \begin{center}
% \begin{table}[htb]
%   \caption{Generated reports - associated Makefile targets and scripts}
%   \begin{tabular}{l*{6}{c}r}
%     Generated report & Makefile target & Script \\
%     \hline
%     Full Test Specification & full_spec & generate_all_spec.py  \\
%     Test Report & test_report & generate_report.py  \\
%     Requirements Coverage & requirements_coverage &
%     generate_requirements_coverage.py   \\
%     API Coverage & api_coverage & generate_api_coverage.py  \\
%   \end{tabular}
%   \label{table:reports}
% \end{table}
% \end{center}


\documentclass[12pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage[left=2.00cm, right=2.00cm, top=2.00cm, bottom=2.00cm]{geometry}

\author{Björn Müller}

\renewcommand{\vec}[1]{\boldsymbol{#1}}
\renewcommand{\matrix}[1]{\boldsymbol{#1}}
\newcommand{\T}[1]{#1^\text{T}}
\newcommand{\dif}[1]{d\mkern-1mu#1}
\newcommand{\diff}[1]{\,\dif{#1}}

\newcommand{\coupling}{\matrix{A}}

\newcommand{\cellTar}{K_1}
\newcommand{\cellSrc}{K_2}
\newcommand{\cellAgg}{K_\text{agg}}

\newcommand{\basisSrc}{\vec{\Phi}_2}
\newcommand{\basisTar}{\vec{\Phi}_1}
\newcommand{\basisAgg}{\vec{\Phi}_\text{agg}}

\newcommand{\solTar}{u_1}
\newcommand{\solSrc}{u_2}
\newcommand{\solAgg}{u_\text{agg}}

\newcommand{\solvecTar}{\vec{u}_1}
\newcommand{\solvecSrc}{\vec{u}_2}
\newcommand{\solvecAgg}{\vec{u}_\text{agg}}

\newcommand{\massTar}{\matrix{M}_1}
\newcommand{\massSrc}{\matrix{M}_2}
\newcommand{\massAgg}{\matrix{M}_\text{agg}}

\begin{document}

\section{Coupling matrix}

Assume that cell $\cellSrc{}$ (source cell) shall be agglomerated to cell $\cellTar{}$ (target cell) to form the agglomerated cell $\cellAgg$. Let $\basisTar$ and $\basisSrc$ be the row vectors of the orthonormal basis functions of $\cellTar$ and $\cellSrc$, respectively. Note that these functions are non-zero in the corresponding cells only. Then the coupling matrix $\matrix{A}$ is chosen such that $\basisSrc \coupling$ is the smooth extension of $\basisTar$ into $\cellSrc$. The basis of the agglomerated cell is then defined as
\begin{equation}
	\basisAgg = \basisTar + \basisSrc \matrix{A}.
\end{equation}

\section{Agglomeration of uncut cell ('ManipulateRHS')}

Let $\solTar = \basisTar \solvecTar$ and $\solSrc = \basisSrc \solvecSrc$ be the current approximations in cell $\cellTar$ and $\cellSrc$, respectively, where $\solvecTar$ and $\solvecSrc$ are column vectors. The agglomerated DOF $\solAgg = \basisAgg \solvecAgg$ can then be computed from
\begin{align}
	(\solvecAgg)_i
	&= \int \limits_{\cellTar} \solTar (\basisTar)_i \diff{V} + \int \limits_{\cellSrc} \solSrc (\basisSrc \coupling))_i \diff{V}\\
	&= (\solvecTar)_j \int \limits_{\cellTar} (\basisTar{})_j (\basisTar)_i \diff{V} + (\solvecSrc)_j \int \limits_{\cellSrc} (\basisSrc{})_j (\basisSrc \coupling)_i \diff{V}\\
	&= (\solvecTar)_j \matrix{I}_{j i} + (\solvecSrc)_j \int \limits_{\cellSrc} (\basisSrc{})_j (\basisSrc)_k (\coupling)_{k i} \diff{V}\\
	&= (\solvecTar)_i + (\solvecSrc)_j (\matrix{I})_{j k} (\coupling)_{k i}\\
	&= (\solvecTar)_i + (\solvecSrc)_j (\coupling)_{j i}\\
	&= (\solvecTar)_i + (\T{\coupling})_{i j} (\solvecSrc)_j
\end{align}
which can also be expressed as
\begin{equation}
	\label{eqn:manipulate_plain}
	\solvecAgg = \solvecTar + \T{\coupling} \solvecSrc.
\end{equation}

\section{Agglomeration of cut cell}

If the source cell is cut by the interface, the integration domain has to be restricted to the sub-domain $A$ where the level set function is positive. The basis $\basisSrc$ is obviously not orthonormal on $\cellSrc \cap A$, which is why we introduce the mass matrix $\massSrc$ which is defined by
\begin{equation}
	(\massSrc)_{i j} = \int \limits_{\cellSrc \cap A} (\basisSrc)_i (\basisSrc)_j \diff{V}.
\end{equation}
Following the same steps as in the previous section, this leads to
\begin{align}
	(\solvecAgg)_i
	&= \int \limits_{\cellTar} \solTar (\basisTar)_i \diff{V} + \int \limits_{\cellSrc} \solSrc (\basisSrc \coupling))_i \diff{V}\\
	&= (\solvecTar)_j \int \limits_{\cellTar} (\basisTar{})_j (\basisTar)_i \diff{V} + (\solvecSrc)_j \int \limits_{\cellSrc} (\basisSrc{})_j (\basisSrc \coupling)_i \diff{V}\\
	&= (\solvecTar)_j \matrix{I}_{j i} + (\solvecSrc)_j \int \limits_{\cellSrc} (\basisSrc{})_j (\basisSrc)_k (\coupling)_{k i} \diff{V}\\
	&= (\solvecTar)_i + (\solvecSrc)_j (\massSrc)_{j k} (\coupling)_{k i}\\
	&= (\solvecTar)_i + (\T{\coupling})_{i k} (\T{\massSrc})_{k j}  (\solvecSrc)_j
\end{align}
and Equation \eqref{eqn:manipulate_plain} is generalized to
\begin{equation}
	\boxed{\solvecAgg = \solvecTar + \T{\coupling} \massSrc \solvecSrc}
\end{equation}
because $\massSrc$ is symmetric

\section{Extrapolation into uncut cell ('Extrapolate')}

In the extrapolation, the solution coefficients $\solvecSrc$ have to be computed from the agglomerated solution coefficients $\solvecAgg$. The coefficients in the target cell, however, simply follow from $\solvecSrc = \solvecAgg$.

In order to compute the coefficients $\solvecSrc$, we simply perform the projection onto the original DG space, i.e.
\begin{align}
	(\solvecSrc)_i
	&= \int \limits_{\cellSrc} \solAgg (\basisSrc)_i \diff{V}\\
	&= \int \limits_{\cellSrc} (\basisAgg)_j (\solvecAgg)_j (\basisSrc)_i \diff{V}\\
	&= \int \limits_{\cellSrc} (\basisSrc \coupling)_j (\solvecAgg)_j (\basisSrc)_i \diff{V}\\
	&= \int \limits_{\cellSrc} (\basisSrc)_k (\coupling)_{k j} (\solvecAgg)_j  (\basisSrc)_i \diff{V}\\
	&= \left(\int \limits_{\cellSrc} (\basisSrc)_i (\basisSrc)_k \diff{V} \right) (\coupling)_{k j} (\solvecAgg)_j\\
	&= (\matrix{I})_{i k} (\coupling)_{k j} (\solvecAgg)_j\\
	&= (\coupling)_{i j} (\solvecAgg)_j,
\end{align}
which is equivalent to
\begin{equation}
	\label{eqn:extrapolate_plain}
	\solvecSrc = \coupling \solvecAgg
\end{equation}

\section{Extrapolation into cut cell}

Again, Equation \eqref{eqn:extrapolate_plain} does not hold if $\cellSrc$ is cut be the interface. Instead, we have to use
\begin{align}
	(\massSrc \solvecSrc)_i
	&= \int \limits_{\cellSrc \cap A} \solAgg (\basisSrc)_i \diff{V}\\
	&= \int \limits_{\cellSrc \cap A} (\basisAgg)_j (\solvecAgg)_j (\basisSrc)_i \diff{V}\\
	&= \int \limits_{\cellSrc \cap A} (\basisSrc \coupling)_j (\solvecAgg)_j (\basisSrc)_i \diff{V}\\
	&= \int \limits_{\cellSrc \cap A} (\basisSrc)_k (\coupling)_{k j} (\solvecAgg)_j  (\basisSrc)_i \diff{V}\\
	&= \left(\int \limits_{\cellSrc \cap A} (\basisSrc)_i (\basisSrc)_k \diff{V} \right) (\coupling)_{k j} (\solvecAgg)_j\\
	&= (\massSrc)_{i k} (\coupling)_{k j} (\solvecAgg)_j
\end{align}
which is equivalent to
\begin{equation}
	\boxed{\solvecSrc = \coupling \solvecAgg.}
\end{equation}

\section{Mass matrix of the agglomerated cell ('ManipulateMassMatrixBlocks')}

Finally, we need the mass matrix of the agglomerated cell. This time, we immediately consider the cut case, i.e.
\begin{align}
	(\massAgg)_{i j}
	&= \int \limits_{\cellAgg} (\basisAgg)_i (\basisAgg)_j \diff{V}\\
	&= \int \limits_{\cellTar} (\basisAgg)_i (\basisAgg)_j \diff{V} + \int \limits_{\cellSrc} (\basisAgg)_i (\basisAgg)_j \diff{V}\\
	&= \int \limits_{\cellTar} (\basisTar)_i (\basisTar)_j \diff{V} + \int \limits_{\cellSrc} (\basisSrc \coupling)_i (\basisSrc \coupling)_j \diff{V}\\
	&= (\matrix{I})_{i j} + \int \limits_{\cellSrc} (\basisSrc)_k (\coupling)_{k i} (\basisSrc)_l (\coupling)_{l j} \diff{V}\\
	&= (\matrix{I})_{i j} + (\coupling)_{k i} \left(\int \limits_{\cellSrc} (\basisSrc)_k (\basisSrc)_l \diff{V}\right) (\coupling)_{l j}\\
	&= (\matrix{I})_{i j} + (\T{\coupling})_{i k} (\massSrc)_{k l} (\coupling)_{l j}
\end{align}
which can also be expressed as
\begin{equation}
	\label{eqn:massmatrix_plain}
	\boxed{\massAgg = \matrix{I} + \T{\coupling} \massSrc \coupling.}
\end{equation}

\end{document}