text:
\section{Introduction}

The understanding of biological processes has been greatly stimulated by systems biology approaches \cite{Kitano2002,Chuang2010,Apweiler2018}.
The integration of mathematical models with the underlying biological knowledge and empirical observations can help us observe emergent systems properties, test new hypotheses, enhance the interpretability of the studied systems and guide innovations in areas such as medicine and drug discovery \cite{Aldridge2006}.
While multiple mathematical modeling frameworks exist, the scarcity of experimental data and the challenges posed by the development of quantitative large-scale biological networks, has favoured the simplicity and intuitiveness of more qualitative approaches, such as logic-based modeling \cite{Morris2010}.

At the heart of the mathematical representation of molecular biological networks lies the concept of regulation.
Regulation of activity, typically by changing the modification state, location or concentration of a biological entity, is a process which can be expressed by a mathematical function that combines the various regulatory inputs that affect the target, with a logic that describes how these regulators are integrated.
In Boolean logic-based modeling, the regulatory inputs are entities which can be expressed in two states: active ($1$) or inactive ($0$).
These entities are combined with logical rules to derive the \textit{Boolean regulatory function} (BRF) of the target entity.
For every possible regulatory input (combination of $0$ and $1$'s) the BRF will produce the end regulatory product, which is the activity of the target ($0$ or $1$).

The construction of a Boolean computational model starts with the assembly of information from literature and experimental observations, in the form of a Prior Knowledge Network (PKN), i.e. a list of network entities and their causal interactions (positive or negative) \cite{Traynard2017, Toure2020}.
The use of a PKN for accurate representation of biological reality and subsequent analysis and simulation requires the definition of the model formalism.
This is one of the most important steps in dynamical modeling since it directly translates to the choice of BRFs, i.e. the logical rules that together with the regulators define the activity state of each network target \cite{Wang2012}.
There have been several approaches related to the choice of BRFs, from using a standardized format \cite{Mendoza2006}, to automatically generating all possible BRFs compatible with the PKN and calibrating the rules in order to fit perturbation data \cite{SaezRodriguez2009, Videla2016, Gjerga2020}.
State-of-the-art approaches involve the automated construction of large-scale logical networks by inferring the logical rules from the topology and semantics of molecular interaction maps \cite{Aghamiri2020}.

Regardless of how a logical model is constructed, it has been shown in practice that expert curation, i.e. the manual fine-tuning of the logical rules to fit experimental data, can result in highly predictive models \cite{Flobak2015, Niederdorfer2020}, yet this is not trivially obtained with automatically constructed networks \cite{Bekkar2018}.
Because of the large function space complemented with a sparsity of observations and inherent noise in existing data, there is a wide range of plausible BRFs.
Thus, it is crucial to properly define function characteristics that can guide the modeler to a more informed function choice.
Our work is focused on explicating some of these metrics and using them to show for example which BRFs can be discarded due to biological inconsistencies with the underlying regulatory topology and which are biased towards specific Boolean outcomes.

The paper is structured as follows: Section \ref{sec:background} provides a list of notations and definitions to be used later in the text.
In Section \ref{sec:DNF}, we discuss the benefits of using the equivalent disjunctive normal form of a Boolean function to delineate its biological interpretability.
In Section \ref{sec:consistency}, we provide a set of properties that characterize the Boolean functions that are consistent with a given regulatory topology and show that several functions under study violate them.
In Section \ref{sec:truth-density}, we present the truth density metric as a means to evaluate if a Boolean function is biased or balanced with increasing number of regulators.
We also discuss the asymptotic properties of different functions relating to the ratio between activators and inhibitors.
Lastly, in Section \ref{sec:network-analysis}, we present evidence that the standardized Boolean functions are indeed biased and show how modelers can exploit such information for their own benefit.
The results are demonstrated in Boolean models derived from a cancer signaling network as well as from scale-free topologies that are applicable to most biological networks.
We close the paper with some discussion points in Section \ref{sec:discuss} and directions for future research in Section \ref{sec:future}.

\section{Background \label{sec:background}}

\subsection{Boolean regulatory functions}

\textit{Boolean regulatory functions} (BRFs) are Boolean functions used in the context of biological networks and modeling.
A mathematical description of such a function associates the activity output of a target biological entity with the Boolean input values of $n$ variables (the \textit{regulators}), such that $f_{BRF}:\{0,1\}^n \rightarrow \{0,1\}$.
Thus, the target's output state is binary, i.e. either 0 ($False$, denoting an inactive or inhibited state) or $1$ ($True$, indicating an active state).

One intuitive representation of a Boolean function is its \textit{truth table}, which is a list of all possible Boolean input configurations of the $n$ regulators along with their associated function output.
Since every regulator can be assigned two possible values ($0$ and $1$), the total number of input configurations (i.e. rows) in a truth table is $2^n$.
For example, a Boolean function $f(x_1,x_2,x_3)$ with $3$ regulators has a total of $2^3=8$ rows in its corresponding truth table, starting from the input configuration $(0,0,0)$ and ending with $(1,1,1)$ (Table \ref{tab:table1}).

The total number of BRFs with $n$ regulators is $2^{2^n}$ since for each of the $2^n$ input configurations (i.e. rows of the truth table) there can be two possible function outcomes ($0$ or $1$).
For example, with $3$ regulators and a total of $8$ rows in the truth table, that would be a total of $2^8=256$ functions, three of which are shown in Table \ref{tab:table1}.

\subsection{Disjunctive normal form}

The most frequently used form of a Boolean function is its analytical expression, where variables are connected with logical operators such as AND ($\land$), OR ($\lor$), NOT ($\lnot$), XOR ($\oplus$), etc. and the output of the function is calculated using basic Boolean algebra.
In Table \ref{tab:table1} for example, we provide the analytical forms for the functions $f_1$ and $f_2$.
Note that there can be multiple analytical forms that essentially compute the same function, e.g. another form of the $f_1$ function is $f_1^{'}=(\lnot x_1 \land x_2 \land \lnot x_3) \lor (x_1 \land \lnot x_2 \land \lnot x_3) \lor (x_1 \land x_2 \land \lnot x_3)$.

\vspace{20pt}
\begin{table}[h!]
  \centering
  \begin{tabular}{cccccc}
    \toprule
    \multicolumn{3}{c}{Truth Table} & \multicolumn{3}{c}{Boolean functions} \\
    \cmidrule(r){1-3}
    \cmidrule(r){4-6}
    $x_1$ & $x_2$ & $x_3$ & $f_1=(x_1 \land \lnot x_3) \lor (x_2 \land \lnot x_3)$ & $f_2=x_1 \lor (\lnot x_2 \land \lnot x_3)$ & $f_3 = 1$ \\
    \midrule
    0&0&0&0&1&1 \\
    0&0&1&0&0&1 \\
    0&1&0&1&0&1 \\
    0&1&1&0&0&1 \\
    1&0&0&1&1&1 \\
    1&0&1&0&1&1 \\
    1&1&0&1&1&1 \\
    1&1&1&0&1&1 \\
    \bottomrule
  \end{tabular}
  \bigskip
  \caption{Truth table of three Boolean functions with three input variables $x_1, x_2$ and $x_3$. Functions $f_1$ and $f_2$ are expressed in disjunctive normal form (DNF) with the minimum possible number of terms. $f_3$ is a tautology.}
  \label{tab:table1}
\end{table}

This brings us to the notion of a general form which could be used to define useful metrics common to all Boolean functions (e.g. complexity), as well as the need to provide minimal forms based on specific criteria.
For example, a more compact function form enhances readability, which can be seen by comparing $f_1$ with $f_1^{'}$.

Every Boolean function can be represented in a \textit{disjunctive normal form} (DNF), requiring only AND ($\land$), OR ($\lor$) and NOT ($\lnot$) operators as building blocks.
In such a representation, \textit{literals}, which are variables (e.g. positive literal $x$) or their logical negations (e.g. negative literal $\text{NOT }x$), are connected by AND's, producing \textit{terms}, which are then in turn connected by OR's \cite{crama2011boolean}.
For example, every function in Table \ref{tab:table1} is expressed in DNF, while the Boolean expressions $\lnot (x_1 \lor x_2)$ and $\lnot (x_1\land x_2)\lor x_3$ are not.
Note that a Boolean function can have multiple DNF formulations.

\subsection{Link operator functions}

We consider the class of BRFs that partitions the input regulators to two sets: the set of positive regulators (\textit{activators}) and the set of negative regulators (\textit{inhibitors}).
Let $f$ be such a Boolean function $f_{BRF}(x,y):\{0,1\}^n \rightarrow \{0,1\}$, with $m \ge 1$ activators $x=\{x_i\}_{i=1}^{m}$ and $k \ge 1$ inhibitors $y=\{y_j\}_{j=1}^{k}$, that is a total of $n = m + k$ regulators.
The \textit{link operator} BRFs have an analytical formula which places the two distinct types of regulators in two separate expressions, while connecting them with a special logical operator that we call a \textit{link operator}.
An example of such a function that has been used extensively in the logical modeling literature is the standardized BRF formula with the "AND-NOT" link operator \cite{Mendoza2006}:

\begin{equation} \label{eq:and-not}
    f_{AND-NOT}(x,y) = \left(\bigvee_{i=1}^{m} x_i\right) \land \lnot \left(\bigvee_{j=1}^{k} y_j\right)
\end{equation}

A variation of the above function is the "OR-NOT" link operator function:

\begin{equation} \label{eq:or-not}
    f_{OR-NOT}(x,y) = \left(\bigvee_{i=1}^{m} x_i\right) \lor \lnot \left(\bigvee_{j=1}^{k} y_j\right)
\end{equation}

Note that the presence of the link operator is what forces the condition $m,k \ge 1$ (at least one regulator in each category).
For the rest of this work, we will not consider BRFs with only one type of regulator, since these can be represented by simple logical functions without loss of biological consistency.
Following the notation introduced in Mendoza et al. \cite{Mendoza2006}, in the case of only positive regulators, the presence of at least one activator makes the target active, i.e. $f(x)=\bigvee_{i=1}^{m} x_i$.
In the case of only inhibitory regulators, the presence of at least one inhibitor is sufficient to make the target inactive, i.e. $f(y)=\lnot \bigvee_{j=1}^{k} y_j=\bigwedge_{j=1}^{k} \lnot y_j$.

Borrowing notation from circuit theory, we will also use other link operators like the "NAND", "NOR", "XNOR" gates, with or without the "NOT" symbol in front.
Note that the logical operator used to connect the same type of regulators (e.g. the activators) is usually OR, but other operators could be used as well.

Another link operator function that we will consider in this work is the "Pairs" function:

\begin{equation} \label{eq:pairs}
    f_{Pairs}(x,y) = \bigvee_{\forall (i,j)}^{m,k}(x_i\land \lnot y_j)
\end{equation}

The intuition behind the name is derived from the fact that the function will return $True$ if there is at least one pair of regulators consisting of a present activator and an absent inhibitor.
For a formulation of the "Pairs" function that is consistent with the link operator terminology as defined above, see (Eq. \ref{eq:pairs-cnf}).

\newpage
\subsection{Threshold functions}

\textit{Threshold functions} are a special type of Boolean functions, the output of which depends on the condition that the sum of (possibly weighted) activities of the input regulators surpasses a given \textit{threshold} value \cite{McCulloch1943, Hopfield1982}.

In this work we will consider two simple threshold functions, which both output $True$ when the number of present activators is larger than the number of present inhibitors.
As such, the activities of the positive and negative regulators are combined in an \textit{additive} manner, with their respective assigned weights set to $\pm1$ and the threshold parameter to $0$, formulating thus a \textit{majority rule} which defines the value of the function \cite{Bornholdt2008, Chaouiya2013}.
These functions differ with regards to their output when there is balance between the activities of the positive and negative regulators: the first outputs $1$ (the activators "win") while the second outputs $0$ (the inhibitors "win"):

\begin{align} \label{eq:thres-act}
    f_{Act-win}(x,y) &= \begin{cases}
    1, & \sum_{i=1}^{m} x_i \ge \sum_{j=1}^{k} y_j \\
    0, & \text{otherwise} \end{cases}
    \intertext{}
    f_{Inh-win}(x,y) &= \begin{cases} \label{eq:thres-inh}
    1, & \sum_{i=1}^{m} x_i > \sum_{j=1}^{k} y_j \\
    0, & \text{otherwise} \end{cases}
\end{align}

\section{Disjunctive Normal Form unmasks biological interpretation} \label{sec:DNF}

\subsection{Interpretability issues in Boolean modeling}

Two main features make Boolean modeling attractive to users.
First, transforming conditions for the activation or inhibition of a target biological entity to Boolean equations is a relatively easy task using a qualitative, logic-based modeling formalism.
Second, the reverse is also true, i.e. Boolean equations can be more interpretable and closer to a simplified description of biological reality that "makes sense" than the use of other kinds of formalisms (e.g. kinetic modeling).
For example, consider the simple case of a target entity, which is regulated by one positive regulator $x_1$ and one negative regulator $y_1$.
The use of the "AND-NOT" link operator function in this case (Eq. \ref{eq:and-not}) is very easy to understand and interpret since the formula directly connects to the underlying biology.
Thus, the mathematical formulation is simply written as $f_{AND-NOT}=x_1 \text{ AND NOT } y_1$, while the modeler reads "the target becomes active when $x_1$ (the activator) is present and $y_1$ (the inhibitor) absent".

Issues start arising when considering the \textit{interpretability} of such Boolean expressions in cases where a larger number of regulators act on a target, e.g. in a more complex scenario with three positive ($x_1,x_2,x_3$) and three negative ($y_1,y_2,y_3$) regulators, the mathematical formulation expressing the target's activity output can be easily written using the link operator function form, as $f_{AND-NOT}=(x_1 \text{ OR }x_2\text{ OR }x_3) \text{ AND NOT } (y_1\text{ OR }y_2\text{ OR }y_3)$.
A modeler could read this as "the target becomes active when at least one activator is present, and all of its inhibitory regulators are absent", but a precise semantic description that explicates the conditions under which the target gets activated, can in general be difficult to assess.
A similar issue arises when reflecting on the use of a different link operator instead of the standard "AND-NOT" or even of an entirely different regulatory function, for which the biological interpretation might be difficult to derive from the expression itself.

\renewcommand{\arraystretch}{1.4}
\begin{table}[H]

  \scriptsize

  \resizebox{\textwidth}{!}{
  \begin{tabular}{|M{5cm}|M{4.7cm}|M{3.4cm}|M{1.4cm}|c|}
  \hline
     \thead{BRF (standard form)} &
     \thead{BRF (CDNF)} &
     \thead{Biological Interpretation} &
     \thead{Consistent} &
     \thead{Complexity} \\
     \hline
        $(x_1 \text{ OR }x_2)\textbf{ NOR }(y_1 \text{ OR }y_2)$ &
        $\text{NOT }x_1\text{ AND NOT }x_2\text{ AND}$ \newline $\text{NOT }y_1\text{ AND NOT }y_2$ \hphantom{mmmm} &
        Absence of all regulators &
        \cellcolor[HTML]{FF4136} NO &
        $1$ (always) \\
    \hline
        $(x_1 \text{ OR }x_2)\textbf{ NAND }(y_1 \text{ OR }y_2)$ &
        $(\text{NOT }x_1\text{ AND NOT }x_2)\textbf{ OR}$ \newline $(\text{NOT }y_1\text{ AND NOT }y_2)$ \hphantom{mmm} &
        Absence of all activators \textbf{or} absence of all inhibitors &
        \cellcolor[HTML]{FF4136} NO &
        $2$ (always) \\
    \hline
        $(x_1 \text{ OR }x_2)\textbf{ AND NOT }(y_1 \text{ OR }y_2)$ \newline \newline "AND-NOT" (Eq. \ref{eq:and-not}) &
        $(x_1\text{ AND NOT }y_1\text{ AND NOT }y_2)\textbf{ OR}$ \newline $(x_2\text{ AND NOT }y_1\text{ AND NOT }y_2)$ \hphantom{mm} &
        Presence of at least one activator \textbf{and} absence of all inhibitors &
        \cellcolor[HTML]{2ECC40} YES &
        $2$ ($m$) \\
    \hline
        $(x_1 \text{ OR }x_2)\textbf{ NOR NOT }(y_1 \text{ OR }y_2)$ &
        $(y_1\text{ AND NOT }x_1\text{ AND NOT }x_2)\textbf{ OR}$ \newline $(y_2\text{ AND NOT }x_1\text{ AND NOT }x_2)$ \hphantom{mm} &
        Presence of at least one inhibitor \textbf{and}  absence of all activators &
        \cellcolor[HTML]{FF4136} NO &
        $2$ ($k$) \\
    \hline
        $(x_1 \text{ OR }x_2)\textbf{ OR NOT }(y_1 \text{ OR }y_2)$ \newline \newline "OR-NOT" (Eq. \ref{eq:or-not}) &
        \hphantom{mm} $x_1\textbf{ OR }x_2\textbf{ OR}$ \newline $(\text{NOT }y_1\text{ AND}$ $\text{NOT }y_2)$ &
        Presence of any activator \textbf{or} absence of all inhibitors &
        \cellcolor[HTML]{2ECC40} YES &
        $3$ ($m+1$) \\
    \hline
        $(x_1 \text{ OR }x_2)\textbf{ NAND NOT }(y_1 \text{ OR }y_2)$ &
        \hphantom{mm} $y_1\textbf{ OR }y_2\textbf{ OR}$ \newline $(\text{NOT }x_1\text{ AND NOT }x_2)$ &
        Presence of any inhibitor \textbf{or} absence of all activators &
        \cellcolor[HTML]{FF4136} NO &
        $3$ ($k+1$) \\
    \hline
        $(x_1 \text{ OR }x_2)\textbf{ XOR }(y_1 \text{ OR }y_2)$ &
        $(x_1\text{ AND NOT }y_1\text{ AND NOT }y_2)\textbf{ OR}$ \newline $(x_2\text{ AND NOT }y_1\text{ AND NOT }y_2)\textbf{ OR}$ \newline $(\text{NOT }x_1\text{ AND NOT }x_2\text{ AND }y_1)\textbf{ OR}$ \newline $(\text{NOT }x_1\text{ AND NOT }x_2\text{ AND }y_2)$ \hphantom{mm} &
        Presence of at least one activator and absence of all inhibitors \textbf{or} presence of at least one inhibitor and absence of all activators &
        \cellcolor[HTML]{FF4136} NO &
        $4$ ($m+k$) \\
    \hline
        $(x_1 \text{ OR }x_2)\text{ AND }(\text{NOT }y_1\text{ OR NOT }y_2)$ \newline \newline "Pairs" (Eq. \ref{eq:pairs}) &
        $(x_1\text{ AND NOT }y_1)\textbf{ OR }$ \newline $(x_1\text{ AND NOT }y_2)\textbf{ OR }$ \newline $(x_2\text{ AND NOT }y_1)\textbf{ OR }$ \newline $(x_2\text{ AND NOT }y_2)$ \hphantom{mmmmm} &
        Presence of at least one activator \textbf{and} absence of at least one inhibitor &
        \cellcolor[HTML]{2ECC40} YES &
        $4$ ($m \times k$) \\
    \hline
        $(x_1 \text{ OR }x_2)\textbf{ XNOR }(y_1 \text{ OR }y_2)$ &
        $(x_1\text{ AND }y_1)\textbf{ OR }$ \hphantom{mm} \newline $(x_1\text{ AND }y_2)\textbf{ OR }$ \hphantom{mm} \newline $(x_2\text{ AND }y_1)\textbf{ OR }$ \hphantom{mm} \newline $(x_2\text{ AND }y_2)\textbf{ OR }$ \hphantom{mm} \newline \hphantom{m} $(\text{NOT }x_1\text{ AND NOT }x_2\text{ AND}$ \newline $\text{NOT }y_1\text{ AND NOT }y_2)$ \hphantom{mm} &
        Presence of at least one activator and inhibitor pair \textbf{or} absence of all regulators &
        \cellcolor[HTML]{FF4136} NO &
        $5$ ($m \times k + 1$) \\
    \hline
        $True$ when $x_1+x_2 > y_1+y_2$ \newline \newline "Inh-win" (Eq. \ref{eq:thres-inh}) &
        $(x_1\text{ AND }x_2\text{ AND NOT }y_1)\textbf{ OR}$ \hphantom{mm} \newline $(x_1\text{ AND }x_2\text{ AND NOT }y_2)\textbf{ OR}$ \hphantom{mm} \newline $(x_1\text{ AND NOT }y_1\text{ AND NOT }y_2)\textbf{ OR}$ \newline $(x_2\text{ AND NOT }y_1\text{ AND NOT }y_2)$ \hphantom{mm} &
        Number of present activators is larger than the number of present inhibitors &
        \cellcolor[HTML]{2ECC40} YES &
        4 \\
    \hline
        $True$ when $x_1+x_2 \ge y_1+y_2$ \newline \newline "Act-win" (Eq. \ref{eq:thres-act}) &
        $(x_1\text{ AND }x_2)\textbf{ OR}$ \hphantom{mmm} \newline $(x_1\text{ AND NOT }y_1)\textbf{ OR}$ \newline $(x_1\text{ AND NOT }y_2)\textbf{ OR}$ \newline $(x_2\text{ AND NOT }y_1)\textbf{ OR}$ \newline $(x_2\text{ AND NOT }y_2)$ \hphantom{mmmmm} &
        Number of present activators is larger than or equal to the number of present inhibitors &
        \cellcolor[HTML]{2ECC40} YES &
        5 \\
    \hline
  \end{tabular}}
  \bigskip
  \caption{Several Boolean regulatory functions with four regulators ($m=2$ positive $\{x_1,x_2\}$, $k=2$ negative $\{y_1,y_2\}$) and some metrics are presented.
  The two first columns provide two different function forms: a standard one, i.e. either the link operator form distinguishing activating and inhibiting regulators or a simple description in the case of the threshold functions, and the CDNF which is a special case of DNF (Section \ref{subsec:consistency}).
  The "Biological Interpretation" states in words the conditions that make a BRF become $True$, and is explicitly translated from the terms in the corresponding CDNF.
  The "Consistent" column states if the functions satisfy the properties 1-3 from Section \ref{subsec:consistency} (YES, green-colored) or there are inconsistencies with the underlying regulatory structure (NO, red-colored), i.e. if an activator (resp. inhibitor) appears as a negative (resp. positive) literal in the corresponding CDNF.
  The functions are sorted according to an increasing complexity metric ("Complexity" column), which is the number of terms in each respective, minimum-length CDNF expression.
  In parentheses we provide the generalized formula for the number of CDNF terms of the link operator functions with $m$ activators and $k$ inhibitors.}
  \label{tab:table2}
\end{table}

\subsection{DNF links to biological semantics}

We argue here that the DNF is the most adequate function form to help us address the aforementioned issues.
Every Boolean regulatory function expressed in DNF, has a biological characterization that is directly derived from the formula itself: each term in the DNF is an activation condition, i.e. a list of regulators, some present (the positive literals) and some absent (the negative literals), which, when combined, make the target (output of the function) active.
Further merging of all the conditions using OR-semantics into a description of how the regulators influence the target's output, facilitates the biological interpretation of any Boolean regulatory function.

In Table \ref{tab:table2}, we show a list of BRFs with two positive and two negative regulators.
Most of the BRFs presented have a different link operator separating the activators from the inhibitors.
Using the functions standard expressions (1st column) makes it very hard to derive a meaningful biological characterization as expressed in the 3rd column of Table \ref{tab:table2}.
For example, defining a meaningful description of the "NOR" or "NAND-NOT" equations using only their standard expression, is a very difficult task.
In contrast, by using the equivalent DNFs (2nd column) we can make an explicit, "1-1" correspondence between mathematical formulation and biological interpretation and use it to compare the different functions' meanings.
Thus, by expressing the "NAND-NOT" equation in DNF, we can precisely identify the conditions that make the outcome of the function $True$ and translate these into a meaningful description such as "Presence of any inhibitor or absence of all activators".
Consequently, we are led to a generalized and independent of the number of regulators description of this link operator function.
Such a description is intuitive to human interpretation and reasoning, in terms of the function's applicability, e.g. in comparing the "AND-NOT" and "NAND-NOT" biological interpretations, we see that the first is semantically plausible while the second completely contradicts the underlying biology.

\section{Characterizing consistent regulatory functions} \label{sec:consistency}

\subsection{The 3 consistency properties} \label{subsec:consistency}

As observed in Table \ref{tab:table2}, not only can the DNF be used to uncover the biological interpretation of any BRF and subsequently help determine its plausibility, but it also provides a means to compare the different function meanings.
Still, we need a more refined, technical description that is able to express the implausibility of the "NAND-NOT" or "NOR" cases directly from their mathematical formulas, and which would be applicable to every BRF.
We define the \textit{consistency} attribute of a BRF to describe its compliance with the underlying regulatory network structure.

The first step in making a Boolean model is to build a graph (PKN), assembling the regulatory entities of interest from various databases or the scientific literature, and use causality information to connect them through their regulatory action on other entities.
As such, a network structure can be defined, in which entities can regulate (either positively or negatively) some of the other entities.
Using such a simple network-driven formalization, we define a set of three properties that describe the set of all the \textit{consistent Boolean regulatory functions}, i.e. the functions that comply with the underlying regulatory structure.
So, for a consistent BRF, the following propositions are satisfied \cite{Cury2019}:

\begin{enumerate}
    \item Its regulators can be partitioned into two disjoint sets: the set of \textit{activators} (positive regulators, enhance target's activity) and the set of \textit{inhibitors} (negative regulators, suppress target's activity).
    This stems from the fact that every interaction in the PKN has a fixed sign (either positive or negative).
    As such, there are no dual regulations, i.e. a regulator cannot activate and inhibit a target at the same time.
    This property essentially makes the set of consistent BRFs a subset of the \textit{monotone} Boolean functions \cite{crama2011boolean}.
    \item All regulators are \textit{essential}: for every regulatory input, inverting their values, will also, in at least one configuration of states of other regulators, change the output of the function.
    This means that all regulators are indispensable for deriving the target's activity output.
    \item A consistent BRF can be represented in a unique \textit{complete} DNF (CDNF) which is also known as Blake's Canonical Form \cite{Blake1937}.
    This is a consequence of property (1), since monotone Boolean functions expressed in any DNF, can be further simplified by removing redundant literals, resulting in the equivalent unique CDNF expression \cite{crama2011boolean}.
    This property is really important since it allows us to identify which regulatory entities are activators and which are inhibitors from the corresponding CDNF expression of a consistent BRF: an activator will always appear as a positive literal, whereas an inhibitor will always appear as a negative literal.
\end{enumerate}

We provide an example to delineate the difference between the DNF and CDNF forms and show violations of the consistency properties.
In Table \ref{tab:table3}, we present three Boolean functions, expressing the output of a target regulated by one activator ($x_1$) and one inhibitor ($y_1$).
The functions $f_2$ and $f_3$ are in CDNF whereas $f_1$ is in normal DNF, since the positive regulator $x_1$ appears both as a positive and a negative literal (i.e. it acts as a dual regulator, making $f_1$ inconsistent).
Notice that $f_1$ reduces to $f_2$ by removing the redundant negative literal ($\lnot x_1$) in the term $(\lnot x_1 \land y_1)$: $y_1$ "absorbs" the larger term and thus a shorter expression manifests, one that covers more $True$ outcomes (i.e. $1$'s) in the truth table.
In addition, we observe that $f_2$ is inconsistent, since inhibitor $y_1$ appears as a positive literal.
On the other hand, using a negative literal for inhibitor $y_1$ and a positive one for activator $x_1$, makes $f_3$ consistent.

\begin{table}[h!]
  \centering
  \begin{tabular}{cccccc}
    \toprule
    \multicolumn{2}{c}{Truth Table} & \multicolumn{1}{c}{Term} & \multicolumn{3}{c}{Boolean functions} \\
    \cmidrule(r){1-2}
    \cmidrule(r){3-3}
    \cmidrule(r){4-6}
    $x_1$ & $y_1$ & $(\lnot x_1 \land y_1) $ & $f_1=x_1 \lor (\lnot x_1 \land y_1)$ & $f_2=x_1 \lor y_1$ & $f_3=x_1 \lor \lnot y_1 $ \\
    \midrule
    0&0&0&0&0&1 \\
    0&1&1&1&1&0 \\
    1&0&0&1&1&1 \\
    1&1&0&1&1&1 \\
    \bottomrule
  \end{tabular}
  \bigskip
  \caption{Truth table of three different Boolean regulatory functions with two input regulators, one positive ($x_1$) and one negative ($y_1$). All functions are expressed in DNF. $f_1$ and $f_2$ result in the same target Boolean output, with $f_2$ expressed in CDNF. Activator $x_1$ regulates the target both positively and negatively in $f_1$, making the function non-monotone and thus inconsistent. Inhibitor $y_1$ is a positive literal in $f_2$'s CDNF, making it inconsistent as well. Function $f_3$ is consistent since it's written in CDNF with the activator $x_1$ and inhibitor $y_1$ appearing as positive and negative literals respectively.}
  \label{tab:table3}
\end{table}

\subsection{Most link operator functions are inconsistent}

Examining Table \ref{tab:table2}, we note that the 2nd column presents not just any DNF expression of the studied Boolean regulatory functions, but precisely the CDNF.
Thus we can immediately identify which BRFs violate at least one of the three properties discussed in Section \ref{subsec:consistency} and are therefore inconsistent with the regulatory topology (this information is presented in the 4th column, labeled "Consistent").
Two examples of such inconsistencies include the "NAND-NOT" and "XOR" link operator functions, which have terms in their corresponding CDNF in which an activator $x_i$ appears as a negative literal ($\text{NOT }x_i$) and an inhibitor $y_j$ as a positive literal (as itself).
In total, from all the BRFs presented in Table \ref{tab:table2}, only the standardized "AND-NOT", the "OR-NOT", the "Pairs" and the two threshold functions respect the underlying regulatory topology, as can be verified by examining their respective CDNFs.
The rest of the link operator functions presented are inconsistent and will not be considered for further analysis in this paper.

\section{Truth Density as a measure of expected function output} \label{sec:truth-density}

In this section we present another interesting Boolean function metric, whose properties can be used to add further knowledge about a Boolean function's behavior.
This metric, which we call \textit{truth density}, allows us to project what the regulatory target's output will most likely be when the number of input regulators changes and investigate how the ratio between activators and inhibitors may affect that output.
From a modeler's perspective, this metric is useful to check if an assigned model parameterization (i.e. use of a specific BRF) can asymptotically predefine the activity state of some targets.
Equipped with this knowledge, a modeler can verify the degree of fitness with the observations that such a parameterization allows, and thus discard a specific function in favor of another, if the latter has a truth density value that better matches the outcome observed in the data.

\subsection{Truth Density}

We define the truth density ($TD$) of a Boolean function as the fraction of all input configurations in its corresponding truth table that yield a $True$ ($1$) outcome.
As such, $TD \in [0,1]$.
This quantity was first introduced in \cite{kauffman1993} and more recently in \cite{Gherardi2016} under the name of \textit{bias} and was similarly defined as the probability that a Boolean function takes on the value $1$.
Using the example with the three Boolean functions from Table \ref{tab:table1}, we have $TD_{f_1}=3/8=0.375$, $TD_{f_2}=5/8=0.625$ and $TD_{f_3}=8/8=1$, where the last function is a tautology, with the maximum possible truth density.
Colloquially, we can say that a Boolean function is \textit{biased}, when it's truth density is close to $0$ or $1$.
Since the size of a truth table grows exponentially with the number of inputs of the Boolean function ($n$ inputs correspond to $2^n$ rows), the existence of bias conveys the information that most of the input regulatory configurations result in either an activated or inhibited target (bias towards $1$ or $0$ respectively).
On the other hand, we shall say that a Boolean function is \textit{balanced}, if it takes on the values $0$ and $1$ equally often, or equivalently, it's truth density is approximately centered around $1/2$ \cite{Benjamini2005}.

\subsection{Asymptotic truth density results of the consistent regulatory functions}

In Appendix \ref{appx:a}, we present a list of propositions and proofs that provide the exact truth density formulas for the generic forms of the five consistent BRFs we studied in previous sections, namely the "AND-NOT", "OR-NOT" and "Pairs" link operator functions, and the two threshold functions, "Act-win" and "Inh-win".
A very important element that enables the straightforward derivation of these formulas, is the use of the equivalent DNF expressions in the proofs, especially for the case of the link operator Boolean functions.
We also noted that the truth densities of all the aforementioned BRFs depend on two variables: the number of activators and the number of inhibitors (the total number of regulators also appears as a separate variable but it depends on the first two, i.e. it is just their sum).
Thus, we logically asked if a BRF's truth density asymptotically tends towards specific values in the $[0,1]$ interval (e.g. the function could be biased or balanced), when the number of its input regulators increases or the ratio between activators and inhibitors changes.
The results of the asymptotic behavior of the truth density formulas are analytically presented in Appendix \ref{appx:b}.

The asymptotic analysis of the truth density formulas confirmed the intuitive perception that the link operator "AND-NOT" and "OR-NOT" functions show a characteristically opposite behavior with increasing number of regulators: the standardized "AND-NOT" formula depends only on the number of inhibitors and its output tends towards $0$, whereas the "OR-NOT" formula depends only on the number of activators and is biased towards $1$.
On the other hand, the "Pairs" and threshold functions truth densities don't have an asymptotic limit since they depend on both the number of activators and inhibitors.
Therefore, we proceeded in clarifying the role of the \textit{activator-to-inhibitor} ratio by investigating three scenarios which explicitly reveal the functions truth density behavior for a significantly large number of regulators:

\begin{itemize}
    \item A $1:1$ activator-to-inhibitor ratio, where approximately half of the regulators are activators and half are inhibitors.
    \item A high activator-to-inhibitor ratio, where all regulators are activators except one inhibitor.
    \item A low activator-to-inhibitor ratio, where all regulators are inhibitors except one activator.
\end{itemize}

In the $1:1$ ratio scenario, where there is an equal number of activators and inhibitors, the asymptotic behavior of the "AND-NOT" and "OR-NOT" functions corresponds to absolute inhibition ($0$) and activation ($1$) respectively, following the biased behavior shown previously.
The "Pairs" function behaves similarly to the "OR-NOT" function and therefore is also biased towards $1$.
Only the threshold functions show balanced behavior with their truth density value reaching asymptotically $1/2$, since the majority rule does favor neither activators nor inhibitors in this scenario.
On the other hand, in the two extremely unbalanced scenarios, where one set of regulators completely outweighs the other, the asymptotic truth density results of the "AND-NOT" and "OR-NOT" functions depend on each respective scenario.
Specifically, when the inhibitors dominate over the activators, the "OR-NOT" is balanced and the "AND-NOT" is biased, since the former has been shown to depend exclusively on the number of activators (which is just one in this case) for increasingly more regulators, whereas the latter on the number of inhibitors.
Their behavior is reversed when the activators outbalance the inhibitors.
In contrast, the "Pairs" function behaves in a balanced manner, having a truth density asymptotically equal to $1/2$ in both these scenarios, since the single minority regulator is paired with every regulator from the dominant group in the respective DNF expression (Eq. \ref{eq:pairs}) and as a result, it significantly influences the function's output.
Lastly, the asymptotic results for the threshold functions follow the larger size regulatory group, being biased towards $0$ with significantly more inhibitors and biased towards $1$ with significantly more activators.

\subsection{Validation of asymptotic behavior}

One key issue of immense practical importance for the modeler, which arises when analyzing the asymptotic behavior of the truth density formulas, is the actual number of regulators that effectively make each of the studied functions exhibit the demonstrated behavior.
We noticed that most of the truth density formulas (Eq. \ref{td-asymp:and-not}, \ref{td-asymp:or-not} and \ref{td-asymp:pairs}) are the sum of two to three terms, with only one of them depending exclusively on the number of regulators $n$.
Also, this term is usually $1/2^n$, and can be omitted when considering values larger than $n = 10$ regulators since it's insignificant ($1/2^{10} \approx 0.001$).
This suggests that the limit value of the truth density formulas may be already derived from a much smaller number of regulators than what is implied by the study of asymptotes.
Therefore, we need to have a more data-centric view of the results from our previous asymptotics analysis of the different BRFs, one that will enable us to verify the mathematically observed behaviors but also identify an approximate range for the number of regulators where the asymptotics decide the outcome of the studied functions.

We generated the complete truth tables for the five consistent BRFs of Table \ref{tab:table2}, from $2$ up to $20$ regulators, accounting for every possible activator-to-inhibitor ratio.
For example, for $n = 10$ regulators, every combination of at least one activator and one inhibitor that adds up to $10$ ($1$ activator + $9$ inhibitors, $2$ activators + $8$ inhibitors, etc.) resulted in a different truth table for each considered Boolean function.
Subsequently, using the generated truth tables, we could easily calculate the exact truth density value for each function at every considered ratio.
The results are shown in Figures \ref{fig:1A} and \ref{fig:1B} for the link operator and threshold functions, respectively.

\clearpage

\begin{figure}[!ht]
    \centering

    \begin{subfigure}{0.72\textwidth}
        \centering

        \caption{} \label{fig:1A}
        \includegraphics[width=\textwidth]{images/td-link-operator.pdf}
    \end{subfigure}

    \begin{subfigure}{0.72\textwidth}
        \centering

        \caption{} \label{fig:1B}
        \includegraphics[width=\textwidth]{images/td-thres.pdf}
    \end{subfigure}

    \caption{Comparing the truth densities of five different Boolean regulatory functions for different numbers of regulators and activator-to-inhibitor ratios. For each specific number of regulators, every possible combination of at least one activator and one inhibitor that add up to that number, results in a different truth table output with its corresponding truth density value. All such possible configurations up to $20$ regulators are shown. (\textbf{A}) The standardized "AND-NOT" function, along with the "OR-NOT" and "Pairs" functions, show an increasingly biased behavior with more regulators. (\textbf{B}) The two threshold functions "Act-win" and "Inh-win" show a more balanced behavior, since they respect the activator-to-inhibitor ratio and thus demonstrate a larger spectrum of possible truth density values even for higher numbers of regulators.}
    \label{fig:1}
\end{figure}

The data in general shows that the different regulatory functions demonstrate quite dissimilar behaviors with regard to their asymptotic outcome.
In particular, we recapitulate the findings from the asymptotics analysis, namely the bias of the link operator functions, which is evident even from $7$ to $10$ input regulators.
Interestingly, the "Pairs" function follows asymptotically the behavior of the "OR-NOT" function but is in general less biased.
We note that the outliers in Figure \ref{fig:1A} with truth density values closer to $1/2$, represent imbalanced activator-to-inhibitor ratio scenarios, i.e. either considerably more activators than inhibitors for the "AND-NOT" function and the reverse for the "OR-NOT" function, or any imbalanced ratio for the "Pairs" function.
Lastly, Figure \ref{fig:1B} shows that the threshold functions exhibit a more balanced behavior, expressed as a higher spectrum of truth density values for any single number of regulators and with the median truth density asymptotically reaching $1/2$.
This result is due to the fact that threshold functions faithfully follow the activator-to-inhibitor ratio, i.e. with more activators the outcome is biased towards $1$ whereas with more inhibitors the function outcome tends towards $0$.

\section{Link operator parameterization determines activity state in biological networks} \label{sec:network-analysis}

In this section we investigate if a model's parameterization can effectively decide the activity state of nodes in biological networks.
In more detail, we will use the "AND-NOT" link operator function \cite{Mendoza2006} and its symmetric function "OR-NOT" (Eq. \ref{eq:and-not} and \ref{eq:or-not}), to build Boolean models from prior causal knowledge and check if their activity state profile as determined by dynamic attractor analysis, shows the biased behavior that we observed in Section \ref{sec:truth-density}.

A major motivation for this analysis is the fact that the "AND-NOT" function is extensively used by logical modelers and thus the knowledge of its bias, made possible through the lens of the truth density metric, should be clearly demonstrated in practical use cases, e.g. biological network targets should mostly be in an inhibited state when the "AND-NOT" parameterization is used in their respective Boolean equations and in an active state in the case of the "OR-NOT".
As such, a modeler could make use of the link operator function bias to select the appropriate model parameterization which statistically guarantees an activity state profile that best matches the one supported by experimental evidence.

\subsection{From topology to link operator Boolean models}

In order to define Boolean models with the "AND-NOT" and "OR-NOT" link operator parameterization forms, we implemented the software \textit{abmlog}, which stands for "\textbf{A}ll possible \textbf{B}oolean \textbf{M}odels \textbf{L}ink \textbf{O}perator \textbf{G}enerator" (\nameref{sec:soft}).
Given a simple interaction (.sif) format file \cite{Toure2020}, representing a PKN with clearly defined, positive and negative causal interactions, the abmlog software outputs all combinatorially possible Boolean models where each link operator equation (deciding the state of a \textit{link operator node}, i.e. one whose Boolean activity state is determined by both positive and negative regulators) will have either the "AND-NOT" or the "OR-NOT" function form.
The models are saved in both the widely-used BoolNet (.bnet) \cite{Mussel2010} format and the gitsbe format \cite{gitsbe-format}, with the latter additionally including the attractors of the Boolean model, calculated via the BioLQM Java library \cite{Naldi2018}.
A simple overview of the software is presented in Figure \ref{fig:2}.

By default, abmlog generates all possible Boolean models with the two link operator parameterizations, the number of which depends on the number of link operator nodes.
For example, if a network has $12$ nodes with both activating and inhibitory regulators, then a total of $2^{12} = 4096$ Boolean models will be generated.
In case the number of all possible Boolean models is very large or space restrictions do not allow the storage of that many models, the software can also be used to generate a random sample of link operator Boolean models from the total parameterization space.
In summary, abmlog is a useful tool that can generate a large pool of Boolean models for subsequent analyses, each with a unique link operator parameterization.

\begin{figure
    \centering
    \includegraphics{images/abmlog.pdf}
    \caption{Data-flow overview diagram of the abmlog software and its related contingency table between output model parameterization and stable state activity. A simple interaction file is given as an input to produce a series of Boolean models where equations with both activating and inhibitory regulators have either the "AND-NOT" or the "OR-NOT" formulation. Two link operator equations give rise to a total of $2^2 = 4$ different Boolean models. Each unique parameterization can be represented by a single binary model number, where a "0" corresponds to an equation with the "AND-NOT" link operator and a "1" to an equation with the "OR-NOT". This representation of parameterization can be directly compared to each of the models' stable states, which enables the creation of a contingency table for the data pertaining to nodes B and C and the derivation of measures of agreement (see Section \ref{subsec:agree}).}
    \label{fig:2}
\end{figure}

\subsection{Measuring agreement between parameterization and stable state} \label{subsec:agree}

In order to quantify the link operator function bias, we use measures of agreement between parameterization and stable state.
The idea is that the more biased the link operator parameterization is, the higher the expected agreement will be between a target node's link operator assignment and its corresponding stable state.
For the rest of this work, we shall use two measures of agreement, namely the \textit{percent agreement} and Cohen's \textit{kappa statistic} \cite{Cohen1960}.

In more detail, using the Boolean model data generated by abmlog, we focus in two categorical variables related to a particular node of interest: its link operator parameterization ("AND-NOT"/"0" or "OR-NOT"/"1") and its corresponding stable state activity ("inhibition" or "activation"), obtained via attractor analysis.
We shall say that these two variables "agree" when a node whose target Boolean equation has the "AND-NOT" link operator (resp. "OR-NOT") ends up with an inhibited (resp. active) state in the corresponding attractor.
In the case of a Boolean model with multiple attractors, each of the stable states is used separately to measure the agreement between the two aforementioned variables, since the activity of a node might change between the different attractors, but its parameterization stays the same.

To define measures of agreement between the two proposed categorical variables, we visualize their interrelation using a contingency table.
A total of four data comparison counts can be used to fill in the table's cells: two where the parameterization and stable state match (i.e. node had the "AND-NOT" link operator form and an inhibited stable state or the "OR-NOT" form and an active state) and two where they differ (i.e. node had the "OR-NOT" form and its state was inhibited, or the "AND-NOT" form and an active state).
The percent agreement is then simply defined as the total number of matches divided by the total number of comparisons and is directly interpreted as the percentage of data that the two variables agree upon.
In the example of Figure \ref{fig:2}, the corresponding contingency table counts all the matches and mismatches between the link operator assignments for nodes B and C and their corresponding activity state ($12$ comparisons in total).
Since there are only two mismatches, the percent agreement is equal to $10/12 = 0.83$, meaning that in $83\%$ of the presented data, the link operator parameterization dictated function outcome.
Naturally, a value of $0$ is the absolute minimum score and indicates complete disagreement between the two variables while a perfect agreement score is equal to $1$ or $100\%$.

A more robust statistic that we also apply in the Boolean model data is Cohen's kappa $(\kappa)$ coefficient \cite{Cohen1960}.
This statistic is used to measure the extent to which data collectors (raters) assign the same score to the same variable (inter-rater reliability) and takes into account the possibility of agreement occurring by chance.
In our case, this can be conceived as one rater that assigns link operator parameterization ("AND-NOT" or "OR-NOT") and another that assigns stable state activity ("inhibition" or "activation").
Both variables are converted to a binary outcome ($0$ or $1$), allowing the creation of a contingency table and subsequently the calculation of Cohen's formula for $\kappa$.
The kappa statistic ranges from $-1$ to $+1$, where a value of $0$ represents the amount of agreement that can be expected from random chance, and a value of $1$ (resp. $-1$) indicates perfect agreement (resp. disagreement) between the raters.
In the example contingency table of Figure \ref{fig:2}, $\kappa = 0.657$, which is a considerable reduction in the level of congruence compared to the $0.83$ percent agreement.

\subsection{Truth Density bias in biological networks}

\subsubsection{Bias guides model parameterization in a cancer signaling network}

We used abmlog on a cancer signaling network, consisting of $77$ nodes and a total of $149$ curated causal interactions that cover a variety of pathways linked to prosurvival and antisurvival cell signaling (e.g. cyclin expression and caspase activation).
This PKN, named CASCADE (\textbf{CA}ncer \textbf{S}ignaling \textbf{CA}usality \textbf{D}atabas\textbf{E}), was successfully used to build a Boolean model able to predict anti-cancer drug combination effects in gastric cell lines \cite{Flobak2015}.
We used the CASCADE version from the Flobak paper (version CASCADE 1.0), with some node naming changes for compatibility with the newest versions \cite{cascade2020}.
The number of nodes with both activating and inhibiting regulators in the CASCADE 1.0 topology is $23$, while the rest of the nodes have regulators that belong to only one of the two regulatory categories.
Thus, using abmlog, we generated all $2^{23}$ possible Boolean models with the "AND-NOT" and "OR-NOT" link operator parameterizations.
The resulting stable state distribution across all produced models is presented in Figure \ref{fig:3A}.
For our subsequent analysis we will use only the \num{2802224} Boolean models that had exactly one stable state, as it makes the calculation of agreement between a node's assigned link operator and its corresponding activity state across all the selected models more straightforward.

The agreement results between link operator parameterization and stable state activity across all the selected CASCADE models are presented in Figures \ref{fig:3B} (percent agreement, per node) and \ref{fig:3C} (Cohen's $\kappa$, nodes with the same number of regulators are grouped together).
The percent agreement results show a high variability across the link operator nodes and range from a minimum of $53\%$ to a perfect agreement ($100\%$).
This suggests that for all nodes, across any selected CASCADE 1.0 Boolean model, there is a higher than random probability that the assignment of the "AND-NOT" (resp. "OR-NOT") link operator formula in the associated Boolean equations will result in the inhibition (resp. activation) of the target nodes.
So, even though none of the nodes have more than $5$ regulators, we already start seeing signs of the truth density bias in the link operator regulatory functions across a wide range of Boolean models.

When applying Cohen's $\kappa$ to evaluate level of agreement, we chose a conservative threshold equal to $0.6$, corresponding empirically to a substantial level of agreement \cite{Landis1977, McHugh2012}.
We found that $60\%$ ($14$ out of $23$) of the nodes have a $\kappa$ value below the specified threshold.
Our conclusion is that biological networks with higher in-degree nodes (i.e. more than $7-10$ regulators) are needed to properly assess if there is a truly high level of agreement between Boolean parameterization and function state outcome in the case of the link operator regulatory functions, providing thus conclusive proof of their bias (Section \ref{subsec:scale-free}).

\begin{figure}[!ht]
    \centering

    \makebox[\linewidth][c]{
        \begin{subfigure}{0.55\textwidth}
            \caption{} \label{fig:3A}
            \includegraphics[width=\textwidth]{images/ss_dist_cascade1.pdf}
        \end{subfigure}
        \hfill
        \begin{subfigure}{0.55\textwidth}
            \caption{} \label{fig:3B}
            \includegraphics[width=\textwidth]{images/ss-lo-agreement-per-node.pdf}
        \end{subfigure}
    }

    \vskip\baselineskip

    \makebox[\linewidth][c]{
        \begin{subfigure}{0.55\textwidth}
            \caption{} \label{fig:3C}
            \includegraphics[width=\textwidth]{images/ss-lo-cohen-per-node-group.pdf}
        \end{subfigure}
        \hfill
        \begin{subfigure}{0.55\textwidth}
            \caption{} \label{fig:3D}
            \includegraphics[width=\textwidth]{images/ss-lo-agreement-per-node-split.pdf}
        \end{subfigure}
    }

    \caption{(\textbf{A}) Stable states distribution across all link operator parameterized Boolean models generated by the abmlog software using the CASCADE 1.0 signaling topology. (\textbf{B}) Percent agreement scores between parameterization and activity state across all single stable state CASCADE 1.0 models, for $23$ nodes with both inhibiting and activating regulators. Nodes are sorted according to the total number of input regulators. (\textbf{C}) Same as (\textbf{B}), with the difference that the link operator nodes are now grouped into categories based on the total number of input regulators and Cohen's $\kappa$ is used as an agreement statistic. (\textbf{D}) Same as (\textbf{B}), with the agreement now calculated as the proportion of matches between a node's link operator and its activity state, in the models that had the specific parameterization. The link operator nodes are sorted according to the average activity state across the considered CASCADE models and the colored node labels indicate literature curated activity profiles from Flobak et al. \cite{Flobak2015}}
    \label{fig:3}
\end{figure}

Regardless of the presence of bias or not, the agreement results can be used to show how experimental data and topological regulatory knowledge (e.g. the activator-to-inhibitor ratio) can be coupled with the truth density metric to guide the choice of regulatory functions.
In one example scenario, a modeler asks what the most probable link operator parameterization is among the "AND-NOT" and "OR-NOT" forms that matches available experimental evidence. We used a literature curated activity profile derived for the AGS cell line from \cite{Flobak2015}, to annotate $7$ of the link operator nodes in Figure \ref{fig:3D} according to their experimentally validated state (activation or inhibition).
To clearly identify which of the two parameterizations best fits the observed data, for each node we split the CASCADE models in two model pools, representing the "AND-NOT" and "OR-NOT" node parameterizations, and calculated the proportion of models within each pool whose link operator matched the expected state outcome.
For example, in the contingency table of Figure \ref{fig:2}, the equivalent calculation would be to divide the number of matches in each row with the corresponding row total sum, resulting in $6/7 = 85.7\%$ of the "AND-NOT" Boolean equations with an inhibited stable state and $4/5 = 80\%$ of the "OR-NOT" equations with an active target node.
Moreover, the link operator nodes of Figure \ref{fig:3D} are sorted in increasing order by their average stable state activity in the considered CASCADE 1.0 Boolean models.
It is evident that nodes with higher average activity in the stable state have a higher agreement with the "OR-NOT" parameterization whereas nodes with lower average activity, a higher agreement with the "AND-NOT" parameterization ($0.85$ and $-0.74$ Pearson correlation coefficients with $p_{corr}^{\text{\tiny{OR-NOT}}}=2.6\times 10^{-7}$ and $p_{corr}^{\text{\tiny{AND-NOT}}}=5\times 10^{-5}$ respectively, see \nameref{sec:soft}).

More specifically, we observe that for all experimentally validated nodes, a modeler could a priori set the link operator to the appropriate form and get a stable state activation profile that matches the observations ("AND-NOT" to match an inhibition node profile or "OR-NOT" for an activation profile) with a higher probability than if he was randomly choosing one of the two.
For example, the data shows that 90\% of the models with an "OR-NOT" Boolean equation for the target family node \texttt{TCF7\_f}, had the node as active in their respective stable state.
The same is observed for the \texttt{CTNNB1} (92\%) and \texttt{ERK\_f} active nodes (74\%), as well as for the \texttt{TP53} (65\%) and \texttt{PTEN} (85\%) inhibited nodes with the choice of the "AND-NOT" parameterization.
Additionally, all the aforementioned nodes have two regulators (one activator and one inhibitor) and using the respective truth density formulas (Eq. \ref{td:and-not} and \ref{td:or-not}) with $n=2$ and $m=k=1$, we have that $TD_{AND-NOT} = 0.25$ (closer to $0$ or inhibition) and $TD_{OR-NOT} = 0.75$ (closer to $1$ or activation), as was also shown in Figure \ref{fig:1A}.
As such, the nodes observed output matches the statistically expected binary outcomes, showing that even with a low number of regulators, the BRF bias can be used to guide function choice.

In another scenario, a modeler knows that a particular node has a skewed activator-to-inhibitor ratio and wants to exploit such knowledge to make the node conform to a particular activity state of his choice.
A nice example from our data is the family node \texttt{LRP\_f}, with four activators and one inhibitor.
Using the truth density formulas for the two link operator parameterizations (Eq. \ref{td:and-not} and \ref{td:or-not}) with $n=5$, $m=4$ and $k=1$, we have that $TD_{AND-NOT} = 0.47$ and $TD_{OR-NOT} = 0.97$.
So, if the modeler wants to have an active \texttt{LRP\_f} in the stable state, the "OR-NOT" parameterization should be preferred since the "AND-NOT" has an approximate $50\%$ probability for this to happen from a statistical point of view.
These truth density values also match the results from Figure \ref{fig:3D}, since only half of the models that use the "AND-NOT" parameterization end up with an inhibited \texttt{LRP\_f} in the stable state while all of them have an active \texttt{LRP\_f} ($100\%$ agreement) in the case where the "OR-NOT" form is used.
Also, the average activity of \texttt{LRP\_f} across all models is one of the highest in the data, suggesting that imbalanced activator-to-inhibitor ratios could be a direct proxy for predicting regulation outcome.
In a similar situation, but at the other range of the activity spectrum, we have the \texttt{TSC\_f} family node with one activator and four inhibitors.
The truth density values (now using $n=5$, $m=1$ and $k=4$) are $TD_{AND-NOT} = 0.03$ and $TD_{OR-NOT} = 0.53$ respectively.
Therefore, the "AND-NOT" parameterization guarantees the inhibition of the \texttt{TSC\_f} node (data shows $100\%$ agreement) and it should be a modeler's first choice if that is the desired outcome.
On the other hand, if the activation of \texttt{TSC\_f} was a modeler's preference, then the choice of the "OR-NOT" form would be the most statistically appropriate according to the truth density metric.
We observe though that there was no model having \texttt{TSC\_f} inhibited in the stable stable, indicating that the complex dynamics of the cancer network can also play a significant role in the function outcome.
In general, we note that the particular configuration of activating and inhibiting regulators of a target in a specific model instance, can influence the dynamics attributable to the parameterization, causing several results from our analysis to differ from the expected behavior of the Boolean functions studied.

\subsubsection{Hub node bias in random scale-free networks} \label{subsec:scale-free}

\vspace{5pt}
In the previous section we showed that the truth density bias can be used to predict regulatory function outcome in a specific cancer signaling network, but the question still remains open for general biological networks.
Also, we found evidence suggesting that Boolean dynamics also plays a significant role in deciding each node's state in the attractors and in some cases activity state results may contradict what is expected from the use and asymptotic interpretation of the truth density formulas.
Therefore, we now proceed to investigate if networks with higher in-degree nodes (i.e. more input regulators) have stable states that can be unquestionably decided a priori by the truth density metric, using the respective $TD$ formulas for the "AND-NOT" and "OR-NOT" link operator parameterizations.

We study the specific class of scale-free networks \cite{Barabasi1999}, based on the hypothesis that most biological networks exhibit that property, i.e. their node degree distribution follows asymptotically a power law $P(k)\sim k^{-\gamma}$, with $k$ the number of regulators and $\gamma$ the scale-free exponent.
We note that the CASCADE 1.0 model also exhibits the scale-free property (see \nameref{sec:soft}) and there has been evidence in the literature both in favor and against this hypothesis.
In particular, earlier studies showed that many complex networks (including metabolic ones) are approximately scale-free \cite{Jeong2000, Wuchty2001, Albert2005, Aldana2007}, whereas more recent efforts demonstrated that not all cellular biological networks may share that property \cite{Khanin2006}, but those that do, exhibit the strongest level of evidence of scale-free structure \cite{Broido2019}.
Consequently, we shall use scale-free topologies as acceptable substitutes of real biological networks in our analysis.

The methodology is as follows: we start by generating scale-free topology files with a total of $50$ nodes each and a maximum in-degree $k_{max} = 50$ \cite{Mussel2010}.
For each network, the number of input regulators per node is drawn from a Riemann Zeta distribution with parameter $\gamma$ \cite{Aldana2003}.
The choice of regulators for each network node, as well as the type of regulation (positive or negative), is uniformly random.
The Zeta distribution allows the creation of in-degree values that far exceed the average connectivity in a network, giving rise to the highest-degree nodes (often called "hubs"), which are the most defining characteristic of the scale-free networks.
The value of the scale-free exponent influences the number of hubs and their in-degree distribution.
More specifically, we created scale-free networks with $\gamma = 2$ and $\gamma = 2.5$, since most of the studied networks have an exponent  between $2$ and $3$ \cite{Aldana2003, Albert2002}.
Comparing the networks built with the above methodology, we found that those with $\gamma = 2$ have more nodes with both activating and inhibiting regulators and higher degree hubs than networks with $\gamma = 2.5$ (Figures \ref{fig:4A} and  \ref{fig:4B}).
These two characteristics suggest that the scale-free networks with $\gamma = 2$ are better suited for use with the abmlog software, since the larger the number of link operator nodes, the more Boolean models can be generated and thus more data comparisons can be made between node parameterization and stable state activity.
Additionally, the presence of higher degree hubs is the perfect testbed for the link operator function bias, which manifests especially for nodes with more than $7-10$ regulators, as we found from our earlier truth density asymptotics analysis (Figure \ref{fig:1A}).

\begin{figure}[!ht]
    \centering

    \makebox[\linewidth][c]{
        \begin{subfigure}{0.55\textwidth}
            \caption{} \label{fig:4A}
            \includegraphics[width=\textwidth]{images/cmp_gamma_lo.pdf}
        \end{subfigure}
        \hfill
        \begin{subfigure}{0.55\textwidth}
            \caption{} \label{fig:4B}
            \includegraphics[width=\textwidth]{images/cmp_gamma_max_degree.pdf}
        \end{subfigure}
    }

    \vskip\baselineskip

    \makebox[\linewidth][c]{
        \begin{subfigure}{0.55\textwidth}
            \caption{} \label{fig:4C}
            \includegraphics[width=\textwidth]{images/agree-stats-gamma2-1.pdf}
        \end{subfigure}
        \hfill
        \begin{subfigure}{0.55\textwidth}
            \caption{} \label{fig:4D}
            \includegraphics[width=\textwidth]{images/agree-stats-gamma2-2.pdf}
        \end{subfigure}
    }

    \caption{(\textbf{A})-(\textbf{B}) Network statistics for scale-free topologies with different degree exponents. Every network tested has $50$ nodes and a maximum in-degree $k_{max} = 50$. A total of $100$ topologies for $\gamma = 2$ and $1000$ topologies for $\gamma = 2.5$ are compared. Networks with $\gamma = 2$ have a higher median number of nodes with both activating and inhibiting regulators and higher degree hubs. (\textbf{C})-(\textbf{D}) Agreement statistics between link operator parameterization and stable state activity. The data is taken from Boolean models generated with the abmlog software, using scale-free topologies with exponent $\gamma = 2$. A total of $757$ link operator nodes were compared across multiple link operator parameterization configurations with their corresponding stable states. Nodes are grouped in buckets, where each bucket indicates a different range of input regulators. Both the percent agreement and Cohen's $\kappa$ show considerable congruence between link operator assignment ("AND-NOT" or "OR-NOT") and resulting stable state (inhibition or activation respectively) for nodes with more than $10$ input regulators.}
    \label{fig:4}
\end{figure}

Our methodology proceeds with using each of the scale-free topologies with $\gamma = 2$ as input to the abmlog software, and generating ensembles of Boolean models parameterized with every possible mix of the "AND-NOT" and "OR-NOT" regulatory functions along with the calculation of their stable states (as demonstrated in Figure \ref{fig:2}).
The produced Boolean models had zero, one, or more stable states.
Interestingly, we observed that around half of the tested scale-free topologies generated Boolean models with no stable states, no matter which combination of link operators was used to define the model parameterization.
Therefore, the randomly assigned regulators, regulatory effects, and Zeta distribution in-degree values, may result in networks which do not have stable phenotypes, suggesting that alternative parameterizations might be more suitable in modeling scenarios which specifically examine stable dynamics.
Nonetheless, we discarded the models with no stable states and used the rest that had single or multiple attractors in our analysis.
Then, for each model node with both activating and inhibiting regulators, we compared its assigned link operator with the activity state value in the corresponding stable state(s), across all the link operator parameterization spectrum that yielded models with stable phenotypes.
The agreement results between parameterization and stable state activity are presented in Figure \ref{fig:4C} for the percent agreement and in Figure \ref{fig:4D} for Cohen's kappa statistic.

We observe that both presented statistics show a large variation of agreement for nodes with less than $10$ regulators and an increasing agreement with more regulators.
This agreement manifests in link operator nodes parameterized with the "AND-NOT" or "OR-NOT" Boolean functions, while at the same time exhibiting inhibited or active states respectively in the associated model attractors.
Therefore, we conclude that the considered standardized Boolean regulatory functions are biased and their outcomes can be determined a priori from the choice of the corresponding link operator parameterization, especially for nodes with more than $7-10$ regulators.

\section{Discussion} \label{sec:discuss}

The specification of mathematical rules that describe the behavior of biological systems is one of the core aspects of computational modeling.
It is therefore of considerable value to have a list of metrics that can be used to compare different model parameterizations and make an informed decision with regard to the selection of an appropriate regulatory function that better matches the expected behavior in a specific modeling application.

We specifically discussed two characterizations that can assist modelers in comparing various regulatory functions and select the most plausible ones with regard to the causal interaction-based knowledge at hand.
Expressing Boolean functions in DNF makes biological interpretation concrete by explicitly specifying the conditions (presence or absence of the positive and negative regulators, respectively) that make a target active.
Expressing the functions in CDNF allows to easily check for compliance with the underlying regulatory topology and subsequently, the rejection of functions that violate such consistency.
The difference between these two characterizations lies in the fact that the consistency terminology stems from the mathematical world, while biological interpretability is tightly connected to the world of language semantics and thus closer to the modeler's point of view.
Finally, truth density is an informative measure which can be used to verify if the function parameterization dictates biased Boolean outcomes.
It can also be used as a test metric to understand how a function behaves when the number of regulators increases or the balance between the number of activators and inhibitors changes.

Using the truth density metric, we showed the presence of link operator function bias in the hubs of randomly constructed scale-free networks.
A potential application of this finding could be to dramatically decrease the time needed to train Boolean models to fit observations via various optimization methods, by pre-assigning the parameterization of link operator nodes with sufficiently many regulators.
The pruning of the searchable parameterization space, guided by the truth density metric, can result in more efficient automated methods and can enable the training of larger models against data from numerous resources (e.g. large cell line panels).
The hub node bias has also interesting links to the presence of order in biological networks \cite{Kauffman1969}.
The dynamics of a Boolean network can exhibit ordered or chaotic behavior.
Ordered dynamics is characterized by the presence of less stable states and limit cycle attractors with smaller mean length (number of states in a complex attractor) and transition times (number of steps needed to reach an attractor starting out from an arbitrary configuration) \cite{Aldana2003}.
It is also known that the truth density (probability of target expression) as well as the degree exponent $\gamma$ (related to network connectivity) can modulate the dynamic transition between the ordered and chaotic phases.
Moreover, it has been shown that above the critical value of $\gamma_c \sim 2.47$, ordered behavior in the form of stable state dynamics manifests independently of the truth density, whereas for values closer to $\gamma = 2$, order coincides with the presence of high biased nodes (see Fig. 4 in \cite{Aldana2003}).
Our work confirms this phenomenon, since the use of the link operator parameterization guarantees the presence of biased hubs, which enable the scale-free networks to exhibit stability and homogeneity in terms of regulatory output, and thus stay in the ordered dynamic regime.

Searching for other function metrics that are applicable to logical modeling, the \textit{sensitivity} of a Boolean function is one of the most relevant \cite{Shmulevich2004}.
As its name suggests, it measures how sensitive the output of the function is to small changes of its inputs.
Sensitivity is tightly linked to the truth density metric, since a highly homogeneous Boolean function (i.e. a biased one), is unlikely to change its value between similar regulatory input configurations and so, its sensitivity is relatively low.
To compute the average sensitivity value for an arbitrary Boolean function we need to sum over all the \textit{influences} of the input variables, which essentially represent a way to measure individual variable importance.
In the context of regulatory functions, a regulator's influence is defined as the probability that a random toggle on its activity (from active to inactive and vice-versa) will change the value of the Boolean function \cite{Shmulevich2002}.
Therefore, by calculating the influence of every regulator, the modeler can gain knowledge of which ones are more important and control the respective function's outcome.
This transition of perspective from the function level to the regulator level might be advantageous in cases where the modeler's intention is to compare different parameterizations and choose the one for which a particular regulator is labeled as significantly more important than the others, based on the available biological knowledge.

Lastly, an important addition to a universal list of Boolean function metrics for modeling purposes, is the notion of function \textit{complexity}.
A recent definition is given by Gherardi et al. \cite{Gherardi2016}, where the authors defined it as the number of terms in the shortest possible DNF expression of a given Boolean function, divided by the total number of rows in the corresponding truth table.
We presented this information in the last column of Table \ref{tab:table2}, where the BRFs are sorted from lower to higher complexity (note that the CDNF has the minimum number of terms for every BRF included in the table).
One useful observation is that the standardized "AND-NOT" formula \cite{Mendoza2006} is the function with the lowest complexity that is also consistent and thus biologically plausible - all properties that make it a good choice from the modeler's perspective.
Assessing the complexity of the studied regulatory functions using the derived formulas for the minimum number of CDNF terms for any number of activators $m$ and inhibitors $k$ (see last column of Table \ref{tab:table2}), we comment on the fact that all BRFs have very low complexity since $\mathcal{O}(m \times k) \ll 2^{m+k}$, i.e. the number of function terms does not grow as fast as the number of rows in the corresponding truth table.
Same observation has been shown to be true in manually-tuned, experimentally-validated Boolean functions \cite{Gherardi2016}, providing us with another confirmation that the consistent functions from Table \ref{tab:table2} are good candidates for logic-based modeling approaches.

\section{Future work} \label{sec:future}

In this work we make an attempt to address the logical rule specification problem, which can be simply stated as: "Many functions may fit the available observations, which one is the most proper to use?"
Of course what is "proper" can be fairly subjective, but the main point is that a careful consideration of the underlying application context (i.e. what output do I expect in a specific scenario of interest) along with a list of metrics that explicate a Boolean function's behavior and semantics, provides the user with the appropriate framework to decide on the function parameterization that sets the basis for further model analysis and simulation.
In that regard, interesting directions for further research include the application of the metrics presented in this work in different published biological models, and the subsequent comparison of different regulatory functions within this framework.
Such meta-analyses could potentially indicate regulatory functions that achieve a higher degree of fitness with the observed data or general properties that are common in all Boolean functions used to model biological systems.

An interesting study for example would be to analyze Boolean functions from published biological models that have extreme activator-to-inhibitor ratios.
If such imbalanced ratios also result in proportionally skewed Boolean outcomes (i.e. with more activators, the truth density is closer to $1$ and the reverse with more inhibitors), suggesting that target outcome follows the majority regulatory groups, then the use of threshold functions could be a more proper parameterization alternative, as was shown in Figure \ref{fig:1B}.
Of course, we note that each individual case must be examined with care, since there might be high influence nodes, whose activity defines the target's output even in the presence of a much larger regulatory group with opposite effects.
For example, \texttt{CASP3} is a biological entity that, when activated, will almost certainly result in the cell's death even in the presence of a majority of proliferation-positive regulators at any given time.
Subsequently, a more appropriate choice based on the results of this study can be made, either by choosing between the biased functions, which demonstrate a more balanced behavior for such extreme activator-to-inhibitor ratios (e.g. using the "Pairs" or the "AND-NOT" functions which are balanced vs using the "OR-NOT" which would make the target activated most of the time, see \nameref{scenario:2}) or by using refined threshold functions, in which each regulator's weight will differ in order to match the influence that it has on the target.

There have been only a handful examples of published logical models \cite{Li2004, Davidich2008} and research papers \cite{Chaouiya2013, Wagner1994, Oikonomou2006, Kaderali2009, Greenbury2010, Jack2011, Zanudo2011, Sharan2013} that use the threshold modeling framework in biological systems.
This is partly due to the lack of tools that make threshold functions accessible to the average user, and the availability of such software in open-source environments such as the CoLoMoTo Interactive Notebook \cite{Naldi2018a}.
We believe that the existence of such novel software will enable the construction and configuration of generic Boolean threshold models and provide users of the logical-modeling community and beyond with the necessary toolbox to further study these models.
This will enable applications that depend on the dynamical analysis of Boolean threshold models (identification of attractors, reachability properties, formal verification and control) and the use of optimization methods to calibrate the threshold function parameters to best fit the available experimental data, as is done currently with analytical logic-based functions \cite{Gjerga2020}.

\section*{Software and Data Availability} \label{sec:soft}

The \textit{abmlog} software that was used to generate Boolean models with the "AND-NOT" and "OR-NOT" Boolean regulatory functions is available at \url{https://github.com/druglogics/abmlog} under the MIT License.
We used the version 1.6.0 for this analysis, which is also offered as a standalone package at \url{https://github.com/druglogics/abmlog/packages}.

An extended analysis accompanying the results of this paper is available at \url{https://druglogics.github.io/brf-bias}.
It includes links to the produced model datasets and scripts to reproduce the results and figures of this paper.
In particular, the correlation analysis between average node state in the CASCADE 1.0 models and percent agreement per each link operator is available at \url{https://druglogics.github.io/brf-bias/cascade-1-0-data-analysis.html#node-state-and-percent-agreement-correlation}.
The degree distribution of the CASCADE 1.0 topology and other network statistics are examined in \url{https://druglogics.github.io/brf-bias/cascade-1-0-data-analysis.html#network-properties}.

\section*{Funding}

This work was supported by ERACoSysMed Call 1 project COLOSYS (JZ, MK), project UIDB/50021/2020 from Fundação para a Ciência e a Tecnologia - INESC-ID multi-annual funding (PM), the Norwegian University of Science and Technology's Strategic Research Area 'NTNU Health' and The Joint Research Committee between St. Olavs hospital and the Faculty of Medicine and Health Sciences, NTNU - FFU (ÅF).

\textit{Conflict of Interest}: none declared.

\section*{Acknowledgments}

The authors acknowledge Dr. Vasundra Touré for her contribution in the improvement of Figure \ref{fig:2} and its caption.

\newpage

redpajama_set_name: RedPajamaArXiv


To share or copy and paste a page or image, get its web address, or URL. Chrome app: Tap Cut or Select all Copy. Depending on what browser you're using, select the image URL by tapping the links below.
Black Sunday is the collaboration album by American rapper and producer Sutter Kain with Donnie Darko, released in If album is bought via iTunes Track. Find a DJ Bless Aka Sutter Kain And Donnie Darko - Black Sunday first pressing or reissue. Complete your DJ Bless Aka Sutter Kain And Donnie Darko. Black Sunday by Sutter Kain & Donnie Darko, released 15 August 1. The Making of 2.
This page contains the VCP drivers currently available for FTDI devices. For D2XX Direct drivers, please click here. Installation guides are available from the. The FTR is the latest device to be added to FTDI's range of USB UART interface and 6MHz clock output signal Options for driving external MCU or FPGA.
The following is a list of episodes of South Korean variety game show television series X-Man. It was broadcast on SBS from November 8, to April 8, Info - English hardsubbed, Sub: cyxion, Size: 9 MB, Frame: x, Extension: AVI, Source: dbskarchives. SBS X-Man #69 - Ep.
All Eyez on Me is the fourth studio album by American rapper 2Pac released on February 13, 1 Background; 2 Recording and production . and hungry with All Eyez on Me, the first double-disc album of original material in hip-hop history. Find a 2Pac - All Eyez On Me first pressing or reissue. , All Bout U ..
3 Feb From the outset I should say I really wanted to like Hack Slash Loot more than I end up doing. It is a procedurally generated turn based. 5 Apr Interface, Full Audio, Subtitles. English Hack, Slash, Loot is a single-player turn -based dungeon crawler.
28 Oct 10 best zombie games for Android. Dead Effect 2. Price: Free with in-app purchases. Dead Trigger 2.
A grande tela de 5,7" aproveita totalmente a tecnologia In-cell para proporcionar mais clareza, cores brilhantes e maior. weekly victorgimenez.com weekly weekly victorgimenez.com weekly .com/radio/options/Remember-Jes%c3%bas-Cenamor-p/ weekly victorgimenez.com%c3%a3-Bossa-Nova-p/ . Can't Hold Us · Can't Remember To Forget You Cast of 'Juniors ' · Cast Of 'Oliver' ..
USB flash drive/pen drive corrupted or failed. Windows Chkdsk utility and EaseUS hard drive recovery software are coming together as the best USB repair tool to help you repair USB pen drive without losing important data. When it happens, you have to repair USB flash drive or pen. 5 Apr Use this application to backup and restore presentation, pictures, songs and applications from and to USB Flash Drive devices and take them.
SPYBOT download download full version with crack serial SPYBOT Cracks. - Underground Downloads Crack Serial Keygen. Direct Neon - Wikipedia .


Taking a break from dining at Michelin star restaurants as a way to keep my finances in check this year, I've opted to infuse home cooking into Bumascloset. The perfect night in entail laying out utensils, gathering all of the necessary ingredients to produce a meal from a carefully thought-out recipe. It's an exciting experience in trying something new that caught your attention and made your mouth water on a cooking channel or recreating a dish you enjoyed while on holiday, hoping that it will transport you to that place. That is if you know how.
The overwhelm of at-home cooking combined with the convenience of a takeaway available at your disposal, on your apps, fast food restaurants on every corner, and the alluring vibe of a Michelin star restaurant makes it tempting to go out. But we are aware that six months of dining out, takeaways add up to a four-night trip to Greece. With each calculation, indulging in luxuries should be limited if you're a student surviving on student finances or your priorities in life has forced the lifestyle change. These suggestions will help you kick your expensive habit and save money in the process.
As a child, I hated food shopping hopping in and out of the cab, going to different supermarkets and markets were exceptionally exhausting. But my mum had a unique approach to food shopping, rounding our regular shopping in Tescos, getting meat shopping in Upton Park, fish from Billingsgate Market, a stop to her favourite African store in Deptford Market for stock cubes, vegetable oil, a box of vegetables, yam and peppers, and much more.
When I got older and in my first year of university, supporting myself with saved up money from my summer job and student finance, I began to assess the products on their quality, comparing the supermarkets for the cheapest deals. Making sure I had the essentials such as Spice, Condiment, Oil, Cheese, Flour, Eggs in the fridge. Soon I became my mother, gathering ingredients from different shops to increase my creativity and save money.
I quickly figured that Aldi is the cheapest supermarket to shop for your essentials, Deptford market sells cheap fresh fish and chicken as it was the closest market to my student hall. So, I advice to always compare your supermarkets that fits your budget.
Secondly, meal plans are a daunting task that saves money and time. It makes you completely aware of the recipes that can stretch the budget. Ideally, before heading to shop for cheap always make a roundup of inexpensive food ingredients with nutritional value that can last you a week or month. Depending on your needs. For example; Rice, Beans, Sweet Potatoes, Chicken, Fish, Tofu, Mushrooms, Eggs, Lentils, Vegetables are versatile food that creates different meals which are available to buy at your local market. Best ways to help look for cheap recipes is via google. There are lots of accessible site offering cheap meals and recipes to follow.
Thirdly, at work lunchtime temptation is real. Getting lunch is the perfect escape to leave the office and get a breather. The meal deals prices are alluring, offering the options to get the main course, snack and a drink. However, this weekly routine adds up, costing you money rather than saving. Hence why you need to plan your meal in order to eat healthily and save some money.


IEE International Electricity Exhibition is an international trade fair for electricity and will take place in Tehran. At the exhibition a number of exhibitors from home and abroad shows their products and services for the industry. It offers its visitors the opportunity to learn about the latest trends to share with others and make new contacts.
Already for the nineteenth time the IEE Iran International Electricity Exhibition invites on 4 days from Wed., 30.10.2019 to Sat., 02.11.2019 in Tehran.
No.2, Shahid Saghafi alley (Golestan 1st), Derakhti Blvd.
Upload a picture from your booth on the IEE Iran International Electricity Exhibition or other fair impressions!


This protein-packed salad blends quinoa with healthful dandelion greens, fresh snow peas, walnuts, feta cheese and light, citrus-herb dressing.
Dressing: place lemon juice, olive oil, dill, lemon pepper, tarragon, garlic and remaining salt in a blender and blend for 30 seconds.
Salad: Rinse quinoa in a fine mesh strainer under running water to remove bitterness. Place in a pan with water and ½ tsp salt. Cover and bring to boil; reduce heat and simmer 15-20 minutes. Remove from heat and let cool completely before adding other ingredients. Place cooled quinoa into a large bowl. Pour dressing over top and toss. Add peas, dandelion greens, feta and chives and gently toss.


text:
Q: Difference between bound imports and delayed imports in PE header could any one please explain the differences between the bound imports  and delayed imports in PE header.I have referred few books but i cannot get to understand the concept of it clearly.can anyone help me out.

A: Bound imports means that in PE-import table store fixed (bound) addresses of import functions for a specific version of DLL with those function. Bound addresses are calculated and written to import table by linker during program compilation and linking phase.
Delayed imports means that in import table instead of import functions addresses contains addresses of a special program part called "delay load helper" (sometimes is called also "thunk"), which substitutes real imported function address when the function is called for the first time. And subsequent function calls use real function address written by delay load helper.
It is the concept. Details you can find in Iczelion's PE tutorial, for example

redpajama_set_name: RedPajamaStackExchange


text:
Nboris the spider children's book
Boris the spider would come towards you and you had to shoot him to stop the spider. American libraries canadian libraries universal library community texts project gutenberg biodiversity heritage library childrens. Boris the spider leapfrog rhyme time hardcover october 11, 2007. If the spider got to your end of the web he won but if he didnt, you won. Boris the spider is a song written by the whos bass guitarist, john entwistle. Leapfrog rhyme time features fun, rhyming stories of no more than 180 words for children who are starting to read on their own. Owning a big pet spider is only fun if you like spiders. Films released after 1959 should generally only be listed in the list of neonoir titles. Baroness hale becomes the unlikely star of new childrens book. The first asks children to fill in the missing rhyming words and the second encourages children to make. Full text of the bible, the missal, and the breviary. Look, hes crawling up my wall black and hairy, very small now hes up above my head hanging by a little thread. Diterlizzi born september 6, 1969 is an american fantasy artist, childrens book.
The boris johnson cut out book by nick mackie isbn. Boris johnson was raised by a chainsmoking nanny who became part of the family while his mother was in and out of hospital with crippling mental health issues the nanny became the former. Boris and bella is a spooky halloween book by carolyn crimi along with illustrations by gris. Especially when the spider runs all over the house. Boris the spider rhyming poetry worksheets teaching.
Can you guess the children s book from these emojis. This song is claimed to be entwistles first composition, and became a staple of live shows. Full text of transactions of the kansas academy of science. Because the 1940s and 1950s are universally regarded as the classic period of american film noir, films released prior to 1940 are listed under the caption precursors early noirlike films. There is a sample spread from the book and two activities. H undreds of years ago there was a king of scotland and his name was robert the bruce. The very busy spider by eric carle animated children s books is a channel dedicated to bringing everyones favorite books as a kid to life. This worksheet is based on the funny rhyming poem, boris the spider written by damian harvey and illustrated by daniel postgate, in the leapfrog rhyme time series. Boris johnsons dad admits leaving his newborn son in the. Inspire a love of reading with prime book box for kids discover delightful childrens books with prime book box, a subscription that delivers new books every 1. Boris the spider eats mosquitoes and flies, but his friends say he should eat burgers and fries. Mayor of london s dad admits leaving newborn son in the back of the car. Henry, a deaf field mouse, and his friend boris, a spider with one lam. Listen to boris the spider childrenary series by jeffrey p.
Boris the spider rhyming poetry worksheets teaching resources. Boris johnson was raised by a chainsmoking nanny their. If you enjoyed this book please subscribe for future. Jan 9, 2019 attachment is so fundamental to human development is should not be negated. Many good fantasy artists will tell you their influences are frazetta or boris vallejo. Parenting can be very tough and i know you want to do the best for your child. See more ideas about circle of security, attachment theory and social work. It was a good thing that he was both brave and wise, because the times in which he lived were wild and dangerous. He won a caldecott honor for his adaptation of the spider and the fly. Apr 14, 2019 boris johnson was raised by a chainsmoking nanny who became part of the family while his mother was in and out of hospital with crippling mental health issues the nanny became the former. Boris the spider has 540 books on goodreads, and is currently reading summer frost by blake crouch, the gone world by tom sweterlitsch, and end of the wi. It appears as the second track of their 1966 album a quick one. Can you guess the childrens book from these emojis. The very busy spider animated childrens book youtube.
590 466 310 528 303 1581 1144 306 997 995 195 626 1203 1592 935 344 1557 1064 608 307 878 1334 639 1504 877 422 771 881 1221 233 405 541 1644 634 204 1583 1561 1196 1116 434 655 36 879 758 1362 1069 1368 719 1282 801 71

redpajama_set_name: RedPajamaCommonCrawl


Have you been to the VM & Display show 2015 yet? No? We've included some of our favourite bits just for you but these things are naturally best experienced first hand. Yesterday was rocking and as always its so fantastic to see a great combination of the old and the new. New products and old friends. This is the biggest show we have seen here in a long time with new and exciting suppliers so its definitely a must for your diary. Just today and tomorrow left...!


Video Marketing is currently an integral part of business processes today. Any business with an active marketing plan will ensure to make video marketing a huge aspect of the plan. With video marketing strategies, it doesn't matter if you are a fortune 500 company or a small startup investment. It is important to realize that internet marketing keeps changing tides and the current trend is video marketing. This is why the invention of video glasses and HD sunglasses keeps upgrading. Inclusive is the fact that we are in the era of smartphones that has ensured access to video cameras which will continuously push the video marketing context.
You might be tempted to say your business or niche doesn't fall within the areas of video publicity. Just understand that the internet only recognizes the content that falls within the majority class. If your business doesn't really need video publicity and majority of the internet users are all about video, it's better to just jump in on it. As long as you wish to generate organic leads and conversions, it's time to add videos to your marketing toolkit.
The likes of YouTube have been able to make waves when it first went public and is still relevant till date because, what can you possibly not be able to learn on YouTube? From the most complex of things to the easiest of them all, YouTube has its users uploading the "How-To" videos regularly.
The marketing benefit of this implies that, YouTube isn't going out of the market anytime soon.
Would you want your business to have the same certainty as that of YouTube or Vimeo? Then this is the right time to mimic all they are doing right and attract the same traffic to your business. Another plus side to video marketing is that there are numerous free apps, available on the internet to help you reach your video marketing goal. Because they are free to use, small businesses can make some huge slashes with respect to their existing marketing budgets.
Increased Conversion Rate: Videos improves and attracts the human attention you need to keep your visitors strung to your website within the first 8 seconds. If the video on your page has a compelling illustration, then your visitors can turn into the necessary conversion that you need to boost your sales.
Better Search Engine Visibility: Ensure to identify the perfect SEO keywords in the lines of your business or niche. Most videos on the internet has a transcribed version associated to it, to help search engines read what is being displayed. These keywords will boost the visibility of your page on the likes of Google to help with suggestions for internet users who are actually searching for your contents.
Build Customer Trust: Video content allows your consumers to see how real your products and services are. That is why major video marketing platforms like YouTube, Instagram, Facebook, etc. has options for going live. If you are rendering your service to a client, it'll be beneficial if you go live streaming, so other prospects can easily believe. As the saying goes, seeing-is-believing! The more videos you make available to help educate your customers, the more they'll respect and take your business to heart.
Social Media Marketing: Allow video shares to social media platforms on your page, to enable website visitors show their own audience the exact thing they just saw. Leverage on your website visitors and continue to grow your conversions in areas and locations that are not limited by you.
It is essential to stay finding new ways to uniquely expand your sales, conversion and build a strong marketing list. The internet has moved sides to engage more of video contents, seize the opportunity to make your product or service stand out in the crowd. You can also make it more fun and create video contents for users who would love to use video glasses or HD sunglasses to view these videos. You might just be adding a new set of target market to your campaign. Video marketing is not new, but it has bought the heart of the millennials and is continually proving to be a powerful and effective part of business marketing strategy.