\chapter{Boolean Expressions}\label{ch05}
\section{Introduction}

Electronic circuits that do not require any memory devices (like flip-flops or registers) are created using what is called ``Combinational Logic.'' These systems can be quite complex, but all outputs are determined solely by input signals that are processed through a series of logic gates. Combinational circuits can be reduced to a Boolean Algebra expression, though it may be quite complex; and that expression can be simplified using methods developed in this chapter and Chapter \ref{ch06}, \nameref{ch06}, page \pageref{ch06}, and Chapter \ref{ch07}, \nameref{ch07}, page \pageref{ch07}. Combinational circuits are covered in Chapter \ref{ch08}, \nameref{ch08}, page \pageref{ch08}.

In contrast, electronic circuits that require memory devices (like flip-flops or registers) use what is called ``Sequential Logic.'' Those circuits often include feedback loops, so the final output is determined by input signals plus the feedback loops that are processed through a series of logic gates. This makes sequential logic circuits much more complex than combinational and the simplification of those circuits is covered in Chapter \ref{ch09}, \nameref{ch09}, page \pageref{ch09}.

Finally, most complex circuits include both combinational and sequential sub-circuits. In that case, the various sub-circuits would be independently simplified using appropriate methods. Several examples of these types of circuits are analyzed in Chapter \ref{ch10}, \nameref{ch10}, page \pageref{ch10}.

\section{Creating Boolean Expressions}

A circuit designer is often only given a written (or oral) description of a circuit and then asked to build that device. Too often, the designer may receive notes scribbled on the back of a dinner napkin, along with some verbal description of the desired output, and be expected to build a circuit to accomplish that task. Regardless of the form of the request, the process that the designer follows, in general, is:

\begin{enumerate}
  \item \textsc{Write The Problem Statement}. The problem to be solved is written in a clear, concise statement. The better this statement is written the easier each of the following steps will be, so time spent polishing the problem statement is worthwhile.

  \item \textsc{Construct A Truth Table}. Once the problem is clearly defined, the circuit designer constructs a truth table where all inputs/outputs are included. It is essential that all possible input combinations that lead to a \emph{True} output are identified.

  \item \textsc{Write A Boolean Expression}. When the truth table is completed, it is easy to create a Boolean expression from that table as covered in Example \ref{05:subsec:example}.

  \item \textsc{Simplify the Boolean Expression}. The expression should be simplified as much as possible, and that process is covered in this chapter, Chapter \ref{ch06}, \nameref{ch06}, page \pageref{ch06}, and Chapter \ref{ch07}, \nameref{ch07}, page \pageref{ch07}.

  \item \textsc{Draw The Logic Diagram}. The logic diagram for a circuit is constructed from the simplified Boolean expression.

  \item \textsc{Build The Circuit}. If desired, a physical circuit can be built using the logic diagram.
\end{enumerate}

\subsection{Example}
\label{05:subsec:example}

A machine is to be programmed to help pack shipping boxes for the ABC Novelty Company. They are running a promotion so if a customer purchases any two of the following items, but not all three, a free poster will be added to the purchase: joy buzzer, fake blood, itching powder. Design the logic needed to add the poster to appropriate orders.

\begin{enumerate}
  \item \textsc{Problem Statement}. The problem is already fairly well stated. A circuit is needed that will activate the ``drop poster'' machine when any two of three inputs (joy buzzer, fake blood, itching powder), but not all three, are \emph{True}.

  \item \textsc{Truth Table}. Let $ J $ be the Joy Buzzer, $ B $ be the Fake Blood, and $ P $ be the Itching Powder; and let the truth table inputs be \emph{True} (or $ 1 $) when any of those items are present in the shipping box. Let the output $ D $ be for ``Drop Poster'' and when it is \emph{True} (or $ 1 $) then a poster will be dropped into the shipping box. The Truth table is illustrated in Table  \ref{05:tab:truth_table_for_example}.

  \begin{table}[H]
    \sffamily
    \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
    \begin{center}
      \rowcolors{2}{gray!10}{white} % Color every other line a light gray
      \begin{tabular}{cccc}
        \rowcolor{black!75}
        \multicolumn{3}{c}{\head{Inputs}} & \head{Output} \\
        J & B & P & D \\
        \hline
        0 & 0 & 0 & 0 \\
        0 & 0 & 1 & 0 \\
        0 & 1 & 0 & 0 \\
        0 & 1 & 1 & 1 \\
        1 & 0 & 0 & 0 \\
        1 & 0 & 1 & 1 \\
        1 & 1 & 0 & 1 \\
        1 & 1 & 1 & 0
      \end{tabular}
    \end{center}
    \caption{Truth Table for Example}
    \label{05:tab:truth_table_for_example}
  \end{table}

  \item \textsc{Write Boolean Expression}. According to the Truth Table, the poster will be dropped into the shipping box in only three cases (when output $ D $ is \emph{True}). Equation \ref{05:eq:example} was generated from the truth table.

  \begin{align}
  \label{05:eq:example}
  BP + JP + JB &= D
  \end{align}

  \item \textsc{Simplify Boolean Expression}. The Boolean expression for this problem is already as simple as possible so no further simplification is needed.

  \item \textsc{Draw Logic Diagram}. Figure \ref{fig:05_01} was drawn from the switching equation.

	\begin{figure}[H]
		\centering
		\includegraphics[width=\maxwidth{.95\linewidth}]{gfx/05_01}
		\caption{Logic Diagram From Switching Equation}
		\label{fig:05_01}
	\end{figure}

  \item \textsc{Build The Circuit}. This circuit could be built (or ``realized'') with three \textsf{AND}  gates and one 3-input \textsf{OR}  gate.

\end{enumerate}

%***************************************************************************
% Section: Minterms and Maxterms
%***************************************************************************
\section{Minterms and Maxterms}
\label{05:sec:minterms_and_maxterms}

\subsection{Introduction}
\label{05:subsec:introduction_to_minterms_and_maxterms}

The solution to a Boolean equation is normally expressed in one of two formats: \ac{SOP} or \ac{POS}.

\subsection{Sum Of Products (SOP) Defined}
\label{05:subsec:sum_of_products_sop_defined}

Equation \ref{05:eq:sop_example} is an example of a \ac{SOP} expression. Notice that the expression describes four inputs ($ A $, $ B $, $ C $, $ D $) that are combined through two \textsf{AND} gates and then the output of those \textsf{AND} gates are combined through an \textsf{OR} gate.

\begin{align}
  \label{05:eq:sop_example}
  (A'BC'D)+(AB'CD) &= Y
\end{align}

Each of the two terms in this expression is a \emph{minterm}. Minterms can be identified in a Boolean expression as a group of inputs joined by an \textsf{AND} gate and then two or more minterms are combined with an \textsf{OR} gate. \marginpar{Notice the inverting bubble on three of the AND gate inputs.}The circuit illustrated in Figure \ref{fig:05_02} would realize Equation \ref{05:eq:sop_example}.

\begin{figure}[H]
	\centering
	\includegraphics[width=\maxwidth{.95\linewidth}]{gfx/05_02}
	\caption{Logic Diagram For SOP Example}
	\label{fig:05_02}
\end{figure}

\subsection{Product of Sums (POS) Defined}
\label{05:subsec:product_of_sums_pos_defined}

Equation \ref{05:eq:pos_example} is an example of a \ac{POS} expression. Notice that the expression describes four inputs ($ A $, $ B $, $ C $, $ D $) that are combined through two \textsf{OR} gates and then the output of those \textsf{OR} gates are combined through an \textsf{AND} gate.

\begin{align}
  \label{05:eq:pos_example}
  (A'+B+C+'D)(A+B'+C+D) &= Y
\end{align}

Each term in this expression is called a \emph{maxterm}. Maxterms can be identified in a Boolean expression as a group of inputs joined by an \textsf{OR} gate; and then two or more maxterms are combined with an \textsf{AND} gate. \marginpar{Notice the inverting bubble on three of the OR gate inputs.}The circuit illustrated in Figure \ref{fig:05_03} would realize Equation \ref{05:eq:pos_example}.

\begin{figure}[H]
	\centering
	\includegraphics[width=\maxwidth{.95\linewidth}]{gfx/05_03}
	\caption{Logic Diagram For POS Example}
	\label{fig:05_03}
\end{figure}

\subsection{About Minterms}
\label{05:subsec:about_minterms}

A term that contains all of the input variables in one row of a truth table joined with an \textsf{AND} gate is called a minterm. Consider truth table \ref{05:tab:truth_table_for_first_minterm_example} which is for a circuit with three inputs ( $ A $,  $ B $, and  $ C $) and one output ( $ Q $).

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{ccc|cc}
      \rowcolor{black!75}
      \multicolumn{3}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      A & B & C & Q & m\\
      \hline
      0 & 0 & 0 & 0 & 0 \\
      0 & 0 & 1 & 0 & 1 \\
      0 & 1 & 0 & 0 & 2 \\
      0 & 1 & 1 & 1 & 3 \\
      1 & 0 & 0 & 0 & 4 \\
      1 & 0 & 1 & 1 & 5 \\
      1 & 1 & 0 & 0 & 6 \\
      1 & 1 & 1 & 0 & 7
    \end{tabular}
  \end{center}
  \caption{Truth Table for First Minterm Example}
  \label{05:tab:truth_table_for_first_minterm_example}
\end{table}

The circuit that is represented by this truth table would output a \emph{True} in only two cases, when the inputs are $ A'BC $ or $ AB'C $. Equation \ref{05:eq:first_minterm_example} describes this circuit.

\begin{align}
  \label{05:eq:first_minterm_example}
  (A'BC)+(AB'C) &= Q
\end{align}

The terms $ A'BC $ and $ AB'C $ are called \emph{minterms} and they contain every combination of input variables that outputs a \emph{True} when the three inputs are joined by an \textsf{AND} gate. Minterms are most often used to describe circuits that have fewer \emph{True} outputs than \emph{False} (that is, there are fewer $ 1 $'s than $ 0 $'s in the output column). In the example above, there are only two \emph{True} outputs with six \emph{False} outputs, so minterms describe the circuit most efficiently.

Minterms are frequently abbreviated with a lower-case  $ m $ along with a subscript that indicates the decimal value of the variables. For example, $ A'BC $, the first of the \emph{True} outputs in the truth table above, has a binary value of $ 011 $, which is a decimal value of $ 3 $; thus, the minterm is $ m_3 $. The other minterm in this equation is $ m_5 $ since its binary value is $ 101 $, which equals decimal $ 5 $. It is possible to verbally describe the entire circuit as: $ m_3 $ \textsf{OR} $ m_5 $. For convenience, each of the minterm numbers are indicated in the last column of the truth table.

As another example, consider truth table \ref{05:tab:truth_table_for_second_minterm_example}.

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{cccc|cc}
      \rowcolor{black!75}
      \multicolumn{4}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      A & B & C & D & Q & m \\
      \hline
      0 & 0 & 0 & 0 & 1 & 0 \\
      0 & 0 & 0 & 1 & 0 & 1 \\
      0 & 0 & 1 & 0 & 0 & 2 \\
      0 & 0 & 1 & 1 & 0 & 3 \\
      0 & 1 & 0 & 0 & 0 & 4 \\
      0 & 1 & 0 & 1 & 1 & 5 \\
      0 & 1 & 1 & 0 & 1 & 6 \\
      0 & 1 & 1 & 1 & 0 & 7 \\
      1 & 0 & 0 & 0 & 0 & 8 \\
      1 & 0 & 0 & 1 & 0 & 9 \\
      1 & 0 & 1 & 0 & 0 & 10 \\
      1 & 0 & 1 & 1 & 0 & 11 \\
      1 & 1 & 0 & 0 & 0 & 12 \\
      1 & 1 & 0 & 1 & 0 & 13 \\
      1 & 1 & 1 & 0 & 1 & 14 \\
      1 & 1 & 1 & 1 & 0 & 15
    \end{tabular}
  \end{center}
  \caption{Truth Table for Second Minterm Example}
  \label{05:tab:truth_table_for_second_minterm_example}
\end{table}

Equation \ref{05:eq:second_minterm_example} describes this circuit because these are the rows where the output is \emph{True}.

\begin{align}
  \label{05:eq:second_minterm_example}
  (A'B'C'D')+(A'BC'D)+(A'BCD')+(ABCD') &= Q
\end{align}

These would be minterms $ m_0 $, $ m_5 $, $ m_6 $, and $ m_{14} $. Equation \ref{05:eq:sigma_notation_second_minterm_example} shows a commonly used, more compact way to express this result.

\begin{align}
  \label{05:eq:sigma_notation_second_minterm_example}
  \int(A,B,C,D) &= \sum(0,5,6,14)
\end{align}

Equation \ref{05:eq:sigma_notation_second_minterm_example} would read: ``For the function of inputs  $ A $,  $ B $,  $ C $, and  $ D $, the output is \emph{True} for minterms $ 0 $, $ 5 $, $ 6 $, and $ 14 $ when they are combined with an \textsf{OR}  gate.'' This format is called \emph{Sigma Notation}, and it is easy to derive the full Boolean equation from it by remembering that $ m_0 $ is $ 0000 $, or $ A'B'C'D' $; $ m_5 $ is $ 0101 $, or $ A'BC'D $; $ m_6 $ is $ 0110 $, or $ A'BCD' $; and $ m_{14} $ is $ 1110 $, or $ ABCD' $. Therefore, the Boolean equation can be quickly created from the Sigma Notation.

A logic equation that is created using minterms is often called the \emph{Sum of Products} (or \emph{SOP}) since each term is composed of inputs \textsf{AND}ed together (``products'') and the terms are then joined by \textsf{OR} gates (``sums'').

\subsection{About Maxterms}
\label{05:subsec:about_maxterms}

A term that contains all of the input variables joined with an \textsf{OR} gate (``added together'') for a \emph{False} output is called a \emph{maxterm}. Consider Truth Table \ref{05:tab:truth_table_for_first_maxterm_example}, which has three inputs ( $ A $,  $ B $, and  $ C $) and one output ( $ Q $):

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{ccc|cc}
      \rowcolor{black!75}
      \multicolumn{3}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      A & B & C & Q & M \\
      \hline
      0 & 0 & 0 & 1 & 0 \\
      0 & 0 & 1 & 1 & 1 \\
      0 & 1 & 0 & 0 & 2 \\
      0 & 1 & 1 & 1 & 3 \\
      1 & 0 & 0 & 1 & 4 \\
      1 & 0 & 1 & 1 & 5 \\
      1 & 1 & 0 & 0 & 6 \\
      1 & 1 & 1 & 1 & 7
    \end{tabular}
  \end{center}
  \caption{Truth Table for First Maxterm Example}
  \label{05:tab:truth_table_for_first_maxterm_example}
\end{table}

The circuit that is represented by this truth table would output a \emph{False} in only two cases. Since there are fewer \emph{False} outputs than \emph{True}, it is easier to create a Boolean equation that would generate the \emph{False} outputs. Because the equation describes the \emph{False} outputs, each term is built by \emph{complementing} the inputs for each of the \emph{False} output lines. After the output groups are defined, they are joined with an \textsf{AND} gate. Equation \ref{05:eq:first_maxterm_example} is the Boolean equation for Truth Table \ref{05:tab:truth_table_for_first_maxterm_example}.

\begin{align}
  \label{05:eq:first_maxterm_example}
  (A+B'+C)(A'+B'+C) &= Q
\end{align}

The terms $ A+B'+C $ and $ A'+B'+C $ are called $ maxterms $, and they contain the complement of the input variables for each of the \emph{False} output lines. Maxterms are most often used to describe circuits that have fewer \emph{False} outputs than \emph{True}. In Truth Table \ref{05:tab:truth_table_for_first_maxterm_example}, there are only two \emph{False} outputs with six \emph{True} outputs, so maxterms describe the circuit most efficiently.

Maxterms are frequently abbreviated with an upper-case  $ M $ along with a subscript that indicates the decimal value of the complements of the variables. For example, the complement of $ A+B'+C $, the first of the \emph{False} outputs in Truth Table \ref{05:tab:truth_table_for_first_maxterm_example}, is $ 010 $, which is a decimal value of 2; thus, the maxterm would be $ M_2 $. This can be confusing, but remember that the \emph{complements} of the inputs are used to form the expression. Thus, $ A+B'+C $ is $ 010 $, not $ 101 $. The other maxterm in this equation is $ M_6 $ since the binary value of its complement is $ 110 $, which equals decimal 6. It is possible to describe the entire circuit as a the product of two groups of maxterms: $ M_2 $ and $ M_6 $.

As another example, consider Truth Table \ref{05:tab:truth_table_for_second_maxterm_example}.

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{cccc|cc}
      \rowcolor{black!75}
      \multicolumn{4}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      A & B & C & D & Q & M \\
      \hline
      0 & 0 & 0 & 0 & 1 & 0 \\
      0 & 0 & 0 & 1 & 1 & 1 \\
      0 & 0 & 1 & 0 & 1 & 2 \\
      0 & 0 & 1 & 1 & 0 & 3 \\
      0 & 1 & 0 & 0 & 1 & 4 \\
      0 & 1 & 0 & 1 & 1 & 5 \\
      0 & 1 & 1 & 0 & 1 & 6 \\
      0 & 1 & 1 & 1 & 1 & 7 \\
      1 & 0 & 0 & 0 & 1 & 8 \\
      1 & 0 & 0 & 1 & 0 & 9 \\
      1 & 0 & 1 & 0 & 1 & 10 \\
      1 & 0 & 1 & 1 & 1 & 11 \\
      1 & 1 & 0 & 0 & 0 & 12 \\
      1 & 1 & 0 & 1 & 1 & 13 \\
      1 & 1 & 1 & 0 & 1 & 14 \\
      1 & 1 & 1 & 1 & 1 & 15
    \end{tabular}
  \end{center}
  \caption{Truth Table for Second Maxterm Example}
  \label{05:tab:truth_table_for_second_maxterm_example}
\end{table}

Equation \ref{05:eq:second_maxterm_example} describes this circuit.

\begin{align}
  \label{05:eq:second_maxterm_example}
  (A+B+C'+D')(A'+B+C+D')(A'+B'+C+D) &= Q
\end{align}

These would be maxterms $ M_3 $, $ M_9 $, and $ M_{12} $. Equation \ref{05:eq:pi_notation_second_maxterm_example} shows a commonly used, more compact way to express this result.

\begin{align}
  \label{05:eq:pi_notation_second_maxterm_example}
  \int(A,B,C,D) &= \prod(3,9,12)
\end{align}

Equation \ref{05:eq:pi_notation_second_maxterm_example} would read: ``For the function of  $ A $,  $ B $,  $ C $,  $ D $, the output is \emph{False} for maxterms $ 3 $, $ 9 $, and $ 12 $ when they are combined with an \textsf{AND}  gate.'' This format is called \emph{Pi Notation}, and it is easy to derive the Boolean equation from it. Remember that $ M_3 $ is $ 0011 $, or $ A+B+C'+D' $ (the complement of the inputs), $ M_9 $ is $ 1001 $, or $ A'+B+C+D' $, and $ M_{12} $ is $ 1100 $, or $ A'+B'+C+D $. The original equation can be quickly created from the Pi Notation.

A logic equation that is created using maxterms is often called the \emph{Product of Sums} (or \emph{POS}) since each term is composed of inputs \textsf{OR} ed together (``sums'') and the terms are then joined by \textsf{AND}  gates (``products'').

\subsection{Minterm and Maxterm Relationships}
\label{05:subsec:minterm_and_maxterm_relationships}

The minterms and maxterms of a circuit have three interesting relationships: equivalence, duality, and inverse. To define and understand these terms, consider Truth Table \ref{05:tab:minterm_and_maxterm_relationships_truth_table} for some unspecified ``black box'' circuit:

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{ccc|cc|cc}
      \rowcolor{black!75}
      \multicolumn{3}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} & \multicolumn{2}{c}{\head{Terms}} \\
      A & B & C & Q & Q' & minterm & Maxterm \\
      \hline
      0 & 0 & 0 & 0 & 1 & $ A'B'C' $ ($ m_0 $) & $ A+B+C $    ($ M_0 $) \\
      0 & 0 & 1 & 1 & 0 & $ A'B'C $  ($ m_1 $) & $ A+B+C' $   ($ M_1 $) \\
      0 & 1 & 0 & 0 & 1 & $ A'BC' $  ($ m_2 $) & $ A+B'+C $   ($ M_2 $) \\
      0 & 1 & 1 & 0 & 1 & $ A'BC $   ($ m_3 $) & $ A+B'+C' $  ($ M_3 $) \\
      1 & 0 & 0 & 0 & 1 & $ AB'C' $  ($ m_4 $) & $ A'+B+C $   ($ M_4 $) \\
      1 & 0 & 1 & 1 & 0 & $ AB'C $   ($ m_5 $) & $ A'+B+C' $  ($ M_5 $) \\
      1 & 1 & 0 & 0 & 1 & $ ABC' $   ($ m_6 $) & $ A'+B'+C $  ($ M_6 $) \\
      1 & 1 & 1 & 1 & 0 & $ ABC $    ($ m_7 $) & $ A'+B'+C' $ ($ M_7 $)
    \end{tabular}
  \end{center}
  \caption{Minterm and Maxterm Relationships}
  \label{05:tab:minterm_and_maxterm_relationships_truth_table}
\end{table}

\subsubsection{Equivalence}
\label{05:subsubsec:equivalence}

The minterms and the maxterms for a given circuit are considered equivalent ways to describe that circuit. For example, the circuit described by Truth Table \ref{05:tab:minterm_and_maxterm_relationships_truth_table} could be defined using minterms (Equation \ref{05:eq:sigma_notation_minterm_and_maxterm_equivalence}).

\begin{align}
  \label{05:eq:sigma_notation_minterm_and_maxterm_equivalence}
  \int(A,B,C) &= \sum(1,5,7)
\end{align}

However, that same circuit could also be defined using maxterms (Equation \ref{05:eq:pi_notation_minterm_and_maxterm_equivalence}).

\begin{align}
  \label{05:eq:pi_notation_minterm_and_maxterm_equivalence}
  \int(A,B,C) &= \prod(0,2,3,4,6)
\end{align}

These two functions describe the same circuit and are, consequently, equivalent. The \emph{Sigma Function} includes the terms $ 1 $, $ 5 $, and $ 7 $ while the \emph{Pi Function} includes all other terms in the truth table ($ 0 $, $ 2 $, $ 3 $, $ 4 $, and $ 6 $). To put it a slightly different way, the \emph{Sigma Function} describes the truth table rows where Q = $ 1 $ (minterms) while the \emph{Pi Function} describes the rows in the same truth table where Q =  $ 0 $ (maxterms). Therefore, Equation \ref{05:eq:minterm_and_maxterm_equivalence} can be derived.

\begin{align}
  \label{05:eq:minterm_and_maxterm_equivalence}
  \sum(1,5,7) &\equiv \prod(0,2,3,4,6)
\end{align}

\subsubsection{Duality}
\label{05:subsubsec:duality}

Each row in Truth Table \ref{05:tab:minterm_and_maxterm_relationships_truth_table} describes two terms that are considered duals. For example, minterm $ m_5 $ ($ AB'C $) and maxterm $ M_5 $ ($ A'+B+C' $) are duals. Terms that are duals are complements of each other ($ Q $ vs. $ Q' $) and the input variables are also complements of each other; moreover, the inputs for the three minterms are combined with an \textsf{AND}  while the maxterms are combined with an \textsf{OR}. The output of the circuit described by Truth Table \ref{05:tab:minterm_and_maxterm_relationships_truth_table} could be defined using minterms (Equation \ref{05:eq:q_output_dual}).

\begin{align}
  \label{05:eq:q_output_dual}
  Q &= \sum(1,5,7)
\end{align}

The dual of the circuit would be defined by using the maxterms for the same output rows. However, those rows are the \emph{complement} of the circuit (Equation \ref{05:eq:q_not_output_dual}).

\begin{align}
  \label{05:eq:q_not_output_dual}
  Q' &= \prod(1,5,7)
\end{align}

This leads to the conclusion that the complement of a \textit{Sigma Function} is the \textit{Pi Function} with the same inputs, as in Equation \ref{05:eq:dual} (the overline was used to emphasize the the fact that the \textit{PI Function} is complemented).

\begin{align}
  \label{05:eq:dual}
  \sum(1,5,7) &= \overline{\prod(1,5,7)}
\end{align}

\subsubsection{Inverse}
\label{05:subsubsec:inverse}

The complement of a function yields the opposite output. For example the following functions are inverses because one defines $ Q $ while the other defines $ Q' $ using only minterms of the same circuit (or truth table).

\begin{align}
  \label{05:eq:q_output_inverse}
  Q &= \sum(1,5,7)
\end{align}

\begin{align}
  \label{05:eq:q_not_output_inverse}
  Q' &= \sum(0,2,3,4,6)
\end{align}

\subsubsection{Summary}
\label{05:subsubsec:summary_minterm_and_maxterm_relationships}

These three relationships are summarized in the following table. Imagine a circuit with two or more inputs and an output of  $ Q $. Table \ref{05:tab:minterm_and_maxterm_relationships} summarizes the various relationships in the Truth Table for that circuit.

\begin{table}[H]
  \sffamily
  \begin{center}
    \begin{tabular}{|l|l|}
      \hline
      Minterms where Q is 1 & Minterms where Q' is 1 \\
      \hline
      Maxterms where Q is 1 & Maxterms where Q' is 1 \\
      \hline
    \end{tabular}
  \end{center}
  \caption{Minterm-Maxterm Relationships}
  \label{05:tab:minterm_and_maxterm_relationships}
\end{table}

The adjacent items in a single column are equivalent (that is,  $ Q $ Minterms are equivalent to  $ Q $ Maxterms), items that are diagonal are duals ( $ Q $ Minterms and  $ Q' $ Maxterms are duals), and items that are adjacent in a single row are inverses ( $ Q $ Minterms and  $ Q' $ Minterms are inverses).

\subsection{Sum of Products Example}
\label{05:subsubsec:sum_of_products_example}

\subsubsection{Given}
\label{05:subsubsec:given_sop_example}

A ``vote-counter'' machine is designed to turn on a light if any two or more of three inputs are \emph{True}. Create a circuit to realize this machine.

\subsubsection{Truth Table}
\label{05:subsubsec:truth_table_sop_example}

When realizing a circuit from a verbal description, the best place to start is constructing a truth table. This will make the Boolean expression easy to write and then make the circuit easy to realize. For the ``vote-counter'' problem, start by defining variables for the truth table: Inputs  $ A $,  $ B $,  $ C $ and Output  $ Q $.

Next, construct the truth table by identifying columns for each of the three input variables and then indicate the output for every possible input condition (Table \ref{05:tab:truth_table_for_sop_example}).

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{ccc|cc}
      \rowcolor{black!75}
      \multicolumn{3}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      A & B & C & Q & m\\
      \hline
      0 & 0 & 0 & 0 & 0 \\
      0 & 0 & 1 & 0 & 1 \\
      0 & 1 & 0 & 0 & 2 \\
      0 & 1 & 1 & 1 & 3 \\
      1 & 0 & 0 & 0 & 4 \\
      1 & 0 & 1 & 1 & 5 \\
      1 & 1 & 0 & 1 & 6 \\
      1 & 1 & 1 & 1 & 7
    \end{tabular}
  \end{center}
  \caption{Truth Table for SOP Example}
  \label{05:tab:truth_table_for_sop_example}
\end{table}

\subsubsection{Boolean Equation}
\label{05:subsubsec:boolean_equation_sop_example}

In a truth table, if there are fewer \emph{True} outputs than \emph{False}, then it is easiest to construct a Sum-of-Products equation. In that case, a Boolean expression can be derived by creating the minterms for all \emph{True} outputs and combining those minterms with \textsf{OR}  gates. Equation \ref{05:eq:sigma_notation_sop_example} is the \emph{Sigma Function} of this circuit.

\begin{align}
  \label{05:eq:sigma_notation_sop_example}
  \int(A,B,C) &= \sum(3,5,6,7)
\end{align}

% Pull Quote - Marginal Note - Sidebar
\marginpar{It may be possible to simplify the Boolean expression, but that process is covered elsewhere in this book.}

At this point, a circuit could be created with four 3-input \textsf{AND}  gates combined into one 4-input \textsf{OR}  gate.

\subsection{Product of Sums Example}
\label{05:subsubsec:product_of_sums_example}

\subsubsection{Given}
\label{05:subsubsec:given_pos_example}

A local supply company is designing a machine to sort packages for shipping. All packages go to the post office \emph{except} packages going to the local ZIP code containing chemicals (they are shipped by courier) and packages going to a distant ZIP code containing only perishables (they are shipped via air freight).

\subsubsection{Truth Table}
\label{05:subsubsec:truth_table_pos_example}

When realizing a circuit from a verbal description, the best place to start is constructing a truth table. This will make the Boolean expression easy to write and then make the circuit easy to realize. For the sorting machine problem, start by defining variables for the truth table:

\begin{itemize}
  \item Ship via post office (the Output):  $ O $ is \emph{True} if ship by Post Office
  \item Zip Code:  $ Z $ is \emph{True} if the zip code is local
  \item Chemicals:  $ C $ is \emph{True} if the package contains chemicals
  \item Perishable:  $ P $ is \emph{True} if the package contains perishables
\end{itemize}

Next, construct the truth table by identifying columns for each of the three input variables and then indicate the output for every possible input condition (Table \ref{05:tab:truth_table_for_sop_example}).

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{ccc|cc}
      \rowcolor{black!75}
      \multicolumn{3}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      Z & C & P & O & M \\
      \hline
      0 & 0 & 0 & 1 & 0 \\
      0 & 0 & 1 & 0 & 1 \\
      0 & 1 & 0 & 1 & 2 \\
      0 & 1 & 1 & 0 & 3 \\
      1 & 0 & 0 & 1 & 4 \\
      1 & 0 & 1 & 1 & 5 \\
      1 & 1 & 0 & 0 & 6 \\
      1 & 1 & 1 & 0 & 7
    \end{tabular}
  \end{center}
  \caption{Truth Table for POS Example}
  \label{05:tab:truth_table_for_pos_example}
\end{table}

\subsubsection{Boolean Equation}
\label{05:subsubsec:boolean_equation_pos_example}

In the truth table, if there are fewer \emph{False} outputs than \emph{True} then it is easiest to construct a Products-of-Sums equation. In that case, a Boolean expression can be derived by creating the maxterms for all \emph{False} outputs and combining the complement those maxterms with \textsf{AND}  gates. Equation \ref{05:eq:pi_notation_pos_example} is the \emph{Pi Expression} of this circuit.

\begin{align}
  \label{05:eq:pi_notation_pos_example}
  \int(Z,C,P) &= \prod(1,3,6,7)
\end{align}

% Pull Quote - Marginal Note - Sidebar
\marginpar{It may be possible to simplify the Boolean expression, but that process is covered elsewhere in this book.}

At this point, a circuit could be created with four 3-input \textsf{OR}  gates combined into one 4-input \textsf{AND}  gate.

\subsection{Summary}
\label{05:subsubsec:summary_minterms_and_maxterms}

\ac{SOP} Boolean expressions may be generated from truth tables quite easily, by determining which rows of the table have an output of \emph{True}, writing one minterm for each of those rows, and then summing all of the minterms. The resulting expression will lend itself well to implementation as a set of \textsf{AND} gates (products) feeding into a single \textsf{OR} gate (sum).

\ac{POS} Boolean expressions may be generated from truth tables quite easily, by determining which rows of the table have an output of \emph{False}, writing one maxterm for each of those rows, and then multiplying all of the maxterms. The resulting expression will lend itself well to implementation as a set of \textsf{OR} gates (sums) feeding into a single \textsf{AND} gate (product).

%***************************************************************************
% Section: Canonical Form
%***************************************************************************
\section{Canonical Form}
\label{05:sec:canonical_form}

\subsection{Introduction}
\label{05:subsec:introduction_to_canonical_form}

The word ``canonical'' simply means ``standard'' and it is used throughout mathematics and science to denote some standard form for equations. In digital electronics, Boolean equations are considered to be in canonical form when each of the terms in the equation includes all of the possible inputs and those terms appear in the same order as in the truth table. Using the canonical form is important when simplifying a Boolean equation. For example, imagine the solution to a given problem generated table \ref{05:tab:canonical_example_truth_table}.

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{ccc|cc}
      \rowcolor{black!75}
      \multicolumn{3}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      A & B & C & Q & m \\
      \hline
      0 & 0 & 0 & 0 & 0 \\
      0 & 0 & 1 & 1 & 1 \\
      0 & 1 & 0 & 0 & 2 \\
      0 & 1 & 1 & 1 & 3 \\
      1 & 0 & 0 & 0 & 4 \\
      1 & 0 & 1 & 0 & 5 \\
      1 & 1 & 0 & 0 & 6 \\
      1 & 1 & 1 & 1 & 7
    \end{tabular}
  \end{center}
  \caption{Canonical Example Truth Table}
  \label{05:tab:canonical_example_truth_table}
\end{table}

Minterm equation \ref{05:eq:canonical_example} is derived from the truth table and is presented in canonical form. Notice that each term includes all possible inputs ( $ A $,  $ B $, and  $ C $), and that the terms are in the same order as they appear in the truth table.

\begin{align}
  \label{05:eq:canonical_example}
  (A'B'C)+(A'BC)+(ABC) &= Q
\end{align}

Frequently, though, a Boolean equation is expressed in standard form, which is not the same as canonical form. Standard form means that some of the terms have been simplified and not all of the inputs will appear in all of the terms. For example, consider Equation \ref{05:eq:standard_form_example}, which is the solution for a 4-input circuit.

\begin{align}
  \label{05:eq:standard_form_example}
  (A'C)+(B'CD) &= Q
\end{align}

This equation is in standard form so the first term, $ A'C $, does not include inputs  $ B $ or  $ D $ and the second term, $ B'CD $, does not include input  $ A $. However, all inputs must be present in every term for an equation to be in canonical form.

Building a truth table for an equation in standard form raises an important question. Consider the truth table for Equation \ref{05:eq:standard_form_example}.

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{cccc|cc}
      \rowcolor{black!75}
      \multicolumn{4}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      A & B & C & D & Q & m \\
      \hline
      0 & 0 & 0 & 0 & 0 & 0 \\
      0 & 0 & 0 & 1 & 0 & 1 \\
      0 & 0 & 1 & 0 & 0 & 2 \\
      0 & 0 & 1 & 1 & ? & 3 \\
      0 & 1 & 0 & 0 & 0 & 4 \\
      0 & 1 & 0 & 1 & 0 & 5 \\
      0 & 1 & 1 & 0 & 0 & 6 \\
      0 & 1 & 1 & 1 & 0 & 7 \\
      1 & 0 & 0 & 0 & 0 & 8 \\
      1 & 0 & 0 & 1 & 0 & 9 \\
      1 & 0 & 1 & 0 & 0 & 10 \\
      1 & 0 & 1 & 1 & ? & 11 \\
      1 & 1 & 0 & 0 & 0 & 12 \\
      1 & 1 & 0 & 1 & 0 & 13 \\
      1 & 1 & 1 & 0 & 0 & 14 \\
      1 & 1 & 1 & 1 & 0 & 15
    \end{tabular}
  \end{center}
  \caption{Truth Table for Standard Form Equation}
  \label{05:tab:truth_table_for_standard_form_equation}
\end{table}

In what row would $ B'CD $, the second term in Equation \ref{05:eq:standard_form_example}, be placed? $ B'CD $ is $ 011 $ (in binary), but since the $ A $ term is missing would it be a $ 0 $ or $ 1 $; in other words, would $ B'CD $ generate an output of $ 1 $ for row $ 0011\;(m_3)$ or $ 1011\;(m_{11})$? (The output for these two rows are marked with a question mark in Table \ref{05:tab:truth_table_for_standard_form_equation}.) In fact, the output for \emph{both} of these rows must be considered \emph{True} in order to ensure that all possible combinations of input are covered. Thus, the final equation for this circuit must include at least these two terms: $ (A'B'CD) + (AB'CD) $. In the same way, the term $ A'C $ means that the output is \emph{True} for $ m_2 $, $ m_3 $, $ m_6 $, and $ m_7 $ since input $ A'C $ is \emph{True} and any minterm that contains those two value is also considered \emph{True}. Thus, the final equation for this circuit must include at least these four terms: $ (A'B'CD') + (A'B'CD) + (A'BCD') + (A'BCD) $.

\subsection{Converting Terms Missing One Variable}
\label{05:subsec:converting_terms_missing_one_variable}

To change a standard Boolean expression that is missing one input term into a canonical Boolean expression, insert both \emph{True} and \emph{False} for the missing term into the original standard expression. As an example, consider the term $ B'CD $. Since term $ A $ is missing, both $ A $ and $ A' $ must be included in the converted canonical expression. Equation \ref{05:eq:canonical_expanding_3-variable_term} proves that $ B'CD $ can be expanded to include both possible values for $ A $ by using the Adjacency Property (page \pageref{BF:subsec:adjacency_property}).

\begin{align}
  \label{05:eq:canonical_expanding_3-variable_term}
  (B'CD) \rightarrow (AB'CD)+(A'B'CD)
\end{align}

A term that is missing one input variable will expand into two terms that include all variables. For example, in a system with four input variables (as above), any standard term with only three variables will expand to a canonical expression containing two groups of four variables.

Expanding a standard term that is missing one variable can also be done with a truth table. To do that, fill in an output of $ 1 $ for every line where the \emph{True} inputs are found while ignoring all missing variables. As an example, consider Truth Table \ref{05:tab:truth_table_for_standard_form_equation} where the outputs for $ m_3 $ and $ m_11 $ are marked with a question mark. However, the output for both of these lines should be marked as \emph{True} because  $ B'CD $ is \emph{True} (input $ A $ is ignored). Then, those two minterms lead to the Boolean expression $ AB'CD + A'B'CD $.

\subsection{Converting Terms Missing Two Variables}
\label{05:subsec:converting_terms_missing_two_variables}

It is easiest to expand a standard expression that is missing two terms by first inserting one of the missing variables and then inserting the other missing variable in two distinct steps. The process for inserting a single missing variable is found in Section \ref{05:subsec:converting_terms_missing_one_variable}. Consider the term $ A'C $ in a four-variable system. It is missing both the $ B $ and $ D $ variables. To expand that term to its canonical form, start by inserting either of the two missing variables. For example, Equation \ref{05:eq:canonical_expanding_2-variable_term_step_one} illustrates entering $ B $ and $ B' $ into the expression.

\begin{align}
  \label{05:eq:canonical_expanding_2-variable_term_step_one}
  (A'C) &\rightarrow (A'BC)+(A'B'C)
\end{align}

Then, Equation \ref{05:eq:canonical_expanding_2-variable_term_step_two} illustrates inserting $ D $ and $ D' $ into the expression.

\begin{align}
  \label{05:eq:canonical_expanding_2-variable_term_step_two}
  (A'BC) &\rightarrow (A'BCD)+(A'BCD') \\
  \nonumber
  (A'B'C) &\rightarrow (A'B'CD)+(A'B'CD')
\end{align}

In the end, $ A'C $ expands to Equation \ref{05:eq:canonical_expanding_2-variable_term_step_three}:

\begin{align}
  \label{05:eq:canonical_expanding_2-variable_term_step_three}
  (A'C) \rightarrow &(A'BCD) + (A'BCD') \\
  \nonumber
  &+ (A'B'CD) + (A'B'CD')
\end{align}

Thus, in a four-variable system, any standard term with only two variables will expand to a canonical expression with four groups of four variables.

Expanding a standard term that is missing two variables can also be done with a truth table. To do that, fill in an output of $ 1 $ for every line where the \emph{True} inputs are found while ignoring all missing variables. As an example, consider a Table \ref{05:tab:truth_table_for_expression_with_missing_variables}, where $ A'C $ is marked as \emph{True}:

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{cccc|cc}
      \rowcolor{black!75}
      \multicolumn{4}{c}{\head{Inputs}} & \multicolumn{2}{c}{\head{Outputs}} \\
      A & B & C & D & Q & m \\
      \hline
      0 & 0 & 0 & 0 & 0 & 0 \\
      0 & 0 & 0 & 1 & 0 & 1 \\
      0 & 0 & 1 & 0 & 1 & 2 \\
      0 & 0 & 1 & 1 & 1 & 3 \\
      0 & 1 & 0 & 0 & 0 & 4 \\
      0 & 1 & 0 & 1 & 0 & 5 \\
      0 & 1 & 1 & 0 & 1 & 6 \\
      0 & 1 & 1 & 1 & 1 & 7 \\
      1 & 0 & 0 & 0 & 0 & 8 \\
      1 & 0 & 0 & 1 & 0 & 9 \\
      1 & 0 & 1 & 0 & 0 & 10 \\
      1 & 0 & 1 & 1 & 0 & 11 \\
      1 & 1 & 0 & 0 & 0 & 12 \\
      1 & 1 & 0 & 1 & 0 & 13 \\
      1 & 1 & 1 & 0 & 0 & 14 \\
      1 & 1 & 1 & 1 & 0 & 15
    \end{tabular}
  \end{center}
  \caption{Truth Table for Standard Form Equation}
  \label{05:tab:truth_table_for_expression_with_missing_variables}
\end{table}

Notice that outputs for $ m_2 $, $ m_3 $, $ m_6 $, and $ m_7 $ are \emph{True} because for each of those minterms $ A'C $ is \emph{True} (inputs $ B $ and $ D $ are ignored). Then, those four minterms lead to the Boolean expression $ A'B'CD'+A'B'CD+A'BCD'+A'BCD $.

\subsection{Summary}
\label{05:subsec:summary_of_canonical_forms}

This discussion started with Equation \ref{05:eq:standard_form_example_repeated}, which is in standard form.

\begin{align}
  \label{05:eq:standard_form_example_repeated}
  (A'C)+(B'CD) &= Q
\end{align}

After expanding both terms, Equation \ref{05:eq:canonical_form_summary_equation_not_simplified} is generated.

\begin{align}
  \label{05:eq:canonical_form_summary_equation_not_simplified}
  &(A'BCD)+(A'BCD')+(A'B'CD) \\
  \nonumber
  +&(A'B'CD')+(AB'CD)+(A'B'CD) = Q
\end{align}

Notice, though, that the term $ A'B'CD $ appears two times, so one of those can be eliminated by the Idempotence property (page \pageref{BF:subsec:idempotence}), leaving Equation \ref{05:eq:canonical_form_summary_equation_simplified}.

\begin{align}
  \label{05:eq:canonical_form_summary_equation_simplified}
  &(A'BCD)+(A'BCD') \\
  \nonumber
  +&(A'B'CD')+(AB'CD)+(A'B'CD) = Q
\end{align}

To put the equation in canonical form, which is important for simplification; all that remains is to rearrange the terms so they are in the same order as they would appear in a truth table, which results in Equation \ref{05:eq:canonical_form_summary_equation_final}.

\begin{align}
  \label{05:eq:canonical_form_summary_equation_final}
  &(A'B'CD')+(A'B'CD) \\
  \nonumber
  +&(A'BCD')+(A'BCD)+(AB'CD) = Q
\end{align}

\subsection{Practice Problems}
\label{05:subsec:practice_with_canonical_forms}

\marginpar{Parenthesis were not used in order to save space; however, the variable groups are evident.}

\begin{table}[H]
  \sffamily
  \begin{center}
    \begin{tabular}{c c p{6cm} }
      \multirow{2}{*}{\textbf{1}}
        & Standard (A,B,C) & $ A'B+C+AB' $ \\
        & \cellcolor{gray!10} Cannonical
        & \cellcolor{gray!10} $ A'B'C+A'BC'+A'BC+AB'C'+AB'C+ABC $ \\
      \hline
      \multirow{2}{*}{\textbf{2}}
      & Standard (A,B,C,D) & $ A'BC+B'D $ \\
      & \cellcolor{gray!10} Cannonical
      & \cellcolor{gray!10} $ A'B'C'D'+A'B'CD'+A'BCD'+A'BCD+AB'C'D'+AB'CD' $ \\
      \hline
      \multirow{2}{*}{\textbf{3}}
      & Standard (A,B,C,D) & $ A'+D $ \\
      & \cellcolor{gray!10} Cannonical
      & \cellcolor{gray!10} $ A'B'C'D'+A'B'C'D+A'B'CD'+A'B'CD+A'BC'D'       +A'BC'D+A'BCD'+A'BCD+AB'C'D+AB'CD+ABC'D+ABCD $ \\
      \hline
      \multirow{2}{*}{\textbf{4}}
      & Standard (A,B,C) & $ A(B'+C) $ \\
      & \cellcolor{gray!10} Cannonical
      & \cellcolor{gray!10} $ AB'C'+AB'C+ABC $ \\
    \end{tabular}
  \end{center}
  \caption{Canonical Form Practice Problems}
  \label{05:tab:canonical_form_practice_problems}
\end{table}

%***************************************************************************
% Section: Simplification Using Algebraic Methods
%***************************************************************************
\section{Simplification Using Algebraic Methods}
\label{05:sec:simplification_using_algebraic_methods}

\subsection{Introduction}
\label{05:subsec:introduction_to_algebraic_methods}

One method of simplifying a Boolean equation is to use common algebraic processes. It is possible to reduce an equation step-by-step using the various properties of Boolean algebra in the same way that real-number equations can be simplified.

\subsection{Starting From a Circuit}
\label{05:subsec:starting_from_a_circuit}

Occasionally, the circuit designer is faced with an existing circuit and must attempt to simplify it. In that case, the first step is to find the Boolean equation for the circuit and then simplify that equation.

\subsubsection{Generate a Boolean Equation}
\label{05:subsubsec:generate_a_boolean_equation}

In the circuit illustrated in Figure \ref{fig:05_04}, the  $ A $,  $ B $, and  $ C $ input signals are assumed to be provided from switches, sensors, or perhaps other sub-circuits. Where these signals originate is of no concern in the task of gate reduction.

\begin{figure}[H]
	\centering
	\includegraphics[width=\maxwidth{.95\linewidth}]{gfx/05_04}
	\caption{Example Circuit}
	\label{fig:05_04}
\end{figure}

To generate the Boolean equation for a circuit, write the output of each gate as determined by the input signals and type of gate, working from the inputs to the final output. Figure \ref{fig:05_05} illustrates the result of this process.

\begin{figure}[H]
	\centering
	\includegraphics[width=\maxwidth{.95\linewidth}]{gfx/05_05}
	\caption{Example Circuit With Gate Outputs}
	\label{fig:05_05}
\end{figure}

This process leads to Equation \ref{05:eq:equation_from_circuit}.

\begin{align}
  \label{05:eq:equation_from_circuit}
  AB+BC(B+C) &= Y
\end{align}

\subsection{Starting From a Boolean Equation}
\label{05:subsec:starting_from_a_boolean_equation}

If a logic circuit's function is expressed as a Boolean equation, then algebraic methods can be applied to reduce the number of logic gates, resulting in a circuit that performs the same function with fewer components. As an example, Equation \ref{05:soln:equation_to_simplify_one_solution} simplifies the circuit found in Figure \ref{05:fig:circuit_with_boolean_expression}.

\begin{align}
\label{05:soln:equation_to_simplify_one_solution}
  AB+BC(B+C) && \text{Original Expression} \\
  \nonumber
  AB+BBC+BCC && \text{Distribute BC} \\
  \nonumber
  AB+BC+BC && \text{Idempotence: BB=B and CC=C} \\
  \nonumber
  AB+BC && \text{Idempotence: BC+BC=BC} \\
  \nonumber
  B(A+C) && \text{Factor}
\end{align}

The final expression, $ B(A + C) $, requires only two gates, and is much simpler than the original, yet performs the same function. Such component reduction results in higher operating speed (less gate propagation delay), less power consumption, less cost to manufacture, and greater reliability.

As a second example, consider Equation \ref{05:eq:equation_to_simplify_two}.

\begin{align}
  \label{05:eq:equation_to_simplify_two}
  A+AB &= Y
\end{align}

This is simplified below.

\begin{align}
\label{05:soln:equation_to_simplify_two}
  A+AB && \text{Original Expression} \\
  \nonumber
  A(1+B) && \text{Factor} \\
  \nonumber
  A(1) && \text{Annihilation (1+B=1)} \\
  \nonumber
  A && \text{Identity A1=A}
\end{align}

The original expression, $ A+AB $ has been reduced to  $ A $ so the original circuit could be replaced by a wire directly from input  $ A $ to output  $ Y $. Equation \ref{05:eq:equation_to_simplify_three} looks similar to Equation \ref{05:eq:equation_to_simplify_two}, but is quite different and requires a more clever simplification.

\begin{align}
  \label{05:eq:equation_to_simplify_three}
  A+A'B &= Y
\end{align}

This is simplified below.

\begin{align}
  \label{05:soln:equation_to_simplify_three}
  A+A'B && \text{Original Expression} \\
  \nonumber
  A+AB+A'B && \text{Expand A to A+AB (Absorption)} \\
  \nonumber
  A+B(A+A') && \text{Factor B out of the last two terms} \\
  \nonumber
  A+B(1) && \text{Complement Property} \\
  \nonumber
  A+B && \text{Identityt: B(1)=B}
\end{align}

Note how the Absorption Property ($ A + AB = A $) is used to ``un-simplify'' the first  $ A $ term, changing  $ A $ into $ A + AB $. While this may seem like a backward step, it ultimately helped to reduce the expression to something simpler. Sometimes ``backward'' steps must be taken to achieve the most elegant solution. Knowing when to take such a step is part of the art of algebra.

As another example, simplify this \ac{POS} expression equation:

\begin{align}
  \label{05:eq:equation_to_simplify_four}
  (A+B)(A+C) &= Y
\end{align}

This is simplified below.

\begin{align}
  \label{05:soln:equation_to_simplify_four}
  (A+B)(A+C) && \text{Original Expression} \\
  \nonumber
  AA+AC+AB+BC && \text{Distribute A+B} \\
  \nonumber
  A+AC+AB+BC && \text{Idempotence: AA=A} \\
  \nonumber
  A+AB+BC && \text{Absorption: A+AC=A} \\
  \nonumber
  A+BC && \text{Absorption: A+AB=A}
\end{align}

In each of the examples in this section, a Boolean expression was simplified using algebraic methods, which led to a reduction in the number of gates needed and made the final circuit more economical to construct and reliable to operate.

\subsection{Practice Problems}
\label{05:subsec:practice_problems_for_simplifying_with_algebra}

Table \ref{05:tab:simplifying_boolean_expressions} shows a Boolean expression on the left and its simplified version on the right. This is provided for practice in simplifying expressions using algebraic methods.

\begin{table}[H]
  \sffamily
  \newcommand{\head}[1]{\textcolor{white}{\textbf{#1}}}
  \begin{center}
    \rowcolors{2}{gray!10}{white} % Color every other line a light gray
    \begin{tabular}{cc}
      \rowcolor{black!75}
      \head{Original Expression} & \head{Simplified} \\
      $ A(A'+B) $ & $ AB $ \\
      $ A+A'B $ & $ A+B $ \\
      $ (A+B)(A+B') $ & $ A $ \\
      $ AB+A'C+BC $ & $ AB+A'C $ \\
    \end{tabular}
  \end{center}
  \caption{Simplifying Boolean Expressions}
  \label{05:tab:simplifying_boolean_expressions}
\end{table}


\section{Background}
In this section we describe MAPF and ASP. Our definition of MAPF follows closely that of \acite{SternSFK0WLA0KB19}.

\subsection{Multi-Agent Pathfinding}
A MAPF instance is defined by a tuple $(G,\mathcal{A},init,goal)$, where $G=(V,E)$ is a graph, $A$ is the set of agents, and $init:A\rightarrow V$ and $goal:A\rightarrow V$ are functions used to denote the initial and goal vertex for each of the agents.

At each time instant each agent at vertex $v$ can either move to any of its successors in $G$ or not move at all. When the graph $G$ is a 4-connected grid, as we assume in the rest of the paper, there are 5 possible moves at each time instant: $\{\Up,\Down,\Left,\Right,\Wait\}$, where the first four move the agent in one of the four cardinal positions, and the \Wait move results in the agent staying at its current vertex. A \emph{path} over $G$ is a sequence of vertices in  $V$, $v_1v_2\ldots v_n$, where either $v_i=v_{i+1}$ (i.e., a \Wait move is performed) or $(v_i,v_{i+1})\in E$ (i.e., a non wait action is performed) for every $i\in\{1,\ldots,n-1\}$. Given a path $\pi$ we denote by $\pi[i]$ the $i$-th element in $\pi$, where $i\in\{1,\ldots,|\pi|\}$.

A solution to MAPF is a function $sol:\mathcal{A}\rightarrow V^*$, which associates a path to each of the agents, such that the first and last vertices of $sol(a)$ are, respectively, $init(a)$ and $goal(a)$. Without loss of generality henceforth we assume that all paths in $sol$ have the same size, since wait actions may be used at the end of any action sequence to remain on the same vertex. Below we denote by $\mathcal{T}$ the set $\{1,\ldots,M\}$, where $M$ is the size of any of the paths in $sols$. We also refer to $M-1$ as the \emph{makespan} of the solution.

In addition, $sol$ must be \emph{conflict-free}, which means that if $\pi$ and $\rho$ are the paths in $sol$ followed by two different agents, none of the following conflicts should arise.
\begin{itemize}
    \item \textbf{Vertex Conflict}. Two agents cannot be at the same vertex at the same time instant. Formally, there is a vertex conflict iff $\pi[i]=\rho[i]$, for some $i\in\mathcal{T}$.
    \item \textbf{Swap Conflict}. Two agents cannot swap their positions. Intuitively, this conflict is justified by that fact that we assume that size of the agents prevent the connection between two vertices in opposite directions. Formally, there is a swap conflict iff $(\pi[i],\rho[i])=(\rho[i+1],\pi[i+1])$, for some $i\in\{1,\ldots,M-1\}$.
\end{itemize}
Following most of the literature in MAPF (e.g., \nbcite{SharonSFS12}), we consider only these two types of conflicts, and no other conflicts. For example, we do not consider so-called \emph{following conflicts} \cite{SternSFK0WLA0KB19}, which do not allow an agent to occupy at time $t+1$ the position of another agent had at time $t$.

A standard assumption in MAPF is that all moves cost one unit except for waits performed at the goal when no other action is planned in the future. Thus the cost of path $\pi$ for agent $a$ is written as $|\rho|$, when $\rho$ is the shortest sequence such that $\pi=\rho(goal(a))^k$, for some $k$. The cost of a solution $sol$ is defined as $\sum_{a\in A} c(sol(a))$.

A solution $sol$ is \emph{optimal under sum-of-costs}, or simply \emph{cost-optimal}, if no other solution $sol'$ exists such that $c(sol')<c(sol)$. A solution $sol$ is makespan-optimal, if no other solution exists whose makespan is smaller than the makespan of $sol$. A makespan-optimal solution is not necessarily a cost-optimal solution. This is illustrated in Figure~\ref{fig:makespancost}.

\begin{figure}
    \centering
    \includegraphics[width=0.24\textwidth]{graphs/makespangrid.PNG}
      \caption{Problem instance where the increase of makespan yields better \emph{sum-of-costs} solutions. The problem has 3 agents:  $a_0$, who needs to go from $(0,1)$ to $(3,1)$. And agents $a_1$ and $a_2$ who are already at the goal at the positions $(1,1)$ and $(2,1)$ respectively. The optimal cost solution with makespan $3$ is $8$ as it involves $a_1$ and $a_2$ moving away from their goal to make space for $a_0$. In contrast the optimal cost solution with makespan $5$ is $5$ as it only needs to move $a_0$.}
    \label{fig:makespancost}
\end{figure}

\subsection{Answer-Set Programming}
ASP \cite{Lifschitz08} is a logic-based framework for solving optimization problems. For space limitations, here we describe a subset of an ASP standard that is relevant to this paper.

An ASP \emph{basic program} is a set of rules of the form:
\begin{equation}\label{asprule}
p\leftarrow q_1,q_2,\ldots,q_n
\end{equation}
where $n\geq 0$, and $p$, $q_1,\ldots,q_n$ are so-called \emph{atoms}. The intuitive interpretation of this rule is as follows ``p is true/provable if so are $q_1,q_2,\ldots,q_n$''. When $n=0$ rule \eqref{asprule} is considered to have an empty body. Such rules are called facts and are usually written as `$p$' instead of `$p\leftarrow $'.

A \emph{model} of an ASP basic program is a set of atoms $M$ that intuitively contains all and only the atoms that are provable. Formally, $M$ is a model of basic program $\Pi$ iff $M$ is the subset-minimal set such that for every rule $p\leftarrow q_1,q_2,\ldots,q_n \in \Pi$ such that $\{q_1,\ldots,q_n\}\subseteq M$, then $p\in M$.

An important syntactic element relevant to our paper is the so-called \emph{negation as failure}. Rules containing such negated atoms look like:
\begin{equation}\label{negasprule}
p\leftarrow q_1,q_2,\ldots,q_n, not\: r_1,not\: r_2, \ldots, not\: r_k .
\end{equation}
Intuitively, rule \eqref{negasprule} should be interpreted as `$p$ is provable if $q_1,\ldots q_n$ are provable a none of $r_1,\ldots,r_k$ are provable'. The semantics of programs that include negation as failure is simple but a little more involved, require the introduction of so-called \emph{stable models}, whose formal definition we omit from this paper. We direct the interested reader to the paper by \acite{FerrarisL05}.

%e a of set of ntatoms $M$ is a model of a program $P$ containing such rules if the \emph{reduct} of $\Pi$ with respect to $M$, denoted as $\Pi^M$ also has $M$ as a model. Intuitively $P^M$ is a basic program which results from simplifying away every occurrence of a $not\:p$ in the obvious way (i.e., eliminating the rule if $p$ is in the model $M$, and removing $not\:p$ from the rule if $p$ is not in the model $M$). For example $M=\{p\}$ is a model of $\Pi=\{p\leftarrow not\: q\}$, since $\Pi^M=\{p\leftarrow \}$ has $M$ as a model. $M'=\{q\}$ is not a model of $\Pi$ since $\Pi^{M'}=\{\}$ does not have $M'$ as a model.

Another relevant type of rule for our paper is:
\begin{equation*}
    |\{p_1,p_2,\ldots,p_n\}|=k \leftarrow q_1,q_2,\ldots,q_n.
\end{equation*}
Intuitively here we say that if $q_1,q_2,\ldots,q_n$ are all provable, then $k$ of the elements in $\{p_1,p_2,\ldots,p_n\}$ must appear in the model. This definition allows programs to have multiple models\footnote{Technically, negation as failure alone also allows the user to create programs with multiple models, but in our translation we define multiple models exploiting this type of rule.}. For example, the program:
\begin{align*}
    &s,\\
    &|{p,q,r}|=1\leftarrow s
\end{align*}
has three models: $\{p,s\}$, $\{q,s\}$, and $\{r,s\}$.

Another type of rule that is relevant to our translation is:
\begin{equation}
    \leftarrow p_1,p_2,\ldots,p_n
\end{equation}
which is a constraint that prohibits the occurrence of $\{p_1,p_2,\ldots,p_n\}$ in the model.

Finally, ASP programs may contain variables to represent rule \emph{schemas}. As such, a rule like:
\begin{equation}\label{aspwithvariables}
    p(X)\leftarrow q(X)
\end{equation}
where uppercase letters represent \emph{variables}. Intuitively a variable can take any value among the set of \emph{terms}. As such, rule \eqref{aspwithvariables} represents that when $c$ is a term and $q(c)$ is provable, so is $p(c)$. Intuitively a term represents an object that can be named in the program. The set of terms for a program is syntactically determined from the program using the constants mentioned in it. The set of terms has a theoretical counterpart, the so-called \emph{Herbrand base}, whose definition we omit here, since it is not key for understanding the rest of the paper.

What is key to understand for the rest of the paper, because it affects our conclusions is that, in the process of finding a model for a program, an initial step that is carried out is \emph{grounding}. Grounding instantiates rules with variables, effectively removing all variables from the program. Since there are plenty of optimizations that solvers employ during grounding, it is not easy to describe the grounding process with complete precision here, and therefore we will just describe it intuitively. For example, the grounded version of program:
\begin{equation}
    \{q(a),q(b),p(X)\leftarrow q(X)\}
\end{equation} may be
\begin{align*}
&\{q(a),q(b),p(a)\leftarrow q(a),p(b)\leftarrow q(b)\} \text{ or } \\
&\{q(a),q(b),p(a),p(b)\},
\end{align*}
depending on the optimizations applied at grounding time. What is however unavoidable is that grounding generates two instances for the rule $p(X)\leftarrow q(X)$ because the number of objects that satisfy predicate $q$ is two. Thus, if we had declared $n$ objects satisfying $q$ we would expect the grounding process to generate $n$ instances for $p(X)\leftarrow q(X)$. As we will see in the rest of the paper, the size of the grounded version of the program is key for performance.


\hypertarget{inference}{%
\chapter{Inference}\label{inference}}

\href{https://colab.research.google.com/github/AllenDowney/ElementsOfDataScience/blob/master/11_inference.ipynb}{Click
here to run this notebook on Colab} or
\href{https://github.com/AllenDowney/ElementsOfDataScience/raw/master/11_inference.ipynb}{click
here to download it}.

This chapter introduces \textbf{statistical inference}, which is the
process of using a sample to make inferences about a population.

The \textbf{population} is the group we are interested in. Sometimes it
is a group of people, but in general it can be any kind of group.

If we can observe the entire group, we might not need statistical
inference. If not, sometimes we can observe a \textbf{sample} (or
subset) of the population and use those observations to make claims
about the population.

If you have studied statistics before, you might have encountered some
of these ideas before: hypothesis testing, p-values, estimation,
standard error, and confidence intervals.

In this chapter we'll approach these topics using computation and
simulation, as opposed to mathematical analysis. I hope this approach
makes the ideas clearer.

If you have not seen these ideas before, don't worry. That might even be
better.

We'll look at three examples:

\begin{itemize}
\item
  Testing whether a coin is ``fair''.
\item
  Testing whether first babies are more like to be born early (or late).
\item
  Estimating the average height of men in the U.S., and quantifying the
  precision of the estimate.
\end{itemize}

\hypertarget{the-euro-problem}{%
\section{The Euro problem}\label{the-euro-problem}}

In \emph{Information Theory, Inference, and Learning Algorithms}, David
MacKay writes, "A statistical statement appeared in \emph{The Guardian}
on Friday January 4, 2002:

\begin{quote}
When spun on edge 250 times, a Belgian one-euro coin came up heads 140
times and tails 110. `It looks very suspicious to me', said Barry
Blight, a statistics lecturer at the London School of Economics. `If the
coin were unbiased the chance of getting a result as extreme as that
would be less than 7\%'.*
\end{quote}

But do these data give evidence that the coin is biased rather than
fair?"

Before we answer MacKay's question, let's unpack what Dr.~Blight said:

``If the coin were unbiased the chance of getting a result as extreme as
that would be less than 7\%''.

To see where that comes from, I'll simulate the result of spinning an
``unbiased'' coin, meaning that the probability of heads is 50\%.

Here's an example with 10 spins:

\begin{lstlisting}[language=Python,style=source]
import numpy as np

spins = np.random.random(10) < 0.5
spins
\end{lstlisting}

\begin{lstlisting}[style=output]
array([ True,  True,  True, False,  True,  True,  True,  True,  True,
       False])
\end{lstlisting}

\passthrough{\lstinline!np.random.random!} returns numbers between 0 and
1, uniformly distributed. So the probability of being less than 0.5 is
50\%.

The sum of the array is the number of \passthrough{\lstinline!True!}
elements, that is, the number of heads:

\begin{lstlisting}[language=Python,style=source]
np.sum(spins)
\end{lstlisting}

\begin{lstlisting}[style=output]
8
\end{lstlisting}

We can wrap that in a function that simulates
\passthrough{\lstinline!n!} spins with probability
\passthrough{\lstinline!p!}.

\begin{lstlisting}[language=Python,style=source]
def spin(n, p):
    return np.sum(np.random.random(n) < p)
\end{lstlisting}

Here's an example with the actual sample size (250) and hypothetical
probability (50\%).

\begin{lstlisting}[language=Python,style=source]
heads, tails = 140, 110
sample_size = heads + tails
\end{lstlisting}

\begin{lstlisting}[language=Python,style=source]
hypo_prob = 0.5
spin(sample_size, hypo_prob)
\end{lstlisting}

\begin{lstlisting}[style=output]
119
\end{lstlisting}

Since we are generating random numbers, we expect to see different
values if we run the experiment more than once.

Here's a loop that runs \passthrough{\lstinline!spin!} 10 times.

\begin{lstlisting}[language=Python,style=source]
n = 250
p = 0.5

for i in range(10):
    print(spin(n, p))
\end{lstlisting}

\begin{lstlisting}[style=output]
114
121
116
131
141
139
135
129
127
133
\end{lstlisting}

As expected, the results vary from one run to the next.

Now let's run the simulated experiment 10000 times and store the results
in a NumPy array.

\begin{lstlisting}[language=Python,style=source]
outcomes = np.empty(1000)

for i in range(len(outcomes)):
    outcomes[i] = spin(n, p)
\end{lstlisting}

\passthrough{\lstinline!np.empty!} creates an empty array with the given
length. Each time through the loop, we run
\passthrough{\lstinline!spin!} and assign the result to an element of
\passthrough{\lstinline!outcomes!}.

The result is an array of 10000 integers, each representing the number
of heads in a simulated experiment.

The mean of \passthrough{\lstinline!outcomes!} is about 125:

\begin{lstlisting}[language=Python,style=source]
np.mean(outcomes)
\end{lstlisting}

\begin{lstlisting}[style=output]
124.967
\end{lstlisting}

Which makes sense. On average, the expected number of heads is the
product of the hypothetical probability and the sample size:

\begin{lstlisting}[language=Python,style=source]
expected = hypo_prob * sample_size
expected
\end{lstlisting}

\begin{lstlisting}[style=output]
125.0
\end{lstlisting}

Now let's see how much the values in \passthrough{\lstinline!outcomes!}
differ from the expected value:

\begin{lstlisting}[language=Python,style=source]
diffs = outcomes - expected
\end{lstlisting}

\passthrough{\lstinline!diffs!} is an array that contains the deviation
of each experiment from the expected value, 125.

Here's the mean of the absolute deviations:

\begin{lstlisting}[language=Python,style=source]
np.mean(abs(diffs))
\end{lstlisting}

\begin{lstlisting}[style=output]
6.673
\end{lstlisting}

So a typical experiment deviates from the mean by about 6.

To see the whole distribution of deviations, we can plot a histogram.
The following function uses Matplotlib to plot a histogram and adjust
some of the settings.

\begin{lstlisting}[language=Python,style=source]
import matplotlib.pyplot as plt

def plot_hist(values):
    xs, ys, patches = plt.hist(values,
                               density=True,
                               histtype='step',
                               linewidth=2,
                               alpha=0.5)

    plt.ylabel('Density')
    plt.tight_layout()
    return patches[0]
\end{lstlisting}

Here's what the distribution of deviations looks like:

\begin{lstlisting}[language=Python,style=source]
plot_hist(diffs)

plt.title('Sampling distribution (n=250)')
plt.xlabel('Deviation from expected number of heads');
\end{lstlisting}

\begin{figure}
\centering
\includegraphics{11_inference_files/11_inference_29_0.pdf}
\caption{png}
\end{figure}

This is the ``sampling distribution'' of deviations. It shows how much
variation we should expect between experiments with this sample size (n
= 250).

\hypertarget{p-values}{%
\section{P-values}\label{p-values}}

Getting get back to the Euro example, Dr.~Bright reported:

``If the coin were unbiased the chance of getting a result as extreme as
that would be less than 7\%''.

The article doesn't say so explicitly, but this is a ``p-value''. To
understand what that means, let's count how many times, in 10000
attempts, the outcome is ``as extreme as'' the observed outcome, 140
heads.

The observed deviation is the difference between the observed and
expected number of heads:

\begin{lstlisting}[language=Python,style=source]
observed_diff = heads - expected
observed_diff
\end{lstlisting}

\begin{lstlisting}[style=output]
15.0
\end{lstlisting}

Let's see how many times the simulated \passthrough{\lstinline!diffs!}
exceed the observed deviation:

\begin{lstlisting}[language=Python,style=source]
np.mean(diffs >= observed_diff)
\end{lstlisting}

\begin{lstlisting}[style=output]
0.039
\end{lstlisting}

It's around 3\%. But Dr.~Blight said 7\%. Where did that come from?

So far, we only counted the cases where the outcome is \emph{more} heads
than expected. We might also want to count the cases where the outcome
is \emph{fewer} than expected.

Here's the probability of falling below the expected number by 15 or
more.

\begin{lstlisting}[language=Python,style=source]
np.mean(diffs <= -observed_diff)
\end{lstlisting}

\begin{lstlisting}[style=output]
0.033
\end{lstlisting}

To get the total probability of a result ``as extreme as that'', we can
use the absolute value of the simulated differences:

\begin{lstlisting}[language=Python,style=source]
np.mean(abs(diffs) >= observed_diff)
\end{lstlisting}

\begin{lstlisting}[style=output]
0.072
\end{lstlisting}

So that's consistent with what Dr.~Blight reported.

To show what that looks like graphically, I'll use the following
function, which fills in the histogram between
\passthrough{\lstinline!low!} and \passthrough{\lstinline!high!}.

\begin{lstlisting}[language=Python,style=source]
def fill_hist(low, high, patch):
    fill = plt.axvspan(low, high,
                       clip_path=patch,
                       alpha=0.5,
                       color='C0')
\end{lstlisting}

The following plot shows the sampling distribution of
\passthrough{\lstinline!diffs!} with two regions shaded. These regions
represent the probability that an unbiased coin yields a deviation from
the expected as extreme as 15.

\begin{lstlisting}[language=Python,style=source]
patch = plot_hist(diffs)

# fill the right tail of the hist
low = observed_diff
high = diffs.max()
fill_hist(low, high, patch)

# fill the left tail of the hist
low = diffs.min()
high = -observed_diff
fill_hist(low, high, patch)

plt.title('Sampling distribution (n=250)')
plt.xlabel('Deviation from expected number of heads')
plt.ylabel('Density');
\end{lstlisting}

\begin{figure}
\centering
\includegraphics{11_inference_files/11_inference_42_0.pdf}
\caption{png}
\end{figure}

These results show that there is a non-negligible chance of getting a
result as extreme as 140 heads, even if the coin is actually fair.

So even if the results are ``suspicious'' they don't provide compelling
evidence that the coin is biased.

\textbf{Exercise:} There are a few ways to make ``crooked'' dice, that
is, dice that are more likely to land on one side than the others.
Suppose you run a casino and you suspect that a patron is using a die
that comes up 3 more often than it should.

You confiscate the die, roll it 300 times, and 63 times it comes up 3.
Does that support your suspicions?

\begin{itemize}
\item
  To answer this question, use \passthrough{\lstinline!spin!} to
  simulate the experiment, assuming that the die is fair.
\item
  Use a for loop to run \passthrough{\lstinline!spin!} 1000 times and
  store the results in a NumPy array.
\item
  What is the mean of the results from the simulated experiments?
\item
  What is the expected number of 3s if the die is fair?
\item
  Use \passthrough{\lstinline!plot\_hist!} to plot the results. The
  histogram you plot approximates the sampling distribution.
\end{itemize}

\textbf{Exercise:} Continuing the previous exercise, compute the
probability of seeing a deviation from the expected value that is ``as
extreme'' as the observed difference.

For this context, what do you think is the best definition of ``as
extreme''?

Plot the histogram of the random deviations again, and use
\passthrough{\lstinline!fill\_hist!} to fill the region of the histogram
that corresponds to the p-value you computed.

\hypertarget{are-first-babies-more-likely-to-be-late}{%
\section{Are first babies more likely to be
late?}\label{are-first-babies-more-likely-to-be-late}}

The examples so far have been based on coins and dice, which are
relatively simple. In this section we'll look at an example that's based
on real-world data.

Here's the motivation for it: When my wife and I were expecting our
first baby, we heard that first babies are more likely to be late. We
also hear that first babies are more likely to be early. Neither claim
was supported by evidence.

Fortunately, I am a data scientist! Also fortunately, the CDC runs the
National Survey of Family Growth (NSFG), which ``gathers information on
family life, marriage and divorce, pregnancy, infertility, use of
contraception, and men's and women's health.''

I got the data from their web page,
https://www.cdc.gov/nchs/nsfg/index.htm, and wrote some code to get it
into a Pandas Dataframe:

\begin{lstlisting}[language=Python,style=source]
import pandas as pd

nsfg = pd.read_hdf('nsfg.hdf5')
nsfg.shape
\end{lstlisting}

\begin{lstlisting}[style=output]
(9358, 11)
\end{lstlisting}

The \passthrough{\lstinline!nsfg!} DataFrame contains 9358 rows, one for
each recorded pregnancy, and 11 columns, one of each of the variables I
selected.

Here are the first few lines.

\begin{lstlisting}[language=Python,style=source]
nsfg.head()
\end{lstlisting}

\begin{tabular}{lrrrrrrrrrrr}
\toprule
{} &  caseid &  outcome &  birthwgt\_lb1 &  birthwgt\_oz1 &  prglngth &  nbrnaliv &  agecon &  agepreg &  birthord &  hpagelb &  wgt2013\_2015 \\
\midrule
0 &   60418 &        1 &           5.0 &           4.0 &        40 &       1.0 &    2000 &   2075.0 &       1.0 &     22.0 &   3554.964843 \\
1 &   60418 &        1 &           4.0 &          12.0 &        36 &       1.0 &    2291 &   2358.0 &       2.0 &     25.0 &   3554.964843 \\
2 &   60418 &        1 &           5.0 &           4.0 &        36 &       1.0 &    3241 &   3308.0 &       3.0 &     52.0 &   3554.964843 \\
3 &   60419 &        6 &           NaN &           NaN &        33 &       NaN &    3650 &      NaN &       NaN &      NaN &   2484.535358 \\
4 &   60420 &        1 &           8.0 &          13.0 &        41 &       1.0 &    2191 &   2266.0 &       1.0 &     24.0 &   2903.782914 \\
\bottomrule
\end{tabular}

The variables we need are \passthrough{\lstinline!birthord!}, which
indicates birth order, and \passthrough{\lstinline!prglength!}, which is
pregnancy length in weeks.

I'll make two boolean Series, one for first babies and one for others.

\begin{lstlisting}[language=Python,style=source]
firsts = (nsfg.birthord == 1)
others = (nsfg.birthord > 1)

np.sum(firsts), np.sum(others)
\end{lstlisting}

\begin{lstlisting}[style=output]
(3067, 3422)
\end{lstlisting}

We can use the boolean Series to select pregnancy lengths for the two
groups and compute their means:

\begin{lstlisting}[language=Python,style=source]
prglngth = nsfg['prglngth']

mean_first = prglngth[firsts].mean()
mean_other = prglngth[others].mean()

mean_first, mean_other
\end{lstlisting}

\begin{lstlisting}[style=output]
(38.57124225627649, 38.36908240794857)
\end{lstlisting}

Here's the difference in means, in weeks.

\begin{lstlisting}[language=Python,style=source]
diff = mean_first - mean_other
diff
\end{lstlisting}

\begin{lstlisting}[style=output]
0.20215984832792344
\end{lstlisting}

Here it is converted to days:

\begin{lstlisting}[language=Python,style=source]
diff * 7
\end{lstlisting}

\begin{lstlisting}[style=output]
1.415118938295464
\end{lstlisting}

It looks like first babies are born 1.4 days later than other babies, on
average.

\hypertarget{hypothesis-testing}{%
\section{Hypothesis testing}\label{hypothesis-testing}}

The apparent difference between these groups is based on a random sample
that is much smaller than the actual population. So we can't be sure
that the difference we see in the sample reflects a real difference in
the population. There are two other possibilities we should keep in
mind:

\begin{itemize}
\item
  Systematic errors: The sample might be more likely to include some
  pregancies, and less likely to include others, in a way that causes an
  apparent difference in the sample, even if there is no such difference
  in the population.
\item
  Sampling errors: Even if every pregnancy is equally likely to appear
  in the sample, it is still possible to see a difference in the sample
  that is not in the population, just because of random variation.
\end{itemize}

We can never rule out the possibility of systematic errors, but we
\emph{can} test whether an apparent effect could be explained by random
sampling.

Here's how:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  First we choose a ``test statistic'' that measures the size of the
  effect; the test statistic in this example is the difference in mean
  pregnancy length.
\item
  Next we define a model of the population under the assumption that
  there is actually no difference between the groups. This assumption is
  called the ``null hypothesis''.
\item
  Then we use the model to compute the distribution of the test
  statistic under the null hypothesis.
\end{enumerate}

We have already done step 1, but to make it easier to repeat, I'll wrap
it in a function.

\begin{lstlisting}[language=Python,style=source]
def test_stat(group1, group2):
    """Difference in means.

    group1: sequence of values
    group2: sequence of values

    returns: float difference in means
    """
    diff = np.mean(group1) - np.mean(group2)
    return diff
\end{lstlisting}

\passthrough{\lstinline!test\_stat!} takes two sequences and computes
the difference in their means.

Here's how we use it to compute the actual difference in the sample.

\begin{lstlisting}[language=Python,style=source]
group1 = prglngth[firsts]
group2 = prglngth[others]

actual_diff = test_stat(group1, group2)
actual_diff
\end{lstlisting}

\begin{lstlisting}[style=output]
0.20215984832792344
\end{lstlisting}

Now we have to define the null hypothesis, which is a model of the world
where there is no difference in pregnancy length between first babies
and others.

One way to do that is to put the two groups together and then divide
them up again at random. That way the distribution of pregnancy lengths
is the same for both groups.

I'll use \passthrough{\lstinline!concatenate!} to pool the groups.

\begin{lstlisting}[language=Python,style=source]
len(group1), len(group2)
\end{lstlisting}

\begin{lstlisting}[style=output]
(3067, 3422)
\end{lstlisting}

\begin{lstlisting}[language=Python,style=source]
pool = np.concatenate([group1, group2])
pool.shape
\end{lstlisting}

\begin{lstlisting}[style=output]
(6489,)
\end{lstlisting}

I'll use \passthrough{\lstinline!shuffle!} to reorder them.

\begin{lstlisting}[language=Python,style=source]
np.random.shuffle(pool)
\end{lstlisting}

Then I'll use \passthrough{\lstinline!split!} to make two simulated
groups, the same size as the originals.

\begin{lstlisting}[language=Python,style=source]
n = len(group1)
sim_group1, sim_group2 = np.split(pool, [n])
\end{lstlisting}

\begin{lstlisting}[language=Python,style=source]
len(sim_group1), len(sim_group2)
\end{lstlisting}

\begin{lstlisting}[style=output]
(3067, 3422)
\end{lstlisting}

Now we can compute the test statistic for the simulated data.

\begin{lstlisting}[language=Python,style=source]
test_stat(sim_group1, sim_group2)
\end{lstlisting}

\begin{lstlisting}[style=output]
-0.014237551111094149
\end{lstlisting}

In the simulated data, the distribution of pregnancy lengths is the same
for both groups, so the difference is usually close to 0.

But because it is based on a random shuffle of the groups, we get a
different value each time we run it.

To see what the whole distribution looks like, we can run the simulation
many times and store the results.

\begin{lstlisting}[language=Python,style=source]
diffs = np.empty(1000)

for i in range(len(diffs)):
    np.random.shuffle(pool)
    sim_group1, sim_group2 = np.split(pool, [n])
    diffs[i] = test_stat(sim_group1, sim_group2)
\end{lstlisting}

The result is the ``sampling distribution of the test statistic under
the null hypothesis''.

The mean of this distribution should close to zero, because it is based
on the assumption that there is actually no difference between the
groups.

\begin{lstlisting}[language=Python,style=source]
np.mean(diffs)
\end{lstlisting}

\begin{lstlisting}[style=output]
0.0021146729470806775
\end{lstlisting}

And here's what the whole distribution looks like.

\begin{lstlisting}[language=Python,style=source]
plot_hist(diffs)

plt.xlabel('Difference in mean (weeks)')
plt.title('Distribution of test statistic under null hypothesis');
\end{lstlisting}

\begin{figure}
\centering
\includegraphics{11_inference_files/11_inference_79_0.pdf}
\caption{png}
\end{figure}

If there were actually no difference between the groups, we would expect
to see a difference as big as 0.15 weeks by chance, at least
occasionally. But a difference as big as 0.2 would be rare.

To quantify that surprise, we can estimate the probability that the test
statistic, under the null hypothesis, exceeds the observed differences
in the means.

The result is a ``p-value''.

\begin{lstlisting}[language=Python,style=source]
p_value = np.mean(diffs >= actual_diff)
p_value
\end{lstlisting}

\begin{lstlisting}[style=output]
0.0
\end{lstlisting}

In this example the result is 0, which is to say that in 1000
simulations of the null hypothesis, we never saw a difference as big as
0.2.

\hypertarget{interpreting-p-values}{%
\section{Interpreting p-values}\label{interpreting-p-values}}

To interpret this result, remember that we started with three possible
explanations for the observed difference between the groups:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  The observed difference might be ``real''; that is, there might be an
  actual difference in pregnancy length between first babies and others.
\item
  There might be no real difference between the groups, and the observed
  difference might be because of a systematic error in the sampling
  process or the data collection process. For example, maybe reported
  pregnancy lengths are less accurate for first time mothers.
\item
  There might be no real difference between the groups, and the observed
  difference might be due to random variation in the sampling process.
\end{enumerate}

By computing a p-value, we have established that it would be rare to see
a difference as big as 0.2 due to sampling alone. So we can conclude
that the third explanation is unlikely.

That makes it more likely that the difference is real, but we still
can't rule out the second possibility.

\textbf{Exercise:} The test statistic we chose is the difference in
means between the two groups.

But suppose we would like to know whether first babies are more
unpredictable than other babies. In that case the test statistic we
choose might be the standard deviation of pregnancy length, which is one
way to quantify unpredictability.

As an exercise:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  Write a version of \passthrough{\lstinline!test\_stat!} that computes
  the difference in standard deviation between the groups.
\item
  Write a loop that estimates the distribution of this test statistic
  under the null hypothesis.
\item
  Compute a p-value.
\end{enumerate}

\hypertarget{estimation}{%
\section{Estimation}\label{estimation}}

Suppose we want to estimate the average height of men in the U.S.

We can use data from the
\href{https://www.cdc.gov/brfss/index.html}{BRFSS}:

``The Behavioral Risk Factor Surveillance System (BRFSS) is the nation's
premier system of health-related telephone surveys that collect state
data about U.S. residents regarding their health-related risk behaviors,
chronic health conditions, and use of preventive services.''

\begin{lstlisting}[language=Python,style=source]
from os.path import basename, exists

def download(url):
    filename = basename(url)
    if not exists(filename):
        from urllib.request import urlretrieve
        local, _ = urlretrieve(url, filename)
        print('Downloaded ' + local)

download('https://github.com/AllenDowney/' +
         'ElementsOfDataScience/raw/master/brfss.hdf5')
\end{lstlisting}

\begin{lstlisting}[language=Python,style=source]
import pandas as pd

brfss = pd.read_hdf('brfss.hdf5', 'brfss')
brfss.shape
\end{lstlisting}

\begin{lstlisting}[style=output]
(100000, 9)
\end{lstlisting}

We can use \passthrough{\lstinline!SEX!} to select male respondents.

\begin{lstlisting}[language=Python,style=source]
male = (brfss.SEX == 1)
np.mean(male)
\end{lstlisting}

\begin{lstlisting}[style=output]
0.48589
\end{lstlisting}

Then we select height data.

\begin{lstlisting}[language=Python,style=source]
heights = brfss['HTM4']
data = heights[male]
\end{lstlisting}

We can use \passthrough{\lstinline!isnan!} to check for NaN values:

\begin{lstlisting}[language=Python,style=source]
np.mean(np.isnan(data)) * 100
\end{lstlisting}

\begin{lstlisting}[style=output]
4.338430508962934
\end{lstlisting}

About 4\% of the values are missing.

Here are the mean and standard deviation, ignoring missing data.

\begin{lstlisting}[language=Python,style=source]
print('Mean male height in cm =', np.nanmean(data))
print('Std male height in cm =', np.nanstd(data))
\end{lstlisting}

\begin{lstlisting}[style=output]
Mean male height in cm = 177.53804780447925
Std male height in cm = 8.350063691943435
\end{lstlisting}

\hypertarget{quantifying-precision}{%
\section{Quantifying precision}\label{quantifying-precision}}

At this point we have an estimate of the average adult male height. We'd
like to know how accurate this estimate is, and how precise. In the
context of estimation, these words have a
\href{https://en.wikipedia.org/wiki/Accuracy_and_precision}{technical
distinction}:

\begin{quote}
Given a set of data points from repeated measurements of the same
quantity, the set can be said to be precise if the values are close to
each other, while the set can be said to be accurate if their average is
close to the true value of the quantity being measured.
\end{quote}

Usually accuracy is what we really care about, but it's hard to measure
accuracy unless you know the true value. And if you know the true value,
you don't have to estimate it.

Quantifying precision is not as useful, but it is much easier. Here's
one way to do it:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  Use the data you have to make a model of the population.
\item
  Use the model to simulate the data collection process.
\item
  Use the simulated data to compute an estimate.
\end{enumerate}

By repeating these steps, we can quantify the variability of the
estimate due to random sampling.

To model the population, I'll use \textbf{resampling}; that is, I will
treat the observed measurements as if they were taken from the entire
population, and I will draw random samples from them.

We can use \passthrough{\lstinline!np.random.choice!} to resample the
data:

\begin{lstlisting}[language=Python,style=source]
size = len(data)
sim_data = np.random.choice(data, size, replace=True)
sim_data.shape
\end{lstlisting}

\begin{lstlisting}[style=output]
(48589,)
\end{lstlisting}

With \passthrough{\lstinline!replace=True!}, we sample with replacement,
which means that some measurements might be chosen more than once, and
some might not be chosen at all.

(If we sample \emph{without} replacement, the resampled data is always
identical to the original, so that's no good.)

Now we can use \passthrough{\lstinline!nanmean!} to compute the mean of
the simulated data, ignoring missing values.

\begin{lstlisting}[language=Python,style=source]
np.nanmean(sim_data)
\end{lstlisting}

\begin{lstlisting}[style=output]
177.52799586999075
\end{lstlisting}

If we repeat this process 1000 times, we can see how much the results
vary.

\begin{lstlisting}[language=Python,style=source]
outcomes = np.empty(1000)
size = len(data)

for i in range(len(outcomes)):
    sim_data = np.random.choice(data, size, replace=True)
    outcomes[i] = np.nanmean(sim_data)
\end{lstlisting}

The result is the ``sampling distribution'', which shows how much the
results of the experiment would vary if we ran it many times. Here's
what it looks like:

\begin{lstlisting}[language=Python,style=source]
plot_hist(outcomes)
plt.title('Sampling distribution of the mean')
plt.xlabel('Mean adult male height, U.S.');
\end{lstlisting}

\begin{figure}
\centering
\includegraphics{11_inference_files/11_inference_105_0.pdf}
\caption{png}
\end{figure}

The width of this distribution shows how much the results vary from one
experiment to the next.

We can quantify this variability by computing the standard deviation of
the sampling distribution, which is called ``standard error''.

\begin{lstlisting}[language=Python,style=source]
std_err = np.std(outcomes)
std_err
\end{lstlisting}

\begin{lstlisting}[style=output]
0.038376132495234826
\end{lstlisting}

We can also summarize the sampling distribution with a ``confidence
interval'', which is a range that contains a specified fraction, like
90\%, of the values in \passthrough{\lstinline!sampling\_dist\_mean!}.

The central 90\% confidence interval is between the 5th and 95th
percentiles of the sampling distribution.

\begin{lstlisting}[language=Python,style=source]
ci_90 = np.percentile(outcomes, [5, 95])
ci_90
\end{lstlisting}

\begin{lstlisting}[style=output]
array([177.47655879, 177.60152771])
\end{lstlisting}

The following function plots a histogram and shades the 90\% confidence
interval.

\begin{lstlisting}[language=Python,style=source]
def plot_sampling_dist(outcomes):
    """Plot sampling distribution.

    outcomes: sequence of values
    """
    patch = plot_hist(outcomes)
    low, high = np.percentile(outcomes, [5, 95])
    fill_hist(low, high, patch)
    print('Mean = ', np.mean(outcomes))
    print('Std error = ', np.std(outcomes))
    print('90% CI = ', (low, high))
\end{lstlisting}

Here's what it looks like for the sampling distribution of mean adult
height:

\begin{lstlisting}[language=Python,style=source]
plot_sampling_dist(outcomes)
plt.xlabel('Mean adult male height, U.S. (%)');
\end{lstlisting}

\begin{lstlisting}[style=output]
Mean =  177.5401473153372
Std error =  0.038376132495234826
90% CI =  (177.47655879241216, 177.60152770795776)
\end{lstlisting}

\begin{figure}
\centering
\includegraphics{11_inference_files/11_inference_113_1.png}
\caption{png}
\end{figure}

For an experiment like this, we can compute the standard error
analytically.

\begin{lstlisting}[language=Python,style=source]
size = len(data)
analytic_std_err = np.std(data) / np.sqrt(size)
\end{lstlisting}

The result is close to what we observed computationally.

\begin{lstlisting}[language=Python,style=source]
analytic_std_err, std_err
\end{lstlisting}

\begin{lstlisting}[style=output]
(0.03788094526418763, 0.038376132495234826)
\end{lstlisting}

This result indicates that our estimate of the mean is \emph{precise};
that is, if we ran this experiment many times, the results would fall in
a narrow range.

But this range reflects only variability due to random sampling. If
there are systematic errors in the sampling process, or in the
measurement process, the result would not be \emph{accurate}.

Computing a standard error or confidence interval can be useful, but it
only quantifies variability due to random sampling, not other sources of
error.

\textbf{Exercise:} One nice thing about using resampling is that it is
easy to compute the sampling distribution for other statistics.

For example, suppose we want to estimate the coefficient of variation
(standard deviation as a fraction of the mean) for adult male height.
Here's how we can compute it.

\begin{lstlisting}[language=Python,style=source]
cv = np.nanstd(data) / np.nanmean(data)
cv
\end{lstlisting}

\begin{lstlisting}[style=output]
0.04703253074596872
\end{lstlisting}

So the standard deviation is about 5\% of the mean.

Write a loop that uses resampling to estimate the sampling distribution
of \passthrough{\lstinline!cv!}; store the results in an array named
\passthrough{\lstinline!outcomes!}.

Then use \passthrough{\lstinline!plot\_sampling\_dist!} to plot the
sampling distribution of the coefficient of variation.

What is the standard error of the estimated coefficient of variation?

\hypertarget{summary}{%
\section{Summary}\label{summary}}

This chapter presents computational methods for computing p-values,
standard errors, and confidence intervals. The two processes are
similar, but they answer different questions.

The following diagram outlines the hypothesis testing process:

Again, the key steps are

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  Choose a test statistic that quantifies the observed effect.
\item
  Define a model of the null hypothesis and use it to generate simulated
  data.
\item
  Compute the distribution of the test statistic under the null
  hypothesis.
\item
  Compute a p-value, which is probability, under the null hypothesis, of
  seeing an effect as extreme as what you saw.
\end{enumerate}

The following figure shows the similar process for computing standard
errors and confidence intervals.

The essential steps are:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  Choose a sample statistic that quantifies the thing you want to
  estimate.
\item
  Use the data to make a model of the population, assuming that the
  estimate is accurate.
\item
  Use the model to simulate the sampling process and generate simulated
  data.
\item
  Compute the sampling distribution of the estimate.
\item
  Use the sampling distribution to compute the standard error,
  confidence interval, or both.
\end{enumerate}

Finally, remember that both processes only account for variability due
to random sampling. They don't tell us anything about systematic errors
in the sampling process, measurement error, or other sources of error.


\chapter{Cluster Analysis: Alternative}

\section{DBSCAN}
Two core points within \emph{Radius} are put into the same cluster
\begin{description}
\item[Core point] has more than \emph{MinPoint} within \emph{Radius}
\item[Border point] has fewer than \emph{MinPoint} within \emph{Radius}, but is in the neighborhood of a core point.
\item[Noise point] other points
\end{description}

\section{CURE}
Hierarchical Approach. Representative points are found by selecting constant number of points from a cluster and then "shrinking" them toward the center of the cluster.

\section{Graph-Based Clustering}
Builds a graph using proximity matrix, then breaks the graph using sparsification. \emph{Sparsification} keeps the connections to the most similar neighbors while breaking the connections to less similar points. Clusters are connected components in the graph.

\subsection{Chameleon}
Use a dynamic model to measure the similarity between clusters

\begin{table}[h!]
\begin{tabular}{r p{12cm}}
\hline
    1: & Build a k-nearest neighbor graph\\
    2: & partition the graph using a multilevel graph partitioning algorithm\\
    3: & \textbf{repeat} merge the clusters that best preserve the cluster self-similarity with respect to relative inter-connectivity and relative closeness\\
    4: & \textbf{until} no more cluster can be merged \\
\hline
\end{tabular}
\end{table}

\subsection{Shared Near Neighbor Approach}
\begin{description}
\item[SNN Graph] The weight of an edge is the number of shared neighbours between vertices
\end{description}
Computing shared nearest neighbor similarity:
\begin{table}[h!]
\begin{tabular}{r p{12cm}}
\hline
    1: & Find the k-nearest neighbors of all points \\
    2: & \textbf{if} two points, $x$ and $y$ are not among the k-nearest neighbors of eat other \textbf{then} \\
    3: & \ \ $similarity(x, y) = 0$ \\
    4: & \textbf{else} \\
    5: & \ \ $similarity(x, y) = $ number of shared neighbors\\
\hline
\end{tabular}
\end{table}

\subsection{Jarvis-Patrick Clustering}

\begin{description}
\item A threshold is used to sparsify SNN similarity matrix
\item A pair of points is put in the same cluster if:
    \begin{enumerate}
        \item share more than $T$ neighbours
        \item in each others $k$ nearest neighbour list
    \end{enumerate}
\end{description}

\begin{table}[h!]
\begin{tabular}{r p{12cm}}
\hline
    1: & Compute the SNN similarity graph \\
    2: & Sparsify the SNN similarity graph by applying a similarity threshold \\
    3: & Find the connected components of the sparsified SNN similarity graph \\
\hline
\end{tabular}
\end{table}

\section{Cluster Validity}

\begin{description}
\item[External Index] measure the extent to which cluster labels match externally supplied class labels (Entrophy)
\item[Internal Index] measure the goodness of clustering structure without respect to external information (SSE)
\item[Relative Index] compare two different clustering or clusters
\end{description}

\subsection{Measuring Cluster Validity via Correlation}
Compute the correlation between proximity matrix and incidence matrix. High correlation indicates that points that belong to the same cluster are close to each other.

\subsection{Internal Measures: Cohesion and Separation}
Cohesion, within cluster SSE:
$$WSS=\sum_{i} \sum_{x \in C_i} (x-m_i)^{2}$$
Separation, between cluster SSE:
$$BSS=\sum_{i} \mid C_i \mid (m-m_{i})^{2}$$
\begin{center}where $\mid C_i \mid$ is the size of cluster $i$\end{center}
\begin{center}and $m$ is true mean of all point\end{center}
\subsection{Internal Measures: Silhouette Coefficient}
$$s=1-\frac{a}{b}$$
$i$ = individual point
$a$ = average distance of $i$ to the points in its cluster\\
$b$ = min (average distance of $i$ to points in another cluster)\\
Closer to 1 is better


\section{Structural model}
\label{sec:structuralModel}
This section deals with the first term of the right hand side in eq.\ref{eq:nlmeModel}
\begin{align*}
	f(x_{ij}, \psi_{i})
\end{align*}
i.e. the model prediction.

It can be formulated as a simple algebraic equation (e.g. Hill equation) or complex physiology-based PK model implemented as system of ODEs. When defined in such framework, this deterministic model for an individual will later be embedded in a statistical model. Other approaches, such as SDE-based structural models are not supported in this specification.

%\begin{figure}[htbp]
%\begin{center}
%\includegraphics[width=.45\textwidth]{CTS1_smallPKPD}
%\caption{Simulated combined model as defined in the example with PK (blue) and PD (green) time courses for one subject. Here three doses were administered every $48h$.}
%\label{fig:simplePKPD}
%\vspace{-20pt}
%\end{center}
%\end{figure}
%  \vspace{-20pt}
%  \caption{A gull}
%  \vspace{-10pt}
%\end{wrapfigure}
\paragraph{Example}
As an example of a structural model we consider a combined PK/PD model, % see figure \ref{fig:simplePKPD},
\begin{itemize}
\item
PK -- oral one-compartmental model
\item
PD -- turnover model, so called $I_{max}$ model
\end{itemize}
with the following model parameters $ka$, $V$, $CL$, $Imax$, $IC50$, $Rin$ and $kout$.
\begin{align*}
k&=CL/V  \\
\frac{dAd}{dt} &=-ka\times Ad  \\
\frac{dAc}{dt}&=ka \times Ad - k \times Ac  \\
\frac{dE}{dt}&=Rin \times \Bigg(1-\frac{Imax \times Cc}{Cc+IC50}\Bigg) - kout \times E \\
\mbox{Initial condition: } & E(t=0) = Rin/kout   \\
& Ad(t=0) = DoseSize  \\
& Ac(t=0) = 0;  \\
Cc &= Ac/V
\end{align*}

\paragraph{Alternative formulation}
The PK model can, in this case, be formulated as an algebraic equation because an analytic solution exists, i.e.
\begin{align*}
C(t) = \frac{D}{V}  \frac{ka}{ka - k} \Big(e^{-k(t-t_D)} - e^{-ka(t-t_D)} \Big)
\end{align*}

%
%\begin{eqnarray}
%\frac{d\textbf{y}(t)}{dt} &=& F(t,\textbf{y}(t)) \nonumber \\
%with  && \textbf{y}(t_0) = \textbf{y}_0  \nonumber
%\end{eqnarray}
%
%i.e. using explicit, first order ODEs.
%
%\paragraph*{Note on initial conditions}
%Some models have long algebraic expressions for every initial condition, e.g. here the value for glucose in liver:
%\begin{eqnarray}
%G_{L,0}=(1/Q_{G,L})*(Q_{G,A}*G_{H,0} + Q_{G,G}*G_{G,0} + Q_{G,PN}*G_{PN,0} + r_{B_{HGP}} - r_{B_{HGU}}) \nonumber
%\end{eqnarray}
%with
%Initial conditions (defined/calculated before)
%\begin{eqnarray}
%&&G_{H,0} = a; \;\;\;G_{G,0} = b; \;\;\;G_{PN,0} = c; \quad with \quad a,b,c \in R \nonumber
%\end{eqnarray}
%and other user defined functions
%\begin{eqnarray}
%&& r_{B_{HGP}} = f_1(y) \nonumber \\
%&& r_{B_{HGU}} = f_2(y) \nonumber
%\end{eqnarray}