<file_sep>**Grade:** 98/100

## Comments

Problem 1:

--- Part A:   2/2

--- Part B:   3/3

--- Part C:   5/5

--- Part D:   5/5

--- Part E:   5/5

--- Part F:   5/5

Problem 2:

--- Part A:   4/4

--- Part B:   4/4

--- Part C:   4/4 

--- Part D:   4/4 

--- Part E:   4/4 

Problem 3:

--- Part A:  10/10

--- Part B:   8/8 

--- Part C:   7/7

--- Part D:   5/5

Problem 4:

--- Part A:   1/3 

--- Part B:   2/2 

--- Part C:   5/5 

--- Part D:   5/5 

--- Part E:   5/5 

--- Part F:   5/5 

--- Comments: Wrong text model.
<file_sep>**Grade:** 100/100

## Comments

Problem 1:

--- Part A:   3/3

--- Part B:   4/4

--- Part C:   3/3

Problem 2:

--- Part A:  12/12

--- Part B:  12/12

--- Part C:   6/6 

Problem 3:

--- Part A:   5/5

--- Part B:  10/10

--- Part C:   5/5

--- Part D:   5/5

--- Part E:   5/5

Problem 4:

--- Part A:   5/5 

--- Part B:  10/10

--- Part C:  10/10

--- Part D:   5/5
<file_sep># Markov Chain Examples!!!

Before you go and get your hands dirty in my code, I'd suggest running the following command:

```sh
pip install -r requirements.txt
```

Have fun!
<file_sep><p align="center">
  <h1 align="center">CSCI 4622 - Undergraduate Machine Learning</h1>
</p>

**Warning:** For all jupyter notebooks, they might look funny because GitHub has a crappy jupyter rendering system. I'd suggest cloning the repo and opening the notebooks.

**Notes:** This class was a special topics course, in Spring 2018, which covered a variety of stuff, but only section 002, taught by <NAME>, taught undergraduate ML. Also, this course will most likely be renamed **CSCI 4622** in future semesters, but when I took it, it had the course number of **CSCI 4831**.

This is a code dump for this class... Don't cheat!
<file_sep># Imports + Styles
import sys
import pickle, gzip
import numpy as np
import pandas as pd
from tqdm import tqdm
import matplotlib.pylab as plt
plt.style.use('bmh')
# Get the command line argument passed into the program
args = list(sys.argv[1:])
type_of_network = args[0]
print('You have chosen a', type_of_network, 'neural network')
# Import the correct network class
if type_of_network == 'basic':
    from network import Network as Network
if type_of_network == 'tanh':
    from network import tanhNetwork as Network
# Load the data
X_train, y_train, X_valid, y_valid = pickle.load(gzip.open("neural-nets-data/mnist21x21_3789_one_hot.pklz", "rb"))
# Train
etas = [0.01, 0.25, 1.5]
accuracies = []
for eta in etas:
    print('Training the network with eta =', eta)
    my_neural_network = Network([441, 100, 4])
    my_neural_network.train(X_train, y_train, X_valid, y_valid, eta=eta, num_epochs=50, isPrint=False)
    accuracies.append(my_neural_network.saved_valid_accuracies)
    print('For eta = ', eta, ', the accuracy is ', my_neural_network.accuracy(X_valid, y_valid), ' on the validation data', sep='')
<file_sep># Imports + Styles
import pickle, gzip
import numpy as np
import pandas as pd
from tqdm import trange
import matplotlib.pylab as plt
plt.style.use('bmh')
# basic network class definition
class Network:
    def __init__(self, sizes):
        """
        Initialize the neural network

        :param sizes: a list of the number of neurons in each layer
        """
        # save the number of layers in the network
        self.L = len(sizes)

        # store the list of layer sizes
        self.sizes = sizes

        # initialize the bias vectors for each hidden and output layer
        self.b = [np.random.randn(n) for n in self.sizes[1:]]

        # initialize the matrices of weights for each hidden and output layer
        self.W = [np.random.randn(n, m) for (m,n) in zip(self.sizes[:-1], self.sizes[1:])]

        # initialize the derivatives of biases for backprop
        self.db = [np.zeros(n) for n in self.sizes[1:]]

        # initialize the derivatives of weights for backprop
        self.dW = [np.zeros((n, m)) for (m,n) in zip(self.sizes[:-1], self.sizes[1:])]

        # initialize the activities on each hidden and output layer
        self.z = [np.zeros(n) for n in self.sizes]

        # initialize the activations on each hidden and output layer
        self.a = [np.zeros(n) for n in self.sizes]

        # initialize the deltas on each hidden and output layer
        self.delta = [np.zeros(n) for n in self.sizes]

        # initialize the arrays that will contain the accuracies
        self.saved_train_accuracies = []
        self.saved_valid_accuracies = []

    def g(self, z):
        """
        sigmoid activation function

        :param z: vector of activities to apply activation to
        """
        z = np.clip(z, -20, 20)
        return 1.0/(1.0 + np.exp(-z))

    def g_prime(self, z):
        """
        derivative of sigmoid activation function

        :param z: vector of activities to apply derivative of activation to
        """
        return self.g(z) * (1.0 - self.g(z))

    def gradC(self, a, y):
        """
        evaluate gradient of cost function for squared-loss C(a,y) = (a-y)^2/2

        :param a: activations on output layer
        :param y: vector-encoded label
        """
        return (a - y)

    def forward_prop(self, x):
        """
        take an feature vector and propagate through network

        :param x: input feature vector
        """
        # Initialize activation on initial layer to x
        self.a[0] = x
        # Loop over layers and compute activities and activations
        for ll in range(self.L-1):
            self.z[ll+1] = np.dot(self.W[ll], self.a[ll]) + self.b[ll]
            self.a[ll+1] = self.g(self.z[ll+1])

    def predict(self, X):
        """
        Predicts on the the data in X. Assume at least two output neurons so predictions
        are one-hot encoded vectorized labels.

        :param X: a matrix of data to make predictions on
        :return y: a matrix of vectorized labels
        """
        yhat = np.zeros((X.shape[0], self.sizes[-1]), dtype=int)
        # Populate yhat with one-hot-coded predictions
        for i, row in enumerate(X):
            for ll in range(self.L-1):
                z = np.dot(self.W[ll], row) + self.b[ll]
                row = self.g(z)
            yhat[i][np.argmax(row)] = 1
        return yhat

    def accuracy(self, X, y):
        """
        compute accuracy on labeled training set

        :param X: matrix of features
        :param y: matrix of vectorized true labels
        """
        yhat = self.predict(X)
        return np.sum(np.all(np.equal(yhat, y), axis=1)) / X.shape[0]

    def back_prop(self, x, y):
        """
        Back propagation to get derivatives of C wrt weights and biases for given training example

        :param x: training features
        :param y: vector-encoded label
        """
        # Forward prop training example to fill in activities and activations
        self.forward_prop(x)
        # Compute deltas on output layer
        self.delta[-1] = self.gradC(self.a[-1], y)*self.g_prime(self.z[-1])
        # Loop backward through layers, backprop deltas, compute dWs and dbs
        for ll in range(self.L-2, -1, -1):
            self.dW[ll] = np.outer(self.delta[ll+1], self.a[ll])
            self.db[ll] = self.delta[ll+1]
            self.delta[ll] = np.dot(self.W[ll].T, self.delta[ll+1])*self.g_prime(self.z[ll])

    def train(self, X_train, y_train, X_valid=None, y_valid=None, eta=0.25, lam=0.0, num_epochs=10, isPrint=True):
        """
        Train the network with SGD

        :param X_train: matrix of training features
        :param y_train: matrix of vector-encoded training labels
        :param X_train: optional matrix of validation features
        :param y_train: optional matrix of vector-encoded validation labels
        :param eta: learning rate
        :param lam: regularization strength
        :param num_epochs: number of epochs to run
        :param isPrint: flag indicating to print training progress or not
        """
        # initialize shuffled indices
        shuffled_inds = list(range(X_train.shape[0]))

        # loop over training epochs
        for ep in trange(num_epochs):

            # shuffle indices
            np.random.shuffle(shuffled_inds)

            # loop over training examples
            for ind in shuffled_inds:

                # back prop to get derivatives
                self.back_prop(X_train[ind, :], y_train[ind, :])
                # update weights and biases
                self.W = [Wll - eta*(dWll + lam*Wll) for Wll, dWll in zip(self.W, self.dW)]
                self.b = [bll - eta*dbll for bll, dbll in zip(self.b, self.db)]

            # occasionally print accuracy
            if isPrint and ((ep+1)%5)==1:
                self.epoch_report(ep, num_epochs, X_train, y_train, X_valid, y_valid)

        # print final accuracy
        if isPrint:
            self.epoch_report(ep, num_epochs, X_train, y_train, X_valid, y_valid)

    def epoch_report(self, ep, num_epochs, X_train, y_train, X_valid, y_valid):
        """
        Print the accuracy for the given epoch on training and validation data

        :param ep: the current epoch
        :param num_epochs: the total number of epochs
        :param X_train: matrix of training features
        :param y_train: matrix of vector-encoded training labels
        :param X_train: optional matrix of validation features
        :param y_train: optional matrix of vector-encoded validation labels
        """
        print("epoch {:3d}/{:3d}: ".format(ep+1, num_epochs), end="")
        print("  train acc: {:8.3f}".format(self.accuracy(X_train, y_train)), end="")
        self.saved_train_accuracies.append(self.accuracy(X_train, y_train))
        if X_valid is not None: print("  valid acc: {:8.3f}".format(self.accuracy(X_valid, y_valid)))
        else: print("")
        if X_valid is not None: self.saved_valid_accuracies.append(self.accuracy(X_valid, y_valid))
# tanh network class definition
class tanhNetwork:
    def __init__(self, sizes):
        """
        Initialize the neural network

        :param sizes: a list of the number of neurons in each layer
        """
        # save the number of layers in the network
        self.L = len(sizes)

        # store the list of layer sizes
        self.sizes = sizes

        # initialize the bias vectors for each hidden and output layer
        self.b = [np.random.randn(n) for n in self.sizes[1:]]

        # initialize the matrices of weights for each hidden and output layer
        self.W = [np.random.randn(n, m) for (m,n) in zip(self.sizes[:-1], self.sizes[1:])]

        # initialize the derivatives of biases for backprop
        self.db = [np.zeros(n) for n in self.sizes[1:]]

        # initialize the derivatives of weights for backprop
        self.dW = [np.zeros((n, m)) for (m,n) in zip(self.sizes[:-1], self.sizes[1:])]

        # initialize the activities on each hidden and output layer
        self.z = [np.zeros(n) for n in self.sizes]

        # initialize the activations on each hidden and output layer
        self.a = [np.zeros(n) for n in self.sizes]

        # initialize the deltas on each hidden and output layer
        self.delta = [np.zeros(n) for n in self.sizes]

        # initialize the arrays that will contain the accuracies
        self.saved_train_accuracies = []
        self.saved_valid_accuracies = []

    def g(self, z):
        """
        sigmoid activation function

        :param z: vector of activities to apply activation to
        """
#         z = np.clip(z, -20, 20)
        return np.tanh(z)

    def g_prime(self, z):
        """
        derivative of sigmoid activation function

        :param z: vector of activities to apply derivative of activation to
        """
        return 1 - self.g(z)**2

    def gradC(self, a, y):
        """
        evaluate gradient of cost function for squared-loss C(a,y) = (a-y)^2/2

        :param a: activations on output layer
        :param y: vector-encoded label
        """
        return (a - y)

    def forward_prop(self, x):
        """
        take an feature vector and propagate through network

        :param x: input feature vector
        """
        # Initialize activation on initial layer to x
        self.a[0] = x
        # Loop over layers and compute activities and activations
        for ll in range(self.L-1):
            self.z[ll+1] = np.dot(self.W[ll], self.a[ll]) + self.b[ll]
            self.a[ll+1] = self.g(self.z[ll+1])

    def predict(self, X):
        """
        Predicts on the the data in X. Assume at least two output neurons so predictions
        are one-hot encoded vectorized labels.

        :param X: a matrix of data to make predictions on
        :return y: a matrix of vectorized labels
        """
        yhat = np.zeros((X.shape[0], self.sizes[-1]), dtype=int)
        # Populate yhat with one-hot-coded predictions
        for i, row in enumerate(X):
            for ll in range(self.L-1):
                z = np.dot(self.W[ll], row) + self.b[ll]
                row = self.g(z)
            yhat[i][np.argmax(row)] = 1
        return yhat

    def accuracy(self, X, y):
        """
        compute accuracy on labeled training set

        :param X: matrix of features
        :param y: matrix of vectorized true labels
        """
        yhat = self.predict(X)
        return np.sum(np.all(np.equal(yhat, y), axis=1)) / X.shape[0]

    def back_prop(self, x, y):
        """
        Back propagation to get derivatives of C wrt weights and biases for given training example

        :param x: training features
        :param y: vector-encoded label
        """
        # Forward prop training example to fill in activities and activations
        self.forward_prop(x)
        # Compute deltas on output layer
        self.delta[-1] = self.gradC(self.a[-1], y)*self.g_prime(self.z[-1])
        # Loop backward through layers, backprop deltas, compute dWs and dbs
        for ll in range(self.L-2, -1, -1):
            self.dW[ll] = np.outer(self.delta[ll+1], self.a[ll])
            self.db[ll] = self.delta[ll+1]
            self.delta[ll] = np.dot(self.W[ll].T, self.delta[ll+1])*self.g_prime(self.z[ll])

    def train(self, X_train, y_train, X_valid=None, y_valid=None, eta=0.25, lam=0.0, num_epochs=10, isPrint=True):
        """
        Train the network with SGD

        :param X_train: matrix of training features
        :param y_train: matrix of vector-encoded training labels
        :param X_train: optional matrix of validation features
        :param y_train: optional matrix of vector-encoded validation labels
        :param eta: learning rate
        :param lam: regularization strength
        :param num_epochs: number of epochs to run
        :param isPrint: flag indicating to print training progress or not
        """
        # initialize shuffled indices
        shuffled_inds = list(range(X_train.shape[0]))

        # loop over training epochs
        for ep in trange(num_epochs):

            # shuffle indices
            np.random.shuffle(shuffled_inds)

            # loop over training examples
            for ind in shuffled_inds:

                # back prop to get derivatives
                self.back_prop(X_train[ind, :], y_train[ind, :])
                # update weights and biases
                self.W = [Wll - eta*(dWll + lam*Wll) for Wll, dWll in zip(self.W, self.dW)]
                self.b = [bll - eta*dbll for bll, dbll in zip(self.b, self.db)]

            # occasionally print accuracy
            if isPrint and ((ep+1)%5)==1:
                self.epoch_report(ep, num_epochs, X_train, y_train, X_valid, y_valid)

        # print final accuracy
        if isPrint:
            self.epoch_report(ep, num_epochs, X_train, y_train, X_valid, y_valid)

    def epoch_report(self, ep, num_epochs, X_train, y_train, X_valid, y_valid):
        """
        Print the accuracy for the given epoch on training and validation data

        :param ep: the current epoch
        :param num_epochs: the total number of epochs
        :param X_train: matrix of training features
        :param y_train: matrix of vector-encoded training labels
        :param X_train: optional matrix of validation features
        :param y_train: optional matrix of vector-encoded validation labels
        """
        print("epoch {:3d}/{:3d}: ".format(ep+1, num_epochs), end="")
        print("  train acc: {:8.3f}".format(self.accuracy(X_train, y_train)), end="")
        self.saved_train_accuracies.append(self.accuracy(X_train, y_train))
        if X_valid is not None: print("  valid acc: {:8.3f}".format(self.accuracy(X_valid, y_valid)))
        else: print("")
        if X_valid is not None: self.saved_valid_accuracies.append(self.accuracy(X_valid, y_valid))
<file_sep># Imports
import markovify
# Get the data from a text file containing past senate bills
with open("senate-bills.txt") as f:
    text = f.read()
# Initialize the Markov Chain with the senate bills data
text_model = markovify.Text(text)
# Print five randomly-generated sentences
print('RANDOMLY GENERATING 5 SENTENCES')
for i in range(5):
    print('Sentence', i+1, '-', text_model.make_sentence())
# Print three randomly-generated sentences of no more than 140 characters
print('RANDOMLY GENERATING 3 SENTENCES THAT OR NO MORE THAN 140 CHARACTERS LONG')
for i in range(3):
    print('Sentence', i+1, '-', text_model.make_short_sentence(140))
<file_sep>**Grade:** 95/100

## Comments

Problem 1:

--- Part A:   2/2

--- Part B:   2/2

--- Part C:   2/2

--- Part D:   2/2

--- Part E:   7/7

Problem 2:

--- Part A:   3/3

--- Part B:  12/12

--- Part C:   9/9 

--- Part D:   3/3 

--- Part E:   6/8 

--- Comments:  No commentary on optimal K in E.

Problem 3:

--- Part A:   8/8

--- Part B:   4/4

--- Part B:   3/3

Problem 4:

--- Part A:   5/5 

--- Part B:  15/15

--- Part C:   5/5 

--- Part D:  7/10

--- Comments: No commentary in part D.
<file_sep>**Grade:** 96/100

## Comments

Problem 1:

--- Part A:   5/5

--- Part B:  10/10

Problem 2:

--- Part A:   3/3

--- Part B:   5/5

--- Part C:   7/7 

--- Part D:   5/5 

Problem 3:

--- Part A:   5/5

--- Part B:   8/8 

--- Part C:  10/10

--- Part D:  10/10

--- Part E:   7/7

Problem 4:

--- Part A:   5/5 

--- Part B:   5/5 

--- Part C:   3/5 

--- Part D:   3/5 

--- Part E:   3/5 

--- Comments: C-E: No commentary.    

Extra Credit: 2/10 

--- Comments: No commentary.
<file_sep>**Grade:** 100/100

## Comments

Problem 1:

--- Part A:   2/2

--- Part B:   8/8

--- Part C:   5/5

Problem 2:

--- Part A:  15/15

--- Part B:  15/15

--- Part C:   5/5 

Problem 3:

--- Part A:   5/5

--- Part B:   5/5 

--- Part C:  10/10

--- Part D:  10/10

Problem 4:

--- Part A:  10/10

--- Part B:  10/10
