Regression with Neural Networks in Tensorflow problem
I have recently started learning Neural networks and Python. I am trying out linear regression for a dataset with 14 features and 1 outcome. I have divided the data into training and test data. 458
I have experimented with many parameters (learning rate, nodes per layer, number of layers, number of steps and optimization algorithm) but my test errors are as high as 150%.
I have posted my code below along with the cost curve (cost vs epochs). It'll be of great help to me if some of you could go through the short code and let me know where I am committing a mistake and what to change. Or if you could please suggest some other important checks. I have added relevant comments as well.
Thank you!
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
# importing features and observations data for training and validation
training_filename_X = "training_set_X.csv"
training_filename_Y = "training_set_Y.csv"
test_filename_X = "test_set_X.csv"
test_filename_Y = "test_set_Y.csv"
training_features = np.loadtxt(training_filename_X, delimiter=',')
training_observations = np.loadtxt(training_filename_Y, delimiter=',')
test_features = np.loadtxt(test_filename_X, delimiter=',')
test_observations = np.loadtxt(test_filename_Y, delimiter=',')
# normalizing training data
training_features_stddev_arr = np.std(training_features, axis=0)
training_features_mean_arr = np.mean(training_features, axis=0)
normalized_training_features = (training_features-training_features_mean_arr)/training_features_stddev_arr
# normalizing validation data with training set mean and standard deviation
normalized_validation_features = (validation_features-training_features_mean_arr)/training_features_stddev_arr
# normalizing test data with training set mean and standard deviation
normalized_test_features = (test_features-training_features_mean_arr)/training_features_stddev_arr
# layer parameters
n_nodes_hl1 = 20
n_nodes_hl2 = 20
n_nodes_hl3 = 20
no_features = 14
learning_rate = 0.01
epochs = 200
cost_history = np.empty(shape=[1], dtype=float)
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)
# defining weights for each layer taken from a normal distribution with variance 2/n
hl1_weight = tf.Variable(tf.random_normal([no_features, n_nodes_hl1], stddev=np.sqrt(2/no_features)))
hl2_weight = tf.Variable(tf.random_normal([n_nodes_hl1, n_nodes_hl2], stddev=np.sqrt(2/n_nodes_hl1)))
hl3_weight = tf.Variable(tf.random_normal([n_nodes_hl2, n_nodes_hl3], stddev=np.sqrt(2/n_nodes_hl2)))
output_weight = tf.Variable(tf.random_normal([n_nodes_hl3, 1], stddev=np.sqrt(2/n_nodes_hl3)))
# defining biases for each layer
hl1_bias = tf.Variable(tf.random_uniform([n_nodes_hl1], -1.0, 1.0))
hl2_bias = tf.Variable(tf.random_uniform([n_nodes_hl2], -1.0, 1.0))
hl3_bias = tf.Variable(tf.random_uniform([n_nodes_hl3], -1.0, 1.0))
output_bias = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
# defining activation functions for each layer
hl1 = tf.sigmoid(tf.matmul(X, hl1_weight) + hl1_bias)
hl2 = tf.sigmoid(tf. 422