Neural Network unable to track training data
I am new to ML and this is my first Tensorflow project. I am doing regression with Neural Networks on a dataset with 17 features and 1 outcome. But for some reason my network is unable to follow the training data. I am getting massive errors in results, as can be seen from the plots below. 458
I have also tried experimenting with different parameters (learning rate, nodes per layer, number of layers etc) but nothing seems to work. I have pasted the Tensorflow code here. I have also provided my cost and training plots as well as a link to my datasets. I'd be grateful if someone could please help me figure out what I am doing wrong. Thank you!
Links to the datasets- Features arranged in columns- https://drive.google.com/file/d/1U182Lhf67WygeSbv6BNEx5LHyL7Ba13O/view?usp=sharing Output column - https://drive.google.com/file/d/10XWo1d5mhIsxccQBgAyGDWDAVgu2BjAA/view?usp=sharing
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
# importing features and observations data for training and validation
training_filename_X = "training_set_X.csv"
training_filename_Y = "training_set_Y.csv"
validation_filename_X = "validation_set_X.csv"
test_filename_X = "test_set_X.csv"
test_filename_Y = "test_set_Y.csv"
validation_filename_Y = "validation_set_Y.csv"
training_features = np.loadtxt(training_filename_X, delimiter=',')
training_observations = np.loadtxt(training_filename_Y, delimiter=',')
validation_features = np.loadtxt(validation_filename_X, delimiter=',')
validation_observations = np.loadtxt(validation_filename_Y, delimiter=',')
test_features = np.loadtxt(test_filename_X, delimiter=',')
test_observations = np.loadtxt(test_filename_Y, delimiter=',')
# normalizing training data
training_features_stddev_arr = np.std(training_features, axis=0)
training_features_mean_arr = np.mean(training_features, axis=0)
normalized_training_features = (training_features-training_features_mean_arr)/training_features_stddev_arr
# layer parameters
n_nodes_hl1 = 5
n_nodes_hl2 = 5
n_nodes_hl3 = 3
no_features = 17
learning_rate = 0.001
epochs = 2000
cost_history = []
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)
# defining weights for each layer taken from a normal distribution with variance 2/n
hl1_weight = tf.Variable(tf.random_normal([no_features, n_nodes_hl1], stddev=np.sqrt(2/no_features)))
hl2_weight = tf.Variable(tf.random_normal([n_nodes_hl1, n_nodes_hl2], stddev=np.sqrt(2/n_nodes_hl1)))
hl3_weight = tf.Variable(tf.random_normal([n_nodes_hl2, n_nodes_hl3], stddev=np.sqrt(2/n_nodes_hl2)))
output_weight = tf.Variable(tf.random_normal([n_nodes_hl3, 1], stddev=np.sqrt(2/n_nodes_hl3)))
# defining biases for each layer
hl1_bias = tf.Variable(tf.random_uniform([n_nodes_hl1], -1.0, 1.0))
hl2_bias = tf.Variable(tf.random_uniform([n_nodes_hl2], -1.0, 1.0))
hl3_bias = tf.Variable(tf.random_uniform([n_nodes_hl3], -1.0, 1.0))
output_bias = tf.Variable(tf.random_uniform([1], -1. 422