Body: <p>I am new to neural networks. I am trying to solve a binary classification problem. Specifically, I want to determine whether a patient has or not a certain disease based on the dataset.</p>
  <p>The dataset has <strong>about 700 samples</strong> of different patients. I divided the sets into training and test (<strong>test size = 0.3</strong>).
  My model has 1 input layer, 5 hidden layers, and 1 output layer. I used ReLU for the input and hidden layers, and I used the sigmoid for the output layer.</p>
  <p>During the compilation of the model, I used stochastic gradient descent (SGD) as the optimizer and the <strong>mean squared logarithmic error</strong> for the loss.  I used mini-batch gradient descend (<strong>batch size = 4</strong>) for the training.</p>
  <p>I am trying to calculate the accuracy on the test set I created previously.</p>
  <ul>
  <li><p>The model evaluation for <strong>train set</strong> is about: 0.07 (loss) 0.76 (accuracy).</p>
  </li>
  <li><p>The model evaluation for <strong>test set</strong> is about: 0.07 (loss) 0.74 (accuracy).</p>
  </li>
  </ul>
  <p>Firstly, I would like to know if this is a good value for a model. Is the accuracy too small?</p>
  <p>Plus, I would like to know if there's a way to improve accuracy based on my model.</p>
  <p>I am trying to work on a project, so I was wondering if these values are acceptable.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-26T11:50:43.027Z
Comments.Text: Hello. Welcome to AI SE. Please, take the time to go through our [on-topic page](https://ai.stackexchange.com/help/on-topic), which describes the type of question that you can ask here, and [How do I ask a good question?](https://ai.stackexchange.com/help/how-to-ask). Now, you should [**edit your post**](https://ai.stackexchange.com/posts/32189/edit) in order to describe more in detail the 1. **task you're solving** (I suppose you're solving a classification problem, but what are you classifying?), 2. **how much data you're using for training, testing, etc.** [continuing on the next comment]
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-26T11:51:06.443Z
Comments.Text: 3. **are you computing the accuracy on a separate dataset (i.e. the test dataset) or training dataset**, and 4. **describe the architecture of your neural network and other hyper-parameters, e.g. the optimizer**, 5. **what is your ultimate goal?** do you want to deploy your model in production or are you just playing with neural networks?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-26T13:20:44.047Z
Comments.Text: I edited my post. Thank you for your adviceses.
Comments.UserDisplayName: FateMintTrio
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-26T16:00:15.863Z
Comments.Text: Thanks! Now your question looks a lot better than before. So, you used 700*0.3 = 210 samples for testing and to compute the accuracy of 0.74 (and the remaining for training)? If that's the case, your dataset is very small. Unless the problem is simple, I don't think this amount of data is enough. So, you could probably get a higher test accuracy if you use more data. 5 hidden layers for your neural network seems also too many for such a small amount of data. Have you looked at the training/test/validation curves in order to determine whether your model has potentially overfitted the data?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-26T16:02:24.713Z
Comments.Text: You should also make sure that you randomly shuffled the data before splitting it into training and test datasets.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-28T07:05:43.970Z
Comments.Text: Yes, the dataset is too small but I need it for my project. I used 5 hidden layers because with a few less hidden layers the loss and the accuracy are worse. I tried to create a plot to see graphically what's going on but I didn't see overfitting.
Comments.UserDisplayName: FateMintTrio
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-02T08:17:53.747Z
Comments.Text: is always hard to give suggestions without having access to data examples. From what you posted the only reasonable feedback someone can give is that the model seems to be learning fine since training and test accuracy are close in values. Might very well be the limit this approach can reach. Of course there are many things you could try. Larger batch and adding dropout (if not already part of your model architecture) are quick examples. But again, for specific suggestions we need as most detailed information as possible.
Comments.UserDisplayName: Edoardo Guerriero
Tags: <neural-networks><machine-learning><accuracy><binary-classification>
Title: Is a test accuracy of 0.74 good enough, given a dataset of about 700 samples, and, if not, how can I improve it?
UserDisplayName: FateMintTrio
ThreadText: # Is a test accuracy of 0.74 good enough, given a dataset of about 700 samples, and, if not, how can I improve it?
  
  (Asked by: username_0 on 2021-10-26)
  
  I am new to neural networks. I am trying to solve a binary classification problem. Specifically, I want to determine whether a patient has or not a certain disease based on the dataset.
  
  The dataset has **about 700 samples** of different patients. I divided the sets into training and test (**test size = 0.3**).
  My model has 1 input layer, 5 hidden layers, and 1 output layer. I used ReLU for the input and hidden layers, and I used the sigmoid for the output layer.
  
  During the compilation of the model, I used stochastic gradient descent (SGD) as the optimizer and the **mean squared logarithmic error** for the loss. I used mini-batch gradient descend (**batch size = 4**) for the training.
  
  I am trying to calculate the accuracy on the test set I created previously.
  
  * The model evaluation for **train set** is about: 0.07 (loss) 0.76 (accuracy).
  * The model evaluation for **test set** is about: 0.07 (loss) 0.74 (accuracy).
  
  Firstly, I would like to know if this is a good value for a model. Is the accuracy too small?
  
  Plus, I would like to know if there's a way to improve accuracy based on my model.
  
  I am trying to work on a project, so I was wondering if these values are acceptable.
  
  **Tags:** neural-networks, machine-learning, accuracy, binary-classification
  
  > **Comments:**
  > **username_1**: Hello. Welcome to AI SE. Please, take the time to go through our [on-topic page](https://ai.stackexchange.com/help/on-topic), which describes the type of question that you can ask here, and [How do I ask a good question?](https://ai.stackexchange.com/help/how-to-ask). Now, you should [**edit your post**](https://ai.stackexchange.com/posts/32189/edit) in order to describe more in detail the 1. **task you're solving** (I suppose you're solving a classification problem, but what are you classifying?), 2. **how much data you're using for training, testing, etc.** [continuing on the next comment]  
  > **username_1**: 3. **are you computing the accuracy on a separate dataset (i.e. the test dataset) or training dataset**, and 4. **describe the architecture of your neural network and other hyper-parameters, e.g. the optimizer**, 5. **what is your ultimate goal?** do you want to deploy your model in production or are you just playing with neural networks?  
  > **username_0**: I edited my post. Thank you for your adviceses.  
  > **username_1**: Thanks! Now your question looks a lot better than before. So, you used 700*0.3 = 210 samples for testing and to compute the accuracy of 0.74 (and the remaining for training)? If that's the case, your dataset is very small. Unless the problem is simple, I don't think this amount of data is enough. So, you could probably get a higher test accuracy if you use more data. 5 hidden layers for your neural network seems also too many for such a small amount of data. Have you looked at the training/test/validation curves in order to determine whether your model has potentially overfitted the data?  
  > **username_1**: You should also make sure that you randomly shuffled the data before splitting it into training and test datasets.  
  > **username_0**: Yes, the dataset is too small but I need it for my project. I used 5 hidden layers because with a few less hidden layers the loss and the accuracy are worse. I tried to create a plot to see graphically what's going on but I didn't see overfitting.  
  > **username_2**: is always hard to give suggestions without having access to data examples. From what you posted the only reasonable feedback someone can give is that the model seems to be learning fine since training and test accuracy are close in values. Might very well be the limit this approach can reach. Of course there are many things you could try. Larger batch and adding dropout (if not already part of your model architecture) are quick examples. But again, for specific suggestions we need as most detailed information as possible.  
  
  ---
  
  ## 0 answers
  
  