I am rather new to coding and I appreciate any help, but be gentle with me.

I am playing around with the MNIST database for neural networks, since I want to transfer the results to another problem. What I am trying do is to manipulate the MNIST training data set by including a set of images to the image to be classified. Allow me to structure the approach:

* While training the neural network the MNIST db provides an image of a handwritten digit (x\_train) and its label/class (y\_train)
* However I want to train the neural network not only with a single image input, but also feed the neural network with optional images to choose from
* So if I want the machine to classify the digit "5", I would have the image of the digit "5" and also a set of random images, which should have a random quantity, as the input:

-> Input = 
Image to classify "5" | Images to refer to "1", "4", "5", the next would be 
Image to classify "0" | Images to refer to "0", "9", "3", "5", "6" and so on...

* The "images to refer to" should always include the "digit to classify" but not the "image to classify". Meaning the index of "Image to classify "5"" should not be the same as the index of "Images to refer to ... "5""
* So far I managed to choose random images (digit\_randomizer()) of digits of random quantity (random\_with\_N\_digits()). What I miss is:

	1. Indices which exclude themselves: Index of "5" to classify is not the index of "5" to choose from
	2. The images to refer to should not have repeating digits

To 1.: Below you can see my function digit\_randomizer(). I currently have no idea how to approach this problem, but with a nested loop checking for "np.where(j != i)"

To 2.: I am considering splitting y\_train into 10 different sets of labels (each for one digit). However I don't know what kind of command I should write, since I would need to define a random quantity of images. Pick a random image from the 10 sets at random, while watching out for the index.

Here is my code so far:

```
import keras as k
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Conv2D, Dropout, Flatten, MaxPooling2D
import matplotlib.pyplot as plt

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

print('')
print('x_train shape:', x_train.shape)

# Reshaping the array to 4-dims so that it can work with the Keras API
x_train = x_train.reshape(x_train.shape[0], 28, 28, 1)
x_test = x_test.reshape(x_test.shape[0], 28, 28, 1)
input_shape = (28, 28, 1)
# Making sure that the values are float so that we can get decimal points after division
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
# Normalizing the RGB codes by dividing it to the max RGB value.
x_train /= 255
x_test /= 255
print('x_train shape reshaped:', x_train.shape)
print('Number of images in x_train', x_train.shape[0])
print('Number of images in x_test', x_test.shape[0])

classes = set(y_train)
print('Number of classes =', len(classes),'\n')
print('Classes: \n', classes, '\n')

print('y_train shape:', y_train.shape)
print('y_test shape:', y_test.shape)

import random
import warnings
warnings.filterwarnings("ignore")
from random import randint

#select a random image from the training data
def digit_select():
    for j in range(1):
        j = np.random.choice(np.arange(0, len(y_train)), size = (1,))
        digit = (x_train[j] * 255).reshape((28, 28)).astype("uint8")
        imgplot = plt.imshow(digit)
        plt.title(y_train[j])
        imgplot.set_cmap('Greys')
        plt.show()

# return between 1 or 10 images
def random_with_N_digits():
    range_start = 0
    range_end = 9
    return randint(range_start, range_end)

# return 1 or 10 random images
def digit_randomizer():  
    for i in range(random_with_N_digits()):
        i = np.random.choice(np.arange(0, len(y_train)), size = (1,))
        image = (x_train[i] * 255).reshape((28, 28)).astype("uint8")
        imgplot = plt.imshow(image)
        imgplot.set_cmap('Greys')
        plt.title(y_train[i])
        plt.show()

```

Somehow digit\_select should be excluded from digit\_randomizer and digit\_randomizer should only pick one image per class from y\_train. 

Any ideas are very much appreciated.

***CODE Edit:***

```
def digit_label_randselect():
    j = np.random.choice(np.arange(0, len(y_train)), size=(1,))
    return int(y_train[j])
print('Randomly selected label:', digit_label_randselect())

Output: Randomly selected label: 4

def n_reference_digits(input_digit_label):
    other_digits = list(np.unique(y_train)) #create list with all digits
    other_digits.remove(input_digit_label) #remove the input digit label
    sample = random.sample(other_digits, len(np.unique(y_train))-1) #Take a sample of size n of the digits
    sample.append(input_digit_label)
    random.shuffle(sample)
    return sample
print('Randomly shuffled labels:', n_reference_digits(digit_label_randselect()))

Output: Randomly shuffled labels: [8, 0, 6, 2, 7, 4, 3, 5, 9, 1]

'''returns a list of 10 random indices.
necessary to choose random 10 digits as a set, which will be used to train the NN.
the set needs to contain a single identical y_train value (label),
meaning that a random digit definitely has the same random digit in the set.
however their indices have to differ. moreover all y_train values (labels) have to be different,
meaning that the set contains a single representation of every digit.'''
def digit_indices_randselect():
    listi = []
    for i in range(10):
        i = np.random.choice(np.arange(0, len(y_train)), size = (1,))
        listi.append(i)
    return listi
listindex = digit_indices_randselect()
print('Random list of indices:', listindex)

Output: Random list of indices: [array([53451]), array([31815]), array([4519]), array([21354]), array([14855]), array([45147]), array([42903]), array([37681]), array([1386]), array([9584])]

'''for every index in listindex return the corresponding index, pixel array and label'''
#TO DO: One Hot Encode the labels
def array_and_label_for_digit_indices_randselect():
    listi = []
    digit_data = []
    labels = []
    for i in listindex:
        digit_array = x_train[i] #digit data (image array) is the data from index i
        label = y_train[i] #corresponding label
        listi.append(i)
        digit_data.append(digit_array)
        labels.append(label)
    list3 = list(zip(listi, digit_data, labels))
    return list3
array_and_label_for_digit_indices_randselect()

Output:[(array([5437]),
  array([[[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   4,  29,  29,  29,
            29,  29,  29,  29,  92,  91, 141, 241, 255, 228,  94,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,  45, 107, 179, 252, 252, 252,
           253, 252, 252, 252, 253, 252, 252, 252, 253, 252, 224,  19,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,  45, 240, 252, 253, 252, 252, 252,
           253, 252, 252, 252, 253, 252, 252, 252, 253, 252, 186,   6,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0, 157, 252, 252, 253, 252, 252, 252,
           253, 252, 252, 252, 241, 215, 252, 252, 253, 202,  19,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,  41, 253, 253, 253, 255, 234, 100,   0,
             0,   0,   0,   0,   0,  70, 253, 253, 251, 125,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,  66, 252, 252, 252, 253, 133,   0,   0,
             0,   0,   0,   0,   0, 169, 252, 252, 200,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   7, 130, 168, 168, 106,  19,   0,   0,
             0,   0,   0,   0,  10, 197, 252, 252, 113,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0, 128, 252, 252, 252,  63,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,  13, 204, 253, 253, 241,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,  88, 253, 252, 233, 109,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0, 225, 253, 252, 234,  22,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,  38, 237, 253, 252, 164,  15,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,  26, 172, 253, 254, 228,  31,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0, 114, 234, 252, 253, 139,  19,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           111, 234, 252, 252,  94,  19,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           241, 252, 252, 202,  13,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  76,
           254, 253, 253,  78,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 225,
           253, 252, 233,  22,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 225,
           253, 233,  62,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  38, 187,
           241,  59,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0],
          [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
             0,   0,   0,   0]]], dtype=uint8),
  array([7], dtype=uint8)),...

'''for every index in x_train return a random index, its pixel array, label.
also return a list of 10 random indices (digit_indices_randselect())'''
def digit_with_set():
    for i in x_train:
        i = random.randrange(len(y_train)) #returns a random index of a digit between 0 - 60000
        digit_data = x_train[i] #digit data (image array) is the data from index i
        label = y_train[i] #corresponding label
        #digit_randomizer() #returns a random set of 10 images
        print("Index of digit to classify:", i), \
        print("Digit to classify:", label), \
        print("Corresponding array:", digit_data), \
        print("Set of 10 Images:", array_and_label_for_digit_indices_randselect())
        print("")
        print("Next:")
digit_with_set()```

***PURPOSE Edit:*** The purpose of this approach is to research, whether a neural network can devise a model, which not only classifies the input, but also recognizes the possibility from choosing a label from the optional set. Meaning that the model not only classifies the "5" as the "5", but also looks into its options and finds a fit there as well.

This may not make much sense for an image classification problem. However I am working on a sequence to sequence problem in another project. The input sequences are in multiple columns in a .csv file. The output is another sequence. The issue lies within the very heterogenous input, so the accuracy is low and loss is very high.

This is how the data is structured:

**Input**: | AA_10| 31.05.2019 | CW20 | Project1 |   **Output**: AA_Project1_[11]

**Input**: |      | CW19       |      | Project2 |   **Output**: AA_Project2_[3]

**Input**: | y550 | 01.06.2019 | AA12 | Project1 |   **Output**: AA_Project1_[12]

The AA_ProjectX_[Value] within the output is the main issue since its range varies from project to project. Project1 can have [0-12], Project 2 can have [0-20], ProjectX [0-N].

By adding a range of values to the input data I hope to restrict the network from learning values which are not part of the project.

Input: | AA_10| 31.05.2019 | CW20 | Project1 | [0,1,2,3,4,5,6,7,8,9,10,11,12] |  Output: AA_Project1_[11]

So when I want to classify the digit 5, I give the machine a range of possibile classes and corresponding images to derive the output class from.

```