Question: I'm trying to predict movie genres using a neural network. I initially considered using a softmax layer as my output layer, but since a movie can have multiple genre labels, how should my output be?

Also, how would I have to format my data to make it work with Keras?

Answer: 
> 
> how would I have to format my data to make it work with Keras?
> 
> 
> 

Your training labels in the output layer should be a binary vector that is 1 for class which is present and 0 for class which is not. For example, let us assume you have 3 classes of genres - comedy, romantic and horror. There are many ways to make it and Scikit-learn has a method which makes it very easy which I show below.

```
>>> from sklearn.preprocessing import MultiLabelBinarizer
>>> mlb = MultiLabelBinarizer()
>>> y = mlb.fit_transform([[0,2],[1]])
array([[1, 0, 1],
   [0, 1, 0]])

```

> 
> I initially considered using a softmax layer as my output layer, but since a movie can have multiple genre labels, how should my output be?
> 
> 
> 

This is a simple Keras example I suggest.

```
>>> from keras.models import Sequential
>>> from keras.layers import Dense, Activation

>>> model = Sequential([
    Dense(32, input_dim=784),
    Activation('relu'),
    Dense(10),
    Activation('sigmoid'),
    ])
>>> model.compile(optimizer='rmsprop', loss='binary_crossentropy')
>>> model.fit(X_train, y_train)

```

Refer [this](https://datascience.stackexchange.com/questions/13949/activation-method-and-loss-function-for-multilabel-multiclass-classification) for more info. I used sigmoid because it is better for multilabel classification.
You have label of classes which are not mutually exclusive which means as the label of each sample, your data won't be in one-hot-encoding format. Each output vector may have multiple ones. In such occasions you shouldn't use soft-max as the output layer. You have to use `Sigmoid` activation function for each neuron in the last layer. Suppose you have ten labels and for a typical movie each of them may be activated. So, in the last layer use a dense layer with ten `Sigmoid` activation function. You can see [here](https://datascience.stackexchange.com/q/25315/28175) which may help you.

And as a side answer, each cost function that you are going to use should be categorical, because you have different categories. I respect so much for the other answer and I thank him/her for the answer, but I guess you should use categorical cost function otherwise the code won't work because your output matrix actually is a *matrix* of samples and not a vector of samples.