How to extract features from the encoded layer of an autoencoder?
I have done some research on autoencoders, and I have come to understand that they can also be used for feature extraction (see this question on this site as an example). Most of the examples out there seem to focus on autoencoders applied to image data, but I would like to apply them to a more general data set.
Therefore, I have implemented an autoencoder using the keras framework in Python. 856
For simplicity, and to test my program, I have tested it against the Iris Data Set, telling it to compress my original data from 4 features down to 2, to see how it would behave.
The encoder seems to be doing its job in compressing the data (the output of the encoder layer does indeed show only two columns). However, the values of these two columns do not appear in the original dataset, which makes me think that the autoencoder is doing something in the background, selecting/combining the features in order to get to the compressed representation.
Here is the complete working example:
``` from pandas import read_csv from numpy.random import seed from sklearn.model_selection import train_test_split from keras.layers import Input, Dense from keras.models import Model
Get input data and separate features from labels
df = read_csv("iris.data") Y = df.iloc[:,4] X = df.iloc[:, : 4]
Split data set in train and test data
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, train_size=0.5, random_state=seed(1234))
Input information
col_num = X.shape[1] input_dim = Input(shape=(col_num,))
Encoding information
encoding_dim = 2 encoded = Dense(encoding_dim, activation='relu')(input_dim)
Decoding information
decoded = Dense(col_num, activation='sigmoid')(encoded)
Autoencoder information (encoder + decoder)
autoencoder = Model(input=input_dim, output=decoded)
Train the autoencoder
autoencoder.compile(optimizer='adadelta', loss='mean_squared_error') autoencoder.fit(X_train, X_train, nb_epoch=50, batch_size=100, shuffle=True, validation_data=(X_test, X_test))
Encoder information for feature extraction
encoder = Model(input=input_dim, output=encoded) encoded_input = Input(shape=(encoding_dim,)) encoded_output = encoder.predict(X_test)
Show the encoded values
print(encoded_output[:5]) ```
This is the output from this example:
[[ <PHONE_NUMBER> 6.151131 ] <PHONE_NUMBER> ] <PHONE_NUMBER> ] [ 3.0278277 2.7351477 ] [ 0.<PHONE_NUMBER>.064625 ]]
Basically, my idea was to use the autoencoder to extract the most relevant features from the original data set. However, so far I have only managed to get the autoencoder to compress the data, without really understanding what the most important features are though.
My question is therefore this: is there any way to understand which features are being considered by the autoencoder to compress the data, and how exactly they are used to get to the 2-column compressed representation? 422