Very crudely speaking, number of features refer to the size of the vectors/tensors.
Outside end-to-end neural networks, the term feature had a more tangible meaning, since feature engineering was an import processing step you had to do “manually”. For example, to use a SVM to classify a text document had extract the text document into a set of features. This could some very naive features such as #words, #characters. In this case, each document would be represented by 2 numerical features. In practice, you would have more meaningful features but they would have a clear semantic meaning.
With end-to-end neural networks, this semantic meaning is usually latent and not obvious. For example, each word in a text may be represented by a 300-dim vector of numerical values, i.e., the word has 300 features. This vector places a word in a 300-dim space in relation to other words. However, you usually don’t really know what an individual value in the vector means. For example, The 25th entry in the 300-dim vector does not tell you that the word is a noun, verb, adjective, etc.
So in case you use an LSTM for text processing where the LSTM processes a sequences of words, each words is represented by a vector of size input_size. This representation is needed since word are symbolic representations. For example, the words “cat” and “kitten” are only similar to you because your mental models of both concepts (animal, 4 legs, furry, meows, etc.) are similar. For a computer, these are completely different things. A vector representation now allows to map “cat” and “kitten” (and all other word) to a numerical representation where the vector for “cat” and the vector for “kitten” are closer together compared to, say, the vector of “cat” and the vector of “train”.
In contrast, if you use an LSTM for time series prediction of already scalar numerical values, input_size is just 1.
For the resulting dimensions of the hidden state, you best consult the PyTorch docs, but it definitely does not depend on the input_size.