It is hard for Linear layers to learn periodic functions like Sine and Cosine. RNN would work better in these cases. Here is an example 4 of using LSTM to approximate Sine waves.
Edit: I just noticed that your network also have a very limited amount of parameters(only 4), the example you saw has 20. This is also another important problem. What I mean is that as my example and yours demonstrated, RNN can approximate Sine functions better than FCs: this is also why we don’t use FC for sequence modeling.