Well, you should at least feed it things like month, day of week, hour of day (one-hot encoding is the simplest way). But I don’t think you can do much better than your second plot.
In general, that’s not a good task for neural networks. Usual forecasting methods decompose seasonal data (nb: explicitly defined) and use relatively simple formulas to forecast the remainder stepwise. And there is not much else you can do, that would generalize to unseen data.
Use of RNNs may be justified when many “features” with complex interactions are present, but in your case deseasoned fluctuations are likely “unexplainable”.
Maybe you should try something like Prophet 5. For more basic ideas, you may check online book 5