I get a result in ~1 second using your code on the CPU:
model = LSTMnetwork()
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr = 0.001)

#training the model
epochs = 10

#to monitor training time
start_time = time.time()

for epoch in range(epochs):
    #reset the parameters and hidden states
    optimizer.zero_grad()        
    model.hidden1 = (torch.zeros(1,1,model.hidden_size1),
                   torch.zeros(1,1,model.hidden_size1))        
    
    model.hidden2 = (torch.zeros(1,1,model.hidden_size2),
                   torch.zeros(1,1,model.hidden_size2))
    
    seq = torch.randn(128, 1, 3)
    label = torch.randn(1, 1)    
    y_pred_train = model(seq)
    
    loss = criterion(y_pred_train, label.reshape(1,1))
    loss.backward()
    optimizer.step()
    
    # print training result
    print(f'Epoch: {epoch+1:2} Loss: {loss.item():10.8f}')

print(f'\nDuration: {time.time() - start_time:.0f} seconds')

so I guess your script might hang somewhere.
Could you remove the data loading part and check if random data also returns quickly?