Hello, I’m currently working to implement a version of the region proposal network on musical audio transformed into autocorrelated frequency spectrums, to try to draw bounds around the different musical sections. (Chorus, verse, intro, etc.)
When I try to run the system, the feature extractor quickly turns the image into nothing, only after 2 epochs on a 550 song dataset. I’m going to try to include all of the feature maps I have. Any tips on how to fix this?? I’m quite desperate at this point!
with no training:

image1141×949 101 KB

after 1 epoch:

image1141×949 100 KB

after 2 epochs

image1141×930 87.1 KB

I train in accordance with the RPN (training backend + rpn first and then fine tuning rpn layers), here are my parameters:
For backend + rpn:
epochs = 10
lr = .001
optimizer = optim.Adam(rpn.parameters(), lr=lr)
scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=550, gamma=0.7)
for rpn fine tuning:
epochs = 10
lr = .0001
optimizer = optim.Adam(rpn.parameters(), lr=lr)
scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=550, gamma=0.6)
also my weights seem to be random as well. but im not sure what a good way to check that is. looking for any help!!