I would like to access the array with something like array[5000][440] meaning 5000ms from the start and 440hz and it would give me a value of the frequency's amplitude at this very position.
I could not find something like that here, if there is, please point me to it.
You're operating under a couple of misconceptions.
You can't get the frequency of a wave at a particular point in time. You need to select a window of time, including many points before and after the point of interest. The more points you include, the more resolution you'll have in your frequency breakdown. You'll need to run some sort of windowing function on those points, then subject them to a FFT.
Once you have the results of the FFT, the numbers will correspond to frequencies but it won't be a simple relationship. You don't have any control over the frequency corresponding to each output, that was already determined by the sampling frequency of your signal combined with the number of samples. I'm afraid I don't have the conversion formula at hand. Each frequency will have two components, a real and an imaginary, and the amplitude will be sqrt(r**2+i**2).
How do I make python load a big(2hours) wave-file and convert it's contents into a time-frequency array?