<|start_filename|>README.html<|end_filename|>
<html>
<head>
<title>Welcome to MMLL</title>
</head>

<body>
    
    
<p>MMLL is a Musical Machine Listening Library for javascript's Web Audio API. It provides a variety of higher level musical listening facilities for computer music, such as onset detection, (major-minor) chord detection, beat tracking and auditory modelling. All listening objects can run live, or could be called in a simulated block by block way offline, faster than realtime.</p>

<p>The library was developed by <a href="http://composerprogrammer.com/index.html"><NAME></a> as part of the AHRC funded MIMIC project (Musically Intelligent Machines Interacting Creatively). MMLL is released under an MIT license, see the included COPYING.txt file.</p>

<p>The library can be used just for the machine listening objects, used within your own audio callback (e.g., as part of a ScriptProcessorNode), or via a quick set-up frontend that hides Web Audio API details and  has you write setup and audio callback functions analogous to Processing's setup and draw.</p>

<p>The latter method is the one explained here, but expert Web Audio API people should find it easy enough to just take the analyzers for their own work. Only including the precompiled MMLL.js file is needed to deploy the library, though from the home directory of the library you can compile it afresh via the shell script provided (it is just a concatenation of the js source files).</p>

<p>The typical expectation of a machine listening object is that we are working at 44.1KHz sampling rate and that a mono (single channel) input block of samples will be provided for analysis. The objects deal with accumulating samples ready for processing (often via an FFT) themselves and the user doesn't have to worry about that part. However, objects should cope at other standard sampling rates such as 48KHz, 88.2KHz and 96 KHz, even if performance is sub-optimal (for example, the onset detector was developed based on evaluation over a corpus of 44.1KHz samples, so works best at this home rate).</p>

<p>The <a href="Examples/minimalexample.html">minimal code example</a> is reproduced below and linked here too. Note how the machine listener object is prefixed with MMLL, and the SetUp function is passed the sampling rate, needed for initialising the listener. The CallBack is where the main action happens, as each new block of input samples is passed in. The input and output arguments hold MMLLInput and MMLLOutpu objects, which make the channels of input and output audio accessible, as well as a special input.monoinput which is a single channel ready for the listener. If a stereo sound file is loaded or two channel live input requested, the monoinput will be the average of the left and right channels. The output object assumes a stereo output for now, exposing the left and right channel data arrays.</p>

<p>var audioblocksize = 256; //lowest latency possible </p>

<p>var setup = function SetUp(sampleRate) { <br>
sensorydissonance = new MMLLSensoryDissonance(sampleRate); <br>
}; </p>

<p>var callback = function CallBack(input,output,n) { </p>

<p>var dissonance = sensorydissonance.next(input.monoinput); </p>

<p>console.log(dissonance); </p>

<p>for (i = 0; i < n; ++i) {<br>
    output.outputL[i] = input.inputL[i];<br>
    output.outputR[i] = input.inputR[i];<br>
    }<br>
    
};</p>

<p>var gui = new MMLLBasicGUISetup(callback,setup,audioblocksize,true,true);</p>



<p>The Examples folder provides a test example for each listener currently available in the library.</p>

<p>Happy MMLLing</p>


</body>
</html>


<|start_filename|>Core/MMLLSpectrumExtractor.js<|end_filename|>
//for non-realtime extraction of the spectrum

//returns an array of complex spectral data frames, given a MONO sound file to analyse

//Sample rates for sound files must match that assumed in overall code, no sample rate conversion is carried out

function MMLLSpectrumExtractor(blocksize=512,sampleRate=44100,fftsize=1024,hopsize=512,windowtype=0) {
 
    var self = this;
    
    self.audioblocksize = blocksize;
    self.inputAudio = new MMLLInputAudio(self.audioblocksize);
  
    self.sampleRate = sampleRate;

    self.numInputChannels = 1;

    self.fftsize = fftsize;
    self.hopsize = hopsize;
    self.windowtype = windowtype;
    
    //context required for sampler's decodeAudioData calls
    
    //can request specific sample rate, but leaving as device's default for now
    //https://developer.mozilla.org/en-US/docs/Web/API/AudioContext/AudioContext
    try {
        self.audiocontext = new webkitAudioContext();
        
    } catch (e) {
        
        try {
            
            self.audiocontext = new AudioContext();
            
        } catch(e) {
            
            alert("Your browser does not support Web Audio API!");
            return;
        }
        
    }
    
    //ignore sampleRate of audiocontext for now, just running through sound files at rate set by user
    //self.sampleRate = self.audiocontext.sampleRate; //usually 44100.0
    //console.log("AudioContext established with sample rate:",self.sampleRate," and now setting up for input type:",self.inputtype); //print
  
    
    //use async, wait?
    self.analyseAudioFile = function(filename,updatefunction) {
        
       
        var spectraldata;
        
        self.sampler = new MMLLSampler();
        
        
        return new Promise(function(resolve, reject) {
        //"/sounds/05_radiohead_killer_cars.wav"
        self.sampler.loadSamples([filename],
                                 function onload() {
                                 
                                 console.log('loaded: ',filename);
                                 
                                 self.sampleplayer = new MMLLSamplePlayer();
                                 self.sampleplayer.reset(self.sampler.buffers[0]);
                                 //self.sampleplayer.numChannels = self.sampler.buffers[0]
                                 
                                 if(self.sampleplayer.numChannels>1) {
                                 //interleaved input
                                 self.numInputChannels = 2;
                                 
                                 self.inputAudio.numChannels = self.numInputChannels;
                                 //self.samplearray = new Float32Array(2*audioblocksize);
                                 
                                 }
                                 //fresh STFT object
                                 var stft = new MMLLSTFT(self.fftsize,self.hopsize,self.windowtype);
                                 
                                 
                                 //samplearray depends on number of Channels whether interleaved
                                 
                                 //include last frame, will be zero padded as needed
                                 var numblocks = Math.floor(self.sampleplayer.lengthinsampleframes/self.audioblocksize);
                                 
                                 
                                 //starts with fftsize-hopsize in the bank
                                 var numspectralframes = Math.floor(((numblocks*self.audioblocksize) - self.hopsize)/self.hopsize) + 1;
                                 
                                 //self.processSoundFile(self.audioblocksize);
                              
                                 
                                 spectraldata = new Array(numspectralframes);
                                 
                                 
                                 var j=0, i = 0;
                                 
                                 //complex data not in packed form, but dc and nyquist real values as complex numbers at 0 and fftsize indices
                                 for (j = 0; j < numspectralframes; ++j)
                                 spectraldata[j] = new Array(self.fftsize+2);
                                 
                                 console.log('Extracting spectrum for: ',filename); //debug console message
                                 
                                 var spectralframecount = 0;
                                 
                                 for (j = 0; j < numblocks; ++j) {
          
                                    updatefunction(j,numblocks);
                                 
                                 //needed since player accumulates to its output
                                    for (i = 0; i < self.audioblocksize; ++i) self.inputAudio.monoinput[i] = self.inputAudio.inputL[i] = self.inputAudio.inputR[i] = 0.0;
                                 
                                    self.sampleplayer.render(self.inputAudio,self.audioblocksize);
                                 
                                    var newframe = stft.next(self.inputAudio.monoinput);
                                 
                                 if(newframe) {
                                 
                                 if(spectralframecount>=numspectralframes) {
                                 
                                 console.log("what the spectrum?",numblocks,numspectralframes,spectralframecount,self.hopsize,self.fftsize);
                                 }
                                 
                                 
                                 // var fftdata = stft.complex;
                                 
                                  for (i = 0; i < (self.fftsize+2); ++i)
                                    spectraldata[spectralframecount][i] = stft.complex[i];
                                 
                                    ++spectralframecount;
                                 }
                                 
                                
                                 
                                 }
                                 
                                 
                                 resolve(spectraldata); //return via Promise
                                 
                                 
                                 },self.audiocontext);
                           
                           })
        
        
    };


    
}

