We live in busy environments , and our surrounds continuously flood our sensory system with complex information that needs to be analyzed in order to make sense of the world around us ., This process , labeled scene analysis , is common across all sensory modalities including vision , audition and olfaction 1 ., It refers to the ability of humans , animals and machines alike to parse the mixture of cues impinging on our senses , organize them into meaningful groups and map them onto relevant foreground and background objects ., Our brain relies on innate dispositions that aid this process and help guide the organization of patterns into perceived objects 2 ., These dispositions , referred to as Gestalt principles , inform our current understanding of the perceptual organization of scenes 3 , 4 ., In most theoretical accounts , the role of Gestalt principles in parsing a scene can be conceptualized in two stages: segregation ( or analysis ) and grouping ( or fusion ) 5 ., In the first stage , the sensory mixture is decomposed into feature elements , believed to be the building blocks of the scene ., These features reflect the physical nature of sources in the scene , the state and structure of the environment itself , as well as perceptual mappings of these attributes as viewed by the sensory system ., These features vary in complexity along a continuum from basic attributes ( e . g . edges or frequency components ) to more complex characteristics of the scene ( e . g . shapes or timbral profiles ) ., The ubiquitous nature of these profiles often conceals the multiplexed structures that underlie this analysis of scene features in the brain ., In most computational accounts , this segregation stage is modeled using feature analyses which map the sensory signal into its building blocks ranging from simple components ( e . g . frequency channels ) to dimensionally-complex kernels 6 , 7 ., Processing the distinctive features of a scene is generally followed by a fusion stage which integrates the state and behavior of the scene’s building blocks using grouping mechanisms that reflect the local and global distribution and dynamics of the features ., This stage employs ‘rules’ that guide how grouped elements give rise to perceptually coherent structures forming objects or streams 2 , 8 , 9 ., In many mathematical models , these grouping cues are often leveraged in back-end classifiers that are tuned to capture patterns and relationships within specific object classes ( e . g . speech , music , faces , etc ) 10–13 ., In doing so , these models effectively capture the inter-dependencies between object attributes and learn their mapping onto an integrated representational space 14–16 ., Ultimately , success in tackling scene analysis depends on two key components 17:, ( i ) obtaining a rich and robust feature representation that can capture object specific details present in the scene;, ( ii ) grouping the feature elements such that their spatial and temporal associations match the dynamics of objects within the scene ., Vision models have been very successful in mining these two aspects of scene analysis ., Intricate hierarchical systems have leveraged inherent structure in static and dynamic images to extract increasingly elaborate features from a scene that are then used to segment it , interpret its objects or track them over time 18–20 ., Data-driven approaches have shown that high dimensional feature spaces are very effective in extracting meaningful semantics from arbitrary natural images 20–22; while hand-engineered features like scale-invariant feature transform ( SIFT ) 23 , histogram of oriented gradients ( HOG ) 24 , and Bag-of-visual-word descriptor 25 among others have also enjoyed a great deal of success in tackling computer vision problems like image classification and object detection ., Recent advances in deep layered architectures have resulted in a flurry of rich representational spaces showing selectivity to contours , corners , angles and surface boundaries in images 26–29 ., The deep nature of these architectures has also led to a natural evolution from low-level features to more complex , higher-level embeddings that capture scene semantics or syntax 30 , 31 ., In audition , computational approaches to tackle auditory scene organization have mostly taken advantage of physiological and perceptual underpinnings of sound processing 17 ., A large body of work has built on knowledge of the auditory pathway , particularly the peripheral system to build sophisticated analysis models of auditory scenes ., These systems extract relevant cues from a scene , such as its spectral content , spatial structure as well as temporal dynamics; hence allowing sound events with uncorrelated acoustic behavior to occupy different subspaces in the analysis stage ., These models are quite effective in replicating perceptual results of stream segregation especially using simple tone and noise stimuli 32–37 ., Some models also extend beyond early acoustic features to examine feature binding mechanisms that can be used as an effective strategy in segregating wide range of stimuli from simple tone sequences to spectro-temporally complex sounds like speech and music 38–40 ., In most approaches however , the models are built around hand-crafted feature representations , hence limiting their scope to specific mappings of the acoustic space ., With the emergence of deep belief architectures , recent efforts started learning rich feature spaces from natural soundscapes in a data driven fashion , and subsequently using these spaces in domains like music genre classification , phoneme classification and speaker identification 41–44 ., Applications of deep learning have also successfully tackled the problem of speech separation even with monaural inputs by learning embeddings of a speaker’s time-frequency dynamics against other speakers 45 , 46 ., The current study also leverages neural network theory to ‘learn’ Gestalt principles directly from sound ., The work examines what kind of cues can one infer from natural sounds; how well do these learned cue reflect the known Gestalt components of auditory streams; and how effective are these cues in explaining perceptual organization of auditory scenes with varying degrees of complexity ., The model is devised as a hierarchical structure that generally follows the two-stage pipeline of analysis then fusion , in line with prototypical scene analysis theories 5 ., This system analyzes the incoming acoustic signal with a multitude of granularities , hence allowing both local and global acoustic attributes to emerge ., The short-term analysis performs a local tiling of the spectro-temporal space; hence inferring simultaneous grouping cues 47–49 ., A longer-range analysis extends the segregation stage to examine temporal dependencies across acoustic attributes over different time scales; hence exploring emergence of sequential grouping cues 50–54 ., Finally , a fusion stage binds the cues together based on how strongly they correlate with each other across multiple time scales ., This integration is achieved using Hebbian learning which reinforces activity across coherent channels and suppresses activity across incoherent ones 55–57 ., Apart from the basic layout and choice of analysis window sizes , the network is trained in an unsupervised fashion on a rich sound dataset including speech and nature sounds hence offering a general inference architecture of auditory Gestalt cues that are common across many sound environments ., The overall system is tested with a wide range of stimuli where we can quantify the role of each and every component of the network in driving stream segregation processes ., We also contrast the system performance with a set of control experiments where different components of the model are deliberately switched on/off in order to examine their impact on the organization of different acoustic scenes ., These control experiments aim not only to dissect the role of various system components ., They also shed light on how necessary and/or sufficient different grouping cues are to anchor the analysis of different stimuli structures and sound types ., The paper first presents an in-depth description of the proposed architecture , followed by an analysis of the emergent properties of the trained network and their potential neural correlates in the auditory pathway ., The experimental results outline how the network replicates human psychoacoustic behavior in stream segregation and speech intelligibility paradigms ., Finally , we present control experiments that dissect the network architecture and examine the contribution its component ., We discuss the implications of this network in shedding light on ties between observed perceptual performance in various complex auditory scenes and the neural underpinnings of this behavior as implemented in networks of neurons along the auditory pathway ., A number of Gestalt principles have been posited as indispensable anchors used by the brain to guide the segregation of auditory scenes into perceptually meaningful objects 8 , 47 , 58 ., These comprise a wide variety of cues; for instance harmonicity which couples harmonically-related frequency channels together , common fate which favors sound elements that co-vary in amplitude , and common onsets which groups components that share a similar starting time and to a lesser degree a common ending time ., Most of these cues are thought to be innate in our auditory system , and evidence for their role is found across many species 59–63 ., These processes likely take advantage of statistical regularities of sounds in natural environments and reflect the physical constraints of sound generation and propagation ( e . g . two sound sources rarely start at the exactly the same time; periodic vibrations induce resonant modes at integer multiples of the fundamental frequency ) ., Here , we examine whether a statistical inference model can learn these cues directly from natural sounds; and if so , how effective are these learned cues relative to existing hand-tailored segregation systems ., The proposed model is designed as a hierarchical system that explicitly mimics an ‘analysis-then-fusion’ processing pipeline ., The analysis stage is itself laid out in two stages ., First , an analysis of local spectrotemporal cues aims to learn simultaneous Gestalt cues believed to operate over short-time scales in order to locally segregate sound elements ., Second , an analysis of more global cues operates over longer time-scales and aims to learn sequential Gestalt cues that enable tracking dynamics of elements from the first stage at a temporal or melodic level 8 ., Following these stages is a fusion step that combines together segregated elements that constitute different auditory objects , using principles of temporal coherence 39 , 64 , 65 ., The Gestalt analysis stages are learned directly from natural sounds in a generative fashion , allowing each component of the model to represent natural sounds from its own vantage point following principles of stochastic neural networks , as detailed next ., The fusion stage merely organizes or fuses these learned patterns following the concept of temporal coherence , as also detailed later ., Fig 1 depicts a schematic of the overall model ., It takes as input the acoustic waveform of an auditory scene u ( t ) and maps it onto a time-frequency representation , using a biomimetic peripheral model from Yang et al . 66 ., Briefly , this transformation analyzes the acoustic signal u ( t ) using a bank of logarithmically-spaced cochlear filters whose outputs are further sharpened via a first order derivative along the frequency axis , followed by half wave rectification and short term integration over 10ms frames ( see Methods for details ) ., This filterbank analysis results in an auditory spectrogram represented by S ( t , f ) ., The following stage ( called L 1 ) is structured as a two-layer sparse Restricted Boltzmann Machine ( sparse RBM ) with a fully connected visible and hidden layer 67 ., It takes as input 3 consecutive frames of the spectrogram and learns a probability distribution over the set of these short tokens ., RBMs are powerful stochastic neural networks that are conceptually similar to autoencoders but can infer statistical distributions over their input set 68 ., A RBM layer is chosen for this stage in order to explore the space of local spectrotemporal tokens and learn latent cues that represent statistical structures in natural sounds over short time scales ., The visible layer units {xk} are real-valued and characterized by a Gaussian distribution fitted over the input spectrogram S ( t , f ) ; while hidden units {hk} are sampled from a Bernoulli distribution for k = 1 , 2 , … , K where K is the number of nodes in each layer ., The network is parameterized by Θ = {W , A , B} where W represents the interconnected weights between visible and hidden units , and A ( B ) represents the visible ( hidden ) bias , respectively ., The network is trained using a Contrastive Divergence ( CD ) algorithm with the objective to minimize the reconstruction error between x and x ^ = h W + A 69 ., By learning the regularities in local spectrotemporal tokens of natural sounds , the connection weights W effectively span an array of latent cues that reflect the structure of soundscapes ., Our hypothesis is that these latent factors represent the so-called simultaneous cues used as Gestalt principles for sound analysis ., After training , connection weights are transformed into a 2D filter F ( t , f ) , akin to spectro-temporal receptive fields derived from neural activity of biological neurons in the auditory system 70 ., These learned filters are then applied in a convolutional fashion over the incoming spectrogram S ( t , f ) to derive the outputs of layer L 1 nodes ., These responses are further subjected to a neural adaptation stage which imposes a dynamic regulation of the response of each filter hence suppressing units with weak activation ( see Methods for details ) ., L, 1, responses are then processed by the next layer in the model which completes the analysis stage to infer possible sequential cues that extend over longer time constants ., This second layer L 2 is devised as an array of conditional RBMs ( cRBMs ) , which are extended versions of RBMs designed to model temporal dependencies 71 ., Similar to a RBM , a cRBM consists of a visible layer with units {xk} , assumed to arise from a Gaussian distribution fitted over the input , and a hidden layer with {hk} units sampled from a Bernoulli distribution ., Unlike a RBM , a cRBM acts as a dynamical system operating over an entire input history τ taking as input occurrences at times {t , t − 1 , … , t − τ} in order to capture dynamics in the input space over context τ ., In the current model , we explore sequential cues over a range of temporal contexts and construct an array of parallel cRBM networks over multiple histories ranging in temporal resolutions from τ ∼ ( 30–600 ms ) ., L 2 is parameterized by Θ = {W , Aτ , Bτ , Cτ , Dτ} where W represents the interconnected weights between visible and hidden units and capture the interactions across input features over an extended temporal history τ , Aτ and Bτ represent the visible and hidden biases , respectively , while Cτ and Dτ quantify autoregressive weights between past inputs and the current input ( or current hidden unit , respectively ) ., Just like the localized layer L 1 , the contextual layer L 2 is trained in a generative fashion using contrastive divergence ( CD ) in order to best capture the dynamics in natural sounds using the same dataset of realistic sounds spanning speech , music and natural sounds ., Here again , our hypothesis is that the stochastic cRBM learns latent parameters Θ that reflect the sequential cues underlying dynamics of natural sounds over a wide range of temporal contexts ., Once trained , the model parameters are applied to incoming L 1 filter responses in a linear fashion , yielding a multi-resolution output which is then passed over to the next stage in the hierarchy ( see Methods for details ) ., The next layer in the hierarchy focuses on a fusion operation to facilitate the grouping of perceptually-coherent objects ., This binding stage explores co-activations across all L 2 channels within a given context τ and binds together the units that exhibit strong temporal coherence 64 , 72 ., The ‘temporal coherence’ theory posits that emergence of perceptual representations of auditory objects depends upon strong coherence across cues emanating from same object and weaker co-activation across cues from competing objects ., This coherence is not an instantaneous correlation but one that is accumulated over longer time scales , commensurate with the contextual windows explored in the L 2 layer ., We implement this concept in a biologically-plausible fashion via mechanisms of Hebbian learning , which suggests that when two neurons fire together , their synaptic connection gets stronger 73 ., Effectively , Hebbian interactions operate by reinforcing activity across coherent channels , hence grouping them into putative objects and inhibiting activity across incoherent channels 74 ., We implement a synaptic interaction across output channels from layer L 2 by introducing a coherence synaptic weight matrix V . If two units i and j are co-activated at a given time t , their corresponding synaptic connection Vij is reinforced over time ., If the correlation between their activity is weak , the corresponding synaptic weight Vij is reduced accordingly ., These synaptic weights are applied to the output of each channel in a dynamic fashion , hence modulating the activity across an entire ensemble of neurons within each context in layer L 2 ., The net effect gives emergence to perceptual coherent groups that represent auditory objects in a scene ., A final read-out stage is then appended to the model to extract responses to different stimuli and test the degree of segregation of different objects , as viewed by the model outputs ( see Methods for details ) ., In order to examine the emergent sensitivity of learned layers in the network , we derive the tuning characteristics of individual nodes or neurons and explore their filtering properties in the modulation domain 75 , 76 ., Modulation tuning reflects stimulus cues that best drive individual nodes in the model both in terms of temporal variations and dynamics ( i . e . temporal modulations or rates ) as well as spectral span and bandwidth ( i . e . spectral modulations or scales ) ., This approach follows common empirical techniques used in electrophyisology and psychophysics to probe the tuning of a system to specific acoustic cues ., It is specifically used to characterize spectro-temporal receptive fields ( STRFs ) which offer 2-dimensional profiles of filtering characteristics of neurons 70 ., First , we employ a classic transfer function method using probe stimuli in order to derive the tuning of both L 1 and L 2 layers of the network 77–79 ., We present modulated noise signals ( called ripples ) as input to the model with varying spectro-temporal modulation parameters ( Fig 2E ) and characterize the fidelity of the ripple encoding at various stages of the network as the ripple modulation parameters are varied 80 ., Each ripple is constructed as a broadband noise signal whose envelope is modulated both in time and frequency , with temporal modulation parameter ω ( in Hz ) and spectral modulation parameter Ω ( in cyc/oct ) ( see Methods for details ) ., By sweeping through a range of ripple parameters , we compute a normalized modulation transfer function ( MTF ) from the response of layers L 1 and L 2 which quantifies the synchronized response of each layer to the corresponding dynamics in the ripple stimulus ( see Methods for details ) ., L 3 is not a trained layer and hence is not subject to this analysis ., Fig 2A and 2B depict the MTF derived from both L 1 and L 2 ., The functions highlight that both layers exhibit a general low-pass behavior both along temporal and spectral modulations ., As expected , layer L 1 is trained over shorter time-scales and does exhibit faster temporal dynamics along the rate axis , while the contextual layer L 2 is mostly tuned to slower dynamics < 30Hz with a slightly tighter spectral selectivity mostly concentrated below 1 cycles/oct ., This outcome is very reminiscent of similar transfer functions obtained from neurophysiological data showing contrasting tuning characterizations in the midbrain , auditory thalamus and auditory cortex 81–83 , whereby selectivity of individual neurons along the mammalian auditory hierarchy evolves from faster to slower temporal dynamics and from more refined to broader spectral spans along frequency ., We further examine the selectivity of individual neurons and compare emergent tuning characteristics common across nodes in the network by employing an agglomerative clustering algorithm ( see Methods for details ) ., This approach clusters nodes exhibiting similar tuning profiles into common groups hence providing insight into the underlying acoustic cues being processed by each cluster ., Fig 2C and 2D show contour plots from the resulting clusters overlaid on the MTF profiles for layers L 1 and L 2 ., The array of clusters indicates that neurons in each of these layers do indeed exhibit a wide variety of selectivity to spectral and temporal dynamics in the input signal ., We specifically note a cluster of L 1 neurons that is more sensitive to fast transients or ‘onsets’ ., This group is labeled ‘O’ in Fig 2C ., An example time-frequency profile F ( t , f ) of a neuron in the ‘O’ cluster is shown in Fig 2F ( upper-right ) ., We also note a spectrally-structured cluster ( labeled ‘H’ ) centered around spectral modulations ∈ 1-2 cyc/oct corresponding to harmonic peaks present in natural sounds ., An example neuron from this cluster is shown in Fig 2F ( upper-left ) and highlights the selectivity to specific frequency bands in the input spectrogram ., The clustering procedure also reveals the presence of oriented spectro-temporally selective clusters , likely tuned to detect frequency-modulated sweeps in the signal over different spectrotemporal scales; as well as other clusters with special selectivity to spectral or temporal features ., Fig 2F ( lower panels ) shows an example of two L 1 neurons with different temporal dynamics contrasting a slow neuron ‘S’ and a fast neuron ‘F’ ., We test the model’s behavior with a variety of acoustic scenes ranging from classic streaming paradigms using simple tones to experiments using speech signals ., Crucially , all experiments are tested on the same model ( after all layers have been trained ) , without any adjustment to model parameters ., The stimulus parameters are carefully chosen to closely replicate previously published human perceptual experiments hence allowing a direct comparison between the model and human perception ., All stream segregation results are shown in Fig 3 organized in 3 columns: the stimulus on the left , a replica of human perception of the same stimulus reproduced from the corresponding publication in the center , and the model performance on the right ., As outlined earlier , Fig 3 contrasts the model’s performance against reported human perceptual results in a range of stream segregation experiments ., Next , we reexamine our initial hypotheses; namely that the model is able to infer simultaneous and sequential grouping cues by learning statistical regularities in natural soundscapes ., The experimental results shown in the previous section suggest that simultaneous cues ( tonotopic organization , AM rate , harmonicity , temporal synchrony , etc ) , sequential cues and grouping mechanisms play an important role in streaming paradigms ., In order to shed light on their individual contributions , we run a series of control experiments where we look at malfunctions in the model if certain components of the system are disrupted individually ., The analysis of control experiments quantifies the complementarity of rich feature representation and grouping mechanisms in driving scene segregation ., The proposed architecture faithfully replicates human psychoacoustic behavior on steaming paradigms over wide range of stimuli ranging from simple tones to speech utterances as demonstrated in Fig 3 ., In case of two tone streaming paradigm shown in ( Fig 3A ) , the network exhibits stream segregation when two alternating tones are widely separated across tonotopic frequency axis ., This behavior is consistent with well established psychophysical and physiological findings of stream segregation induced by differences in tonotopic cues 129–132; and relies heavily on the activation of different groups of neurons with distinct frequency selectivities as captured in L 1 ., In absence of temporal correlation between these two groups , the temporal coherence layer aided by the adaptation mechanism suppresses the anti-correlated groups of units , hence inducing stream segregation in the final stage of the network ., However when ΔF is small enough , there is high degree of overlap resulting in a single stream percept ., This segregation/integration effect is strongly maintained regardless of a number of manipulations to the model architecture ., The key components crucial to the organization of tone sequences are the presence of tonotopic or frequency selectivity combined with temporal integration that examines activity across neural channels at relatively longer time-scales ., This observation is very much in line with the spatio-temporal view of auditory stream segregation which requires neural channels to be widely separated in addition to temporal asynchrony across these channels 133 ., The interaction of spectral and temporal dynamics during the organization of tone sequences supports the view of stream segregation as a dynamic process ., The buildup effect reported in the current model ( Fig 3B ) is in line with established psychoacoustic behaviors 90 , 134–136 and suggests that segregation of two streams is not instantaneous; but strengthens over time and can lead to segregation when frequency difference ( ΔF ) is large enough ., The current model highlights that this effect is in fact reflecting the competition across neural channels as viewed by the temporal coherence layer ., The binding of correlated groups of neurons strengthens over time while suppressing the anti-correlated units over time in the same process ., Interaction across multiple features is also noted in other simulations that pit against each other harmonicity , onsets and temporal dynamics ( Fig 3C , 3D and 3E ) ., Simulations using complex tones directly examine the role of localized spectro-temporal tuning in L 1 as an encoding of simultaneous cues such as harmonicity , onset and fast amplitude modulations among others ., Sequential cues emergent in L 2 are crucial in tracking the activity emerging in the localized layer over longer amplitude modulations; which are then fused together in the last L 3 layer ., Through this rich selectivity learned directly from natural sounds , the network offers a wide span of selectivity across the spectrotemporal space ., This tuning proves effective in tackling complex auditory scenes composed of speech with various interferers ., In line with human perceptual data , the model shows that speech tokens are harder to identify in presence of utterances from same corpus compared to babble and cafe noise as the signal-to-noise ratio gets smaller ., The model highlights that this variable response is largely caused by the dominance of neural activity from the interfering set relative to the target ., The distinct activation between target and interferer is further blurred in absence of of slow sequential cues which integrate information about the speech utterance beyond just that target number/color ., As shown in the control experiments , a network that lacks slow sequential cues is further impaired in making a judgment about the identity of the target token , likely due a to an enhanced confusion between its representation and that of the interferer ., Once this activity reaches the temporal coherence layer , the weakly responsive neurons get suppressed , hence resulting in the actual number/color token getting wrongly identified as the one in the interfering utterance ., Overall , the proposed model highlights three key results:, ( i ) Using the right configuration , we are able to infer a wide-range of Gestalt cues directly from natural sounds ., The proposed RBM architecture offers a cooperative and nonlinear integration of these cues to result in a multiplexed representation of auditory scenes across various granularities in time and frequency ., By using an unsupervised learning approach , the network is not being optimized for a specific application; rather , it is reflecting the inherent variety of local and global dynamics present in natural sounds ., Possibly , an even deeper neural architecture extending beyond just a few layers could extend the rich feature analysis and fill in the spectrum from local to global hence adding a more refined mapping along with the nonlinear integration naturally offered by the RBM architecture ., ( ii ) Grouping acoustic features is effectively an outlook across all active nodes that allows to piece together the pieces of each auditory object ., This process effectively plays 2 key roles: a grouping role by putting together pieces of a sound object ( effectively integrating together pitch , timbre , rhythm and possibly space information that reflect a common object ) ; and an elimination role by suppressing channels that are irrelevant to the emergence of the foreground object , hence enhancing the signal-to-noise ratio in the network ., Temporal coherence is one such fusion mechanism that has been garnering stronger neural and perceptual evidence 39 , 65 , 125 , 126 ., The current work employs Hebbian learning , a biological simple mechanism that affords such fusion over the rightly chosen time-scales ., ( iii ) Auditory scene segregation is a balancing act of the proper feature analysis along with mechanisms for fusion that give rise of auditory object representations ., While both stages are necessary , neither one is sufficient ., The proposed model offers a unified platform that integrates together these different mechanisms and strategies ., It also bridges the existing physiological theories of scene organization with perceptual accounts of auditory scene analysis ., The proposed model is structured along 4 key stages: initial data pre-processing by transforming the acoustic signal to a time-frequency representation , a local analysis over short time-scales , a global analysis over an array of longer time-scales , then a fusion stage using temporal coherence ., A final readout of the network activity is implemented to extract information from specific streaming experiments to probe segregation of individual streams in the input scene ., Details of each component of the model are outlined next: The acoustic signal is first analyzed through a model of peripheral processing in the mammalian auditory system , following the model by Yang et al . 66 ., Briefly , it transforms the acoustic stimulus sampled at 8KHz into a joint time-frequency representation referred to as auditory spectrogram ., The stage starts with a bank of 128 asymmetric constant-Q filters equally-spaced on a logarithmic axis over 5 . 3 octaves spanning the range 180 Hz to 4000 Hz ( QERB ≈ 4 ) 137 ., By its very nature , the peripheral model uses a non-parametric set of cochlear filters that are fixed over a span of 5 . 3 octaves ( see 66 for details ) ., In the current model , we cap our sampling rate to 8KHz in order to provide ample coverage over lower frequency regions ., After cochlear filtering , the outputs undergo spectral sharpening via first order derivative along frequency , followed by half-wa
Our current understanding of how the brain segregates auditory scenes into meaningful objects is in line with a Gestaltism framework ., These Gestalt principles suggest a theory of how different attributes of the soundscape are extracted then bound together into separate groups that reflect different objects or streams present in the scene ., These cues are thought to reflect the underlying statistical structure of natural sounds in a similar way that statistics of natural images are closely linked to the principles that guide figure-ground segregation and object segmentation in vision ., In the present study , we leverage inference in stochastic neural networks to learn emergent grouping cues directly from natural soundscapes including speech , music and sounds in nature ., The model learns a hierarchy of local and global spectro-temporal attributes reminiscent of simultaneous and sequential Gestalt cues that underlie the organization of auditory scenes ., These mappings operate at multiple time scales to analyze an incoming complex scene and are then fused using a Hebbian network that binds together coherent features into perceptually-segregated auditory objects ., The proposed architecture successfully emulates a wide range of well established auditory scene segregation phenomena and quantifies the complimentary role of segregation and binding cues in driving auditory scene segregation .
In every day life , our brain is able to effortlessly make sense of the cacophony of sounds that constantly enter our ears and organize them into meaningful sound objects ., In this work , we use an architecture based on stochastic neural networks to ‘learn’ from natural sounds which cues are crucial to the process of auditory scene organization ., The computational model delivers a hierarchical architecture that mimics multistage processing in the biological auditory system ., It learns a rich hierarchy of spectral and temporal features that allow the decomposition of an auditory scene into informative components ., These features are then grouped together into coherent objects based on Hebbian learning principles ., Though trained on unrelated datasets of natural sounds , the model is able to replicate human perception of auditory scenes in a wide variety of soundscapes ranging from simple tone sequences to complex speech-in-noise scenes .
acoustics, linguistics, neural networks, engineering and technology, signal processing, social sciences, neuroscience, neuronal tuning, computer and information sciences, animal cells, speech, physics, cellular neuroscience, speech signal processing, cell biology, neurons, biology and life sciences, cellular types, physical sciences, cognitive science, modulation, acoustic signals