summarize the discussion on lpc[SEP] <s>phd a: eh , we should be going . professor b: so ne next week we 'll have , uh , both birger and , uh , mike michael michael kleinschmidt and birger kollmeier will join us . phd d: uh - huh . professor b: um , and you 're you 're probably gon na go up in a couple three weeks or so ? when d when are you thinking of going up to , uh , ogi ? phd d: yeah , like , uh , not next week but maybe the week after . professor b: ok . good . so at least we 'll have one meeting with yo with you still around , and and phd d: uh - huh . professor b: that 's good . phd d: um , yeah . well , maybe we can start with this . mmm . professor b: all today , huh ? phd d: yeah . professor b: oh . phd d: um . yeah . so there was this conference call this morning , um , and the only topic on the agenda was just to discuss a and to come at uh , to get a decision about this latency problem . professor b: no , this i 'm sorry , this is a conference call between different aurora people or just ? phd d: uh , yeah . it 's the conference call between the aurora , uh , group . professor b: it 's the main conference call . ok . phd d: uh , yeah . there were like two hours of discussions , and then suddenly , uh , people were tired , i guess , and they decided on a number , two hundred and twenty , um , included e including everything . uh , it means that it 's like eighty milliseconds less than before . professor b: and what are we sitting at currently ? phd d: um . professor b: yeah . phd d: so , currently d uh , we have system that has two hundred and thirty . so , that 's fine . professor b: two thirty . phd d: yeah . so that 's the system that 's described on the second point of this document . professor b: so it 's we have to reduce it by ten milliseconds somehow . phd d: yeah . but that 's yeah . that 's not a problem , i i guess . professor b: ok . w it 's it 's p d primary primarily determined by the vad at this point , phd d: um . professor b: right ? phd d: yeah . professor b: s so we can make the vad a little shorter . phd d: yeah . at this point , yeah . professor b: that 's phd d: yeah , uh - huh . professor b: yeah . we probably should do that pretty soon so that we do n't get used to it being a certain way . phd d: uh - huh . professor b: yeah . phd d: um . professor b: was hari on the on the phone ? phd d: yeah , sure . professor b: ok . phd d: well , it was mainly a discussion between hari and david , professor b: hmm . phd d: who was like professor b: yeah . phd d: uh , professor b: ok . phd d: mmm uh , yeah . so , the second thing is the system that we have currently . oh , yes . we have , like , a system that gives sixty - two percent improvement , but if you want to stick to the this latency well , it has a latency of two thirty , but if you want also to stick to the number of features that limit it to sixty , then we go a little bit down but it 's still sixty - one percent . uh , and if we drop the tandem network , then we have fifty - seven percent . professor b: uh , but th the two th two thirty includes the tandem network ? phd d: yeah . professor b: ok . and i is the tandem network , uh , small enough that it will fit on the terminal size in terms of ? phd d: uh , no , i do n't think so . professor b: no . phd d: no . professor b: ok . phd d: it 's still in terms of computation , if we use , like , their way of computing the the maps the the mips , i think it fits , professor b: mm - hmm . mm - hmm . phd d: but it 's , uh , m mainly a problem of memory . professor b: right . phd d: um , and i do n't know how much this can be discussed or not , because it 's it could be in rom , so it 's maybe not that expensive . but professor b: ho - how much memory d ? h how many ? phd d: i d i d uh , i i do n't kn remember exactly , but uh . yeah , i c i i have to check that . professor b: yeah . i 'd like to see that , cuz maybe i could think a little bit about it , cuz we maybe we could make it a little smaller or i mean , it 'd be it 'd be neat if we could fit it all . phd d: uh - huh . professor b: uh , i 'd like to see how far off we are . phd d: mm - hmm . professor b: but i guess it 's still within their rules to have have it on the , uh , t uh , server side . right ? phd d: yeah . yeah . professor b: ok . phd d: mmm . professor b: and this is still ? uh , well , y you 're saying here . i c i should just let you go on . phd d: yeah , there were small tricks to make this tandem network work . uh , mmm , and one of the trick was to , um , use some kind of hierarchical structure where the silence probability is not computed by the final tandem network but by the vad network . um , so apparently it looks better when , uh , we use the silence probability from the vad network professor b: huh . phd d: and we re - scale the other probabilities by one minus the silence probability . um . so it 's some kind of hierarchical thing , uh , that sunil also tried , um , on spine and apparently it helps a little bit also . mmm . and . yeah , the reason w why why we did that with the silence probability was that , um professor b: could ? uh , uh , i 'm i 'm really sorry . can you repeat what you were saying about the silence probability ? phd d: mm - hmm . professor b: i only my mind was some phd d: yeah . so there is the tandem network that e e e estimates the phone probabilities professor b: yeah . yeah . phd d: and the silence probabilities also . professor b: right . phd d: and things get better when , instead of using the silence probability computed by the tandem network , we use the silence probability , uh , given by the vad network , professor b: oh . phd d: um , professor b: the vad network is ? phd d: which is smaller , but maybe , um so we have a network for the vad which has one hundred hidden units , and the tandem network has five hundred . um . so it 's smaller but th the silence probability from this network seems , uh , better . professor b: ok . phd d: mmm . uh . well , it looks strange , but professor b: yeah . but phd d: but it professor b: ok . phd d: maybe it 's has something to do to the fact that we do n't have infinite training data and professor b: we do n't ? phd d: well ! and so well , things are not optimal professor b: yeah . phd d: and mmm grad e: are you you were going to say why what made you wh what led you to do that . phd d: yeah . uh , there was a p { comment } problem that we observed , um , that there was there were , like , many insertions in the in the system . professor b: mm - hmm . phd d: mmm . professor b: hmm . phd d: actually plugging in the tandem network was increasing , i i i think , the number of insertions . professor b: mm - hmm . phd d: and , um so it looked strange and then just using the the other silence probability helps . mmm . um yeah . the next thing we will do is train this tandem on more data . professor b: so , you know , in a way what it might i it 's it 's a little bit like combining knowledge sources . phd d: um professor b: right ? because the fact that you have these two nets that are different sizes means they behave a little differently , phd d: mm - hmm . professor b: they find different things . and , um , if you have , um f the distribution that you have from , uh , f speech sounds is w { comment } sort of one source of knowledge . phd d: mm - hmm . professor b: and this is and rather than just taking one minus that to get the other , which is essentially what 's happening , you have this other source of knowledge that you 're putting in there . so you make use of both of them in in what you 're ending up with . maybe it 's better . phd d: yeah . professor b: anyway , you can probably justify anything if what 's use phd d: yeah . professor b: yeah . phd d: and and the features are different also . i mean , the vad does n't use the same features there are . professor b: mm - hmm . grad e: hmm . professor b: oh ! phd d: um professor b: that might be the key , actually . phd d: mm - hmm . professor b: cuz you were really thinking about speech versus nonspeech for that . phd d: mm - hmm . professor b: that 's a good point . phd d: mmm . uh . well , there are other things that we should do but , um , it requires time and we have ideas , like so , these things are like hav having a better vad . uh , we have some ideas about that . it would probably implies working a little bit on features that are more suited to a voice activity detection . professor b: mm - hmm . phd d: working on the second stream . of course we have ideas on this also , but w we need to try different things and uh , but their noise estimation , um uh professor b: i mean , back on the second stream , i mean , that 's something we 've talked about for a while . i mean , i think that 's certainly a high hope . phd d: yeah . mmm . professor b: um , so we have this this default idea about just using some sort of purely spectral thing ? phd d: uh , yeah . professor b: for a second stream ? phd d: but , um , we we did a first try with this , and it it clearly hurts . professor b: but , uh , how was the stream combined ? phd d: uh . it was c it was just combined , um , by the acoustic model . so there was , no neural network for the moment . professor b: right . so , i mean , if you just had a second stream that was just spectral and had another neural net and combined there , that that , uh , might be good . phd d: mm - hmm . yeah . mm - hmm . mm - hmm . mmm . yeah . um yeah , and the other thing , that noise estimation and th um , maybe try to train uh , the training data for the t tandem network , right now , is like i is using the noises from the aurora task and i think that people might , um , try to argue about that because then in some cases we have the same noises in for training the network than the noises that are used for testing , professor b: right . phd d: and so we have t n uh , to try to get rid of these this problem . professor b: yeah . maybe you just put in some other noise , something that 's different . phd d: mm - hmm . yeah . professor b: i mean , it it 's probably helpful to have have a little noise there . but it may be something else phd d: uh - huh . professor b: th at least you could say it was . phd d: yeah . professor b: and then if it does n't hurt too much , though . phd d: uh - huh . professor b: yeah . that 's a good idea . phd d: um . yeah . the last thing is that i think we are getting close to human performance . well , that 's something i would like to investigate further , but , um , i did , like , um i did , uh , listen to the m most noisy utterances of the speechdat - car italian and tried to transcribe them . and , um professor b: so this is a particular human . this is this i this is stephane . phd d: yeah . so that 's that 's grad e: st - stephane . professor b: yeah . phd d: that 's the the flaw of the experiment . this is just i j { comment } it 's just one subject , professor b: yeah . grad e: getting close . phd d: but but still , uh , what happens is is that , uh , the digit error rate on this is around one percent , professor b: yeah . phd d: while our system is currently at seven percent . um , but what happens also is that if i listen to the , um a re - synthesized version of the speech and i re - synthesized this using a white noise that 's filtered by a lpc , uh , filter professor b: yeah . phd d: um , well , you can argue , that , uh that this is not speech , professor b: yeah . phd d: so the ear is not trained to recognize this . but s actually it sound like whispering , so we are professor b: well , i mean , it 's phd d: eh professor b: there 's two problems there . i mean i mean , so so the first is that by doing lpc - twelve with synthesized speech w like you 're saying , uh , it 's i i you 're you 're adding other degradation . phd d: uh - huh . professor b: right ? so it 's not just the noise but you 're adding in fact some degradation because it 's only an approximation . um , and the second thing is which is m maybe more interesting is that , um , { comment } if you do it with whispered speech , you get this number . what if you had done analysis { comment } re - synthesis and taken the pitch as well ? alright ? so now you put the pitch in . phd d: uh - huh . professor b: what would the percentage be then ? phd d: um professor b: see , that 's the question . so , you see , if it 's if it 's if it 's , uh let 's say it 's back down to one percent again . phd d: uh - huh . professor b: that would say at least for people , having the pitch is really , really important , which would be interesting in itself . um , phd d: uh , yeah . but professor b: if i on the other hand , if it stayed up near five percent , then i 'd say `` boy , lpc n twelve is pretty crummy `` . you know ? phd d: uh - huh . professor b: so i i i 'm not sure i 'm not sure how we can conclude from this anything about that our system is close to the human performance . phd d: ye yeah . well , the point is that eh l ey the point is that , um , what i what i listened to when i re - synthesized the lp - the lpc - twelve spectrum is in a way what the system , uh , is hearing , cuz @ @ all the all the , um , excitation all the well , the excitation is is not taken into account . that 's what we do with our system . and professor b: well , you 're not doing the lpc phd d: in this case professor b: i mean , so so what if you did a phd d: well , it 's not lpc , sure , professor b: what if you did lpc - twenty ? phd d: but lpc ? professor b: twenty . right ? i mean , th the thing is lpc is not a a really great representation of speech . phd d: mm - hmm . mm - hmm . professor b: so , all i 'm saying is that you have in addition to the w the , uh , removal of pitch , you also are doing , uh , a particular parameterization , phd d: mm - hmm . professor b: which , um , uh phd d: mmm . professor b: uh , so , let 's see , how would you do ? so , fo phd d: but that 's that 's what we do with our systems . and professor b: no . actually , we d we we do n't , because we do we do , uh , uh , mel filter bank , for instance . right ? phd d: yeah , but is it that is it that different , i mean ? professor b: um , i do n't know what mel , uh , based synthesis would sound like , phd d: i professor b: but certainly the spectra are quite different . phd d: mm - hmm . phd a: could n't you t could n't you , um , test the human performance on just the original audio ? phd d: mm - hmm . this is the one percent number . professor b: yeah , it 's one percent . he 's trying to remove the pitch information phd d: mm - hmm . phd a: oh , oh . ok , phd d: mm - hmm . phd a: i see . professor b: and make it closer to what to what we 're seeing as the feature vectors . phd a: ok . so , y uh , your performance was one percent , and then when you re - synthesize with lpc - twelve it went to five . phd d: uh - huh . yeah . phd a: ok . professor b: i mean we were we were j it it it 's a little bit still apples and oranges because we are choosing these features in order to be the best for recognition . phd d: uh - huh . professor b: and , um , i if you listen to them they still might not be very even if you made something closer to what we 're gon na i it might not sound very good . phd d: yeah . professor b: uh , and i the degradation from that might might actually make it even harder , uh , to understand than the lpc - twelve . so all i 'm saying is that the lpc - twelve puts in synthesis puts in some degradation that 's not what we 're used to hearing , phd d: uh - huh . professor b: and is , um it 's not it 's not just a question of how much information is there , as if you will always take maximum advantage of any information that 's presented to you . phd d: mm - hmm . professor b: in fact , you hear some things better than others . and so it it is n't phd a: but professor b: but , i agree that it says that , uh , the kind of information that we 're feeding it is probably , um , um , a little bit , um , minimal . there 's definitely some things that we 've thrown away . and that 's why i was saying it might be interesting if you an interesting test of this would be if you if you actually put the pitch back in . so , you just extract it from the actual speech and put it back in , and see does that is that does that make the difference ? if that if that takes it down to one percent again , then you 'd say `` ok , it 's it 's in fact having , um , not just the spectral envelope but also the also the the pitch that , uh , { comment } @ @ { comment } has the information that people can use , anyway . `` phd d: uh - huh . mmm . phd a: but from this it 's pretty safe to say that the system is with either two to seven percent away from the performance of a human . right ? so it 's somewhere in that range . professor b: well , or it 's it 's phd a: two two to six percent . professor b: yeah , so it 's it 's one point four times , uh , to , uh , seven times the error , phd d: to f seven times , yeah . professor b: for stephane . phd d: um . professor b: so , uh uh , but i i do n't know . i do do n't wan na take you away from other things . phd d: but { comment } but professor b: but that 's that 's what that 's the first thing that i would be curious about , is , you know , i i when you we phd d: but the signal itself is like a mix of um , of a a periodic sound and , @ @ { comment } uh , unvoiced sound , and the noise professor b: mm - hmm . phd d: which is mostly , uh , noise . i mean not periodic . so , what what do you mean exactly by putting back the pitch in ? because phd a: in the lpc synthesis ? i think professor b: yeah . you did lpc re - synthesis phd d: i professor b: l pc re - synthesis . phd d: uh - huh . professor b: so , uh and you did it with a noise source , rather than with with a s periodic source . phd d: mm - hmm . professor b: right ? so if you actually did real re - synthesis like you do in an lpc synthesizer , where it 's unvoiced you use noise , where it 's voiced you use , uh , periodic pulses . phd d: um . professor b: right ? phd d: yeah , but it 's neither purely voiced or purely unvoiced . esp - especially because there is noise . professor b: well , it might be hard to do it phd d: so professor b: but it but but the thing is that if you um , if you detect that there 's periodic s strong periodic components , then you can use a voiced voice thing . phd d: oh . uh - huh . yeah . professor b: yeah . i mean , it 's probably not worth your time . it 's it 's a side thing and and and there 's a lot to do . phd d: uh - huh , yeah . professor b: but i 'm i 'm just saying , at least as a thought experiment , that 's what i would wan na test . phd d: mm - hmm . professor b: uh , i wan would wan na drive it with a a a two - source system rather than a than a one - source system . phd d: mm - hmm . mm - hmm . professor b: and then that would tell you whether in fact it 's cuz we 've talked about , like , this harmonic tunneling or other things that people have done based on pitch , maybe that 's really a key element . maybe maybe , uh , uh , without that , it 's it 's not possible to do a whole lot better than we 're doing . that that could be . phd d: yeah . that 's what i was thinking by doing this es experiment , professor b: yeah . phd d: like mmm . evi professor b: but , i mean , other than that , i do n't think it 's i mean , other than the pitch de information , it 's hard to imagine that there 's a whole lot more in the signal that that , uh that we 're throwing away that 's important . phd d: yeah , but yeah . mm - hmm . yeah , right . professor b: right ? i mean , we 're using a fair number of filters in the filter bank and uh phd d: mm - hmm . uh , yeah . professor b: hmm . yeah . phd d: um . professor b: yeah . that look phd d: yeah , that 's it . professor b: yeah . that 's that 's i mean , one one percent is sort of what i would i would figure . if somebody was paying really close attention , you might get i would actually think that if , you looked at people on various times of the day and different amounts of attention , you might actually get up to three or four percent error on digits . uh , uh phd d: mm - hmm . um . professor b: so it 's you know , we 're not we 're not incredibly far off . on the other hand , with any of these numbers except maybe the one percent , it 's st it 's not actually usable in a commercial system with a full telephone number or something . phd d: uh - huh . yeah . at these noise levels . professor b: yeah . phd d: yeah . mm - hmm . professor b: right . phd d: well , yeah . these numbers , i mean . mmm . professor b: good . um , while we 're still on aurora stuff maybe you can talk a little about the status with the , uh , wall street journal things for it . phd a: so i 've , um , downloaded , uh , a couple of things from mississippi state . um , one is their software their , uh , lvcsr system . downloaded the latest version of that . got it compiled and everything . um , downloaded the scripts . they wrote some scripts that sort of make it easy to run the system on the wall street journal , uh , data . um , so i have n't run the scripts yet . uh , i 'm waiting there was one problem with part of it and i wrote a note to joe asking him about it . so i 'm waiting to hear from him . but , um , i did print something out just to give you an idea about where the system is . uh , they on their web site they , uh , did this little table of where their system performs relative to other systems that have done this this task . and , um , the mississippi state system using a bigram grammar , uh , is at about eight point two percent . other comparable systems from , uh were getting from , uh , like six point nine , six point eight percent . so they 're professor b: this is on clean test set ? phd a: this is on clean on clean stuff . yeah . they they 've started a table where they 're showing their results on various different noise conditions but they they do n't have a whole lot of it filled in and and i did n't notice until after i 'd printed it out that , um , they do n't say here what these different testing conditions are . professor b:  phd a: you actually have to click on it on the web site to see them . so i i do n't know what those numbers really mean . professor b: what kind of numbers are they getting on these on the test conditions ? phd a: well , see , i was a little confused because on this table , i 'm the they 're showing word error rate . but on this one , i i do n't know if these are word error rates because they 're really big . so , under condition one here it 's ten percent . then under three it goes to sixty - four point six percent . professor b: yeah , that 's probably aurora . phd a: yeah . professor b: i mean phd a: so m i guess maybe they 're error rates but they 're , uh they 're really high . professor b: i i i do n't find that surpri phd a: so professor b: i mean , we w what 's what 's some of the lower error rates on on on uh , some of the higher error rates on , uh , some of these w uh , uh , highly mismatched difficult conditions ? what 's a ? phd d: uh . yeah , it 's around fifteen to twenty percent . phd a: correct ? phd d: and the baseline , eh phd a: accuracy ? phd d: uh , error rate . professor b: yeah . phd d: twenty percent error rate , professor b: yeah . so twenty percent error rate on digits . phd d: and phd a: oh , oh , on digits . professor b: so if you 're doing so if you 're doing , phd d: and phd a: yeah . phd d: on digits . phd a: ok . professor b: you know , phd d: and this is so so still the baseline . professor b: sixty - thousand phd d: right ? phd a: yeah . professor b: yeah , and if you 're saying sixty - thousand word recognition , getting sixty percent error on some of these noise condition not at all surprising . phd a: yeah . phd d: the baseline is sixty percent also on digits , phd a: oh , is it ? phd d: on the m more mismatched conditions . phd a: ok . professor b: yeah . phd d: so . phd a: so , yeah , that 's probably what it is then . yeah . so they have a lot of different conditions that they 're gon na be filling out . professor b: it 's a bad sign when you looking at the numbers , you ca n't tell whether it 's accuracy or error rate . phd a: yeah . yeah . it 's it 's gon na be hard . um , they 're i i 'm still waiting for them to release the , um , multi - cpu version of their scripts , cuz right now their script only handles processing on a single cpu , which will take a really long time to run . so . but their s professor b: this is for the training ? phd a: uh i beli yes , for the training also . and , um , they 're supposed to be coming out with it any time , professor b: ok . phd a: the multi - cpu one . so , as soon as they get that , then i 'll i 'll grab those too professor b: ok . phd a: and so w professor b: yeah . cuz we have to get started , phd a: yeah . professor b: cuz it 's cuz , uh , phd a: yeah . i 'll go ahead and try to run it though with just the single cpu one , professor b: if the phd a: and i they they , um , released like a smaller data set that you can use that only takes like sixteen hours to train and stuff . so i can i can run it on that just to make sure that the the thing works and everything . professor b: oh ! good . yeah . cuz we 'll grad e: hmm . professor b: i guess the actual evaluation will be in six weeks or something . so . is that about right you think ? phd d: uh , we do n't know yet , i i think . professor b: really , we do n't know ? phd d: uh - huh . um . phd a: it was n't on the conference call this morning ? professor b: hmm . phd d: no . phd a: hmm . did they say anything on the conference call about , um , how the wall street journal part of the test was going to be run ? because i i thought i remembered hearing that some sites were saying that they did n't have the compute to be able to run the wall street journal stuff at their place , phd d: no . mmm . phd a: so there was some talk about having mississippi state run the systems for them . and i did did that come up at all ? phd d: uh , no . well , this first , this was not the point at all of this the meeting today phd a: oh , ok . phd d: and , professor b: some phd d: uh , frankly , i do n't know because i d { comment } did n't read also the most recent mails about the large - vocabulary task . but , uh , did you do you still , uh , get the mails ? you 're not on the mailing list or what ? phd a: hmm - mm . the only , um , mail i get is from mississippi state phd d: uh - huh . phd a: so phd d: oh , yeah . so we should have a look at this . phd a: about their system . i i do n't get any mail about professor b: i have to say , there 's uh something funny - sounding about saying that one of these big companies does n't have enough cup compute power do that , so they 're having to have it done by mississippi state . phd a: yeah . professor b: it just just sounds funny . phd a: yeah . it does . professor b: but , phd a: yeah . i 'm i 'm wondering about that professor b: anyway . phd a: because there 's this whole issue about , you know , simple tuning parameters , like word insertion penalties . phd d: mm - hmm . phd a: and whether or not those are going to be tuned or not , and { comment } so . phd d: mm - hmm . phd a: i mean , it makes a big difference . if you change your front - end , you know , the scale is completely can be completely different , so . it seems reasonable that that at least should be tweaked to match the front - end . but phd d: you did n't get any answer from joe ? phd a: i did , but joe said , you know , `` what you 're saying makes sense phd d: uh - huh . phd a: and i do n't know `` . so he does n't know what the answer is . phd d: uh - huh . phd a: i mean , that 's th we had this back and forth a little bit about , you know , are sites gon na are you gon na run this data for different sites ? and , well , if if mississippi state runs it , then maybe they 'll do a little optimization on that parameter , and , uh but then he was n't asked to run it for anybody . so i it 's it 's just not clear yet what 's gon na happen . phd d: mm - hmm . phd a: uh , he 's been putting this stuff out on their web site and for people to grab but i have n't heard too much about what 's happening . professor b: so it could be i mean , chuck and i had actually talked about this a couple times , and and over some lunches , i think , that , um , one thing that we might wan na do the - there 's this question about , you know , what do you wan na scale ? suppose y you ca n't adjust these word insertion penalties and so forth , so you have to do everything at the level of the features . what could you do ? and , uh , one thing i had suggested at an earlier time was maybe some sort of scaling , some sort of root or or something of the , um , uh , features . but the problem with that is that is n't quite the same , it occurred to me later , because what you really want to do is scale the , uh , @ @ { comment } the range of the likelihoods rather than phd d: nnn , the dist yeah . professor b: but , what might get at something similar , it just occurred to me , is kind of an intermediate thing is because we do this strange thing that we do with the tandem system , at least in that system what you could do is take the , um , uh , values that come out of the net , which are something like log probabilities , and scale those . and then , uh , um then at least those things would have the right values or the right the right range . and then that goes into the rest of it and then that 's used as observations . so it 's it 's , um , another way to do it . phd d: mm - hmm . mm - hmm . but , these values are not directly used as probabilities anyway . professor b: i know they 're not . phd d: so there are there is professor b: i know they 're not . but but , you know so because what we 're doing is pretty strange and complicated , we do n't really know what the effect is at the other end . phd d: uh - huh . mm - hmm . professor b: so , um , my thought was maybe i mean , they 're not used as probabilities , but the log probabilities we 're taking advantage of the fact that something like log probabilities has more of a gaussian shape than gaus - than probabilities , and so we can model them better . so , in a way we 're taking advantage of the fact that they 're probabilities , because they 're this quantity that looks kind of gaussian when you take it 's log . so , { comment } uh , maybe maybe it would have a a reasonable effect to do that . phd d: mm - hmm . professor b: i d i do n't know . but , i mean , i guess we still have n't had a a ruling back on this . and we may end up being in a situation where we just you know really ca n't change the word insertion penalty . but the other thing we could do is also we could i mean , this this may not help us , uh , in the evaluation but it might help us in our understanding at least . we might , just run it with different insper insertion penalties , and show that , uh , `` well , ok , not changing it , playing the rules the way you wanted , we did this . but in fact if we did that , it made a a big difference . `` phd a: i wonder if it it might be possible to , uh , simulate the back - end with some other system . so we we get our f front - end features , and then , uh , as part of the process of figuring out the scaling of these features , { comment } you know , if we 're gon na take it to a root or to a power or something , { comment } we have some back - end that we attach onto our features that sort of simulates what would be happening . professor b: mm - hmm . phd a: um , professor b: and just adjust it until it 's the best number ? phd a: and just adjust it until that our l version of the back - end , uh , decides that that professor b: well , we can probably use the real thing , ca n't we ? and then jus just , uh , use it on a reduced test set or something . phd a: yeah . oh , yeah . that 's true . professor b: yeah . phd a: and then we just use that to determine some scaling factor that we use . professor b: yeah . so i mean , i i think that that 's a reasonable thing to do and the only question is what 's the actual knob that we use ? phd a: mm - hmm . professor b: and the knob that we use should uh , uh , unfortunately , like i say , i do n't know the analytic solution to this cuz what we really want to do is change the scale of the likelihoods , phd a: mm - hmm . professor b: not the cha not the scale of the the observations . but but , uh phd d: mm - hmm . phd a: yeah . grad e: out of curiosity , what what kind of recognizer is the one from mississippi state ? phd a: uh , w what do you mean when you say `` what kind `` ? grad e: is it ? um , is it like a gaussian mixture model ? phd a: yeah . gaussian mixture model . grad e: ok . phd a: it 's the same system that they use when they participate in the hub - five evals . it 's a , um sort of came out of , uh uh , looking a lot like htk . i mean , they started off with um , when they were building their system they were always comparing to htk to make sure they were getting similar results . and so , it 's a gaussian mixture system , uh professor b: do they have the same sort of mix - down sort of procedure , where they start off with a small number of some things phd a: i do n't know . yeah . and then divide the mixtures in half . professor b: and ? yeah . phd a: i do n't know if they do that . i 'm not really sure . professor b: yeah . grad e: hmm . professor b: d do you know what kind of tying they use ? are they they sort of some sort of a bunch of gaussians that they share across everything ? or or if it 's ? phd a: yeah , th i have i i i do n't have it up here but i have a the whole system description , that describes exactly what their system is professor b: ok . phd a: and i i 'm not sure . but , um professor b: ok . phd a: it 's some kind of a mixture of gaussians and , uh , clustering and , uh they 're they 're trying to put in sort of all of the standard features that people use nowadays . grad e: mm - hmm . professor b: so the other , uh , aurora thing maybe is i i dunno if any of this is gon na come in in time to be relevant , but , uh , we had talked about , uh , { comment } guenter playing around , uh , uh , over in germany phd d: mm - hmm . professor b: and and , @ @ { comment } uh , possibly coming up with something that would , uh , uh , fit in later . uh , i saw that other mail where he said that he uh , it was n't going to work for him to do cvs . phd d: yeah . yeah . so now he has a version of the software . professor b: so he just has it all sitting there . yeah . phd d: yeah . um mm - hmm . professor b: so if he 'll he might work on improving the noise estimate or on some histogram things , or phd d: yeah . mm - hmm . professor b: yeah . i just saw the eurospeech we we did n't talk about it at our meeting but i just saw the just read the paper . someone , i forget the name , { comment } and and ney , uh , about histogram equalization ? did you see that one ? phd d: um , it was a poster . or professor b: yeah . i mean , i just read the paper . phd d: yeah . professor b: i did n't see the poster . phd d: yeah . um it was something similar to n on - line normalization finally i mean , in the idea of of normalizing professor b: yeah . but it 's a little more it it 's a little finer , right ? so they had like ten quantiles phd d: yeah . professor b: and and they adjust the distribution . phd d: right . professor b: so you you have the distributions from the training set , phd d: n professor b: and then , uh so this is just a a histogram of of the amplitudes , i guess . right ? and then um , people do this in image processing some . phd d: mm - hmm . professor b: you have this kind of of histogram of of levels of brightness or whatever . and and and then , when you get a new new thing that you you want to adjust to be better in some way , you adjust it so that the histogram of the new data looks like the old data . phd a: hmm . professor b: you do this kind of piece - wise linear or , uh , some kind of piece - wise approximation . they did a uh one version that was piece - wise linear and another that had a power law thing between them between the points . and , uh , they said they s they sort of see it in a way as s for the speech case { comment } as being kind of a generalization of spectral subtraction in a way , because , you know , in spectral subtraction you 're trying to get rid of this excess energy . uh , you know , it 's not supposed to be there . uh and , uh , this is sort of adjusting it for for a lot of different levels . and then they have s they have some kind of , uh , a floor or something , grad e: hmm . professor b: so if it gets too low you do n't do n't do it . phd a: hmm . professor b: and they they claimed very nice results , phd d: mm - hmm . phd a: so is this a histogram across different frequency bins ? professor b: and phd a: or ? professor b: um , i think this i you know , i do n't remember that . do you remember ? phd d: i think they have , yeah , different histograms . i uh something like one per frequency band , professor b: one phd a: so , one histogram per frequency bin . professor b: one per critical phd d: or but i did yeah , i guess . phd a: and that 's phd d: but i should read the paper . i just went through the poster quickly , professor b: yeah . phd a: so th professor b: and i do n't remember whether it was filter bank things phd a: oh . phd d: and i did n't professor b: or whether it was fft bins phd a: huh . professor b: or phd a: and and that that , um , histogram represents the different energy levels that have been seen at that frequency ? professor b: i do n't remember that . and how often they you 've seen them . yeah . phd a: uh - huh . grad e: hmm . professor b: yeah . and they do they said that they could do it for the test so you do n't have to change the training . you just do a measurement over the training . and then , uh , for testing , uh , you can do it for one per utterance . even relatively short utterances . and they claim it it works pretty well . phd a: so they , uh is the idea that you you run a test utterance through some histogram generation thing and then you compare the histograms and that tells you what to do to the utterance to make it more like ? professor b: i guess in pri yeah . in principle . phd a: i see . professor b: i did n't read carefully how they actually implemented it , phd a: hmm . yeah . professor b: whether it was some , uh , on - line thing , or whether it was a second pass , or what . but but they that that was sort of the idea . phd a: hmm . professor b: so that that seemed , you know , different . we 're sort of curious about , uh , what are some things that are , u u um , @ @ { comment } conceptually quite different from what we 've done . phd a: mm - hmm . professor b: cuz we you know , one thing that w that , uh , stephane and sunil seemed to find , uh , was , you know , they could actually make a unified piece of software that handled a range of different things that people were talking about , and it was really just sort of setting of different constants . and it would turn , you know , one thing into another . it 'd turn wiener filtering into spectral subtraction , or whatever . but there 's other things that we 're not doing . so , we 're not making any use of pitch , uh , uh , which again , might might be important , uh , because the stuff between the harmonics is probably a schmutz . and and the , uh , transcribers will have fun with that . uh and , um , the , uh , stuff at the harmonics is n't so much . and and , uh and we there 's this overall idea of really sort of matching the the hi distributions somehow . uh , not just , um , um not just subtracting off your estimate of the noise . so . so i guess , uh , guenter 's gon na play around with some of these things now over this next period , phd d: uh , i dunno . professor b: or ? phd d: i do n't have feedback from him , but professor b: yeah . phd d: i guess he 's gon na , maybe professor b: well , he 's got it anyway , so he can . phd d: yeah . professor b: so potentially if he came up with something that was useful , like a diff a better noise estimation module or something , he could ship it to you guys u up there phd d: uh - huh . professor b: and phd d: yeah . professor b: we could put it in . phd d: mm - hmm . mm - hmm . professor b: yeah . yeah . so , that 's good . so , why do n't we just , uh , um i think starting starting a w couple weeks from now , especially if you 're not gon na be around for a while , we 'll we 'll be shifting more over to some other other territory . but , uh , uh , { comment } uh , n not not so much in this meeting about aurora , but but , uh , uh , maybe just , uh , quickly today about maybe you could just say a little bit about what you 've been talking about with michael . and and then barry can say something about what { comment } what we 're talking about . grad c: ok . so michael kleinschmidt , who 's a phd student from germany , showed up this week . he 'll be here for about six months . and he 's done some work using an auditory model of , um , human hearing , and using that f uh , to generate speech recognition features . and he did work back in germany with , um , a toy recognition system using , um , isolated digit recognition as the task . it was actually just a single - layer neural network that classified words classified digits , in fact . um , and he tried that on i think on some aurora data and got results that he thought seemed respectable . and he w he 's coming here to u u use it on a uh , a real speech recognition system . so i 'll be working with him on that . and , um , maybe i should say a little more about these features , although i do n't understand them that well . the i think it 's a two - stage idea . and , um , the first stage of these features correspond to what 's called the peripheral auditory system . and i guess that is like a filter bank with a compressive nonlinearity . and i 'm - i 'm not sure what we have @ @ in there that is n't already modeled in something like , um , plp . i should learn more about that . and then the second stage is , um , the most different thing , i think , from what we usually do . it 's , um it computes features which are , um , based on sort of like based on diffe different w um , wavelet basis functions used to analyze the input . so th he uses analysis functions called gabor functions , um , which have a certain extent , um , in time and in frequency . and the idea is these are used to sample , um , the signal in a represented as a time - frequency representation . so you 're sampling some piece of this time - frequency plane . and , um , that , um , is is interesting , cuz , @ @ for for one thing , you could use it , um , in a a multi - scale way . you could have these instead of having everything like we use a twenty - five millisecond or so analysis window , typically , um , and that 's our time scale for features , but you could using this , um , basis function idea , you could have some basis functions which have a lot longer time scale and , um , some which have a lot shorter , and so it would be like a set of multi - scale features . so he 's interested in , um th - this is because it 's , um there are these different parameters for the shape of these basis functions , um there are a lot of different possible basis functions . and so he he actually does an optimization procedure to choose an an optimal set of basis functions out of all the possible ones . phd a: hmm . h what does he do to choose those ? grad c: the method he uses is kind of funny is , { comment } um , he starts with he has a set of m of them . um , he and then he uses that to classify i mean , he t he tries , um , using just m minus one of them . so there are m possible subsets of this length - m vector . he tries classifying , using each of the m possible sub - vectors . phd d: hmm . grad c: whichever sub - vector , um , works the the best , i guess , he says the the fe feature that did n't use was the most useless feature , professor b: y yeah . gets thrown out . yeah . grad c: so we 'll throw it out and we 're gon na randomly select another feature from the set of possible basis functions . phd a: hmm ! professor b: yeah . phd a: so it 's a professor b: so i so it 's actuall phd a: it 's a little bit like a genetic algorithm or something in a way . professor b: well , it 's it 's much simpler . grad e: it 's like a greedy professor b: but it 's but it 's uh , it 's there 's a lot number of things i like about it , let me just say . phd a: greedy . professor b: so , first thing , well , you 're absolutely right . i mean , i i in truth , both pieces of this are have their analogies in stuff we already do . but it 's a different take at how to approach it and potentially one that 's m maybe a bit more systematic than what we 've done , uh , and a b a bit more inspiration from from auditory things . so it 's so i think it 's a neat thing to try . the primary features , um , are in fact yeah , essentially , it 's it 's , uh , you know , plp or or mel cepstrum , or something like that . you 've you 've got some , uh , compression . we always have some compression . we always have some you know , the the the kind of filter bank with a kind of quasi - log scaling . um , if you put in if you also include the rasta in it i rasta the filtering being done in the log domain has an agc - like , uh , characteristic , which , you know , people typi typically put in these kind of , uh , um , uh , auditory front - ends . so it 's very , very similar , uh , but it 's not exactly the same . um , i would agree that the second one is is somewhat more different but , um , it 's mainly different in that the things that we have been doing like that have been um , had a different kind of motivation and have ended up with different kinds of constraints . so , for instance , if you look at the lda rasta stuff , you know , basically what they do is they they look at the different eigenvectors out of the lda and they form filters out of it . right ? and those filters have different , uh , kinds of temporal extents and temporal characteristics . and so in fact they 're multi - scale . but , they 're not sort of systematically multi - scale , like `` let 's start here and go to there , and go to there , and go to there `` , and so forth . it 's more like , you run it on this , you do discriminant analysis , and you find out what 's helpful . grad c: i it 's multi - scale because you use several of these in parallel , professor b: yeah . they use several of them . grad c: is that right ? of professor b: yeah . grad c: ok . professor b: uh , i mean , you do n't have to but but but , uh , hynek has . um , but it 's also , uh hyn - when hynek 's had people do this kind of lda analysis , they 've done it on frequency direction and they 've done it on the time direction . i think he may have had people sometimes doing it on both simultaneously some two - d and that would be the closest to these gabor function kind of things . uh , but i do n't think they 've done that much of that . and , uh , the other thing that 's interesting the the , uh the feature selection thing , it 's a simple method , but i kinda like it . um , there 's a a old , old method for feature selection . i mean , eh , uh , i remember people referring to it as old when i was playing with it twenty years ago , so i know it 's pretty old , uh , called stepwise linear discriminant analysis in which you which i think it 's used in social sciences a lot . so , you you you you pick the best feature . and then you take y you find the next feature that 's the best in combination with it . and then so on and so on . and what what michael 's describing seems to me much , much better , because the problem with the stepwise discriminant analysis is that you do n't know that you know , if you 've picked the right set of features . just because something 's a good feature does n't mean that you should be adding it . so , um , uh , here at least you 're starting off with all of them , and you 're throwing out useless features . i think that 's that seems , uh that seems like a lot better idea . uh , you 're always looking at things in combination with other features . um , so the only thing is , of course , there 's this this artificial question of of , uh , exactly how you how you a how you assess it and if if your order had been different in throwing them out . i mean , it still is n't necessarily really optimal , but it seems like a pretty good heuristic . so i th i think it 's it 's i think it 's kinda neat stuff . grad e: hmm . professor b: and and and , uh , the thing that i wanted to to add to it also was to have us use this in a multi - stream way . grad e: hmm . professor b: um , so so that , um , when you come up with these different things , and these different functions , you do n't necessarily just put them all into one huge vector , but perhaps you have some of them in one stream and some of them in another stream , and so forth . and , um , um , { comment } um and we 've also talked a little bit about , uh , uh , shihab shamma 's stuff , in which you the way you look at it is that there 's these different mappings and some of them emphasize , uh , upward moving , uh , energy and fre and frequency . and some are emphasizing downward and fast things and slow things and and so forth . so . so there 's a bunch of stuff to look at . but , uh , i think we 're sorta gon na start off with what he , uh , came here with and branch out branch out from there . and his advisor is here , too , at the same time . so , he 'll be another interesting source of wisdom . grad e: hmm . professor b: so . grad e: as as we were talking about this i was thinking , um , whether there 's a relationship between um , between michael 's approach to , uh , some some sort of optimal brain damage or optimal brain surgeon on the neural nets . professor b: yeah . grad c: hmm . grad e: so , like , if we have , um we have our we have our rasta features and and presumably the neural nets are are learning some sort of a nonlinear mapping , uh , from the the the features to to this this probability posterior space . professor b: mm - hmm . grad e: right ? and , um and each of the hidden units is learning some sort of some sort of some sort of pattern . right ? and it could be , like like these , um these auditory patterns that michael is looking at . and then when you 're looking at the the , uh , um , the best features , you know , you can take out you can do the do this , uh , brain surgery by taking out , um , hidden units that do n't really help at all . professor b: mm - hmm . or the or features . grad e: and this is k sorta like professor b: right ? grad e: yeah . professor b: i mean , y actually , you make me think a a very important point here is that , um , if we a again try to look at how is this different from what we 're already doing , uh , there 's a a , uh a nasty argument that could be made th that it 's it 's not different at at all , because , uh if you ignore the the selection part because we are going into a a very powerful , uh , nonlinearity that , uh , in fact is combining over time and frequency , and is coming up with its own you know , better than gabor functions its , you know , neural net functions , grad e: mm - hmm . professor b: its { comment } whatever it finds to be best . grad c:  professor b: um , so you could argue that in fact it but i i do n't actually believe that argument because i know that , um , you can , uh computing features is useful , even though in principle you have n't added anything in fact , you subtracted something , from the original waveform you know , uh , if you 've you 've processed it in some way you 've typically lost something some information . and so , you 've lost information and yet it does better with with features than it does with the waveform . so , uh , i i know that i sometimes it 's useful to to constrain things . so that 's why it really seems like the constraint in in all this stuff it 's the constraints that are actually what matters . because if it was n't the constraints that mattered , then we would 've completely solved this problem long ago , because long ago we already knew how to put waveforms into powerful statistical mechanisms . so . phd d: yeah . well , if we had infinite processing power and data , { comment } i guess , using the waveform could grad e: right . professor b: yeah uh , then it would work . yeah , i agree . yeah . there 's the problem . phd d: so , that 's professor b: yeah . then it would work . but but , i mean , i it 's with finite of those things i mean , uh , we we have done experiments where we literally have put waveforms in and and and , uh , phd d: mm - hmm . professor b: we kept the number of parameters the same and so forth , and it used a lot of training data . and it and it it , uh not infinite but a lot , and then compared to the number parameters and it it , uh it just does n't do nearly as well . so , anyway the point is that you want to suppress phd d: mm - hmm . professor b: it 's not just having the maximum information , you want to suppress , uh , the aspects of the input signal that are not helpful for for the discrimination you 're trying to make . so . so maybe just briefly , uh grad e: well , that sort of segues into what what i 'm doing . professor b: yeah . grad e: um , so , uh , the big picture is k um , come up with a set of , uh , intermediate categories , then build intermediate category classifiers , then do recognition , and , um , improve speech recognition in that way . um , so right now i 'm in in the phase where i 'm looking at at , um , deciding on a initial set of intermediate categories . and i 'm looking for data data - driven methods that can help me find , um , a set of intermediate categories of speech that , uh , will help me to discriminate later down the line . and one of the ideas , um , that was to take a take a neural net train train an ordinary neural net to uh , to learn the posterior probabilities of phones . and so , um , at the end of the day you have this neural net and it has hidden hidden units . and each of these hidden units is um , is learning some sort of pattern . and so , um , what what are these patterns ? phd a: hmm . grad e: i do n't know . um , and i 'm gon na to try to to look at those patterns to to see , um , from those patterns uh , presumably those are important patterns for discriminating between phone classes . and maybe maybe some , uh , intermediate categories can come from just looking at the patterns of um , that the neural net learns . professor b: be - before you get on the next part l let me just point out that s there 's there 's a a pretty nice { comment } relationship between what you 're talking about doing and what you 're talking about doing there . right ? grad e: yeah . professor b: so , it seems to me that , you know , if you take away the the the difference of this primary features , and , say , you use as we had talked about maybe doing you use p - rasta - plp or something for the the primary features , um , then this feature discovery , uh , uh , thing is just what he 's talking about doing , too , except that he 's talking about doing them in order to discover intermediate categories that correspond to these uh , uh , what these sub - features are are are are showing you . and , um , the other difference is that , um , he 's doing this in a in a multi - band setting , which means that he 's constraining himself to look across time in some f relatively limited , uh , uh , spectral extent . right ? and whereas in in this case you 're saying `` let 's just do it unconstrained `` . so they 're they 're really pretty related and maybe they 'll be at some point where we 'll see the the connections a little better and connect them . grad c: hmm . grad e: mm - hmm . um . yeah , so so that 's the that 's the first part uh , one one of the ideas to get at some some patterns of intermediate categories . um , the other one was , um , to , uh , come up with a a a model { comment } um , a graphical model , that treats the intermediate categories as hidden hidden variables , latent variables , that we do n't know anything about , but that through , um , s statistical training and the em algorithm , um , at the end of the day , we have , um we have learned something about these these latent , um latent variables which happen to correspond to intermediate categories . um . yeah , and so those are the the two directions that i 'm i 'm looking into right now . and , uh , um yeah . i guess that 's that 's it . professor b: ok . should we do our digits and get ou get our treats ? grad e: oh , tea time ? professor b: yeah . it 's kind of like , you know , the little rats with the little thing dropping down to them . phd a: that 's ri professor b: we do the digits and then we get our treats . grad e: oops . phd a: ok . </s> [SEP]summarize the discussion on lpc
the team was filtering out white noise from the digit task using lpc . the concern was that lpc was not a great representation of speech . the lpc was adding degradation .