PhD B: OK . We 're on .
Grad E: Hello ?
Professor A: OK , so uh {vocalsound} had some interesting mail from uh Dan Ellis . Actually , I think he {disfmarker} he {vocalsound} redirected it to everybody also so uh {vocalsound} the PDA mikes uh have a big bunch of energy at {disfmarker} at uh five hertz uh where this came up was that uh I was showing off these wave forms that we have on the web and {disfmarker} and uh {vocalsound} I just sort of hadn't noticed this , but that {disfmarker} the major , major component in the wave {disfmarker} in the second wave form in that pair of wave forms is actually the air conditioner .
Grad C: Huh .
Professor A: So . So . I {vocalsound} {vocalsound} I have to be more careful about using that as a {disfmarker} as a {disfmarker} {vocalsound} as a good illustration , uh , in fact it 's not , of uh {disfmarker} {vocalsound} of the effects of room reverberation . It is isn't a bad illustration of the effects of uh room noise . {vocalsound} on {disfmarker} on uh some mikes uh but So . And then we had this other discussion about um {vocalsound} whether this affects the dynamic range , cuz I know , although we start off with thirty two bits , you end up with uh sixteen bits and {vocalsound} you know , are we getting hurt there ? But uh Dan is pretty confident that we 're not , that {disfmarker} that quantization error is not {disfmarker} is still not a significant {vocalsound} factor there . So . So there was a question of whether we should change things here , whether we should {vocalsound} change a capacitor on the input box for that or whether we should
PhD B: Yeah , he suggested a smaller capacitor , right ?
Professor A: Right . But then I had some other uh thing discussions with him
PhD B: For the P D
Professor A: and the feeling was {vocalsound} once we start monk monkeying with that , uh , many other problems could ha happen . And additionally we {disfmarker} we already have a lot of data that 's been collected with that , so .
PhD B: Yeah .
Professor A: A simple thing to do is he {disfmarker} he {disfmarker} he has a {disfmarker} I forget if it {disfmarker} this was in that mail or in the following mail , but he has a {disfmarker} a simple filter , a digital filter that he suggested . We just run over the data before we deal with it .
PhD B: Mm - hmm .
Professor A: um The other thing that I don't know the answer to , but when people are using Feacalc here , uh whether they 're using it with the high - pass filter option or not . And I don't know if anybody knows .
Grad E: Um . {vocalsound} I could go check .
Professor A: But . Yeah . So when we 're doing all these things using our software there is {disfmarker} um if it 's {disfmarker} if it 's based on the RASTA - PLP program , {vocalsound} which does both PLP and RASTA - PLP {vocalsound} um then {vocalsound} uh there is an option there which then comes up through to Feacalc which {vocalsound} um allows you to do high - pass filtering and in general we like to do that , because of things like this and {vocalsound} it 's {disfmarker} it 's pretty {disfmarker} it 's not a very severe filter . Doesn't affect speech frequencies , even pretty low speech frequencies , at all , but it 's
PhD B: What 's the {pause} cut - off frequency it used ?
Professor A: Oh . I don't know I wrote this a while ago
PhD B: Is it like twenty ?
Professor A: Something like that .
PhD B: Yeah .
Professor A: Yeah . I mean I think there 's some effect above twenty but it 's {disfmarker} it 's {disfmarker} it 's {disfmarker} it 's mild . So , I mean it probably {disfmarker} there 's probably some effect up to a hundred hertz or something but it 's {disfmarker} it 's pretty mild . I don't know in the {disfmarker} in the STRUT implementation of the stuff is there a high - pass filter or a pre pre - emphasis or something in the {disfmarker}
PhD F: Uh . I think we use a pre - emphasis . Yeah . Yeah .
Professor A: So . We {disfmarker} we {disfmarker} we want to go and check that in i for anything that we 're going to use the P D A mike for . {vocalsound} uh He says that there 's a pretty good roll off in the PZM mikes so {vocalsound} we don't need {disfmarker} need to worry about them one way or the other but if we do make use of the cheap mikes , {vocalsound} uh we want to be sure to do that {disfmarker} that filtering before we {vocalsound} process it . And then again if it 's uh depending on the option that the {disfmarker} our {disfmarker} our software is being run with , it 's {disfmarker} it 's quite possible that 's already being taken care of . uh But I also have to pick a different picture to show the effects of reverberation . uh
PhD B: Did somebody notice it during your talk ?
Professor A: uh No .
PhD B: Huh .
Professor A: Well . uh Well . If they made output they were {disfmarker} they were , you know {disfmarker} they were nice .
PhD B: Didn't say anything ?
Professor A: But . {vocalsound} I mean the thing is it was since I was talking about reverberation and showing this thing that was noise , it wasn't a good match , but it certainly was still uh an indication of the fact that you get noise with distant mikes . uh It 's just not a great example because not only isn't it reverberation but it 's a noise that we definitely know what to do .
PhD B: Mm - hmm .
Professor A: So , I mean , it doesn't take deep {disfmarker} {vocalsound} a new {disfmarker} bold new methods to get rid of uh five hertz noise , so .
PhD B: Yeah .
Professor A: um {vocalsound} uh But . So it was {disfmarker} it was a bad example in that way , but it 's {disfmarker} it still is {disfmarker} it 's the real thing that we did get out of the microphone at distance , so it wasn't {vocalsound} it w it w wasn't wrong it was inappropriate . So . {vocalsound} So uh , but uh , Yeah , someone noticed it later pointed it out to me , and I went " oh , man . Why didn't I notice that ? "
PhD B: Hmm .
Professor A: um . So . {vocalsound} um So I think we 'll change our {disfmarker} our picture on the web , when we 're @ @ . One of the things I was {disfmarker} I mean , I was trying to think about what {disfmarker} what 's the best {vocalsound} way to show the difference an and I had a couple of thoughts one was , {vocalsound} that spectrogram that we show {vocalsound} is O K , but the thing is {vocalsound} the eyes uh and the {vocalsound} the brain behind them are so good at picking out patterns {vocalsound} from {disfmarker} from noise {vocalsound} that in first glance you look at them it doesn't seem like it 's that bad uh because there 's many features that are still preserved . So one thing to do might be to just take a piece of the spec uh of the spectrogram where you can see {vocalsound} that something looks different , an and blow it up , and have that be the part that 's {disfmarker} just to show as well . You know .
PhD B: Mm - hmm . Mm - hmm .
Professor A: i i Some things are going to be hurt . um {vocalsound} Another , I was thinking of was um {vocalsound} taking some spectral slices , like uh {disfmarker} like we look at with the recognizer , and look at the spectrum or cepstrum that you get out of there , and the {disfmarker} the uh , um , {vocalsound} the reverberation uh does make it {disfmarker} does change that . And so maybe {disfmarker} maybe that would be more obvious .
PhD B: Hmm .
Grad C: Spectral slices ?
Professor A: Yeah .
Grad C: W w what d what do you mean ?
Professor A: Well , I mean um all the recognizers look at frames . So they {disfmarker} they look at {disfmarker}
PhD B: So like one instant in time .
Professor A: Yeah , look at a {disfmarker}
Grad C: OK .
Professor A: So it 's , yeah , at one point in time or uh twenty {disfmarker} over twenty milliseconds or something , {vocalsound} you have a spectrum or a cepstrum .
Grad C: OK .
Professor A: That 's what I meant by a slice .
Grad C: I see .
Professor A: Yeah . And {vocalsound} if you look at {disfmarker}
PhD B: You could just {disfmarker} you could just throw up , you know , uh {vocalsound} the uh {disfmarker} some MFCC feature vectors . You know , one from one , one from the other , and then , you know , you can look and see how different the numbers are .
Professor A: Right . Well , that 's why I saying either {vocalsound} {vocalsound} Well , either spectrum or cepstrum
PhD B: I 'm just kidding .
Professor A: but {disfmarker} {vocalsound} but I think the thing is you wanna {disfmarker}
PhD B: I don't mean a graph . I mean the actual numbers .
Professor A: Oh . I see . Oh . That would be lovely , yeah .
PhD B: Yeah . " See how different these {vocalsound} sequences of numbers are ? "
Professor A: Yeah . Or I could just add them up and get a different total .
PhD B: Yeah . It 's not the square .
Professor A: OK . Uh . What else {disfmarker} wh what 's {disfmarker} what else is going on ?
PhD F: Uh , yeah . Yeah , at first I had a remark why {disfmarker} I am wondering why the PDA is always so far . I mean we are always meeting at the {vocalsound} beginning of the table and {vocalsound} the PDA 's there .
Professor A: Uh . I guess cuz we haven't wanted to move it . We {disfmarker} we could {disfmarker} {vocalsound} we could move us ,
PhD F: Yeah ?
Professor A: and .
PhD F: OK .
Grad E: That 's right .
PhD F: Well , anyway . Um . Yeah , so . Uh . Since the last meeting we 've {disfmarker} we 've tried to put together um {vocalsound} the clean low - pass um downsampling , upsampling , I mean , Uh the new filter that 's replacing the LDA filters , and also {vocalsound} the um delay issue so that {disfmarker} We considered th the {disfmarker} the delay issue on the {disfmarker} for the on - line normalization . Mmm . So we 've put together all this and then we have results that are not um {vocalsound} {vocalsound} very impressive . Well , there is no {vocalsound} real improvement .
Professor A: But it 's not wer worse and it 's better {disfmarker} better latency ,
PhD F: It 's not {disfmarker}
Professor A: right ?
PhD F: Yeah . Yeah . Well . Actually it 's better . It seems better when we look at the mismatched case but {vocalsound} I think we are like {disfmarker} like cheated here by the {disfmarker} th this problem that {vocalsound} uh in some cases when you modify slight {disfmarker} slightly modify the initial condition you end up {vocalsound} completely somewhere air somewhere else in the {disfmarker} in the space , {vocalsound} the parameters .
Professor A: Yeah .
PhD F: So . Well . The other system are for instance . For Italian is at seventy - eight {vocalsound} percent recognition rate on the mismatch , and this new system has eighty - nine . But I don't think it indicates something , really . I don't {disfmarker} I don't think it means that the new system is more robust
Professor A: Uh - huh .
PhD F: or {disfmarker} It 's simply the fact that {disfmarker} Well .
Professor A: Well , the test would be if you then tried it on one of the other test sets , if {disfmarker} if it was {disfmarker}
PhD F: Y
Professor A: Right . So this was Italian , right ?
PhD F: Yeah . Yeah .
Professor A: So then if you take your changes
PhD F: It 's similar for other test sets
Professor A: and then {disfmarker}
PhD F: but I mean {vocalsound} from this se seventy - eight um percent recognition rate system , {vocalsound} I could change the transition probabilities for the {disfmarker} the first HMM and {pause} it will end up to eighty - nine also .
Professor A: Uh - huh .
PhD F: By using point five instead of point six , point four {vocalsound} as in the {disfmarker} the HTK script .
Professor A: Uh - huh . Yeah .
PhD F: So . Well . That 's {disfmarker}
PhD B: Yeah . Yeah I looked at um {disfmarker} {vocalsound} looked at the results when Stephane did that
PhD F: Well . Eh uh {disfmarker}
PhD B: and it 's {disfmarker} it 's really wo really happens .
PhD F: This really happens .
PhD B: I mean th the only difference is you change the self - loop transition probability by a tenth of a percent
PhD F: Yeah .
Professor A: Yeah .
PhD B: and it causes ten percent difference in the word error rate .
Professor A: A tenth of a per cent .
PhD B: Yeah . From point {disfmarker}
PhD F: Even tenth of a percent ?
PhD B: I {disfmarker} I 'm sorry
PhD F: Well , we tried {disfmarker} we tried point one ,
PhD B: f for point {disfmarker} from {disfmarker} You change at point one
PhD F: yeah .
Professor A: Oh !
PhD B: and n not tenth of a percent , one tenth ,
PhD F: Hmm .
Professor A: Yeah .
PhD B: alright ? Um so from point five {disfmarker} so from point six to point five and you get ten percent better .
Professor A: Mm - hmm .
PhD B: And it 's {disfmarker} {vocalsound} I think it 's what you basically hypothesized in the last meeting {vocalsound} about uh it just being very {disfmarker}
PhD F: Mm - hmm .
PhD B: and I think you mentioned this in your email too {disfmarker} it 's just very um {disfmarker}
PhD F: Mmm , yeah .
PhD B: you know get stuck in some local minimum and this thing throws you out of it I guess .
PhD F: Mm - hmm .
Professor A: Well , what 's {disfmarker} what are {disfmarker} according to the rules what {disfmarker} what are we supposed to do about the transition probabilities ? Are they supposed to be point five or point six ?
PhD B: I think you 're not allowed to {disfmarker} Yeah . That 's supposed to be point six , for the self - loop .
PhD F: Yeah .
Professor A: Point {disfmarker} It 's supposed to be point six .
PhD B: Yeah . But changing it to point five I think is {disfmarker} which gives you much better results , but that 's {vocalsound} not allowed .
Professor A: But not allowed ? Yeah . OK .
PhD B: Yeah .
PhD F: Yeah , but even if you use point five , I 'm not sure it will always give you the better results
PhD B: Yeah .
PhD F: on other test set or it
PhD B: Right . We only tested it on the {disfmarker} the medium mismatch ,
PhD F: on the other training set , I mean .
PhD B: right ? You said on the other cases you didn't notice {disfmarker}
PhD F: Yeah . But . I think , yeah . I think the reason is , yeah , I not I {disfmarker} it was in my mail I think also , {vocalsound} is the fact that the mismatch is trained only on the far microphone . Well , in {disfmarker} for the mismatched case everything is um using the far microphone training and testing , whereas for the highly mismatched , training is done on the close microphone so {vocalsound} it 's {disfmarker} it 's clean speech basically so you don't have this problem of local minima probably and for the well - match , it 's a mix of close microphone and distant microphone and {disfmarker} Well .
PhD B: I did notice uh something {disfmarker}
PhD F: So th I think the mismatch is the more difficult for the training part .
PhD B: Somebody , I think it was Morgan , suggested at the last meeting that I actually count to see {vocalsound} how many parameters and how many frames .
Professor A: Mm - hmm .
PhD F: Mm - hmm .
PhD B: And there are uh almost one point eight million frames of training data and less than forty thousand parameters in the baseline system .
Professor A: Hmm .
PhD F: Yeah .
PhD B: So it 's very , very few parameters compared to how much training data .
Professor A: Well . Yes .
PhD D: Mm - hmm .
Professor A: So . And that {disfmarker} that says that we could have lots more parameters actually .
PhD B: Yeah . Yeah .
PhD F: Mm - hmm .
PhD B: I did one quick experiment just to make sure I had everything worked out and I just {disfmarker} {vocalsound} uh f for most of the um {disfmarker} For {disfmarker} for all of the digit models , they end up at three mixtures per state . And so I just did a quick experiment , where I changed it so it went to four and um {vocalsound} it it {disfmarker} it didn't have a r any significant effect at the uh medium mismatch and high mismatch cases and it had {disfmarker} {vocalsound} it was just barely significant for the well - matched better . Uh so I 'm r gonna run that again but {vocalsound} um with many more uh mixtures per state .
Professor A: Yeah . Cuz at forty thou I mean you could you could have uh {disfmarker} Yeah , easily four times as many {vocalsound} parameters .
PhD B: Mm - hmm . And I think also {vocalsound} just seeing what we saw {vocalsound} uh in terms of the expected duration of the silence model ? when we did this tweaking of the self - loop ? The silence model expected duration was really different .
PhD F: Yeah .
PhD B: And so in the case where {vocalsound} um {vocalsound} it had a better score , the silence model expected duration was much longer .
PhD F: Yeah .
PhD B: So it was like {disfmarker} {vocalsound} it was a better match . I think {vocalsound} you know if we make a better silence model I think that will help a lot too um for a lot of these cases so but one one thing I {disfmarker} I wanted to check out before I increased the um {vocalsound} number of mixtures per state was {vocalsound} uh {vocalsound} in their {vocalsound} default training script they do an initial set of three re - estimations and then they built the silence model and then they do seven iterations then the add mixtures and they do another seven then they add mixtures then they do a final set of seven and they quit . Seven seems like a lot to me and it also makes the experiments go take a really long time I mean to do one turn - around of the well matched case takes like a day .
Professor A: Mm - hmm . Mm - hmm .
PhD B: And so {vocalsound} you know in trying to run these experiments I notice , you know , it 's difficult to find machines , you know , compute the run on . And so one of the things I did was I compiled HTK for the Linux {vocalsound} machines
Professor A: Mm - hmm .
PhD B: cuz we have this one from IBM that 's got like five processors in it ?
Professor A: Right .
PhD B: and so now I 'm {disfmarker} you can run stuff on that and that really helps a lot because now we 've got {vocalsound} you know , extra machines that we can use for compute . And if {disfmarker} I 'm do running an experiment right now where I 'm changing the number of iterations ? {vocalsound} from seven to three ?
PhD D: Mm - hmm .
Professor A: Yeah .
PhD B: just to see how it affects the baseline system . And so if we can get away with just doing three , we can do {vocalsound} many more experiments more quickly . And if it 's not a {disfmarker} a huge difference from running with seven iterations , {vocalsound} um , you know , we should be able to get a lot more experiments done .
PhD F: Hmm .
PhD B: And so . I 'll let you know what {disfmarker} what happens with that . But if we can {vocalsound} you know , run all of these back - ends f with many fewer iterations and {vocalsound} on Linux boxes we should be able to get a lot more experimenting done .
Professor A: Mm - hmm .
PhD B: So . So I wanted to experiment with cutting down the number of iterations before I {vocalsound} increased the number of Gaussians .
Professor A: Right . Sorry . So um , how 's it going on the {disfmarker}
PhD F: Um .
Professor A: So . You {disfmarker} you did some things . They didn't improve things in a way that convinced you you 'd substantially improved anything .
PhD F: Yeah .
Professor A: But they 're not making things worse and we have reduced latency , right ?
PhD F: Yeah . But actually {disfmarker} um actually it seems to do a little bit worse for the well - matched case and we just noticed that {disfmarker} Yeah , actually the way the final score is computed is quite funny . It 's not a mean of word error rate . It 's not a weighted mean of word error rate , it 's a weighted mean of improvements .
Professor A: Uh - huh .
PhD F: So . Which means that {vocalsound} actually the weight on the well - matched is {disfmarker} Well I well what what {disfmarker} What happened is that if you have a small improvement or a small if on the well - matched case {vocalsound} it will have uh huge influence on the improvement compared to the reference because the reference system is {disfmarker} is {disfmarker} is quite good for {disfmarker} for the well - ma well - matched case also .
PhD B: So it {disfmarker} it weights the improvement on the well - matched case really heavily compared to the improvement on the other cases ?
PhD F: No , but it 's the weighting of the {disfmarker} of the improvement not of the error rate .
PhD B: Yeah . Yeah , and it 's hard to improve on the {disfmarker} on the best case , cuz it 's already so good , right ?
PhD F: Yeah but {pause} what I mean is that you can have a huge improvement on the H {disfmarker} HMK 's , uh like five percent uh absolute , and this will not affect the final score almost {disfmarker} Uh this will almost not affect the final score because {vocalsound} this improvement {disfmarker} because the improvement {vocalsound} uh relative to the {disfmarker} the baseline is small {disfmarker}
Professor A: So they do improvement in terms of uh accuracy ? rather than word error rate ?
PhD F: Uh . Uh improvement ?
Professor A: So {disfmarker}
PhD F: No , it 's compared to the word er it 's improvement on the word error rate ,
Professor A: OK .
PhD F: yeah . Sorry .
Professor A: So if you have uh ten percent error and you get five percent absolute uh {vocalsound} improvement then that 's fifty percent .
PhD F: Mm - hmm .
Professor A: OK . So what you 're saying then is that if it 's something that has a small word error rate , {vocalsound} then uh a {disfmarker} even a relatively small improvement on it , in absolute terms , {vocalsound} will show up as quite {disfmarker} quite large in this .
PhD F: Mm - hmm .
Professor A: Is that what you 're saying ?
PhD F: Yeah .
Professor A: Yes .
PhD F: Yeah .
Professor A: OK . But yeah that 's {disfmarker} that 's {disfmarker} it 's the notion of relative improvement . Word error rate .
PhD F: Yeah . Sure , but when we think about the weighting , which is point five , point three , point two , {vocalsound} it 's on absolute on {disfmarker} on relative figures ,
Professor A: Yeah .
PhD F: not {disfmarker}
Professor A: Yeah .
PhD F: So when we look at this error rate
Professor A: No . That 's why I 've been saying we should be looking at word error rate uh and {disfmarker} and not {disfmarker} not at {vocalsound} at accuracies .
PhD F: uh {disfmarker} Mmm , yeah . Mmm , yeah .
Professor A: It 's {disfmarker}
PhD F: Mm - hmm .
Professor A: I mean uh we probably should have standardized on that all the way through . It 's just {disfmarker}
PhD B: Well .
PhD F: Mm - hmm .
PhD B: I mean , it 's not {disfmarker} it 's not that different , right ? I mean , just subtract the accuracy .
Professor A: Yeah but you 're {disfmarker} but when you look at the numbers , your sense of the relative size of things is quite different .
PhD B: I mean {disfmarker} Oh . Oh , I see . Yeah .
Professor A: If you had ninety percent uh correct {vocalsound} and five percent , five over ninety doesn't look like it 's a big difference , but {vocalsound} five over ten is {disfmarker} is big .
PhD B: Mm - hmm .
PhD F: Mm - hmm .
Professor A: So just when we were looking at a lot of numbers and {vocalsound} getting sense of what was important .
PhD B: I see . I see . Yeah . That makes sense .
Professor A: Um .
PhD F: Mmm .
Professor A: Um .
PhD F: Well anyway uh . So . Yeah . So it hurts a little bit on the well - match and yeah .
Professor A: What 's a little bit ? Like {disfmarker}
PhD F: Like , it 's difficult to say because again um {vocalsound} {vocalsound} I 'm not sure I have the um {disfmarker}
PhD B: Hey Morgan ? Do you remember that Signif program that we used to use for testing signi ? Is that still valid ? I {disfmarker} I 've been using that .
Professor A: Yeah . Yeah , it was actually updated .
PhD B: OK .
Professor A: Uh . {vocalsound} Jeff updated it some years ago
PhD B: Oh , it was . Oh , I shoul
Professor A: and {disfmarker} and uh cleaned it up made some things better in it . So .
PhD B: OK . I should find that new one . I just use my old one from {vocalsound} ninety - two or whatever
Professor A: Yeah , I 'm sure it 's not that different but {disfmarker} but he {disfmarker} {vocalsound} he uh {disfmarker} he was a little more rigorous , as I recall .
PhD B: OK .
PhD F: Right . So it 's around , like , point five . No , point six {comment} uh percent absolute on Italian {disfmarker}
Professor A: Worse .
PhD F: Worse , yep .
Professor A: Out of what ? I mean . s
PhD F: Uh well we start from ninety - four point sixty - four , and we go to ninety - four point O four .
Professor A: Uh - huh . So that 's six {disfmarker} six point th
PhD F: Uh .
PhD B: Ninety - three point six four , right ? is the baseline .
PhD F: Oh , no , I 've ninety - four . Oh , the baseline , you mean .
PhD B: Yeah .
PhD F: Well I don't {disfmarker} I 'm not talking about the baseline here .
PhD B: Oh . Oh . I 'm sorry .
PhD F: I uh {disfmarker} My baseline is the submitted system .
PhD B: Ah ! OK . Ah , ah .
PhD F: Hmm .
Professor A: Yeah .
PhD B: Sorry .
PhD F:  Oh yeah . For Finnish , we start to ninety - three point eight - four and we go to ninety - three point seventy - four . And for Spanish we are {disfmarker} we were at ninety - five point O five and we go to ninety - three - s point sixty one .
Professor A: OK , so we are getting hurt somewhat .
PhD F: So .
Professor A: And is that wh what {disfmarker} do you know what piece {disfmarker} you 've done several changes here . Uh , do you know what pie
PhD F: Yeah . I guess {disfmarker} I guess it 's {disfmarker} it 's the filter . Because nnn , well uh we don't have complete result , but the filter {disfmarker} So the filter with the shorter delay hurts on Italian well - matched , which {disfmarker} And , yeah . And the other things , like um {vocalsound} downsampling , upsampling , don't seem to hurt and {vocalsound} the new on - line normalization , neither .
PhD B: I 'm {disfmarker}
PhD F: So .
PhD B: I 'm really confused about something . If we saw that making a small change like , you know , a tenth , to the self - loop had a huge effect , {vocalsound} can we really make any conclusions about differences in this stuff ?
PhD F: Mm - hmm . Yeah that 's th Yeah .
PhD B: I mean , especially when they 're this small . I mean .
PhD F: I think we can be completely fooled by this thing , but {disfmarker} I don't know .
Professor A: Well , yeah .
PhD F: So . There is first this thing , and then the {disfmarker} yeah , I computed the um {disfmarker} {vocalsound} like , the confidence level on the different test sets . And for the well - matched they are around um {vocalsound} point six uh percent . For the mismatched they are around like let 's say one point five percent . And for the well - m uh HM they are also around one point five .
Professor A: But {disfmarker} OK , so you {disfmarker} these {disfmarker} these degradations you were talking about were on the well - matched case
PhD F: So .
Professor A: Uh . Do the {disfmarker} does the new filter make things uh better or worse for the other cases ?
PhD F: Yeah . But . Uh . About the same . It doesn't hurt . Yeah .
Professor A: Doesn't hurt , but doesn't get a little better , or something .
PhD F: No .
Professor A: No . OK , so {vocalsound} um I guess the argument one might make is that , " Yeah , if you looked at one of these cases {vocalsound} and you jiggle something and it changes {vocalsound} then uh you 're not quite sure what to make of it . But when you look across a bunch of these and there 's some {disfmarker} some pattern , um {disfmarker} I mean , so eh h here 's all the {disfmarker} if {disfmarker} if in all these different cases {vocalsound} it never gets better , and there 's significant number of cases where it gets worse , {vocalsound} then you 're probably {pause} hurting things , {vocalsound} I would say . So um {vocalsound} I mean at the very least that would be a reasonably prediction of what would happen with {disfmarker} with a different test set , that you 're not jiggling things with . So I guess the question is if you can do better than this . If you can {disfmarker} if we can approximate {vocalsound} the old numbers while still keeping the latency down .
PhD F: Mmm . Yeah .
Professor A: Uh , so . Um . What I was asking , though , is uh {disfmarker} are {disfmarker} what 's {disfmarker} what 's the level of communication with uh {vocalsound} the O G I gang now , about this and {disfmarker}
PhD F: Well , we are exchanging mail as soon as we {disfmarker} {vocalsound} we have significant results .
Professor A: Yeah .
PhD F: Um . Yeah . For the moment , they are working on integrating {vocalsound} the um {vocalsound} spectral subtraction apparently from Ericsson .
Professor A: Mm - hmm .
PhD F: Um . Yeah . And so . Yeah . We are working on our side on other things like {vocalsound} uh also trying a sup spectral subtraction but of {disfmarker} of our own , I mean , another {vocalsound} spectral substraction .
Professor A: Mm - hmm .
PhD F: Um . Yeah . So I think it 's {disfmarker} it 's OK . It 's going {disfmarker}
Professor A: Is there any further discussion about this {disfmarker} this idea of {disfmarker} of having some sort of source code control ?
PhD F: Yeah . Well . For the moment they 're {disfmarker} uh everybody 's quite um {disfmarker} There is this Eurospeech deadline , so .
Professor A: I see .
PhD F: Um . And . Yeah . But yeah . As soon as we have something that 's significant and that 's better than {disfmarker} than what was submitted , we will fix {disfmarker} fix the system and {disfmarker} But we 've not discussed it {disfmarker} it {disfmarker} it {disfmarker} this yet , yeah .
Professor A: Yeah . Sounds like a great idea but {disfmarker} but I think that {disfmarker} that um {vocalsound} he 's saying people are sort of scrambling for a Eurospeech deadline .
PhD F: Mmm .
Professor A: But that 'll be uh , uh done in a week . So , maybe after {vocalsound} this next one .
PhD F: Yeah .
PhD B: Wow ! Already a week ! Man !
Professor A: Yeah .
PhD B: You 're right . That 's amazing .
Professor A: Yeah . Anybo - anybody in the {disfmarker} in this group do doing anything for Eurospeech ?
PhD F: S
Professor A: Or , is that what {disfmarker} is that {disfmarker}
PhD F: Yeah we are {disfmarker} {vocalsound} We are trying to {disfmarker} to do something with the Meeting Recorder digits ,
Professor A: Right .
PhD F: and {disfmarker} But yeah . Yeah . And the good thing is that {pause} there is this first deadline ,
Professor A: Yeah .
PhD F: and , well , some people from OGI are working on a paper for this , but there is also the um {vocalsound} special session about th Aurora which is {disfmarker} {vocalsound} uh which has an extended deadline . So . The deadline is in May .
Professor A: For uh {disfmarker} {vocalsound} Oh , for Eurospeech ?
PhD F: For th Yeah .
Professor A: Oh !
PhD F: So f only for the experiments on Aurora . So it {disfmarker} it 's good ,
Professor A: Oh , a special dispensation .
PhD F: yeah .
Professor A: That 's great .
PhD B: Mm - hmm . Where is Eurospeech this year ?
PhD F: It 's in Denmark .
Professor A: Aalborg {disfmarker} Aalborg uh
PhD B: Oh .
Professor A: So the deadline {disfmarker} When 's the deadline ? When 's the deadline ?
PhD F: Hmm ? I think it 's the thirteenth of May .
Professor A: That 's great ! It 's great . So we should definitely get something in for that .
PhD F: Yeah .
Professor A: But on meeting digits , maybe there 's {disfmarker} Maybe .
PhD F: Yeah .
Professor A: Maybe .
PhD F: So it would be for the first deadline .
Professor A: Yeah .
PhD F: Nnn .
Professor A: Yeah . So , I mean , I {disfmarker} I think that you could certainly start looking at {disfmarker} at the issue uh but {disfmarker} but uh {vocalsound} I think it 's probably , on s from what Stephane is saying , it 's {disfmarker} it 's unlikely to get sort of active participation from the two sides until after they 've {disfmarker}
PhD B: Well I could at least {disfmarker} Well , I 'm going to be out next week but I could {pause} try to look into like this uh CVS over the web . That seems to be a very popular {vocalsound} way of {pause} people distributing changes and {disfmarker} over , you know , multiple sites and things
Professor A: Mm - hmm .
PhD B: so maybe {vocalsound} if I can figure out how do that easily and then pass the information on to everybody so that it 's {vocalsound} you know , as easy to do as possible and {disfmarker} and people don't {disfmarker} it won't interfere with {comment} their regular work , then maybe that would be good . And I think we could use it for other things around here too . So .
Professor A: Good .
Grad C: That 's cool . And if you 're interested in using CVS , I 've set it up here ,
PhD B: Oh great .
Grad C: so .
PhD B: OK .
Grad C: um j
PhD B: I used it a long time ago but it 's been a while so maybe I can ask you some questions .
Grad C: Oh . So . I 'll be away tomorrow and Monday but I 'll be back on Tuesday or Wednesday .
PhD B: OK .
Professor A: Yeah . Dave , the other thing , actually , is {disfmarker} is this business about this wave form . Maybe you and I can talk a little bit at some point about {vocalsound} coming up with a better {vocalsound} uh demonstration of the effects of reverberation for our web page , cuz uh {vocalsound} {disfmarker} the uh {vocalsound} um I mean , actually the {disfmarker} the uh It made a good {disfmarker} good audio demonstration because when we could play that clip the {disfmarker} the {disfmarker} the really {vocalsound} obvious difference is that you can hear two voices and {disfmarker} {vocalsound} {vocalsound} in the second one and only hear {disfmarker}
PhD B: Maybe we could just {pause} like , talk into a cup .
Professor A: Yeah .
PhD B: Some good reverb .
Professor A: No , I mean , it sound {disfmarker} it sounds pretty reverberant , but I mean you can't {disfmarker} when you play it back in a room with a {disfmarker} you know a big room , {vocalsound} nobody can hear that difference really .
Grad C: Yeah .
Professor A: They hear that it 's lower amplitude and they hear there 's a second voice ,
Grad C: Uh - huh .
Professor A: um {vocalsound} but uh that {disfmarker} actually that makes for a perfectly good demo because that 's a real obvious thing , that you hear two voices .
PhD B: But not of reverberation .
Professor A: Yeah .
Grad C: A boom .
Professor A: Well that {disfmarker} that {disfmarker} that 's OK . But for the {disfmarker} the visual , just , you know , I 'd like to have uh {vocalsound} uh , you know , the spectrogram again ,
Grad C: Yeah .
Professor A: because you 're {disfmarker} you 're {disfmarker} you 're visual {vocalsound} uh abilities as a human being are so good {vocalsound} you can pick out {disfmarker} you know , you {disfmarker} you look at the good one , you look at the cru the screwed up one , and {disfmarker} and you can see the features in it without trying to @ @ {disfmarker}
PhD B: I noticed that in the pictures .
Professor A: yeah .
PhD B: I thought " hey , you know th " I {disfmarker} My initial thought was " this is not too bad ! "
Professor A: Right . But you have to {disfmarker} you know , if you look at it closely , you see " well , here 's a place where this one has a big formant {disfmarker} uh uh formant {disfmarker} maj major formants here are {disfmarker} {vocalsound} are moving quite a bit . " And then you look in the other one and they look practically flat .
PhD B: Mm - hmm .
Professor A: So I mean you could {disfmarker} that 's why I was thinking , in a section like that , you could take a look {disfmarker} look at just that part of the spectrogram and you could say " Oh yeah . This {disfmarker} this really distorted it quite a bit . "
PhD B: Yeah . The main thing that struck me in looking at those two spectrograms was the difference in the high frequencies . It looked like {vocalsound} for the one that was farther away , you know , it really {disfmarker} everything was attenuated
Professor A: Right .
PhD B: and {disfmarker} I mean that was the main visual thing that I noticed .
Professor A: Right . But it 's {disfmarker} it 's uh {disfmarker} So . Yeah . So there are {disfmarker} clearly are spectral effects . Since you 're getting all this indirect energy , then a lot of it does have {disfmarker} have uh {vocalsound} reduced high frequencies . But um the other thing is the temporal courses of things really are changed , and {disfmarker} {vocalsound} and uh we want to show that , in some obvious way . The reason I put the wave forms in there was because {vocalsound} uh they {disfmarker} they do look quite different . Uh . And so I thought " Oh , this is good . " but I {disfmarker} {vocalsound} I just uh {disfmarker} After {disfmarker} after uh they were put in there I didn't really look at them anymore , cuz I just {disfmarker} they were different . So {vocalsound} I want something that has a {disfmarker} is a more interesting explanation for why they 're different . Um .
Grad C: Oh . So maybe we can just substitute one of these wave forms and um {vocalsound} then do some kind of zoom in on the spectrogram on an interesting area .
Professor A: Something like that . Yeah .
Grad C: Uh - huh .
Professor A: The other thing that we had in there that I didn't like was that um {vocalsound} the most obvious characteristic of the difference uh when you listen to it is that there 's a second voice , and the {disfmarker} the {disfmarker} the {disfmarker} the {disfmarker} the uh {vocalsound} cuts that we have there actually don't correspond to the full wave form . It 's just the first {disfmarker} I think there was something where he was having some trouble getting so much in , or . I {disfmarker} I forget the reason behind it . But {vocalsound} it {disfmarker} it 's um {disfmarker} {vocalsound} it 's the first six seconds or something {vocalsound} of it and it 's in {vocalsound} the seventh or eighth second or something where @ @ the second voice comes in . So we {disfmarker} we would like to actually see {vocalsound} the voice coming in , too , I think , since that 's the most obvious thing {pause} when you listen to it .
Grad C: Mm - hmm .
Professor A: So . Um .
PhD F: Uh , yeah . Yeah . I brought some {disfmarker} I don't know if {disfmarker} {vocalsound} some {vocalsound} figures here . Well . I start {disfmarker} we started to work on spectral subtraction . And {vocalsound} um {vocalsound} the preliminary results were very bad .
Professor A: Uh - huh .
PhD F: So the thing that we did is just to add spectral subtraction before this , the Wall uh process , which contains LDA on - line normalization . And it hurts uh a lot .
Professor A: Uh - huh .
PhD F: And so we started to look at {disfmarker} at um things like this , which is , well , it 's {disfmarker} Yeah . So you have the C - zero parameters for one uh Italian utterance .
PhD D: You can @ @ .
PhD F: And I plotted this for two channels . Channel zero is the close mic microphone , and channel one is the distant microphone . And it 's perfectly synchronized , so . And the sentence contain only one word , which is " Due " And it can't clearly be seen . Where {disfmarker} where is it ?
Professor A: Uh - huh .
PhD F: Where is the word ?
PhD B: This is {disfmarker} this is ,
Grad E: Hmm .
PhD B: oh , a plot of C - zero ,
PhD F: So .
PhD B: the energy .
PhD F: This is a plot of C - zero , uh when we don't use spectral substraction , and when there is no on - line normalization .
Professor A: Mm - hmm .
PhD F: So . There is just some filtering with the LDA and {vocalsound} and some downsampling , upsampling .
PhD B: C - zero is the close talking ? {disfmarker}
PhD F: So .
PhD B: uh the close channel ?
PhD F: Yeah . Yeah .
PhD B: and s channel one is the {disfmarker}
PhD F: Yeah . So C - zero is very clean , actually .
PhD B: Yeah .
PhD F: Uh then when we apply mean normalization it looks like the second figure , though it is not . Which is good . Well , the noise part is around zero
Professor A: Mm - hmm .
PhD F: and {disfmarker} {vocalsound} {vocalsound} And then the third figure is what happens when we apply mean normalization and variance normalization . So . What we can clearly see is that on the speech portion {vocalsound} the two channel come {disfmarker} becomes very close , but also what happens on the noisy portion is that the variance of the noise is {disfmarker}
Professor A: Mm - hmm .
PhD B: This is still being a plot of C - zero ? OK .
PhD F: Yeah . This is still C - zero .
PhD B: Can I ask um what does variance normalization do ? w What is the effect of that ?
Professor A: Normalizes the variance .
PhD F: So it {disfmarker} it {disfmarker} Yeah .
PhD B: I mean
PhD F: It normalized th the standard deviation .
PhD B: y Yeah .
PhD F: So it {disfmarker}
PhD B: No , I understand that ,
PhD F: You {disfmarker} you get an estimate of the standard deviation .
PhD B: but I mean {disfmarker}
PhD F: That 's
PhD B: No .
PhD F: um {disfmarker}
PhD B: No , I understand what it is , but I mean , what does it {disfmarker} what 's {disfmarker} what is
PhD F: Yeah but .
PhD B: uh {disfmarker}
Professor A: What 's the rationale ?
PhD B: We Yeah . Yeah . Why {disfmarker} why do it ?
PhD F: Uh .
Professor A: Well , I mean , because {vocalsound} everything uh {disfmarker} If you have a system based on Gaussians , everything is based on means and variances .
PhD B: Yeah .
Professor A: So if there 's an overall {vocalsound} reason {disfmarker} You know , it 's like uh if you were doing uh image processing and in some of the pictures you were looking at , uh there was a lot of light uh and {disfmarker} and in some , there was low light ,
PhD B: Mm - hmm .
Professor A: you know , you would want to adjust for that in order to compare things .
PhD B: Mm - hmm .
Professor A: And the variance is just sort of like the next moment , you know ? So uh {vocalsound} what if um one set of pictures was taken uh so that throughout the course it was {disfmarker} went through daylight and night uh {vocalsound} um um ten times , another time it went thr I mean i is , you know , how {disfmarker} how much {disfmarker} {vocalsound} how much vari
PhD B: Oh , OK .
Professor A: Or no . I guess a better example would be {vocalsound} how much of the light was coming in from outside rather than artificial light . So if it was a lot {disfmarker} {vocalsound} if more was coming from outside , then there 'd be the bigger effect of the {disfmarker} of the {disfmarker} of the change in the {disfmarker} So every mean {disfmarker} every {disfmarker} all {disfmarker} all of the {disfmarker} the parameters that you have , especially the variances , are going to be affected by the overall variance .
PhD B: Oh , OK . Uh - huh .
Professor A: And so , in principle , you {disfmarker} if you remove that source , then , you know , you can {disfmarker}
PhD B: I see . OK . So would {disfmarker} the major effect is {disfmarker} that you 're gonna get is by normalizing the means ,
Professor A: That 's the first order but {disfmarker} thing ,
PhD B: but it may help {disfmarker} First - order effects .
Professor A: but then the second order is {disfmarker} is the variances
PhD B: And it may help to do the variance . OK .
Professor A: because , again , if you {disfmarker} if you 're trying to distinguish between E and B
PhD B: OK .
Professor A: if it just so happens that the E 's {vocalsound} were a more {disfmarker} you know , were recorded when {disfmarker} when the energy was {disfmarker} was {disfmarker} was larger or something ,
PhD B: Mm - hmm . Mm - hmm . Mm - hmm .
Professor A: or the variation in it was larger , {vocalsound} uh than with the B 's , then this will be {disfmarker} give you some {disfmarker} some bias .
PhD B:
Professor A: So the {disfmarker} {vocalsound} it 's removing these sources of variability in the data {vocalsound} that have nothing to do with the linguistic component .
PhD B: OK .
PhD F: Mmm .
PhD B: Gotcha . OK . Sorry to interrupt .
Professor A: But the {disfmarker} the uh {disfmarker} but let me as ask {disfmarker} ask you something .
PhD F: Yep . And it {disfmarker} and this {disfmarker}
Professor A: i is {disfmarker} if {disfmarker} If you have a good voice activity detector , isn't {disfmarker} isn't it gonna pull that out ?
PhD F: Yeah . Sure . If they are good . Yeah . Well what it {disfmarker} it shows is that , yeah , perhaps a good voice activity detector is {disfmarker} is good before on - line normalization and that 's what uh {vocalsound} we 've already observed . But uh , yeah , voice activity detection is not {vocalsound} {vocalsound} an easy thing neither .
PhD B: But after you do this , after you do the variance normalization {disfmarker} I mean .
PhD F: Mm - hmm .
PhD B: I don't know , it seems like this would be a lot easier than this signal to work with .
PhD F: Yeah . So . What I notice is that , while I prefer to look at the second figure than at the third one , well , because you clearly see where speech is .
Professor A: Yeah .
PhD B: Yeah .
PhD F: But the problem is that on the speech portion , channel zero and channel one are more different than when you use variance normalization where channel zero and channel one become closer .
Professor A: Right .
PhD B: But for the purposes of finding the speech {disfmarker}
PhD F: And {disfmarker} Yeah , but here {disfmarker}
PhD B: You 're more interested in the difference between the speech and the nonspeech ,
PhD F: Yeah .
PhD B: right ?
PhD F: Yeah . So I think , yeah . For I th I think that it {disfmarker} perhaps it shows that {vocalsound} uh the parameters that the voice activity detector should use {disfmarker} uh have to use should be different than the parameter that have to be used for speech recognition .
Professor A: Yeah . So basically you want to reduce this effect .
PhD F: Well , y
Professor A: So you can do that by doing the voi voice activity detection . You also could do it by spect uh spectral subtraction before the {vocalsound} variance normalization , right ?
PhD F: Yeah , but it 's not clear , yeah .
Professor A: So uh {disfmarker}
PhD F: We So . Well . It 's just to
Professor A: Yeah .
PhD F: the {disfmarker} the number that at that are here are recognition experiments on Italian HM and MM {vocalsound} with these two kinds of parameters . And , {pause} well , it 's better with variance normalization .
Professor A: Yeah . Yeah . So it does get better even though it looks ugly .
PhD F: Uh {disfmarker}
Professor A: OK . but does this have the voice activity detection in it ?
PhD F: Yeah .
Professor A: OK .
PhD F: Um .
Professor A: So .
Grad E: OK .
PhD B: Where 's th
PhD F: But the fact is that the voice activity detector doesn't work on channel one . So . Yeah .
Professor A: Uh - huh .
PhD B: Where {disfmarker} at what stage is the voice activity detector applied ? Is it applied here or a after the variance normalization ?
PhD F: Hmm ?
Professor A: Spectral subtraction , I guess .
PhD B: or {disfmarker}
PhD F: It 's applied before variance normalization . So it 's a good thing ,
PhD B: Oh .
PhD F: because I guess voice activity detection on this should {disfmarker} could be worse .
PhD B: Yeah . Is it applied all the way back here ?
PhD F: It 's applied the um on , yeah , something like this ,
PhD B: Maybe that 's why it doesn't work for channel one .
PhD F: yeah . Perhaps , yeah .
Professor A: Can I {disfmarker}
PhD F: So we could perhaps do just mean normalization before VAD .
PhD B: Mm - hmm .
Professor A: Mm - hmm . Can I ask a , I mean {disfmarker} a sort of top - level question , which is {vocalsound} um " if {disfmarker} if most of what the OGI folk are working with is trying to {vocalsound} integrate this other {disfmarker} other uh spectral subtraction , {vocalsound} why are we worrying about it ? "
PhD F: Mm - hmm . About ? Spectral subtraction ?
Professor A: Yeah .
PhD F: It 's just uh {disfmarker} Well it 's another {disfmarker} They are trying to u to use the um {disfmarker} {vocalsound} the Ericsson and we 're trying to use something {disfmarker} something else . And . Yeah , and also to understand what happens because
Professor A: OK .
PhD F: uh fff Well . When we do spectral subtraction , actually , I think {vocalsound} that this is the {disfmarker} the two last figures .
Professor A: Yeah .
PhD F: Um . It seems that after spectral subtraction , speech is more emerging now uh {vocalsound} than {disfmarker} than before .
Professor A: Mm - hmm .
PhD B: Speech is more what ?
PhD F: Well , the difference between the energy of the speech and the energy of the n spectral subtrac subtracted noise portion is {disfmarker} is larger .
Professor A: Mm - hmm .
PhD F: Well , if you compare the first figure to this one {disfmarker} Actually the scale is not the same , but if you look at the {disfmarker} the numbers um {vocalsound} you clearly see that the difference between the C - zero of the speech and C - zero of the noise portion is larger . Uh but what happens is that after spectral subtraction , {vocalsound} you also increase the variance of this {disfmarker} of C - zero .
Professor A: Mm - hmm .
PhD F: And so if you apply variance normalization on this , it completely sc screw everything . Well .
Professor A: Mm - hmm .
PhD F: Um . Uh . Yeah . So yeah . And what they did at OGI is just {vocalsound} uh they don't use on - line normalization , for the moment , on spectral subtraction and I think {disfmarker} Yeah . I think as soon as they will try on - line normalization {vocalsound} there will be a problem . So yeah , we 're working on the same thing but {vocalsound} I think uh with different {disfmarker} different system and {disfmarker}
Professor A: Right . I mean , i the Intellectually it 's interesting to work on things th uh one way or the other
PhD F: Mm - hmm .
Professor A: but I 'm {disfmarker} I 'm just wondering if um {disfmarker} {vocalsound} on the list of things that there are to do , if there are things that we won't do because {vocalsound} we 've got two groups doing the same thing .
PhD F: Mm - hmm .
Professor A: Um . That 's {disfmarker}
PhD F: Mm - hmm .
Professor A: Um . Just {disfmarker} just asking . Uh . I mean , it 's {disfmarker}
PhD F: Yeah , well ,
PhD B: There also could be {disfmarker} I mean . I can maybe see a reason f for both working on it too
PhD F: uh .
PhD B: if {vocalsound} um you know , if {disfmarker} if {disfmarker} if you work on something else and {disfmarker} and you 're waiting for them to give you {vocalsound} spectral subtraction {disfmarker} I mean it 's hard to know whether {vocalsound} the effects that you get from the other experiments you do will {vocalsound} carry over once you then bring in their spectral subtraction module . So it 's {disfmarker} it 's almost like everything 's held up waiting for this {vocalsound} one thing . I don't know if that 's true or not , but I could see how {disfmarker}
PhD F: Mmm .
Professor A: I don't know .
PhD B: Maybe that 's what you were thinking .
Professor A: I don't know . {vocalsound} I mean , we still evidently have a latency reduction plan which {disfmarker} which isn't quite what you 'd like it to be . That {disfmarker} that seems like one prominent thing . And then uh weren't issues of {disfmarker} of having a {disfmarker} a second stream or something ? That was {disfmarker} Was it {disfmarker} There was this business that , you know , we {disfmarker} we could use up the full forty - eight hundred bits , and {disfmarker}
PhD F: Yeah . But I think they ' I think we want to work on this . They also want to work on this , so . Uh . {vocalsound} yeah . We {disfmarker} we will try MSG , but um , yeah . And they are t I think they want to work on the second stream also , but more with {vocalsound} some kind of multi - band or , well , what they call TRAP or generalized TRAP .
Professor A: Mm - hmm .
PhD F: Um . So .
Professor A: OK . Do you remember when the next meeting is supposed to be ? the next uh {disfmarker}
PhD F: It 's uh in June .
Professor A: In June . OK .
PhD F: Yeah .
Professor A: Yeah . Um . Yeah , the other thing is that you saw that {disfmarker} that mail about uh the VAD {disfmarker} V A Ds performing quite differently ? That that uh So um . This {disfmarker} there was this experiment of uh " what if we just take the baseline ? "
PhD F: Mmm .
Professor A: set uh of features , just mel cepstra , and you inc incorporate the different V A And it looks like the {disfmarker} the French VAD is actually uh better {disfmarker} significantly better .
PhD B: Improves the baseline ?
Professor A: Yeah . Yeah .
PhD F: Yeah but I don't know which VAD they use . Uh . If the use the small VAD I th I think it 's on {disfmarker} I think it 's easy to do better because it doesn't work at all . So . I {disfmarker} I don't know which {disfmarker} which one . It 's Pratibha that {disfmarker} that did this experiment .
PhD D: Yeah .
PhD F: Um . We should ask which VAD she used .
PhD D: I don't @ @ . He {disfmarker} Actually , I think that he say with the good VAD of {disfmarker} from OGI and with the Alcatel VAD . And the experiment was sometime better , sometime worse .
PhD F: Yeah but I {disfmarker} it 's uh {disfmarker} I think you were talking about the other mail that used VAD on the reference features .
Professor A: Yes .
PhD F: Yeah .
Professor A: And on that one , uh the French one is {disfmarker} was better .
PhD D: I don't remember .
Professor A: It was just better .
PhD D: Mm - hmm .
Professor A: I mean it was enough better that {disfmarker} that it would {vocalsound} uh account for a fair amount of the difference between our performance , actually .
PhD F: Mm - hmm .
PhD D: Mm - hmm .
Professor A: So . {vocalsound} Uh . So if they have a better one , we should use it . I mean . You know ? it 's {disfmarker} you can't work on everything .
PhD F: Yeah .
Professor A: Uh . {vocalsound} Uh . Yeah .
PhD F: Yeah , so we should find out if it 's really better . I mean if it {disfmarker} the {disfmarker} compared to the small or the big network .
PhD D: Mm - hmm .
Professor A: Yeah .
PhD F: And perhaps we can easily improve if {disfmarker} if we put like mean normalization before the {disfmarker} before the VAD . Because {disfmarker} {vocalsound} as {disfmarker} as you 've {pause} mentioned .
Professor A: Yeah .
PhD F: Mmm .
Professor A: H Hynek will be back in town uh the week after next , back {disfmarker} back in the country . So . And start {disfmarker} start organizing uh {vocalsound} more visits and connections and so forth ,
PhD F: Mm - hmm .
Professor A: and {disfmarker} uh working towards June .
PhD F: Yeah .
PhD D: Also is Stephane was thinking that {vocalsound} maybe it was useful to f to think about uh {vocalsound} voiced - unvoiced {disfmarker}
PhD F: Mm - hmm .
PhD D: to work uh here in voiced - unvoiced detection .
PhD F: Yeah . Yeah .
PhD D: And we are looking {vocalsound} {vocalsound} in the uh signal .
PhD F: Yeah , my feeling is that um actually {vocalsound} when we look at all the proposals , ev everybody is still using some kind of spectral envelope
Professor A: Right .
PhD F: and um it 's {disfmarker}
Professor A: No use of pitch uh basically . Yeah .
PhD F: Yeah , well , not pitch , but to look at the um fine {disfmarker} at the {disfmarker} at the high re high resolution spectrum .
Professor A: Yeah . Well , it {disfmarker}
PhD F: So . We don't necessarily want to find the {disfmarker} the pitch of the {disfmarker} of the sound but uh {disfmarker} Cuz I have a feeling that {vocalsound} when we look {disfmarker} when we look at the {disfmarker} just at the envelope there is no way you can tell if it 's voiced and unvoiced , if there is some {disfmarker} It 's {disfmarker} it 's easy in clean speech because voiced sound are more low frequency and . So there would be more ,
Professor A: Yeah .
PhD F: uh {disfmarker} there is the first formant , which is the larger and then voiced sound are more high frequencies cuz it 's frication and {disfmarker}
Professor A: Right .
PhD F: But , yeah . When you have noise there is no um {disfmarker} {vocalsound} if {disfmarker} if you have a low frequency noise it could be taken for {disfmarker} for voiced speech and .
Professor A: Yeah , you can make these mistakes ,
PhD F: So .
Professor A: but {disfmarker} but {disfmarker}
PhD B: Isn't there some other
PhD F: S
PhD B: uh d
PhD F: So I think that it {disfmarker} it would be good {disfmarker} Yeah , yeah , well , go {disfmarker} go on .
PhD B: Uh , I was just gonna say isn't there {disfmarker} {vocalsound} aren't {disfmarker} aren't there lots of ideas for doing voice activity , or speech - nonspeech rather , {comment} um by looking at {vocalsound} um , you know , uh {vocalsound} I guess harmonics or looking across time {disfmarker}
Professor A: Well , I think he was talking about the voiced - unvoiced , though ,
PhD F: Mmm .
Professor A: right ? So , not the speech - nonspeech .
PhD B: Yeah . Well even with e
Professor A: Yeah .
PhD B: uh w ah you know , uh even with the voiced - non {pause} voiced - unvoiced
PhD F: Mmm .
PhD B: um {disfmarker} I thought that you or {pause} somebody was talking about {disfmarker}
Professor A: Well . Uh yeah . B We should let him finish what he w he was gonna say ,
PhD F: So .
PhD B: OK .
Professor A: and {disfmarker}
PhD B: So go ahead .
PhD F: Um yeah , so yeah , I think if we try to develop a second stream well , there would be one stream that is the envelope and the second , it could be interesting to have that 's {disfmarker} something that 's more related to the fine structure of the spectrum . And . Yeah , so I don't know . We were thinking about like using ideas from {disfmarker} from Larry Saul , have a good voice detector , have a good , well , voiced - speech detector , that 's working on {disfmarker} on the FFT and {vocalsound} uh
Professor A: U
PhD F: Larry Saul could be an idea . We were are thinking about just {vocalsound} kind of uh taking the spectrum and computing the variance of {disfmarker} of the high resolution spectrum {vocalsound} and things like this .
Professor A: So u s u OK . So {disfmarker} So many {vocalsound} tell you something about that . Uh we had a guy here some years ago who did some work on {vocalsound} um {vocalsound} making use of voicing information uh to {vocalsound} help in reducing the noise .
PhD F: Yeah ?
Professor A: So what he was doing is basically y you {disfmarker} {vocalsound} you do estimate the pitch .
PhD F: Mm - hmm .
Professor A: And um you {disfmarker} from that you {disfmarker} you estimate {disfmarker} or you estimate fine harmonic structure , whichev ei either way , it 's more or less the same . But {vocalsound} uh the thing is that um you then {vocalsound} can get rid of things that are not {disfmarker} i if there is strong harmonic structure , {vocalsound} you can throw away stuff that 's {disfmarker} that 's non - harmonic .
PhD F: Mm - hmm . Mm - hmm .
Professor A: And that {disfmarker} that is another way of getting rid of part of the noise
PhD F: Yeah .
Professor A: So um that 's something {vocalsound} that is sort of finer ,
PhD F: Yeah .
Professor A: brings in a little more information than just spectral subtraction . Um .
PhD F: Mm - hmm .
Professor A: And he had some {disfmarker} I mean , he did that sort of in combination with RASTA . It was kind of like RASTA was taking care of convolutional stuff
PhD F: Mmm .
Professor A: and he was {disfmarker}
PhD F: Mm - hmm .
Professor A: and {disfmarker} and got some {disfmarker} some decent results doing that . So that {disfmarker} that 's another {disfmarker} another way . But yeah , there 's {disfmarker} there 's {disfmarker}
PhD F: Yeah . Mmm .
Professor A: Right . There 's all these cues . We 've actually back when Chuck was here we did some voiced - unvoiced uh {vocalsound} classification using a bunch of these ,
PhD F: But {disfmarker}
Professor A: and {disfmarker} and uh works OK . Obviously it 's not perfect but um {disfmarker}
PhD F: Mm - hmm .
Professor A: But the thing is that you can't {disfmarker} given the constraints of this task , we can't , {vocalsound} in a very nice way , feed {pause} forward to the recognizer the information {disfmarker} the probabilistic information that you might get about whether it 's voiced or unvoiced , where w we can't you know affect the {disfmarker} {vocalsound} the uh distributions or anything .
PhD F: Mm - hmm .
Professor A: But we {disfmarker} what we uh {disfmarker} I guess we could Yeah .
PhD B: Didn't the head dude send around that message ? Yeah , I think you sent us all a copy of the message , where he was saying that {disfmarker} I I 'm not sure , exactly , what the gist of what he was saying , but something having to do with the voice {vocalsound} activity detector and that it will {disfmarker} {vocalsound} that people shouldn't put their own in or something . It was gonna be a {disfmarker}
Professor A: That {disfmarker} But {disfmarker} OK . So that 's voice activity detector as opposed to voicing detector .
PhD F: They didn't .
Professor A: So we 're talking about something a little different .
PhD F: Mmm .
PhD B: Oh , I 'm sorry .
Professor A: Right ?
PhD B: I {disfmarker} I missed that .
PhD F: Mmm .
Professor A: I guess what you could do , maybe this would be w useful , if {disfmarker} if you have {disfmarker} if you view the second stream , yeah , before you {disfmarker} before you do KLT 's and so forth , if you do view it as probabilities , and if it 's an independent {disfmarker} So , if it 's {disfmarker} if it 's uh not so much {vocalsound} envelope - based by fine - structure - based , uh looking at harmonicity or something like that , um if you get a probability from that information and then multiply it by {disfmarker} you know , multiply by all the voiced {vocalsound} outputs and all the unvoiced outputs , you know , then {vocalsound} use that as the
PhD F: Mm - hmm .
Professor A: uh {disfmarker} take the log of that or {vocalsound} uh pre pre uh {disfmarker} pre - nonlinearity ,
PhD F: Yeah . i if {disfmarker}
Professor A: uh and do the KLT on the {disfmarker} on {disfmarker} on that ,
PhD F: Yeah .
Professor A: then that would {disfmarker} that would I guess be uh a reasonable use of independent information . So maybe that 's what you meant . And then that would be {disfmarker}
PhD F: Yeah , well , I was not thinking this {disfmarker} yeah , this could be an yeah So you mean have some kind of probability for the v the voicing
Professor A: R Right . So you have a second neural net .
PhD F: and then use a tandem system
Professor A: It could be pretty small . Yeah . If you have a tandem system and then you have some kind of {disfmarker} it can be pretty small {disfmarker} net {disfmarker}
PhD F: Mm - hmm .
Professor A: we used {disfmarker} we d did some of this stuff . Uh I {disfmarker} I did , some years ago ,
PhD F: Yeah .
Professor A: and the {disfmarker} and {disfmarker} and you use {disfmarker} {vocalsound} the thing is to use information primarily that 's different as you say , it 's more fine - structure - based than {disfmarker} than envelope - based
PhD F: Mm - hmm .
Professor A: uh so then it you {disfmarker} you {disfmarker} you can pretty much guarantee it 's stuff that you 're not looking at very well with the other one , and uh then you only use for this one distinction .
PhD F: Alright .
Professor A: And {disfmarker} and so now you 've got a probability of the cases , and you 've got uh the probability of the finer uh categories on the other side . You multiply them where appropriate and uh {vocalsound} um
PhD F: I see , yeah . Mm - hmm .
Professor A: if they really are from independent {pause} information sources then {vocalsound} they should have different kinds of errors
PhD F: Mm - hmm .
Professor A: and roughly independent errors , and {vocalsound} it 's a good choice for {disfmarker}
PhD F: Mm - hmm . Mm - hmm . Yeah .
Professor A: Uh . Yeah , that 's a good idea .
PhD F: Yeah . Because , yeah , well , spectral subtraction is good and we could u we could use the fine structure to {disfmarker} to have a better estimate of the noise but {vocalsound} still there is this issue with spectral subtraction that it seems to increase the variance of {disfmarker} of {disfmarker} of
Professor A: Yeah .
PhD F: um Well it 's this musical noise which is annoying if you d you do some kind of on - line normalization after .
Professor A: Right .
PhD F: So . Um . Yeah . Well . Spectral subtraction and on - line normalization don't seem to {disfmarker} to go together very well . I
Professor A: Or if you do a spectral subtraction {disfmarker} do some spectral subtraction first and then do some on - line normalization then do some more spectral subtraction {disfmarker} I mean , maybe {disfmarker} maybe you can do it layers or something so it doesn't {disfmarker} doesn't hurt too much or something .
PhD F: Ah , yeah .
Professor A: But it {disfmarker} but uh , anyway I think I was sort of arguing against myself there by giving that example
PhD F: Yeah .
Professor A: uh I mean cuz I was already sort of {vocalsound} suggesting that we should be careful about not spending too much time on exactly what they 're doing In fact if you get {disfmarker} if you go into uh {disfmarker} a uh harmonics - related thing {vocalsound} it 's definitely going to be different than what they 're doing and uh uh
PhD F: Mm - hmm .
Professor A: should have some interesting properties in noise . Um . {vocalsound} I know that when have people have done {pause} um sort of the obvious thing of taking {vocalsound} uh your feature vector and adding {pause} in some variables which are {vocalsound} pitch related or uh that {disfmarker} it hasn't {disfmarker} my impression it hasn't particularly helped . Uh . Has not .
PhD F: It {disfmarker} it i has not ,
Professor A: Yeah .
PhD F: yeah .
Professor A: But I think uh {pause} that 's {disfmarker} that 's a question for this uh you know extending the feature vector versus having different streams .
PhD F: Oh . Was it nois noisy condition ? the example that you {disfmarker} you just
Professor A: And {disfmarker} and it may not have been noisy conditions .
PhD F: Yeah .
Professor A: Yeah . I {disfmarker} I don't remember the example but it was {disfmarker} {vocalsound} it was on some DARPA data and some years ago and so it probably wasn't , actually
PhD F: Mm - hmm . Mm - hmm . Yeah . But we were thinking , we discussed with Barry about this , and {vocalsound} perhaps {vocalsound} thinking {disfmarker} we were thinking about some kind of sheet cheating experiment where we would use TIMIT
Professor A: Uh - huh .
PhD F: and see if giving the d uh , this voicing bit would help in {disfmarker} in terms of uh frame classification .
Professor A: Why don't you {disfmarker} why don't you just do it with Aurora ?
PhD F: Mmm .
Professor A: Just any i in {disfmarker} in each {disfmarker} in each frame
PhD F: Yeah , but {disfmarker} but {disfmarker} B but we cannot do the cheating , this cheating thing .
Grad E: We 're {disfmarker}
Professor A: uh {disfmarker}
Grad E: We need labels .
Professor A: Why not ?
PhD F: Well . Cuz we don't have {disfmarker} Well , for Italian perhaps we have , but we don't have this labeling for Aurora . We just have a labeling with word models
Professor A: I see .
PhD F: but not for phonemes .
PhD D: Not for foreigners .
Grad E: we don't have frame {disfmarker} frame level transcriptions .
Professor A: Um .
PhD D: Right .
PhD F: Um . {vocalsound} Yeah .
Professor A: But you could {disfmarker} I mean you can {disfmarker} you can align so that {disfmarker} It 's not perfect , but if you {disfmarker} if you know what was said and {disfmarker}
PhD B: But the problem is that their models are all word level models . So there 's no phone models {pause} that you get alignments for .
PhD F: Mm - hmm .
Professor A: Oh .
PhD B: You {disfmarker} So you could find out where the word boundaries are but that 's about it .
Professor A: Yeah . I see .
Grad E: S But we could use uh the {disfmarker} the noisy version that TIMIT , which {vocalsound} you know , is similar to the {disfmarker} the noises found in the TI - digits {vocalsound} um portion of Aurora .
PhD F: Yeah . noise , yeah . Yeah , that 's right , yep . Mmm .
Professor A: Yeah .
PhD F: Well , I guess {disfmarker} I guess we can {disfmarker} we can say that it will help , but I don't know . If this voicing bit doesn't help , uh , I think we don't have to {disfmarker} to work more about this because {disfmarker}
Professor A: Uh .
PhD F: Uh . It 's just to know if it {disfmarker} how much i it will help
Professor A: Yeah .
PhD F: and to have an idea of how much we can gain .
Professor A: Right . I mean in experiments that we did a long time ago
PhD F: Mmm .
Professor A: and different ta it was probably Resource Management or something , um , I think you were getting {pause} something like still eight or nine percent error on the voicing , as I recall . And um , so um
Grad E: Another person 's voice .
Professor A: what that said is that , sort of , left to its own devices , like without the {disfmarker} a strong language model and so forth , that you would {disfmarker} {vocalsound} you would make significant number of errors {vocalsound} just with your uh probabilistic machinery in deciding
PhD B: It also {disfmarker}
Professor A: one oh
PhD B: Yeah , the {disfmarker} though I think uh there was one problem with that in that , you know , we used canonical mapping so {vocalsound} our truth may not have really been {pause} true to the acoustics .
Professor A: Uh - huh .
Grad E: Hmm .
PhD B: So .
PhD F: Mmm .
Professor A: Yeah . Well back twenty years ago when I did this voiced - unvoiced stuff , we were getting more like {vocalsound} ninety - seven or ninety - eight percent correct in voicing . But that was {vocalsound} speaker - dependent {vocalsound} actually . We were doing training {vocalsound} on a particular announcer
PhD F: Mm - hmm .
Professor A: and {disfmarker} and getting a {vocalsound} very good handle on the features .
PhD F: Mm - hmm .
Professor A: And we did this complex feature selection thing where we looked at all the different possible features one could have for voicing and {disfmarker} {vocalsound} and {disfmarker} and uh {disfmarker} and exhaustively searched {vocalsound} all size subsets and {disfmarker} and uh {disfmarker} for {disfmarker} for that particular speaker and you 'd find you know the five or six features which really did well on them .
PhD B: Wow !
PhD F: Mm - hmm .
Professor A: And then doing {disfmarker} doing all of that we could get down to two or three percent error . But that , again , was speaker - dependent with {vocalsound} lots of feature selection
PhD F: Mm - hmm .
Professor A: and a very complex sort of thing .
PhD F: Mmm .
Professor A: So I would {disfmarker} I would believe {vocalsound} that uh it was quite likely that um looking at envelope only , that we 'd be {vocalsound} significantly worse than that .
PhD F: Mm - hmm .
Professor A: Uh .
PhD F: And the {disfmarker} all the {disfmarker} the SpeechCorders ? what 's the idea behind ? Cuz they {disfmarker} they have to {disfmarker} Oh , they don't even have to detect voiced spe speech ?
Professor A: The modern ones don't do a {disfmarker} {vocalsound} a simple switch .
PhD F: They just work on the code book
Professor A: They work on the code book excitation .
PhD F: and find out the best excitation .
Professor A: Yeah they do {vocalsound} analysis - by - synthesis . They try {disfmarker} they {disfmarker} they try every {disfmarker} every possible excitation they have in their code book and find the one that matches best .
PhD F: Yeah . Mmm . Alright . Yeah . So it would not help .
Professor A: Yeah .
Grad E: Hmm .
Professor A: Uh . O K .
PhD B: Can I just mention one other interesting thing ?
Professor A: Yeah .
PhD B: Um . One of the ideas that we {pause} had come up with last week for things to try to {vocalsound} improve the system {disfmarker} Um . Actually I {disfmarker} I s we didn't {disfmarker} I guess I wrote this in after the meeting b but {vocalsound} the thought I had was um looking at the language model that 's used in the HTK recognizer , which is basically just a big {vocalsound} loop ,
Grad E: Mm - hmm .
PhD B: right ? So you {disfmarker} it goes " digit "
PhD D: Mm - hmm .
PhD B: and then that can be {disfmarker} either go to silence or go to another digit , which {disfmarker} That model would allow for the production of {vocalsound} infinitely long sequences of digits , right ?
Professor A: Right .
PhD B: So . I thought " well I 'm gonna just look at the {disfmarker} what actual digit strings do occur in the training data . "
Professor A: Right .
PhD B: And the interesting thing was it turns out that there are no sequences of two - long or three - long digit strings {pause} in any of the Aurora training data . So it 's either one , four , five , six , uh up to eleven , and then it skips and then there 's some at sixteen .
Professor A: But what about the testing data ?
PhD B: Um . I don't know . I didn't look at the test data yet .
Professor A: Yeah . I mean if there 's some testing data that has {disfmarker} has {disfmarker} {vocalsound} has two or three {disfmarker}
PhD B: So . Yeah . But I just thought that was a little odd , that there were no two or three long {disfmarker} Sorry . So I {disfmarker} I {disfmarker} just for the heck of it , I made a little grammar which um , you know , had it 's separate path {pause} for each length digit string you could get . So there was a one - long path and there was a four - long and a five - long
Professor A: Mm - hmm .
PhD B: and I tried that and it got way worse . There were lots of deletions .
Professor A: Mm - hmm .
PhD B: So it was {disfmarker} {vocalsound} you know , I {disfmarker} I didn't have any weights of these paths or {disfmarker} I didn't have anything like that .
Professor A: Mm - hmm .
PhD B: And I played with tweaking the {vocalsound} word transition penalties a bunch , but I couldn't go anywhere .
Professor A: Hmm .
PhD B: But um . I thought " well if I only allow {disfmarker} " Yeah , I guess I should have looked at {disfmarker} to see how often there was a mistake where a two - long or a three - long path was actually put out as a hypothesis . Um . But .
Professor A: Hmm .
PhD B: So to do that right you 'd probably want to have {disfmarker} {vocalsound} allow for them all but then have weightings and things . So . I just thought that was a interesting {vocalsound} thing about the data .
Professor A: OK . So we 're gonna read some more digit strings I guess ?
PhD B: Yeah . You want to go ahead , Morgan ?
Professor A: Sure .