Postdoc A: OK .
Grad G: How about channel
Professor C: Yeah , go ahead .
Grad E: We 're recording .
Grad G: Alright .
Professor C: Alright , and no crash .
Postdoc A: Hmm .
Grad E: I pre - crashed it .
Professor C: Yeah .
PhD F: Pre - crashed !
PhD D: It never crashes on me .
Grad E: I think it 's actually {disfmarker}
PhD D: What is {disfmarker} what is that ?
Grad E: it depends on if the temp files are there or not , that {disfmarker} at least that 's my current working hypothesis ,
PhD D: Ah .
Grad E: that I think what happens is it tries to clear the temp files and if they 're too big , it crashes .
PhD D: Ah .
PhD B: When the power went out the other day and I restarted it , it crashed the first time .
Grad E: Oh , that 's right .
PhD B: After the power out
PhD D: So then there would be no temp files .
Professor C: Yeah .
PhD D: OK . {comment} Hmm .
Grad E: Uh , no , it doesn't {disfmarker} it doesn't clear those necessarily ,
PhD D: Oh wait {disfmarker} It {disfmarker} it doesn't clear them , OK .
Grad E: so .
Professor C: Hmm , no connection .
Grad E: It 's {disfmarker} i they 're called temp files , but they 're not actually in the temp directory they 're in the scratch , so . They 're not backed up , but they 're not erased either on power failure .
PhD D: But that 's usually the meeting that I recorded , and it neve it doesn't crash on me .
PhD B: Well this wasn't {disfmarker} Actually , this wasn't a before your meeting , this was , um , Tuesday afternoon when , um , uh , Robert just wanted to do a little recording ,
Grad E: Oh well .
PhD D: Oh {disfmarker} Oh , right .
PhD B: and the power had gone out earlier in the day .
PhD D: OK . Huh , OK .
Professor C: I don't know when would be a good excuse for it , but I just can't wait to be giving a talk t and {disfmarker} and {disfmarker} and use the example from last week with everybody t doing the digits at once .
Grad E: Yeah .
Postdoc A: That was fun .
Professor C: I 'd love to play somebody that .
Postdoc A: That was fun .
PhD D: It was quick .
Professor C: It was . It was really efficient .
PhD B: Talk about a good noise shield . You know ? You wanted to pe keep people from listening in , you could like have that playing outside the room . Nobody could listen in .
Professor C: Yeah .
PhD D: Well , I had this idea we could make our whole meeting faster that way .
Professor C: Yeah . Everybody give the reports about what they were doing at exactly the same time ,
PhD D: And we 'll just all leave ,
PhD B: And then we 'll {disfmarker} we 'll go back later and review the individual channels ,
Professor C: yeah .
PhD D: and {disfmarker}
Grad E: Yep , and then everyone can listen to it later .
PhD B: right ?
Grad E: Yes . Absolutely .
PhD B: If you wanna know what {disfmarker}
Professor C: Actually isn't that what we have been doing ?
PhD D: Yeah .
Grad E: It 's what it sounds like .
PhD B: Practically , huh . With all the overlaps .
Postdoc A: Yeah .
Professor C: What are we doing ?
Grad E: I {disfmarker} Since I 've been gone all week , I didn't send out a reminder for an agenda , so .
Professor C: Yeah , and I 'm just {disfmarker}
Grad E: Do we have anything to talk about or should we just read digits and go ?
PhD B: I wouldn't mind hearing how the conference was .
Professor C: What conference ?
PhD D: Uh , I had one question about {disfmarker}
Grad E: Yeah , really . It 's all a blur .
PhD D: Aren't the UW folks coming this weekend ?
Grad E: Yep .
PhD F: No . The next ,
PhD D: Next weekend ?
Grad E: Next weekend , week from {disfmarker}
PhD F: right ?
Professor C: That is right . The next weekend .
PhD D: Sorry , not {disfmarker} not {disfmarker} not the days coming up , but {disfmarker}
PhD F: It 's like the {disfmarker}
Grad E: A week from Saturday .
PhD D: Yeah ,
Professor C: That 's when they 're coming .
PhD D: within ten days .
Professor C: That 's correct .
PhD D: So , are we {disfmarker} do we have like an agenda or anything that we should be {disfmarker}
Professor C: No , but that would be a good idea .
PhD D: OK .
Professor C: Why don't we w
PhD F: So {disfmarker} so the deal is that I can , um , {vocalsound} uh , I can be available after , uh , like ten thirty or something . I don't know how s how early you wanted to {disfmarker}
Professor C: They 're not even gonna be here until eleven or so .
Grad E: That 's good .
PhD F: Oh , OK . So {disfmarker}
Professor C: Cuz they 're flying up that day .
PhD D: Wait , this is on {disfmarker} on Sunday ?
Professor C: Saturday .
PhD D: Or Saturday ?
Professor C: Saturday .
PhD F: Saturday .
Professor C: S Saturday .
PhD D: OK .
Grad E: Well , y
Professor C: Yeah .
PhD F: Mm - hmm .
Grad E: Eurospeech is due on Friday and then I 'm going down to San {disfmarker} uh , San Jose Friday night , so , if {disfmarker} you know , if we start nice and late Saturday that 's a good thing .
Professor C: No , I mean , they 're flying up from {disfmarker} from {disfmarker}
Grad E: Seattle .
Professor C: down from Seattle .
Grad E: They 're flying from somewhere to somewhere ,
Professor C: Yeah , and they 'll end up here . So b and also Brian Kingsbury is actually flying from , uh , the east coast on that {disfmarker} that morning .
Postdoc A: Excellent .
Professor C: So , i I {disfmarker} I will be {disfmarker} I mean , he 's taking a very early flight
PhD F: Oh .
Professor C: and we do have the time work difference running the right way , but I still think that there 's no way we could start before eleven . It might end up really being twelve . So when we get closer we 'll find people 's plane schedules , and let everybody know . Uh , So . That 's good .
Grad E: But , uh , yeah maybe an agenda , or at least some things to talk about would be a good idea .
Professor C: Well we can start gathering those {disfmarker} those ideas , but then we {disfmarker} we should firm it up by next {disfmarker} next Thursday 's meeting .
Postdoc A: Will we have time to , um , to prepare something that we {disfmarker} in the format we were planning for the IBM transcribers by then , or {disfmarker} ?
Grad E: Oh yeah . Absolutely .
Postdoc A: OK .
Grad E: So have you heard back from Brian about that , Chuck ?
PhD B: Yes , um , he 's {disfmarker} I {disfmarker} I 'm sorry , I should have forwarded that along . Uh , {vocalsound} oh I {disfmarker} I think I mentioned at the last meeting , he said that , um , he talked to them and it was fine {disfmarker} with the beeps they would be {disfmarker} That 's easy for them to do .
Grad E: Great . OK . So , uh , oh , though Thi - Thilo isn't here , um , but , uh , I {disfmarker} I have the program to insert the beeps . What I don't have is something to parse the output of the channelized transcripts to find out where to put the beeps , but that should be really easy to do . So do we have a meeting that that 's been done with ,
Postdoc A: He 's {disfmarker} he 's {disfmarker}
Grad E: that we 've tightened it up to the point where we can actually give it to IBM and have them try it out ?
Postdoc A: He generated , um , a channel - wise presegmented version of a meeting , but it was Robustness rather than EDU so I guess depends on whether we 're willing to use Robustness ?
PhD B: Well for this experiment I think we can use pre pretty much anything .
Grad E: Mm - hmm .
Postdoc A: OK .
PhD B: This experiment of just {disfmarker}
Grad E: Well we had {disfmarker} we had talked about doing maybe EDU as a good choice , though . Well , {vocalsound} whatever we have .
PhD B: Well we 've talked about that as being the next ones we wanted to transcribe .
Grad E: Right .
Postdoc A: OK .
PhD B: But for the purpose of sending him a sample one to {disfmarker} f
Grad E: Yeah , maybe it doesn't matter .
Postdoc A: Great .
PhD B: I {disfmarker} I don't think it matte
Postdoc A: I 'll {disfmarker} I 'll {disfmarker} I 'll , um , get {disfmarker} make that available .
Grad E: OK , and has it been corrected ?
Postdoc A: Oh , well , wait . Um {disfmarker}
Grad E: Hand - checked ? Cuz that was one of the {vocalsound} processes we were talking about as well .
PhD B: Right , so we need to run Thilo 's thing on it ,
Postdoc A: That 's right .
PhD B: and then we go in and adjust the boundaries .
Postdoc A: Yeah that 's right . Yeah , we haven't done that . I {disfmarker} I could set someone on that tomorrow .
PhD B: Right .
Grad E: And time how long it takes .
PhD B: OK .
Postdoc A: I think they 're coming {disfmarker}
PhD B: And we probably don't have to do necessarily a whole meeting for that if we just wanna send them a sample to try .
Postdoc A: OK . What would be a good number of minutes ?
PhD B: I don't know , maybe we can figure out how long it 'll take @ @ to {disfmarker} to do .
Grad E: Um , I don't know , it seems to me w we probably should go ahead and do a whole meeting because we 'll have to transcribe the whole meeting anyway sometime .
Professor C: Yes except that if they had {disfmarker} if there was a choice between having fifteen minutes that was fully the way you wanted it , and having a whole meeting that didn't get at what you wanted for them {disfmarker} It 's just dependent of how much {disfmarker}
Grad E: Like I {disfmarker} I mean I guess if we have to do it again anyway , but , uh
Professor C: Yeah .
PhD B: I guess , the only thing I 'm not sure about is , um , how quickly can the transcribers scan over and fix the boundaries ,
Postdoc A: Mm - hmm .
PhD B: and {disfmarker} I mean , is it pretty easy ?
Grad E: I think it 's gonna be one or two times real time at {disfmarker} Wow , excuse me , two or more times real time , right ? Cuz they have to at least listen to it .
Professor C: Can we pipeline it so that say there 's , uh , the transcriber gets done with a quarter of the meeting and then we {disfmarker} you run it through this other {disfmarker} other stuff ? Uh ,
Grad E: Well the other stuff is I B I 'm just thinking that from a data {disfmarker} keeping - track - of - the - data point of view , it may be best to send them whole meetings at a time and not try to send them bits and pieces .
Professor C: OK , so . Oh , that 's right . So the first thing is the automatic thing , and then it 's {disfmarker} then it 's {disfmarker} then it 's the transcribers tightening stuff up ,
Grad E: Right .
Postdoc A: Mm - hmm .
Professor C: and then it 's IBM .
Postdoc A: Mm - hmm , mm - hmm .
Grad E: Right .
Professor C: OK , so you might as well ha run the automatic thing over the entire meeting , and then {disfmarker} and then , uh , you would give IBM whatever was fixed .
Postdoc A: And have them fix it over the entire meeting too ?
Grad E: Right .
Professor C: Well , yeah , but start from the beginning and go to the end , right ? So if they were only half way through then that 's what you 'd give IBM .
Postdoc A: OK .
Professor C: Right ?
PhD B: As of what point ? I mean . The {disfmarker} I guess the question on my mind is do we wait for the transcribers to adjust the marks for the whole meeting before we give anything to IBM , or do we go ahead and send them a sample ? Let their {disfmarker}
Professor C: Why wouldn't we s @ @ w i if they were going sequentially through it , why wouldn't we give them {disfmarker} I mean i are we trying to get something done by the time Brian comes ?
PhD B: Well I {disfmarker} I {disfmarker} I mean , I don't know .
Grad E: That was the question . Though .
Professor C: So if we {disfmarker} if we were , then it seems like giving them something , whatever they had gotten up to , would be better than nothing .
PhD B: Yeah . Uh . That {disfmarker} I agree . I agree .
Grad E: Well , I don't think {disfmarker} I mean , h they {disfmarker} they typically work for what , four hours , something like that ?
Postdoc A: Hmm , I gue hmm .
Grad E: I think the they should be able to get through a whole meeting in one sitting . I would think , unless it 's a lot harder than we think it is , which it could be , certainly .
Postdoc A: If it 's got like for speakers then I guess {disfmarker} I mean if {disfmarker}
PhD B: We 're just doing the individual channels ,
Grad E: Or seven or eight .
PhD B: right ?
Postdoc A: Individual channels . Yeah .
PhD B: So it 's gonna be , depending on the number of people in the meeting , um ,
Postdoc A: I guess there is this issue of , you know , if {disfmarker} if the segmenter thought there was no speech on {disfmarker} on a particular stretch , on a particular channel ,
Grad E: Well {disfmarker}
Postdoc A: and there really was , then , if it didn't show up in a mixed signal to verify , then it might be overlooked , so , I mean , the question is " should {disfmarker} should a transcriber listen to the entire thing or can it g can it be based on the mixed signal ? " And I th eh so far as I 'm concerned it 's fine to base it on the mixed signal at this point , and {disfmarker}
Grad E: That 's what it seems to me too , in that if they need to , just like in the other cases , they can listen to the individual , if they need to .
Postdoc A: And that cuts down the time . Yeah .
Grad E: But they don't have to for most of it .
Postdoc A: Yeah , that 's good . So . Yeah . Good , good , good .
PhD B: I don't see how that will work , though .
Postdoc A: What {disfmarker} what aspect ?
Professor C: So you 're talking about tightening up time boundaries ?
PhD B: Yeah .
Professor C: So how do you {disfmarker}
Grad E: So , they have the normal channeltrans interface where they have each individual speaker has their own line ,
PhD B: Yeah .
Grad E: but you 're listening to the mixed signal and you 're tightening the boundaries , correcting the boundaries . You shouldn't have to tighten them too much because Thilo 's program does that .
Postdoc A: Should be pretty good , yeah .
PhD D: Except for {vocalsound} it doesn't do well on short things , remember .
Grad E: Right , so {disfmarker} so you 'll have to I {disfmarker}
PhD D: It will miss them . It will miss most of the really short things .
Grad E: Uh - huh .
PhD D: Like that .
Postdoc A: But those would be {disfmarker} those would be {disfmarker}
PhD D: Uh - huh . It will {disfmarker} it will miss {disfmarker}
Grad E: Uh - huh !
PhD D: Yeah , you have to say " uh - huh " more slowly to {disfmarker} to get c
Grad E: Sorry .
PhD D: No , I 'm s I 'm actually serious .
Grad E: I 'll work on that .
PhD D: So it will miss stuff like that which {disfmarker}
PhD B: I {disfmarker}
Grad E: Well , so {disfmarker} so that 's something that the transcribers will have to {disfmarker} have to do .
Postdoc A: Yeah , but presumably , most of those they should be able to hear from the mixed signal unless they 're embedded in the heavil heavy overlap section when {disfmarker} in which case they 'd be listening to the channels anyway .
PhD B: That 's {disfmarker} that 's what I 'm {disfmarker} I 'm concerned about the part .
PhD D: Right , and that 's what I 'm not sure about .
Postdoc A: Yeah , I am too . And I think it 's an empirical question .
PhD B: Can't we {disfmarker} uh couldn't we just have , um , I don't know , maybe this just doesn't fit with the software , but I guess if I didn't know anything about Transcriber and I was gonna make something to let them adjust boundaries , I would just show them one channel at a time , with the marks , and let them adju
Postdoc A: Oh they can {disfmarker}
Grad E: Well , but then they have to do {disfmarker} but then they {disfmarker} for this meeting they would have to do seven times real time , and it would probably be more than that .
Postdoc A: Yeah , that 's it . Yeah .
Grad E: Right ? Because they 'd have to at least listen to each channel all the way through .
Postdoc A: And if {disfmarker}
PhD B: But i but it 's very quick ,
Postdoc A: Uh - huh .
PhD B: right ? I mean , you scan {disfmarker} I mean , if you have a display of the waveform .
Postdoc A: Yeah .
Grad E: Oh , you 're talking about visually .
Postdoc A: w Well , the other problem is the breaths
Grad E: I just don't think {disfmarker}
Postdoc A: cuz you also see the breaths on the waveform . I 've {disfmarker} I 've looked at the int uh , s I 've tried to do that with a single channel , and {disfmarker} and you do see all sorts of other stuff besides just the voice .
PhD B: Uh - huh .
Grad E: Yeah , and I {disfmarker} I think that they 're going much more on acoustics than they are on visuals .
Postdoc A: Well that {disfmarker} that I 'm not sure .
Grad E: So .
Postdoc A: What you {disfmarker} the digital {disfmarker} what the digital task that you had your interface ? Um , I know for a fact that one of those {disfmarker} sh she could really well {disfmarker} she could judge what th what the number was based on the {disfmarker} on the waveform .
Grad E: Yeah , that 's actually true . Yeah , you 're right . You 're absolutely right . Yeah , I found the same thing that when I was scanning through the wave form {vocalsound} I could see when someone started to read digits just by the shapes .
Postdoc A: Yeah , she could tell which one was seven .
Grad E: Um , maybe .
Postdoc A: Yeah .
Professor C: So I don't {disfmarker} I 'm {disfmarker} I 'm now entirely confused about what they do .
Grad E: But {disfmarker}
Professor C: So , they 're {disfmarker} they 're looking at a mixed signal , or they 're looking {disfmarker} what {disfmarker} what are they looking at visually ?
Postdoc A: Well , they have a choice . They could choose any signal to look at . I 've tried lookin but usually they look at the mixed . But I 've {disfmarker} I 've tried looking at the single signal and {disfmarker} and in order to judge when it {disfmarker} when it was speech and when it wasn't ,
Grad E: Oh .
Postdoc A: but the problem is then you have breaths which {disfmarker} which show up on the signal .
Professor C: But the procedure that you 're imagining , I mean , people vary from this , is that they have the mixed signal wave form in front of them ,
Postdoc A: Yes .
PhD F:
Postdoc A: Yes .
Professor C: and they have multiple , uh , well , let 's see , there isn't {disfmarker} we don't have transcription yet . So {disfmarker} but there 's markers of some sort that have been happening automatically ,
Postdoc A: Yes .
Grad E: Right .
Professor C: and those show up on the mixed signal ?
Postdoc A: Oh ,
Professor C: There 's a @ @ clicks ?
Grad E: N the t
Postdoc A: they show up on the separate ribbons . So you have a separate ribbon for each channel ,
Professor C: There 're separate ribbons .
Grad E: Right .
Postdoc A: and {disfmarker} and i i it 'll be {disfmarker} because it 's being segmented as channel at a time with his {disfmarker} with Thilo 's new procedure , then you don't have the correspondence of the times across the bins {disfmarker} uh across the ribbons uh you could have {disfmarker}
Professor C: And is there a line moving across the waveform as it goes ?
Grad E: Yes .
Postdoc A: Yes .
Professor C: OK , so The way you 're imaging is they kind of play it , and they see oh this happened , then this happened , then {disfmarker} and if it 's about right , they just sort of let it slide ,
Postdoc A: Yeah .
Grad E: Right . Right .
Professor C: and if it {disfmarker} if it {disfmarker} there 's a question on something , they stop and maybe look at the individual wave form .
Postdoc A: Oh , well not {disfmarker} not " look " .
Grad E: Right . Well , they wouldn't look at it {pause} at this point . They would just listen .
Professor C: They {disfmarker} they might look at it , right ?
Grad E: Well , the problem is that the {disfmarker} the interface doesn't really allow you to switch visuals .
Postdoc A: Not very quickly .
Grad E: The problem is that {disfmarker} that {disfmarker} the Tcl - TK interface with the visuals , it 's very slow to load waveforms .
Postdoc A: You can but it takes time . That 's it .
Professor C: Uh - huh .
Grad E: And so when I tried {disfmarker} that {disfmarker} that was the first thing I tried when I first started it ,
Postdoc A: Oh , oh . Visually . You can {disfmarker} you can switch quickly between the audio ,
Grad E: right ?
Postdoc A: but you just can't get the visual display to show quickly . So you have to {disfmarker} It takes , I don't know , three , four minutes to {disfmarker} Well , I mean , it takes {disfmarker} it takes long enough {disfmarker}
PhD D: Yeah , it 's very slow to do that .
Postdoc A: It takes long enough cuz it has to reload the I {disfmarker} I don't know exactly what it 's doing frankly cuz {disfmarker} but it t it takes long enough that it 's just not a practical alternative .
PhD D: That w
Grad E: Well it {disfmarker} it does some sort of shape pre - computation so that it can then scroll it quickly ,
Grad G: But you can cancel that .
PhD D: Yeah .
Grad E: yeah . But then you can't change the resolution or scroll quickly .
Grad G: Oh , really ?
Postdoc A: Now you could set up multiple windows , each one with a different signal showing , and then look between the windows .
Grad E: So .
Grad G: Huh !
Postdoc A: Maybe that 's the solution .
Grad E: I mean , we {disfmarker} we could do different interfaces ,
Grad G: What if you preload them all ?
Grad E: right ? I mean , so {disfmarker} so we could use like X Waves instead of Transcriber ,
Postdoc A: Yeah .
Grad E: and it loads faster , certainly .
Grad G: What if you were to preload all the channels or {disfmarker} or initially {disfmarker}
Grad E: Well that 's what I tried originally .
Grad G: like doesn't {disfmarker}
Grad E: So I {disfmarker} I actually before , uh , Dave Gelbart did this , I did an interface which showed each waveform and ea a ribbon for each waveform ,
Grad G: Mm - hmm .
Grad E: but the problem with it is even with just three waveforms it was just painfully slow to scroll . So you just scroll a screen and it would , you know go " kur - chunk ! "
Grad G: Oh , OK .
Postdoc A: Mm - hmm .
Grad E: And so it just was not doable with the current interface .
Postdoc A: You know , I am thinking if we have a meeting with only four speakers and , you know , you could fire up a Transcriber interface for , y you know , in different windows , multiple ones , one for each channel . And it 's sort of a {disfmarker} a hack but I mean it would be one way of seeing the visual form .
Grad E: I think that if we decide that we need {disfmarker} that they need to see the visuals , we need to change the interface so that they can do that .
Postdoc A: Yeah . Yeah .
Professor C: So {disfmarker}
PhD D: That 's actually what I thought of , loading the chopped up waveforms , I mean , you know , that {disfmarker} that would make it faster {disfmarker}
Grad E: An But isn't {disfmarker}
Grad G: Hmm .
Grad E: The chopped up waveforms .
PhD B: The problem is if {disfmarker} if anything 's cut off , you can't expand it from the chopped up {disfmarker}
PhD D: So .
Grad E: Isn't that {disfmarker}
Grad G: Right .
PhD D: Right , but if you a at some point {disfmarker}
Grad E: And wouldn't that be the same {comment} as the mixed signal ?
PhD D: No , I mean the individual channels that were chopped up that {disfmarker} it 'd be nice to be able to go back and forth between those short segments .
Professor C: Mm - hmm .
PhD D: Cuz you don't really nee like nine tenths of the time you 're throwing most of them out , but what you need are tho that particular channel , or that particular location ,
Grad E: Yeah .
PhD D: and ,
Postdoc A: Yeah .
PhD D: um , might be nice , cuz we save those out already , {comment} um , to be able to do that . But it won't work for IBM of course , it only works here cuz they 're not saving out the individual channels .
Postdoc A: Well , I {disfmarker} I do think that this {disfmarker} this will be a doable procedure ,
Professor C: Yeah .
Postdoc A: and have them starting with mixed
Professor C: OK .
Postdoc A: and , um , then when they get into overlaps , just have them systematically check all the channels to be sure that there isn't something hidden from {disfmarker} from audio view .
Grad E: Yeah . Yeah , hopefully , I mean {disfmarker} The mixed signal , the overlaps are pretty audible because it is volume equalized . So I think they should be able to hear . The only problem is {disfmarker} is , you know , counting how many and if they 're really correct or not . So , I don't know .
PhD D: I don't know that you can locate them very well from the mixed signal ,
Grad E: Right but {disfmarker} but once {disfmarker} once you know that they happen , you can at least listen to the close talking ,
PhD D: but you would know that they were there , and then you would switch . Right . And then you would switch into the other {disfmarker}
Grad E: so .
Professor C: But right now , to do this limitation , the switching is going to be switching of the audio ? Is what she 's saying .
Postdoc A: Yeah .
Grad E: Right .
Professor C: So {disfmarker}
Grad E: Right , so {disfmarker} so
Professor C: so they 're using their ears to do these markings anyway .
Grad E: did Dave {disfmarker} Did Dave do that change where you can actually just click rather than having to go up to the menu to listen to the individual channels ?
Postdoc A: Yes . Yes .
PhD D: Yeah .
Postdoc A: Click {disfmarker} Um ,
Grad E: I had suggested it before . I just don't know whether he did it or not .
Postdoc A: I 'm not sure what {disfmarker} click what {disfmarker} click on the ribbon ? Yeah , you can get that {disfmarker}
Grad E: Yeah .
Postdoc A: oh , oh , get {disfmarker} you can get the , uh {disfmarker} you can get it to switch audio ? Uh , not last I tried ,
Grad E: Yeah .
Postdoc A: but , um , maybe he 's changed it again .
Grad E: We should get him to do that because , uh , I think that would be much , much faster than going to the menu .
Postdoc A: I disagree . There 's a reason I disagree , and that is that , uh , you {disfmarker} it 's very good to have a dissociation between the visual and the audio . There 're times when I wanna hear the mixed signal , bu but I want to transcribe on the single channel . So right now {disfmarker}
Grad E: Then maybe just buttons down at the bottom next to it .
Postdoc A: Maybe , I just don't {disfmarker} I don't see that it 's a {disfmarker}
Grad E: Just something so that it 's not in the menu option so that you can do it much faster .
Postdoc A: Well , I mean , that 's the i I {disfmarker} I think that might be a personal style thing . I find it really convenient the way {disfmarker} the way it 's set up right now .
Grad E: Well it just seems to me that if you wanna quickly {disfmarker} " well was that Jane , no , was that Chuck , no , was that Morgan " , right now , you have to go up to the menu , and each time , go up to the menu , select it , listen to that channel then click below , and then go back to the menu , select the next one , and then click below .
Postdoc A: That 's fine . Yeah , it 's true .
Grad E: So you can definitely streamline that with the i with the interface .
Postdoc A: Yeah , it could be faster , but , you know , I mean , th in the ideal world {disfmarker} Yeah .
Grad E: What ?
Postdoc A: No I {disfmarker} I agree that 'd be nice . Yeah . OK .
Grad E: OK .
Professor C: So , um , Done with that ?  Does any {disfmarker} I forget , does anybody , uh , working on any {disfmarker} any Eurospeech submission related to this ?
Grad E: I would like to try to do something on digits but I just don't know if we have time . I mean , it 's due next Friday so we have to do the experiments and write the paper . So , I 'm gonna try , but , uh , we 'll just have to see . So actually I wanna get together with both Andreas and , uh , uh , Stephane with their respective systems .
Professor C: Yeah . Yeah there was that {disfmarker} we that 's right , we had that one conversation about , uh , what {disfmarker} what {disfmarker} what did it mean for , uh , one of those speakers to be pathological , was it a {disfmarker}
Grad E: Right , and I haven't had s chance to sit down and listen .
PhD F: Oh , I haven't {disfmarker} I haven't listened to them either ,
Grad E: I was going to do that this afternoon .
PhD F: but there must be something wrong , I mean ,
Grad E: Well , Morgan and I were {disfmarker} were having a debate {comment} about that .
PhD F: unless our {disfmarker}
Grad E: Whereas I think it it 's probably something pathologic and actually Stephane 's results , I think confirm that . He s he did the Aurora system also got very lousy average error , like fifteen or {disfmarker} or , uh , fifteen to twenty percent average ? But then he ran it just on the lapel , and got about five or six percent word error ? So that {disfmarker} that means to me that somewhere in the other recordings there are some pathological cases . But , you know , we {disfmarker} th that may not be true . It may be just some of the segments they 're just doing a lousy job on . So I 'll {disfmarker} I 'll listen to it and find out since you 'd actually split it up by segment .
Professor C: Right .
Grad E: So I can actually listen to it .
PhD F: Yeah .
PhD B: Did you run the {disfmarker} Andreas {disfmarker} the r SRI recognizer on the digits ?
Grad E: Oh , I thought he had sent that around to everyone ,
PhD F: Yeah .
Grad E: did you just sent that to me ?
PhD F: No , I d I didn't .
Grad E: Oh .
PhD F: Since I considered those preliminary , I didn't .
PhD B: I it wasn't {disfmarker}
PhD F: But , yeah , if you take {disfmarker}
Grad E: It was bimodal .
PhD F: So if you {disfmarker} Yeah , it 's actually , um , it {disfmarker} uh {disfmarker} it was trimodal , actually {disfmarker}
Grad E: Oh , was it trimodal , OK .
Professor C: Yeah .
PhD F: trimodal , so
Professor C: There 's zero , a little bit , and a lot .
PhD F: there were {disfmarker} {vocalsound} t there was {disfmarker} there was one h one bump at ze around zero , which were the native speakers ,
Professor C: Yeah .
PhD B: Zero percent error ?
Professor C: Yeah .
PhD F: the non - pathological native speakers .
Professor C: Y yeah .
PhD F: Then there was another bump at , um , {vocalsound} {vocalsound} oh , like fifteen or something .
PhD B: This is error you 're talking about ?
Professor C: Oh was it fifteen ?
PhD F: whe
PhD B: OK .
Professor C: Yeah .
Grad E: Yeah .
PhD F: Yeah . Those were the non - natives . And then there was another distinct bump at , like , a hundred , {vocalsound} which must have been some problem .
Postdoc A: Oh , wow ! Oh , OK .
PhD F: I can't imagine that {disfmarker}
Grad G: What is patho what do you mean by pathological ?
Grad E: Just {disfmarker} just something really wrong with {disfmarker}
Grad G: I 'm sorry , I don't {disfmarker}
Grad E: A bug is what I mean ,
PhD F: In the recording
Grad G: Oh .
Grad E: so that it 's like {disfmarker}
Grad G: Oh , OK .
PhD F: And there was this one meeting , I forget which one it was , where like , uh , six out of the eight channels were all , like {disfmarker} had a hundred percent error .
Grad G: I see .
Grad E: Which probably means like there was a {disfmarker} th the recording interface crashed ,
Grad G: Right .
Grad E: or there was a short {disfmarker} you know , someone was jiggling with a cord
PhD F: But {disfmarker}
Grad E: or , uh , I extracted it incorrectly ,
PhD F: But {disfmarker}
Grad E: it was labeled {disfmarker}
Grad G: Mm - hmm .
Grad E: it was transcribed incorrectly , something really bad happened , and I just haven't listened to it yet to find out what it was .
Grad G: OK .
PhD F: So , if I excluded the pathological ones , {vocalsound} by definition , those that had like over ninety - five percent error rate , {vocalsound} and the non - natives , then the average error rate was like one point four or something ,
Professor C: What we 're calling .
Postdoc A: Oh . Oh .
PhD F: which {disfmarker} which seemed reasonable given that , you know , the models weren't tuned for {disfmarker} {vocalsound} for it .
Grad G: Hmm !
Professor C: Yeah .
PhD F: And the grammar wasn't tuned either .
PhD B: And it didn't matter whether it was the lapel or whether it was the {disfmarker}
PhD F: It was just a @ @ . I haven't split it up that way ,
PhD D: But there 's no overlap during the digit readings , so it shouldn't really matter .
PhD F: but it would be {disfmarker}
PhD B: Yeah .
PhD F: Right .
Grad G: Right .
Professor C: No , but there 's a little difference ,
PhD F: So it should {disfmarker}
Grad E: There 's a lot .
Professor C: and we haven't looked at it for digits ,
Grad E: Yeah .
Professor C: right ?
PhD B: Yeah , so I was curious about that .
Professor C: And so , cuz {disfmarker} because what he was {disfmarker} what I was saying when I looked at those things is it {disfmarker} it {disfmarker} I was almost gonna call it quadrimodal because {vocalsound} {disfmarker} because there was a whole lot of cases where it was zero percent .
PhD F: Mm - hmm .
Professor C: They just plain got it all right . And then there {disfmarker} and then there was another bunch that were couple percent or something .
PhD F: Yeah . But if you p if you actually histogrammed it , and {disfmarker} it was a nice {disfmarker} uh , you know , it {disfmarker} it was {disfmarker} zero was the most of them ,
Professor C: Yeah .
Grad E: A normal . Yeah .
PhD F: but then there were {disfmarker} the others were sort of decaying from there .
Professor C: Yeah , yeah .
PhD F: And then there was the bump for the non - natives and then the pathological ones ,
Professor C: I see . I see .
PhD F: so .
Grad E: Yeah , cuz some of our non - natives are pretty non - native . So .
Postdoc A: You {disfmarker} did you have , uh , something in the report about , uh , {disfmarker} about , uh , for f uh , forced alignment ?
Professor C: Yeah .
Postdoc A: Have you {disfmarker} have you started on that ?
PhD F: Oh , well , yeah , so I 've been struggling with the forced alignments . Um . {vocalsound} {vocalsound} So the scheme that I drew on the board last time where we tried to , um {vocalsound} allow reject models for the s speech from other speakers , um , {vocalsound} {vocalsound} most of the time it doesn't work very well . So , {vocalsound} um , {vocalsound} and the {disfmarker} I haven't done {disfmarker} I mean , the only way to check this right now was for me to actually {vocalsound} load these into X Waves and , you know , plus the alignments , and s play them and see where the {disfmarker}
Professor C: Hmm .
PhD F: And it looks {disfmarker} And so I looked at all of the utterances from you , Chuck , in that one conversation , I don't know which {disfmarker} You probably know which one I mean , it 's where you were on the lapel {vocalsound} and Morgan was sitting next to you and we can hear everything Morgan says .
Postdoc A: Hmm .
PhD F: But {disfmarker} and {disfmarker} and some of what you {disfmarker} I mean , you also appear quite a bit in that cross - talk . So , {vocalsound} I actually went through all of those , there were I think fifty - five segments , {vocalsound} um , in {disfmarker} in X Waves , and {disfmarker} and sort of did a crude check , and {vocalsound} more often than not , it {disfmarker} it gets it wrong . So there 's either the beginning , mostly the beginning word , {vocalsound} where th you , um , you know , Chuck talks somewhere into the segment , but the first , um , word of what he says , often " I " but it 's very reduced " I , " that 's just aligned {vocalsound} to the beginning of someone else 's speech , uh in that segment , which is cross - talk . So , {vocalsound} um , {vocalsound} I 'm still tinkering with it , but it might well be that we can't get clean alignments out of this {disfmarker} out of those , uh , {vocalsound} channels , so .
Professor C: Unless maybe we do this , uh , um , cancellation business .
PhD D: Right , but that 's {disfmarker} I mean , that was our plan ,
PhD F: Yeah , right .
PhD D: but it 's clear from Dan that this is not something you can do in a short amount of time .
Professor C: Oh , the short amount of time thing , right .
PhD D: So {disfmarker} so we {disfmarker} you know , we had spent a lot of time , um , writing up the HLT paper and we wanted to use that , uh , kind of analysis ,
Professor C: Yeah .
PhD D: but the HLT paper has , you know , it 's a very crude measure of overlap . It 's not really something you could scientifically say is overlap , it 's just whether or not the , um , the segments that were all synchronized , whether there was some overlap somewhere .
Grad E: c High correlation .
PhD D: And , you know , that pointed out some differences , so he thought well if we can do something quick and dirty because Dan said the cross - cancellation , it 's not straight - forward . If it were straight - forward then we would try it , but {disfmarker} so , it 's sort of good to hear that it was not straight - forward , thinking if we can get decent forced alignments , then at least we can do sort of a overall report of what happens with actual overlap in time , but , um {disfmarker}
PhD B: I didn't think that his message said it wasn't straight - forward .
Grad E: Well if we 'd just {disfmarker}
Professor C: Well
PhD B: I thought he 's just saying you have to look over a longer time window when you do it .
Grad E: Um - hmm .
PhD D: and the {disfmarker} but there are some issues of this timing , um , in the recordings
Professor C: Yeah .
PhD B: Right .
PhD D: and {disfmarker}
PhD B: So you just have to look over longer time when you 're trying to align the things , you can't {disfmarker} you can't just look {disfmarker}
Grad E: Well . are you talking about the fact that the recording software doesn't do time - synchronous ? Is that what you 're referring to ?
Professor C:
Grad E: That seems to me you can do that over the entire file and get a very accurate {disfmarker}
PhD F: I don't thi I d I don't think that was the issue .
PhD D: I {disfmarker} yeah , that was sort of a side issue .
Grad E: I didn't think so either .
PhD F: The issue was that you have {disfmarker} to {disfmarker} you have have {disfmarker} you first have to have a pretty good speech detection on the individual channels .
PhD D: And it 's dynamic , so I guess it was more dynamic than some simple models would be able t to {disfmarker} so {disfmarker} so there are some things available , and I don't know too much about this area where if people aren't moving around much than you could apply them , and it should work pretty well if you took care of this recording time difference .
Grad E: Right , which should be pretty straight forward .
PhD D: Which a at least is well defined , and
Grad E: Yeah .
PhD D: um , but then if you add the dynamic aspect of adapting distances , then it wasn't {disfmarker} I guess it just wasn't something that he could do quickly {pause} and not {disfmarker} in time for us to be able to do something by two weeks from now , so . Well less than a week . So {disfmarker} um , so I don't know what we can do if anything , that 's sort of worth , you know , a Eurospeech paper at this point .
PhD B: Well , Andreas , how well did it work on the non - lapel stuff ?
Grad E: Yeah . That 's what I was gonna say .
PhD F: I haven't checked those yet .
Grad E: C
PhD F: It 's very tedious to check these .
PhD B: Mmm .
PhD F: Um , we would really need , ideally , a transcriber {vocalsound} to time mark the {disfmarker} you know , the be at least the beginning and s ends {comment} of contiguous speech . Um , {vocalsound} {vocalsound} and , you know , then with the time marks , you can do an automatic comparison of your {disfmarker} of your forced alignments .
PhD B: Because {disfmarker} really the {disfmarker} the {disfmarker} at least in terms of how we were gonna use this in our system was to get an ideal {disfmarker} an idea , uh , for each channel about the start and end boundaries .
Grad E: Oh , MNCM .
PhD F: Mm - hmm .
PhD B: We don't really care about like intermediate word boundaries , so {disfmarker}
PhD F: No , that 's how I 've been looking at it .
PhD B: Yeah .
PhD D: Right .
PhD F: I mean , I don't care that the individual words are aligned correctly ,
PhD B: Yeah .
PhD F: but {vocalsound} you don't wanna , uh , infer from the alignment that someone spoke who didn't .
PhD B: Right , exactly . So that 's why I was wondering if it {disfmarker}
PhD F: so , so {disfmarker}
PhD B: I mean , maybe if it doesn't work for lapel stuff , we can just not use that
PhD F: Yeah .
PhD B: and {disfmarker}
PhD F: I haven't {disfmarker} I ha just haven't had the time to , um , do the same procedure on one of the {disfmarker} so I would need a k I would need a channel that has {vocalsound} a speaker whose {disfmarker} who has a lot of overlap but s you know , is a non - lapel mike . And , um , {vocalsound} {vocalsound} where preferably , also there 's someone sitting next to them who talks a lot .
Grad E: Hmm !
PhD F: So , I {disfmarker}
Grad E: So a meeting with me in it .
PhD F: maybe someone can help me find a good candidate and then I would be willing to
PhD B: We c you know what ? Maybe the best way to find that would be to look through these .
PhD F: you know , hand
PhD B: Cuz you can see the seat numbers , and then you can see what type of mike they were using . And so we just look for , you know , somebody sitting next to Adam at one of the meetings {disfmarker}
PhD D: Actually y we can tell from the data that we have ,
PhD F: From the insertions , maybe ?
PhD D: um , yeah , there 's a way to tell .
PhD F: fr fr from the {disfmarker}
PhD D: It might not be a single person who 's always overlapping that person but any number of people ,
PhD F: Right .
PhD D: and , um , if you align the two hypothesis files across the channels , you know , just word alignment , you 'd be able to find that . So {disfmarker} so I guess that 's sort of a last {disfmarker} ther there 're sort of a few things we could do . One is just do like non - lapels if we can get good enough alignments . Another one was to try to get {disfmarker} somehow align Thilo 's energy segmentations with what we have . But then you have the problem of not knowing where the words are because these meetings were done before that segmentation . But maybe there 's something that could be done .
PhD B: What {disfmarker} what is {disfmarker} why do you need the , um , the forced alignment for the HLT {disfmarker} I mean for the Eurospeech paper ?
PhD D: Well , I guess I {disfmarker} I wanted to just do something not on recognition experiments because that 's ju way too early , but to be able to report , you know , actual numbers . Like if we {disfmarker} if we had hand - transcribed pe good alignments or hand - checked alignments , then we could do this paper . It 's not that we need it to be automatic . But without knowing where the real words are , in time {disfmarker}
PhD B: So it was to get {disfmarker} it was to get more data and better {disfmarker} to {disfmarker} to squeeze the boundaries in .
PhD D: To {disfmarker} to know what an overlap really {disfmarker} if it 's really an overlap , or if it 's just a {disfmarker} a {disfmarker} a segment correlated with an overlap ,
PhD B: Ah , OK . Yeah .
PhD D: and I guess that 's the difference to me between like a real paper and a sort of , promissory paper . So , um , if we d it might be possible to take Thilo 's output and like if you have , um , like right now these meetings are all ,
Grad E: Ugh ! I forgot the digital camera again .
PhD D: um ,
Grad E: Every meeting !
PhD D: you know , they 're time - aligned , so if these are two different channels and somebody 's talking here and somebody else is talking here , just that word , if Thilo can tell us that there 're boundaries here , we should be able to figure that out
Grad E: Mm - hmm .
PhD D: because the only thing transcribed in this channel is this word . But , um , you know , if there are things {disfmarker}
Grad E: Two words .
PhD D: Yeah , if you have two and they 're at the edges , it 's like here and here , and there 's speech here , then it doesn't really help you , so , um {disfmarker}
PhD B: Thilo 's won't put down two separate marks in that case {disfmarker}
PhD D: Well it w it would , but , um , we don't know exactly where the words are because the transcriber gave us two words in this time bin
Grad E: Thilo 's will . But .
PhD D: and we don't really know , I mean ,
Postdoc A: Well it 's a merging problem . If you had a {disfmarker} if you had a s if you had a script which would {disfmarker}
PhD D: yeah it 's {disfmarker}
Postdoc A: I 've thought about this , um , and I 've discussed {disfmarker} I 've discussed it with Thilo ,
PhD D: I mean , if you have any ideas . I would {disfmarker}
Postdoc A: um , the , I mean , I {disfmarker} I {disfmarker} in principle I could imagine writing a script which would approximate it to some degree , but there is this problem of slippage ,
Grad E: Well maybe {disfmarker} Maybe that will get enough of the cases to be useful .
Postdoc A: yeah .
PhD D: Right . I mean , that {disfmarker} that would be really helpful . That was sort of another possibility .
Grad E: You know s cuz it seemed like most of the cases are in fact the single word sorts , or at least a single phrase
Postdoc A: Well they {disfmarker} they can be stretched .
Grad E: in most of the bins .
PhD F: Mmm .
Postdoc A: I wouldn't make that generalization cuz sometimes people will say , " And then I " and there 's a long pause
PhD D: Yeah .
Postdoc A: and finish the sentence and {disfmarker} and sometimes it looks coherent and {disfmarker} and the {disfmarker} I mean it 's {disfmarker} it 's not a simple problem . But it 's really {disfmarker} And then it 's coupled with the problem that sometimes , you know , with {disfmarker} with a fricative you might get the beginning of the word cut off and so it 's coupled with the problem that Thilo 's isn't perfect either . I mean , we 've i th it 's like you have a merging problem plus {disfmarker} so merging plus this problem of , uh , not {disfmarker}
Grad E: Right . Hmm !
Postdoc A: y i i if the speech - nonspeech were perfect to begin with , the detector , that would already be an improvement , but that 's impossible , you know , i that 's too much to ask .
PhD D: Right .
Grad E: Yes .
Postdoc A: And so i and may you know , I mean , it 's {disfmarker} I think that there always {disfmarker} th there would have to be some hand - tweaking , but it 's possible that a script could be written to merge those two types of things . I 've {disfmarker} I 've discussed it with Thilo and I mean {disfmarker} in terms of not him doing it , but we {disfmarker} we discussed some of the parameters of that and how hard it would be to {disfmarker} in principle {disfmarker} to write something that would do that .
PhD D: I mean , I guess in the future it won't be as much as an issue if transcribers are using the tightened boundaries to start with , then we have a good idea of where the forced alignment is constrained to .
Postdoc A: Well , it 's just , you know , a matter of we had the revolution {disfmarker} we had the revolution of improved , uh , interface , um , one month too late ,
PhD D: So I 'm no I don't know if this
Grad E: Oh . Tools .
Postdoc A: but it 's like , you know , it 's wonderful to have the revolution ,
PhD D: Oh it 's {disfmarker} it 's a {disfmarker}
Postdoc A: so it 's just a matter of {disfmarker} of , you know , from now on we 'll be able to have things channelized to begin with .
PhD D: yeah .
Grad E: Right . And we 'll just have to see how hard that is .
Postdoc A: Yeah , that 's right .
Grad E: So {disfmarker} so whether the corrections take too much time .
Postdoc A: That 's right .
Grad E: I was just thinking about the fact that if Thilo 's missed these short segments , that might be quite time - consuming for them to insert them .
PhD D: Yeah .
Postdoc A: Good point .
PhD D: But he {disfmarker} he also can adjust this minimum time duration constraint and then what you get is noises mostly ,
Postdoc A: Yeah .
Grad E: Spurious .
PhD D: but that might be OK , an
Grad E: It might be easier to delete something that 's wrong than to insert something that 's missing .
PhD D: Right . And you can also see in the waveform {disfmarker} exac
Grad E: What do you think , Jane ?
PhD D: yeah .
Professor C: If you can feel confident that what the {disfmarker} yeah , that there 's actually something {disfmarker}
PhD D: Yeah .
Professor C: that you 're not gonna miss something ,
Grad E: Yeah . Cuz then {disfmarker} then you just delete it , and you don't have to pick a time .
Professor C: yeah .
PhD D: I think it 's {disfmarker}
Postdoc A: Well the problem is I {disfmarker} you know {disfmarker} I {disfmarker} I {disfmarker} it 's a {disfmarker} it 's a really good question , and I really find it a pain in the neck to delete things because you have to get the mouse up there on the t on the text line and i and otherwise you just use an arrow to get down {disfmarker} I mean , i it depends on how lar th there 's so many extra things that would make it one of them harder than the other , or {disfmarker} or vice versa . It 's not a simple question . But , you know , I mean , in principle , like , you know , if one of them is easier then to bias it towards whichever one 's easier .
Grad E: Yeah , I guess the semantics aren't clear when you delete a segment , right ? Because you would say {disfmarker} You would have to determine what the surroundings were .
PhD D: You could just say it 's a noise , though , and write , you know , a post - processor will just {disfmarker} all you have to do is just {disfmarker}
Grad E: If it 's really a noise .
PhD D: or just say it 's {disfmarker} just put " X , " you know , like " not speech " or something ,
Postdoc A: I think it 's easier to add than delete , frankly ,
PhD D: and then you can get {disfmarker} Yeah , or
Postdoc A: because you have to , uh , maneuver around on the {disfmarker} on both windows then .
Grad E: To add or to delete ?
Postdoc A: To delete .
PhD D: Anyways , so I {disfmarker} I guess {disfmarker}
Grad E: OK . That {disfmarker} Maybe that 's an interface issue that might be addressable .
Postdoc A: It 's possible .
Grad E: But I think it 's the semantics that are {disfmarker} that are questionable to me , that you delete something {disfmarker} So let 's say someone is talking to here , and then you have a little segment here . Well , is that part of the speech ? Is it part of the nonspeech ? I mean , w what do you embed it in ?
PhD D: There 's something nice , though , about keeping , and this is probably another discussion , keeping the stuff that Thilo 's detector detected as possible speech and just marking it as not speech than deleting it . Because then when you align it , then the alignment can {disfmarker} you can put a reject model or whatever ,
Grad E: Oh , I see . So then they could just like put {disfmarker} Oh that 's what you meant by just put an " X " there .
PhD D: and you 're consistent with th the automatic system ,
Grad E: Uh , that 's an interesting idea .
PhD D: whereas if you delete it {disfmarker}
Grad E: So {disfmarker} so all they {disfmarker} So that all they would have to do is put like an " X " there .
PhD D: Yeah , or some , you know , dummy reject mod
Grad E: So blank for {disfmarker} blank for silence , " S " " S " for speech , " X " " X " for something else .
PhD D: whatever , yeah . That 's actually a better way to do it cuz the a the forced alignment will probably be more consistent than {disfmarker}
Postdoc A: Well , like , I think there 's a complication which is that {disfmarker} that you can have speech and noise in s
PhD D: I mean if it 's just as easy , but {disfmarker}
Postdoc A: uh , you know , on the same channel , the same speaker , so now sometimes you get a ni microphone pop and , uh , I mean , there 're these fuzzy hybrid cases , and then the problem with the boundaries that have to be shifted around . It 's not a simple {disfmarker} not a simple problem .
PhD D: Anyway , quick question , though , at a high level do people think , let 's just say that we 're moving to this new era of like using the , um , pre - segmented t you know , non - synchronous conversations , does it make sense to try to take what we have now , which are the ones that , you know , we have recognition on which are synchronous and not time - tightened , and try to get something out of those for sort of purposes of illustrating the structure and the nature of the meetings , or is it better to just , you know , forget that and tr I mean , it 's {disfmarker}
Grad E: Well , I think we 'll have to , eventually . And my hope was that we would be able to use the forced alignment to get it .
PhD D: Right . That was everybody 's hope .
Grad E: But if we can't {disfmarker}
PhD D: And maybe we can for the non - lapel , but
Grad E: But if we can't , then maybe we just have to {disfmarker}
PhD D: is it worth {disfmarker} if we can't then we can fake it even if we 're {disfmarker} we report , you know , we 're wrong twenty percent of the time or ten percent of the time .
Grad E: Well , I 'm thinking {disfmarker} are you talking about for a paper , or are talking about for the corpus .
PhD D: Uh {disfmarker} uh , that 's a good question actually .
Grad E: I mean cuz for the corpus it would be nice if everything were {disfmarker}
PhD D: Actually that 's a good question because we 'd have to completely redo those meetings , and we have like ten of them now .
Grad E: We wouldn't have to re - do them , we would just have to edit them .
Postdoc A: Well , and also , I mean , I still haven't {disfmarker} I still haven't given up on forced alignment .
PhD D: No , you 're right , actually {disfmarker}
Postdoc A: I think that when Brian comes , this 'll be uh an interesting aspect to ask him as well b
Grad E: When {disfmarker}
Postdoc A: when Brian Kingsbury comes .
Grad E: Oh , Brian . You s I thought you said Ryan . And it 's like , " Who 's Ryan ? "
Postdoc A: Yeah , good question .
Grad E: OK .
Postdoc A: Well , Ryan could come .
PhD D: Uh , no , that 's a good point , though , because for feature extraction like for prosody or something , I mean , the meetings we have now , it 's a good chunk of data {disfmarker}
Grad E: Yep .
PhD D: we need to get a decent f OK .
Postdoc A: That 's what my hope has been ,
PhD D: So we should at least try it even if we can't ,
Postdoc A: and that 's what {disfmarker} that 's what {disfmarker} you know , ever since the {disfmarker} the February meeting that I transcribed from last year , forced alignment has been on the {disfmarker} on the table as a way of cleaning them up later .
PhD D: right ?
Grad E: On the table , right ?
Postdoc A: And {disfmarker} and so I 'm hopeful that that 's possible . I know that there 's complication in the overlap sections and with the lapel mikes ,
PhD F: There 's {disfmarker}
Postdoc A: but {disfmarker}
PhD F: Yeah .
PhD D: I mean , we might be able , at the very worst , we can get transcribers to correct the cases where {disfmarker} I mean , you sort of have a good estimate where these places are because the recognition 's so poor . Right ?
PhD B: Yeah , we were never just gonna go with these as the final alignments .
PhD D: And so you 're {disfmarker}
Postdoc A: I agree . I agree .
PhD B: We were always gonna run them past somebody .
PhD D: Yeah .
Postdoc A: Absolutely .
PhD D: So we need some way to push these first chunk of meetings into a state where we get good alignments .
PhD F: I 'm probably going to spend another day or so trying to improve things by , um , {vocalsound} {vocalsound} by using , um , acoustic adaptation . Um , the {disfmarker} {vocalsound} Right now I 'm using the unadapted models for the forced alignments , and it 's possible that you get considerably better results if you , uh , manage to adapt the , {vocalsound} uh , phone models to the speaker and the reject model to the {disfmarker} to {disfmarker} to all the other speech . Um , so
PhD B: Could you {disfmarker} could you at the same time adapt the reject model to the speech from all the other channels ?
Professor C: That 's what he just said .
Grad E: That 's what he was saying .
PhD D: Yeah .
PhD F: That 's what I just said .
PhD B: Oh , not just the speech from that {disfmarker} of the other people from that channel ,
PhD F: Right .
PhD B: but the speech from the a actual other channels .
PhD D: Right .
PhD F: Oh , oh , I see . Um ,
Professor C: Oh .
Grad E: I don't think so . I don't think that would work ,
PhD F: No , it {disfmarker}
Grad E: right ? Because you 'd {disfmarker} A lot of it 's dominated by channel properties .
PhD F: th Exactly .
PhD D: But what you do wanna do is take the , even if it 's klugey , take the segments {disfmarker} the synchronous segments , the ones from the HLT paper , where only that speaker was talking .
PhD F: So you want to u
PhD D: Use those for adaptation , cuz if you {disfmarker} if you use everything , then you get all the cross - talk in the adaptation , and it 's just sort of blurred .
PhD F: That 's a good point .
PhD B: If you {disfmarker}
PhD F: Yep .
PhD D: And that we know , I mean , we have that . And it 's about roughly two - thirds , I mean , very roughly averaged .
PhD F: Yeah .
PhD D: That 's not completely negligible . Like a third of it is bad for adaptation or so .
PhD F: Mm - hmm .
Grad E: Cool . I thought it was higher than that , that 's pr
PhD D: It really {disfmarker} it depends a lot . This is just sort of an overall {disfmarker}
PhD F: So .
Professor C: Well I know what we 're not turning in to Eurospeech , a redo of the HLT paper .
Grad E: Right .
Professor C: That {disfmarker} I don't wanna do that ,
Grad E: Yeah , I 'm doing that for AVIOS .
Professor C: but .
PhD D: Yeah . But I think we 're {disfmarker} oh , Morgan 's talk went very well , I think .
Professor C: Bleep .
Grad E: Uh , " bleep " . Yeah , really .
PhD D: I think Morgan 's talk went very well it woke {disfmarker}
Postdoc A: Excellent .
PhD D: you know , it was really a well presented {disfmarker} and got people laughing {disfmarker}
Postdoc A: Yeah .
PhD F: Some good jokes in it ?
Grad E: Especially the batteried meter popping up ,
PhD D: Yeah .
Grad E: that was hilarious . Right when you were talking about that .
Professor C: You know , that wa that was the battery meter saying that it was fully charged ,
Grad E: It 's full . Yeah .
Professor C: yeah .
Postdoc A: You said , " Speaking about energy " , or {vocalsound} something .
Grad E: But that was funny .
Postdoc A: That was very nice .
Professor C: Yeah .
Grad E: He {disfmarker} he {disfmarker} he was onto the bullet points about talking about the {disfmarker} you know {disfmarker} the little hand - held , and trying to get lower power and so on ,
PhD F: Po - low power
Grad E: and Microsoft pops up a little window saying " Your batteries are now fully charged . "
Postdoc A: That 's great .
Professor C: Yeah , yeah , yeah .
Grad E: I 'm thinking about scripting that for my talk , you know , put {disfmarker} put a little script in there to say " Your batteries are low " right when I 'm saying that .
Professor C: Yeah . Yeah . No I mean , i in {disfmarker} in your case , I mean , you were joking about it , but , I mean , your case the fact that your talking about similar things at a couple of conferences , it 's not {disfmarker} these are conferences that have d really different emphases . Whereas HLT and {disfmarker} and Eurospeech , pretty {disfmarker} pretty {disfmarker} pretty similar , so I {disfmarker} I {disfmarker} I can't see really just putting in the same thing ,
Grad E: Are too close , yeah .
PhD D: No , I d I don't think that paper is really {disfmarker}
Professor C: but {disfmarker}
PhD D: the HLT paper is really more of a introduction - to - the - project paper , and , um {disfmarker}
Professor C: Yeah .
Grad E: Yeah , for Eurospeech we want some results if we can get them .
PhD D: Well , yeah , it {disfmarker} it 's {disfmarker} probably wouldn't make sense ,
Professor C: Or some {disfmarker} or some {disfmarker} I mean , I would see Eurospeech {disfmarker} if we have some Eurospeech papers , these will be paper p p uh , submissions .
PhD D: but {disfmarker}
Professor C: These will be things that are particular things , aspects of it that we 're looking at , rather than , you know , attempt at a global paper about it .
PhD D: Right , right .
Grad E: Detail , yeah . Overall .
Postdoc A: I did go through one of these meetings . I had , uh , one of the transcribers go through and tighten up the bins on one of the , uh , NSA meetings , and then I went through afterwards and double - checked it so that one is really very {disfmarker} very accurate .
PhD D: Oh .
Postdoc A: I men I mentioned the link . I sent {disfmarker} You know that one ?
PhD D: Oh , so {disfmarker}
Grad G: The {disfmarker} which one ? I 'm sorry .
Postdoc A: Um , I 'm trying to remember {disfmarker} I don't remember the number off hand .
Grad E: Those are all {disfmarker}
Postdoc A: It 's one of the NSA 's . I sent email before the conference , before last week .
Grad G: Oh , OK .
Postdoc A: Bef - What I mean is Wednesday , Thursday .
PhD D: That might {disfmarker} might have been the one {disfmarker} one of the ones that we did .
Grad G: Mm - hmm .
Postdoc A: I 'm sure that that one 's accurate , I 've been through it {vocalsound} myself .
PhD D: So that might actually be useful but they 're all non - native speakers .
PhD F: So we could compare before and after
Grad G: OK .
PhD F: and see {disfmarker}
Grad E: Yeah . Yeah , that 's what I was gonna say . The problem with those , they 're all German .
PhD F: oh , Darn !
Grad G: Yeah , that 's the problem with the NSA speakers .
PhD D: And e and e and extremely hard to follow , like word - wise ,
Grad E: So .
PhD D: I bet the transcri I mean , I have no idea what they 're talking about ,
Grad G: Yeah .
Postdoc A: I corrected it for a number of the words .
PhD D: so ,
Postdoc A: I 'm sure that , um , they 're {disfmarker} they 're accurate now .
PhD D: um ,
PhD F: Uh , actually I have to {disfmarker} {vocalsound} to go .
PhD D: I mean , this is tough for a language model probably {disfmarker}
Grad G: Right .
PhD D: but {disfmarker} but that might be useful just for speech .
Professor C: Well .
Grad E: OK , Andreas is leaving {disfmarker} leaving the building . Mm - hmm .
Professor C: Yeah .
Grad E: See ya .
Professor C: See ya . I don't think we 'll go much longer .
Grad E: Um , oh , before you l go {disfmarker} I guess it 's alright for you to talk a little without the mike {disfmarker} I noticed you adjusting the mike a lot , did it not fit you well ? Oh .
Postdoc A: Well I won I noticed when you turned your head , it would {disfmarker} it would tilt .
Grad E: Maybe it wasn't just tightened enough , or {disfmarker}
PhD D: Maybe the {disfmarker} yeah , the s thing that you have tightened @ @ ,
PhD B: Actually if {disfmarker} if you have a larger head , that mike 's gotta go farther away which means the {disfmarker} the balance is gonna make it wanna tip down .
PhD D: oh .
Grad E: OK . Anyway .
Professor C: Yeah . OK , see ya .
Grad E: Cuz , I 'm just thinking , you know , we were {disfmarker} we 're {disfmarker} we 've been talking about changing the mikes , uh , for a while ,
Grad G:
Postdoc A: Yeah .
Grad E: and if these aren't {disfmarker} acoustically they seem really good , but if they 're not comfortable , we have the same problems we have with these stupid things .
Postdoc A: I think it 's com This is the first time I 've worn this , I find it very comfortable .
Grad E: I find it very comfortable too , but , uh , it looked like Andreas was having problems , and I think Morgan was saying it {disfmarker}
Professor C: Well , but I had it on {disfmarker} I had it on this morning and it was fine .
PhD B: Can I see that ?
Grad E: Oh , oh you did wear it this morning ?
Professor C: Yeah .
Grad E: OK , it 's off , so you can put it on .
PhD B: I {disfmarker} yeah , I don't want it on , I just {disfmarker} I just want to , um , say what I think is a problem with this . If you are wearing this over your ears and you 've got it all the way out here , then the balance is gonna want to pull it this way .
PhD D: Yeah .
Grad E: Right .
PhD B: Where as if somebody with a smaller head has it back here ,
Grad E: It 's more balanced .
PhD B: right ? Yeah .
Postdoc A: Oh !
PhD B: Then it {disfmarker} then it falls back this way so it 's {disfmarker}
PhD D: So we have to
Grad E: Well wh what it 's supposed to do is the backstrap is supposed to be under your crown , and so that should be {disfmarker} should be {disfmarker}
Postdoc A: Ah .
Grad E: if it 's right against your head there , which is what it 's supposed to be , that balances it so it doesn't slide up .
PhD B: So this is supposed to be under that little protuberance .
Grad E: Yep , right {disfmarker} right below {disfmarker} if you feel the back of your head , you feel a little lump ,
PhD B: Yeah .
Grad E: um , and so it 's supposed to be right under that .
PhD B: So it 's really supposed to go more like this than like this .
Grad E: Yes , exactly .
PhD B: But then isn't that going to {disfmarker} Well , I guess you can control that .
Grad E: That {disfmarker} that {disfmarker} that tilts , right ? In lots and lots of different ways .
PhD D: So I 'm not saying anything about bias towards small headsize ,
Grad E: About heads ?
PhD D: but does seem , uh {disfmarker}
PhD B: It would be an advantage .
Postdoc A: Well , wonder if it 's {disfmarker} if {disfmarker} if he was wearing it over his hair instead of under his hair .
Professor C: Well , we should {disfmarker} We shou we should work on compressing the heads , and {disfmarker}
Grad E: I think probably it was {disfmarker} Yeah . It probably just wasn't tight enough to the back of his head . I mean , so the directions do talk about bending it to your size , which is not really what we want .
PhD B: The other thing that would do it would be to hang a five pound weight off the back .
Professor C: Yeah
PhD D: Right .
Professor C: that 's good !
Postdoc A: What did you say ?
PhD D: A little ,
Grad E: wh
Professor C: Hang a five pound weight off the {disfmarker} off the back .
PhD B: Hang a five pound weight off the back .
PhD D: um ,
Grad E: We did that {disfmarker}
Professor C: Weight .
Grad E: We {disfmarker} at Boeing I used {disfmarker} I was doing augmented reality so they had head - mounts on , and we {disfmarker} we had a little jury - rigged one with a welder 's helmet ,
PhD B: Counter - balance .
Grad E: and we had just a bag with a bunch of marbles in it {vocalsound} as a counter - balance .
Professor C: Or maybe this could be helpful just for evening the conversation between people . If people {disfmarker} those who talk a lot have to wear heavier weights or something , and {disfmarker}
Grad E: Yeah !
Professor C: and {disfmarker} um ,
Grad E: Anyway .
Professor C: um , so , uh , what was I gonna say ? Oh , yeah , I was gonna say , uh , I had these , uh , conversations with NIST folks also while I was there and {disfmarker} and , uh , um , so they {disfmarker} they have their {disfmarker} their plan for a room , uh , with , um , mikes in the middle of the table , and , uh , close - mounted mikes ,
Grad E: Yep .
Professor C: and they 're talking about close - mounted and lapels , just cuz
PhD D: And arrays ,
Professor C: sort of {disfmarker} and the array .
Grad E: And arrays ,
Professor C: Yeah , so they were {disfmarker}
PhD D: which is the i interesting {disfmarker}
Grad E: yep . And cameras .
Professor C: And yeah , like multiple {disfmarker} multiple video cameras coverin covering every {disfmarker} everybody {disfmarker} every place in the room ,
PhD D: and video , right .
Professor C: uh , the {disfmarker} yeah {disfmarker} the {disfmarker} the mikes in the middle , the head - mounted mikes , the lapel mikes , the array , uh , with {disfmarker} well , there 's some discussion of fifty - nine ,
Grad E: Fifty - nine elements .
Professor C: they might go down to fifty - seven Because , uh , there is , uh , some pressure from a couple people at the meeting for them to use a KEMAR head . I forget what KEMAR , uh , stands for ,
PhD D: Mm - hmm .
Professor C: but what it is is it 's dummy head that is very specially designed ,
Grad E: Oh , that 's right .
Professor C: and {disfmarker} and {disfmarker} and , so what they 're actually doing is they 're really {disfmarker} there 's really two recording systems .
PhD D: Right .
Grad E: Yep .
PhD D: That 's a great idea .
Professor C: So they may not be precisely synchronous , but the but there 's two {disfmarker} two recording systems , one with , I think , twenty - four channels , and one with sixty - four channels . And the sixty - four channel one is for the array , but they 've got some empty channels there , and anyway they {disfmarker} like they 're saying they may give up a couple or something if {disfmarker} for {disfmarker} for the KEMAR head if they go {disfmarker} go with that .
Grad E: Right . Yeah , it is a good idea .
Professor C: So .
Grad E: Yeah , h uh , J Jonathan Fiscus did say that , uh , they have lots of software for doing calibration for skew and offset between channels
PhD D: Mm - hmm
Grad E: and that they 've found that 's just not a big deal .
Professor C: Yeah .
Grad E: So .
Professor C: Yeah , I 'm not {pause} too worried about that . I was thinking {disfmarker}
PhD D: But they 're still planning to do like fake {disfmarker}
Grad E: Scenario - based .
PhD D: they have to do something like that ,
Grad E: Y right .
PhD D: right .
Grad E: Their {disfmarker} their legal issues won't allow them to do otherwise .
Professor C: Yeah .
Grad E: But it sounded like they were {pause} pretty well thought out
PhD D: Yeah , th that 's true .
Grad E: and they 're {disfmarker} they 're gonna be real meetings ,
Postdoc A: Mm - hmm .
Grad E: it 's just that they 're with str with people who would not be meeting otherwise .
PhD B: Did {disfmarker} did they give a talk on this or was this informal ?
PhD D: Mm - hmm .
Grad E: So .
PhD D: No .
Grad E: No .
Professor C: No , we just had some discussions , various discussions with them .
Grad E: It 's just informal .
Postdoc A: Mm - hmm . Mm - hmm . Yeah .
Grad E: Yeah , I also sat and chatted with several of the NIST folks . They seemed like a good group .
PhD B: What was the , um {disfmarker} the paper by , um , Lori Lamel that you mentioned ?
Professor C: Um , yeah , we sh we should just have you {disfmarker} have you read it , but , I mea ba i i uh , we 've all got these little proceedings ,
Postdoc A: Mmm , yeah .
Professor C: but , um , basically , it was about , um , uh , going to a new task where you have insufficient data and using {disfmarker} using data from something else , and adapting , and how well that works . Uh , so in {disfmarker} in fact it was pretty related to what Liz and Andreas did , uh , except that this was not with meeting stuff , it was with
Grad E: Right .
Professor C: uh , like I think they s didn't they start off with Broadcast News system ? And then they went to {disfmarker}
Grad E: The - their Broadcast News was their acoustic models and then all the other tasks were much simpler .
Professor C: Yeah .
Grad E: So they were command and control and that sort of thing .
Professor C: TI - digits was one of them , and , uh , Wall Street Journal .
Grad E: Yep .
PhD B: What was their rough {disfmarker} what was their conclusion ?
Grad E: Yeah , read Wall Street Journal . It works .
Professor C: Yeah .
PhD D: Well , it 's {disfmarker} it 's a good paper , I mean {disfmarker}
Professor C: Yeah , yeah .
Grad E: Yeah . Yeah , that was one of the ones that I liked .
PhD D: Bring the {disfmarker}
Grad E: That {disfmarker} It not only works , in some cases it was better , which I thought was pretty interesting , but that 's cuz they didn't control for parameters . So .
Professor C: Probably .
Grad E: You know , the Broadcast News nets were {disfmarker} not nets ,
PhD D: Right .
PhD B: Did they ever try going {disfmarker} going the other direction from simpler task to more complicated tasks ,
Grad E: acoustic models {comment} were a lot more complex .
PhD B: or {disfmarker} ?
Grad E: n Not in that paper .
Professor C: That might be hard .
Grad E: Yeah , well , one of the big problems with that is {disfmarker} is often the simpler task isn't fully {disfmarker} doesn't have all the phones in it ,
Professor C: Yeah .
Grad E: and that {disfmarker} that makes it very hard .
PhD B: Mm - hmm .
Professor C: Yeah .
Grad E: But I 've done the same thing . I 've been using Broadcast News nets for digits ,
Professor C: Yeah .
Grad E: like for the spr speech proxy thing that I did ? That 's what I did .
Professor C: Yeah , sure .
Grad E: So . It works .
Professor C: Yeah . Yeah , and they have {disfmarker} I mean {disfmarker} they have better adaptation than we had than that {disfmarker} that system ,
Grad E: Yep .
Professor C: so they {disfmarker} um ,
Grad E: You mean they have some .
Professor C: yeah , we should probably what would {disfmarker} actually what we should do , uh , I haven't said anything about this , but probably the five of us should pick out a paper or two that {disfmarker} that , uh , you know , got our interest , and we should go around the room at one of the Tuesday lunch meetings and say , you know , what {disfmarker} what was good about the conference ,
Grad E: Present . Yep . Do a trip report .
Professor C: yeah .
PhD D: Well , the summarization stuff was interesting , I mean , I don't know anything about that field , but for this proposal on meeting summarization , um , I mean , it 's sort of a far cry because they weren't working with meeting type data , but he got sort of an overview on some of the different approaches ,
Grad E: Right .
PhD B: Do you remember who the groups were that we 're doing ?
PhD D: so . Well there 're {disfmarker} this was the last day ,
Grad E: A lot of different ones .
Postdoc A: R I think {disfmarker} Mm - hmm .
PhD D: but , I mean , there 's {disfmarker} that 's a huge field and probably the groups there may not be representative of the field , I {disfmarker} I don't know exactly that everyone submits to this particular conference ,
PhD B: Was {disfmarker} were there folks from BBN presenting ?
PhD D: but yet there was , let 's see , this was on the last day , Mitre , BBN , and , um , Prager {disfmarker}
Grad E: Mitre , BBN , IBM . Uh ,
Postdoc A: Maryland .
PhD D: um , I wo it was {disfmarker}
Professor C: Columbia have anything ? No .
PhD D: no it was {disfmarker}
Grad E: Wasn't {disfmarker} Who {disfmarker} who {disfmarker} who did the order one ?
PhD D: this was Wednesday morning . The sentence ordering one , was that Barselou , and these guys ?
Grad E: Ugh ! {comment} I 'm just so bad at that .
Postdoc A: Oh .
PhD D: Anyway , I {disfmarker} I {disfmarker} it 's in the program , I should have read it to remind myself , but that 's sort of useful and I think like when Mari and Katrin and Jeff are here it 'd be good to figure out some kinds of things that we can start doing maybe just on the transcripts cuz we already have {disfmarker}
Professor C: Mm - hmm .
Grad E: Yeah , we do have word transcripts .
PhD D: you know , yeah .
Grad E: So .
Postdoc A: Well , I like the idea that Adam had of {disfmarker} of , um , z maybe generating minutes based on some of these things that we have because it would be easy to {disfmarker} to {disfmarker} to do that just , you know , and {disfmarker} and
PhD D: Right .
Postdoc A: it has to be , though , someone from this group because of the technical nature of the thing .
Grad E: Someone who actually does take notes , um , {vocalsound} I 'm very bad at note - taking .
PhD D: But I think what 's interesting is there 's all these different evaluations , like {disfmarker} just , you know , how do you evaluate whether the summary is good or not ,
Grad E: I always write down the wrong things .
Postdoc A: I do take notes .
PhD D: and that 's what 's {disfmarker} was sort of interesting to me is that there 's different ways to do it ,
Grad E: A judge .
PhD D: and {disfmarker}
PhD B: Was SRA one of the groups talking about summarization , no ?
Grad E: Yep .
PhD D: Hm - umm . No .
Postdoc A: It was an interesting session . One of those w
Grad E: And as I said , I like the Microsoft talk on {pause} scaling issues in , uh , word sense disambiguation ,
PhD D: Yeah .
Grad E: that was interesting .
Professor C: Yeah , that was an interesting discussion ,
Grad E: The {disfmarker}
Professor C: uh , I
Grad E: It {disfmarker} it {disfmarker} it was the only one {disfmarker} It was the only one that had any sort of real disagreement about .
PhD D: The data issue comes up all the ti
Professor C: Well , I didn't have as much disagreement as I would have liked ,
Grad E: So .
Professor C: but I didn't wanna {disfmarker} I wouldn I didn't wanna get into it because , uh , you know , it was the application was one I didn't know anything about ,
Grad E: Yep .
Professor C: uh , it just would have been , you know , me getting up to be argumentative , but {disfmarker} but , uh , I mean , the missing thi so {disfmarker} so what they were saying {disfmarker} it 's one of these things {disfmarker} is {disfmarker} you know , all you need is more data , sort of {disfmarker} But I mea i wh it {disfmarker} @ @ that 's {disfmarker} that 's dissing it , uh , improperly , I mean , it was a nice study . Uh , they were doing this {disfmarker} it wasn't word - sense disambiguation , it was {disfmarker}
PhD D: Yeah {disfmarker} yeah {disfmarker} yeah {disfmarker}
Grad E: Well , it sort of was .
Professor C: was it w was it word - sense ? Yes .
Grad E: But it was {disfmarker} it was a very simple case of " to " versus " too " versus " two " and " there " , " their " , " they 're " {disfmarker}
PhD D: And there and their and {disfmarker}
Professor C: Yeah , yeah . OK .
PhD D: and that you could do better with more data , I mean , that 's clearly statistically {disfmarker}
Professor C: Right .
Grad E: Yeah .
Professor C: And so , what they did was they had these different kinds of learning machines , and they had different amounts of data , and so they did like , you know , eight different methods that everybody , you know , uh , argues about {disfmarker} about , " Oh my {disfmarker} my kind of learning machine is better than your kind of learning machine . " And , uh , they were {disfmarker} started off with a million words that they used , which was evidently a number that a lot of people doing that particular kind of task had been using . So they went up , being Microsoft , they went up to a billion . And then they had this log scale showing a {disfmarker} you know , and {disfmarker} and naturally everything gets {disfmarker}
Grad E: Them being beep , {comment} they went off to a billion .
Professor C: they {disfmarker} well , it 's a big company , I didn't {disfmarker} I didn't mean it as a ne anything negative ,
Grad E: Yeah .
Professor C: but i i i
PhD D: You mean the bigger the company the more words they use for training ?
Grad E: Well , I think the reason they can do that , is that they assumed that text that they get off the web , like from Wall Street Journal , is correct , and edit it .
Professor C: Yeah .
Grad E: So that 's what they used as training data . It 's just saying if it 's in this corpus it 's correct .
Professor C: OK . But , I mean , yes . Of course there was the kind of effect that , you know , one would expect that {disfmarker} uh {disfmarker} that you got better and better performance with more and more data . Um , but the {disfmarker} the real point was that the {disfmarker} the different learning machines are sort of all over the place , and {disfmarker} and by {disfmarker} by going up significantly in data you can have much bigger effect then by switching learning machines and furthermore which learning machine was on top kind of depended on where you were in this picture , so ,
PhD B: This was my concern about the recognizer in Aurora .
Professor C: uh , That {disfmarker}
PhD B: That the differences we 're seeing in the front - end is b
Professor C: Yeah .
Grad E: Are irrelevant .
PhD B: are irrelevant once you get a real recognizer at the back - end .
Professor C: Yeah .
PhD D: If you add more data ? Or {disfmarker}
PhD B: You know ?
Professor C: Yeah .
PhD D: Huh .
Professor C: Yeah , could well be . So {disfmarker} so , I mean , that was {disfmarker} that was kind of , you know , it 's a good point , but the problem I had with it was that the implications out of this was that , uh , the kind of choices you make about learning machines were therefore irrelevant which is not at {disfmarker} n t as for as I know in {disfmarker} in tasks I 'm more familiar with @ @ is not at all true . What i what is {disfmarker} is true is that different learning machines have different properties , and you wanna know what those properties are . And someone else sort of implied that well we s you know , a all the study of learning machine we still don't know what those properties are . We don't know them perfectly , but we know that some kinds use more memory and {disfmarker} and some other kinds use more computation and some are {disfmarker} are hav have limited kind of discrimination , but are just easy to use , and others are {disfmarker}
PhD B: But doesn't their conclusion just sort of {disfmarker} you could have guessed that before they even started ? Because if you assume that these learning things get better and better and better ,
Professor C: You would guess {disfmarker}
PhD B: then as you approach {disfmarker} there 's a point where you can't get any better , right ? You get everything right .
Professor C: Yeah .
PhD D: It 's just no {disfmarker}
Grad E: But {disfmarker}
PhD B: So they 're all approaching .
Grad E: No , but there was still a spread . They weren't all up They weren't converging .
PhD B: But what I 'm saying is that th they have to , as they all get better , they have to get closer together .
Professor C: It w
Grad E: They were all still spread . But they {disfmarker} Right , right . Sure . But they hadn't even come close to that point . All the tasks were still improving when they hit a billion .
Professor C: Yeah .
PhD B: But they 're all going the same way , right ? So you have to get closer .
Professor C: Eventually . O one would
Grad E: But they didn't get closer .
PhD B: Oh they didn't ?
Professor C: Well {disfmarker}
Grad E: They just switched position .
Professor C: well that 's getting cl I mean , yeah , the spread was still pretty wide that 's th that 's true ,
Grad E: Yep .
Professor C: but {disfmarker} but , uh , I think it would be irntu intu intuition that this would be the case , but , uh , to really see it and to have the intuition is quite different , I mean , I think somebody w w let 's see who was talking about earlier that the effect of having a lot more data is quite different in Switchboard than it is in {disfmarker} in Broadcast News ,
PhD D: Well it 's different for different tasks .
Grad E: Yeah . It was Liz . Yeah .
Professor C: yeah .
PhD D: So it depends a lot on whether , you know , it {disfmarker} disambiguation is exactly the case where more data is better , right ? You 're {disfmarker} you 're {disfmarker} you can assume similar distributions ,
Professor C: Yeah .
PhD D: but if you wanted to do disambiguation on a different type of , uh , test data then your training data , then that extra data wouldn't generalize ,
Grad E: Right .
PhD D: so .
Professor C: Right .
Grad E: But , I think one of their p They {disfmarker} they had a couple points . w {comment} Uh , I think one of them was that " Well , maybe simpler algorithms and more data are {disfmarker} is better " . Less memory , faster operation , simpler . Right ? Because their simplest , most brain - dead algorithm did pretty darn well
Professor C: Mm - hmm .
Grad E: when you got {disfmarker} gave it a lot more data . And then also they were saying , " Well , m You have access to a lot more data . Why are you sticking with a million words ? " I mean , their point was that this million - word corpus that everyone uses is apparently ten or fifteen years old . And everyone is still using it , so .
Professor C: Yeah . But anyway , I {disfmarker} I {disfmarker} I think it 's {disfmarker} it 's just the {disfmarker} the i it 's {disfmarker} it 's {disfmarker} it 's not really the conclusion they came to so much , as the conclusion that some of the , uh , uh , commenters in the crowd {vocalsound} came up with
Grad E: But we could talk about this stuff , I think this would be fun to do .  Right .
Professor C: that , uh , you know , this therefore is further evidence that , you know , more data is really all you should care about , and that I thought was just kind of going too far the other way ,
Grad E: Machine - learning .
Professor C: and {disfmarker} and the {disfmarker} the , uh , one {disfmarker} one person ga g g got up and made a {disfmarker} a brief defense , uh , but it was a different kind of grounds , it was that {disfmarker} that , uh , i w the reason people were not using so much data before was not because they were stupid or didn't realize data was important , but in fact th they didn't have it available . Um , but the other point to make a again is that , uh , machine learning still does matter , but it {disfmarker} it matters more in some situations than in others , and it {disfmarker} and also there 's {disfmarker} there 's not just mattering or not mattering , but there 's mattering in different ways . I mean , you might be in some situation where you care how much memory you 're using , or you care , you know , what recall time is ,
Grad E: Right .
Professor C: or you care , you know , and {disfmarker} and {disfmarker}
Grad E: Or you only have a million words {pause} for your {disfmarker} some new task .
Professor C: Yeah , or {disfmarker} or , uh {disfmarker}
PhD D: Or done another language , or {disfmarker} I mean , you {disfmarker} so there 's papers on portability and rapid prototyping and blah - blah - blah ,
Grad E: Yep .
Professor C: Yeah .
Grad E: Right .
Professor C: Yeah .
PhD D: and then there 's people saying , " Oh , just add more data . "
Professor C: And there 's cost !
PhD D: So , these are like two different religions , basically .
Grad E: Mm - hmm . Cost .
Professor C: There 's just plain cost ,
Grad E: Yeah . That 's a big one .
Professor C: you know , so {disfmarker} so these , I mean th the {disfmarker} in the {disfmarker} in the speech side , the thing that @ @ always occurs to me is that if you {disfmarker} if you {disfmarker} uh {disfmarker} one person has a system that requires ten thousand hours to train on , and the other only requires a hundred , and they both do about the same because the hundred hour one was smarter , that 's {disfmarker} that 's gonna be better . because people , I mean , there isn't gonna be just one system that people train on
Grad E: Yep .
Professor C: and then that 's it for the r for all of time . I mean , people are gonna be doing other different things , and so it {disfmarker} these {disfmarker} these things matters {disfmarker} matter .
Postdoc A: Yeah , that 's it .
Grad E: Yeah , so that 's one of the slides they put up .
Postdoc A: So , I mean , this was a very provocative slide . She put this up , and it was like this is {disfmarker} this p people kept saying , " Can I see that slide again ? "
Professor C: Yeah .
PhD D: Yeah ,
Postdoc A: and then they 'd make a comment , and one person said , well - known person said , um , you know , " Before you dismiss forty - five years including my work {disfmarker} "
PhD D: yeah .
Grad E: Forty - five years of research .
PhD D: Yeah .
Grad G: Yeah .
PhD D: But th you know , the same thing has happened in computational linguistics , right ? You look at the ACL papers coming out , and now there 's sort of a turn back towards , OK we 've learned statistic {disfmarker} you know , we 're basically getting what we expect out of some statistical methods , and , you know , the there 's arguments on both sides ,
Grad E: Yep .
PhD D: so {disfmarker}
Grad E: I think the matters is the thing that {disfmarker} that was misleading .
Postdoc A: That was very offending , very offending .
PhD D: Yeah , yeah .
Grad E: Is that {disfmarker} all {disfmarker} all of them are based on all the others , right ? Just , you {disfmarker} you can't say {disfmarker}
PhD B: Maybe they should have said " focus " or something .
Professor C: Right .
Grad E: Yeah . I mean , so . {disfmarker} And I 'm saying the same thing happened with speech recognition , right ? For a long time people were hand - c coding linguistic rules and then they discovered machine - learning worked better . And now they 're throwing more and more data and worrying {disfmarker} perhaps worrying less and less about , uh , the exact details of the algorithms .
PhD D: And {disfmarker} and then you hit this {disfmarker}
Grad E: Except when they have a Eurospeech paper .
Postdoc A: Yeah .
Professor C: Yeah .
Grad E: Anyway .
Professor C: Anyway , tea is {disfmarker} tea is , uh , starting .
Grad E: Shall we read some digits ? Are we gonna do one at a time ? Or should we read them all agai at once again .
Professor C: Let 's do it all at once .
Postdoc A: Yeah , that 's good .
Professor C: We {disfmarker} @ @ {disfmarker} let 's try that again .
PhD D: Yes ! So , and maybe we won't laugh this time also .
Grad E: OK . So remember to read the transcript number so that , uh , everyone knows that {disfmarker} what it is . And ready ?
Postdoc A: Yeah .
Grad E: Three , two , one .
Professor C: Boy , is that ever efficient .
Grad E: Yep . That 's really fast .
Professor C: Yeah . Yeah .