<http://schema.org/articleBody>
"Google has published a video of a snippet of an internal search quality meeting on their blog yesterday.  It is pretty amazing to watch, even though everyone knows they are being recorded and it is only a small snippet of the meeting.Here is the video:Who are the main players at this table?Left - Ben Azose Search Quality Analyst\r\nMiddle - Amit Singhal in charge of search\r\nRight - Matt Cutts\r\nLeft - Scott Huffman testing man\r\nMiddle - Panda Nayak (um, Panda update)\r\nRight - Paul Haahr Ranking LeadLeft - Matt Cutts\r\nMiddle - Ben Gomes\r\nRight - Lars Hellsten Engineer Spell Correction Team\r\n\r\nI bet some of you can even recognize some of the people on the floor?    Hi Tiffany and I see you Alon on the screen from Haifa!Here is the transcript:0:02Singhal: Everyone, thank you for setting this up,\r\n0:04and guys, thank you for putting up\r\n0:05with all the inconvenience we are putting you through.\r\n0:09It so happens that this meeting is the heart of what we do.\r\n0:15What we approve, how we run Search.\r\n0:18This is an experiment.\r\n0:20We will see how the tape comes out.\r\n0:23If I look bad, we will not put it out.\r\n0:25[laughter]\r\n0:28Okay?\r\n0:30If Gomes looks bad,\r\n0:31we will put it on the front page.\r\n0:33[laughter]\r\n0:37Huffman: All right. Spell-correcting long queries.\r\n0:40Singhal: Lars.\r\n0:43Lars: So, to keep our latency low,\r\n0:45spelling has always just corrected\r\n0:4810 terms in long queries,\r\n0:50and we decided to use the first 10 terms,\r\n0:55which was sort of arbitrary,\r\n0:57and so this is a change by Euro in Zurich,\r\n0:59who decided that we could be\r\n1:01a little bit more intelligent about this.\r\n1:03And so we're going to pick the two words\r\n1:06that we think are most likely to be misspelled in the query,\r\n1:09and form intervals of five words around each,\r\n1:12so we're still correcting only 10 words.\r\n1:14And this is just a smarter way\r\n1:18of deciding which words to correct.\r\n1:20Gomes: So, your context is the five words\r\n1:22rather than the whole 10 words.\r\n1:24So, you're more likely to find a match.\r\n1:25man: Well, in general, the context is only three words,\r\n1:27because we use trigrams for this thing,\r\n1:29but they correct five words at a time,\r\n1:32rather than simply the first 10 words.\r\n1:35man: So, if you take a look at the mean scores...\r\n1:36man: This is huge.\r\n1:38man: This is very, very positive.\r\n1:39man: We send both fragments to spelling separately,\r\n1:42or is it strung together?\r\n1:43Lars: No, they're sent together.\r\n1:45We have a way of marking which terms we correct,\r\n1:47and which terms we won't correct.\r\n1:48Cutts: But roughly what percentage\r\n1:49of queries have more than 10 terms?\r\n1:51Lars: Not a lot. So...\r\n1:53[laughter]\r\n1:54man: But it is very annoying\r\n1:56when your misspelling is towards the end of a long query,\r\n2:00and you don't-- you don't get it.\r\n2:01And it's so obviously wrong.\r\n2:03Paul: We've seen these where it was pasted quotes\r\n2:04and the last word is mangled.\r\n2:06Singhal: Why would anything ever go wrong with this?\r\n2:08man: Yeah. man: It does, because you try\r\n2:11to correct something late in the query,\r\n2:13and you'll see some examples\r\n2:15where early in the query\r\n2:16there's also a misspelling which you failed to correct.\r\n2:18Singhal: Oh, so, because of your two-word selection,\r\n2:21you end up picking--\r\n2:22if there are more than two misspellings in a query...\r\n2:24man: Or there's a very rare word that makes you believe\r\n2:27that that's a potential misspelling.\r\n2:29'Cause you don't know it's a misspelling.\r\n2:30Gomes: Why wouldn't you apply the misspelling\r\n2:32across the whole query?\r\n2:33The same misspelling, you're saying,\r\n2:35would get corrected in one place, because of context?\r\n2:36man: No, no, no, it's a different\r\n2:38misspelling at the beginning.\r\n2:39The problem is we--if we could just correct the whole thing,\r\n2:42but then you'd pay in cost.\r\n2:44Right, latency and things, so they don't want to do that.\r\n2:46man: It's mostly the latency, right?\r\n2:48Like, why? I don't know, it seems a little like--\r\n2:50We could do, you know, hundreds of--thousands of QPS,\r\n2:53right, why can't we send--\r\n2:55break the query up into multiple chunks,\r\n2:56and then send them all through parallel so that--\r\n2:59so we can correct the entire query, right?\r\n3:00Lars: we could do that, but I think the traffic effect\r\n3:03would just be a really small slice.\r\n3:05Singhal: So...\r\n3:06But why not just do that right?\r\n3:08Mean, like, take overlapping five-word windows,\r\n3:11and send runs of 10-word queries,\r\n3:14as many as you can make out of a query,\r\n3:16and send them all in parallel?\r\n3:18man: Actually 'cause there's only a 0.1% change.\r\n3:22Singhal: And, you know-- And by the way,\r\n3:24in most cases, you'll be pretty much done.\r\n3:26You will cover up to 15-word queries\r\n3:30with just two.\r\n3:32Paul: I think we should certainly launch this.\r\n3:35I think [indistinct] gets points for a clever idea on it,\r\n3:36but I think it is driving the same--\r\n3:38the idea of splitting it.\r\n3:41That's probably more infrastructure work.\r\n3:43man: I'm sorry, I just want to jump back to this problem\r\n3:45with the beginnings of the queries.\r\n3:47So, these situations where--\r\n3:50if you look at the second one in the second block there.\r\n3:52\"Int he book 'Julius Caesar,'\" et cetera, et cetera, et cetera,\r\n3:57we don't catch--we catch all sorts of misspellings\r\n3:59about Caesar and differences,\r\n4:01but we miss the fact that \"int he\" should be \"in the.\"\r\n4:05We have another query\r\n4:09about sponsoring a child living in Tenerife,\r\n4:11and we want to figure out\r\n4:14whether \"Tenerife\" is misspelled,\r\n4:15but we miss the fact that it's \"cam\" instead of \"can.\"\r\n4:20Gomes: By the way, are you doing this--\r\n4:21but in the course of Suggest--\r\n4:22So, the same thing will work with Suggest?\r\n4:24When we have live-spelling Suggest?\r\n4:27man: I'm sure if-- once you launch this,\r\n4:29Suggest will do the same thing, right?\r\n4:31Gomes: So, Suggest will be actually all from--\r\n4:32man: This is all inside the Spell server,\r\n4:34so there are no multiple calls being made.\r\n4:36It's all embedded inside the Spell server.\r\n4:39Singhal: So, on the sponsor, did we send the context,\r\n4:41left and right?\r\n4:43man: We did.\r\n4:44man: And then why didn't we correct the context?\r\n4:47Lars: Actually, this is sort of\r\n4:48an issue with the current implementation.\r\n4:50If there are--\r\n4:52if there are two intervals that are close enough together,\r\n4:55then we merge them into one,\r\n4:57so what's actually happening is we're correcting, I think,\r\n5:00from \"I\" to \"credit.\"\r\n5:03Paul: So, we just missed one.\r\n5:05Lars: Yeah, so...\r\n5:06Paul: Picked--We picked slightly the wrong window.\r\n5:09Look, that's gonna happen with any of these.\r\n5:11man: I mean, certainly, the original thing\r\n5:13of picking the first 10 was missing a lot of words.\r\n5:15man: That's right, that's right.\r\n5:17Paul: The averages say this is clearly an improvement.\r\n5:20Cutts: But if this is, like, .01% of queries,\r\n5:23why not just correct--\r\n5:24man: No, it's .1. Not .01, it's .1.\r\n5:27Cutts: But how much, resource-wise--\r\n5:30Paul: I think it's more just the--\r\n5:32the infrastructure work on doing it.\r\n5:34Because you now have to have the Spell servers call out\r\n5:35to other Spell servers.\r\n5:37Cutts: Okay.\r\n5:40man: It seems good.\r\n5:42Gomes: I mean, to a large extent, you will be seeing\r\n5:43those spell corrections happening in Suggest,\r\n5:44because you're going to get that initial window...\r\n5:47Paul: I think a lot of these are just pasted queries, though.\r\n5:49Singhal: This is cut and pasted.\r\n5:51These are cut and paste. No one's typing these.\r\n5:54man: We're seeing a lot of people's--\r\n5:56Paul: \"Cam I sponsor--\" man: \"Cam I sponsor.\"\r\n5:59Cutts: The Caesar one is--that's a kid just doing his homework.\r\n6:02Paul: \"Stein, S. et al amino acid analysis\"\r\n6:04is a pasted query.\r\n6:06Cutts: Yeah.\r\n6:08Paul: So, I mean--So... man: Not all of them.\r\n6:10Paul: But not all of these.\r\n6:11Cutts: Like \"how long do you have to wait\r\n6:13to wash your hair after a perm?\"\r\n6:14man: \"Int he book\" is almost certainly not.\r\n6:15Singhal: It may hap--\r\n6:17plenty of pastes do all kinds of funky things.\r\n6:19man: And if you look at the wins,\r\n6:20a lot of those are definitely typed queries.\r\n6:22Paul: Okay.\r\n6:23Anyway, I--Look, this is clearly a good change.\r\n6:26man: Maybe a great change.\r\n6:28Paul: Let's give a recommendation to the team\r\n6:29to actually give up the 10-word limits.\r\n6:31Singhal: No, but I want some follow-up\r\n6:32on that recommendation.\r\n6:34Paul: Yeah.\r\n6:35Singhal: So, how are we gonna get that follow-up?\r\n6:37man: Your recommendation is issuing multiple...\r\n6:40Singhal: Just do it. All of it.\r\n6:41Right in that, by chunking.\r\n6:42man: Yeah, I just think we should have some system\r\n6:43that can handle 100-word queries, right?\r\n6:46I don't know. Singhal: Yeah.\r\n6:48man: We shouldn't die on, like, the hardest query.\r\n6:50Paul: Right, but I think we end up doing that on the front end\r\n6:52and not in--\r\n6:53Singhal: No, but I don't--Paul.\r\n6:54Paul: We don't care. Singhal: I'm sorry.\r\n6:56You're defending something that--\r\n6:57you know, the design is not perfect.\r\n6:58Don't defend it.\r\n7:00Okay?\r\n7:01Paul: I think it's fine to do the recommendation,\r\n7:03but I think this is a good step,\r\n7:04and I think Euro gets points\r\n7:05for getting us to look at that again.\r\n7:07Singhal: No, that's fine, but, you know,\r\n7:08I want to make sure that the team comes back,\r\n7:10or we put some kind of exploding deadline\r\n7:12that, you know, we won't do this.\r\n7:13If you don't do it right within, say, three months,\r\n7:17Gomes: Your Spell server is being used\r\n7:19for running text in other places too?\r\n7:21man: Now they're being used in--\r\n7:24for red underline also, isn't that right?\r\n7:27Lars: We don't use--\r\n7:28We use the same servers-- Yes and no.\r\n7:30man: But a different set-up?\r\n7:32Lars: For part of--\r\n7:33one of the red underline clients is using them, yeah.\r\n7:35man: So, that must be much longer chunks of text.\r\n7:37man: No, no, but I think they break it up into smaller chunks.\r\n7:40Singhal: Treat this as someone's typing an email.\r\n7:42man: Email, right. That's what I was...\r\n7:43man: And bring in all the red underlines.\r\n7:45Gomes: Right.\r\n7:49Singhal: Okay, we can launch this, but...\r\n7:52man: I mean, remember,\r\n7:53we may still have problems even there\r\n7:55because of context and things, if you break things up.\r\n7:58So, they'll always be--\r\n8:01Gomes: Yeah, treat it as running text, right?\r\n8:03Singhal: Okay. man: Okay.Forum discussion at Google+, WebmasterWorld and Matt Cutts Google+."^^<http://www.w3.org/2001/XMLSchema#string>
<https://www.seroundtable.com/search-quality-meeting-14868.html>