ellen voorhees muuhecfkliodhnistgov call participation text retrieval conference trec february 2008 november 2008 conducted national institute standards technology nist support intelligence advanced research projects activity iarpa text retrieval conference trec workshop series encourages research information retrieval related applications providing large test collection uniform scoring procedures forum organizations interested comparing results seventeenth year conference become major experimental effort field participants previous trec conferences examined wide variety retrieval techniques retrieval environments including crosslanguage retrieval retrieval web documents multimedia retrieval question answering details trec found trec web site httptrecnistgov invited participate trec 2008 trec 2008 consist set tasks known tracks track focuses particular subproblem variant retrieval task described organizations may choose participate tracks tracks training test materials available nist tracks use special collections available organizations fee dissemination trec work results publicly available conference proceedings welcomed conditions participation specifically preclude advertising claims based trec results retrieval results submitted nist published proceedings archived trec web site workshop november open participating groups submit retrieval results least one track selected government personnel sponsoring agencies schedule february 21 submit application described nist returning application add active participants mailing list feb 25 nist announce new password active participants portion trec web site included portion web site information regarding permission forms needed obtain trec document disks beginning march 1 document disks used existing trec collections distributed participants returned required forms please note disks shipped march 1 midjulymidaugust results submission deadline tracks results deadline may need even earlier tracks depending assessor availability specific deadlines track included track guidelines finalized spring september 9 estimated speaker proposals due nist september 30 estimated relevance judgments individual evaluation scores due back participants nov 1821 trec 2008 conference nist gaithersburg md usa task description brief summary tasks complete descriptions tasks performed previous years included overview papers trec proceedings publications section web site exact definition tasks performed track trec 2008 still formulated track discussion takes place track mailing list added track mailing list follow instructions contacting mailing list given questions track send mail track coordinator post question track mailing list join trec 2008 one new track four continuing tracks new track relevance feedback track track systematically explore factors affect relevance feedback behavior blog enterprise legal million query tracks continue trec 2008 though specific tasks track may differ year year note qa track moved new text analysis conference tac call participation tac sent trec friends mailing list blog track purpose blog track explore information seeking behavior blogosphere track coordinator iadh ounis mljfzcsglaacuk mailing list send mail message mardfyrxnistgov body consists line subscribe trecblog enterprise track purpose enterprise track study enterprise search satisfying user searching data organization complete task track coordinators nick craswell ukdbbkmicrosoftcom ian soboroff jlzpfohsudxnistgov arjen de vries dxxhpacmorg track web page httpwwwinscwinlprojectstrecent mailing list send mail message mardfyrxnistgov body consists line subscribe trecent legal track goal legal track develop search technology meets needs lawyers engage effective discovery digital document collections track coordinators jason baron zjttadftmknaragov doug oard rshhumdedu track web page httptreclegalumiacsumdedu mailing list contact rshhumdedu added list million query track goal million query track test hypothesis test collection built many incompletely judged topics better tool collection built using traditional trec pooling track web page httpciircsumasseduresearchmillion mailing list follow instructions given track web page join email list etaylhmcsumassedu relevance feedback track goal relevance feedback track provide framework exploring effects different factors success relevance feedback track coordinators chris buckley gbxbtdvhqsabircom steve robertson bhlmicrosoftcom track web page httpgroupsgooglecomgrouptrecrelfeed mailing list follow instructions given track web page join email list conference format conference used forum presentation results including failure analyses system comparisons lengthy system presentations describing retrieval techniques used experiments run using data issues interest researchers information retrieval limited amount time presentations trec program committee determine groups asked speak groups present poster session groups interested speaking slot workshop submit 200300 word abstract september describing experiments performed program committee use abstracts select speakers organizations may wish describe proprietary algorithms trec defines two categories participation category full participation participants expected present full details system algorithms various experiments run using data either talk poster session category c evaluation participants category expected submit results common scoring tabulation present results poster session expected describe systems minute detail expected describe general approach report time effort statistics data many existing trec english collections documents topics relevance judgments available training purposes may also used tracks parts training collection disks 13 assembled linguistic data consortium ldc text signed user agreement required participants documents assorted collection newspapers newswires journals technical abstracts ldc collected recent set newswire material called aquaint collection disks 67 collection also available trec participants covered separate user agreement third agreement needed remaining disks 45 documents typical seen realworld situation ie arcane vocabulary may missing pieces text typographical errors tracks relevance judgments systems output scored made experienced relevance assessors based output trec participants using pooled relevance methodology see overview paper trec8 proceedings trec web site detailed discussion pooling response format submission details organizations wishing participate trec 2008 respond call participation submitting application important note participants previous trecs participate trec 2008 must submit new application application consists following five parts 1 contact information full name main contact person organization full name organization participating member organization please specify self know another group organization also participate trec 2008 example two groups university please give enough qualification organization name distinguish different groups organizationteam name 20 characters used unique identifier group need use identifier correspondence nist requesting data sending permission forms example remember identifier also used tag runs submit complete organization physical mail addresssufficient mail sent address accepted post office fully qualified phone fax numbers main contact person fully qualified valid email address main contact person exactly one fully qualified valid email address use trec 2008 participants mailing list overhead involved maintaining mailing list active participants one email address per participating group added trec 2008 participants list strongly encourage use local mailing list institution distributes trec mail internally project participants involved see mail sent list trec run solely email important address valid mail sent address read timely fashion may use email address main contact person address mailing list please give twice application sure intentions 2 whether participated trec please indicate years participated otherwise indicate new trec 3 oneparagraph description retrieval approach 4 whether participate category category c group 5 list tracks likely participate nonbinding helpful know planning application form simple text file consisting information number application please respond using simple text ie pdf word rtf postscript etc files process application participate trec 2008 unless complete responses mailed lori buckland loribucklandnistgov questions conference participation response format etc sent general trec email address wwxjnistgov tue 05 aug 2008 200053 0500 trec friends call participation trec 2008