ohwwlejspbfotbplistscmswaikatoacnz send wekalist mailing list submissions iuxxdkqklistscmswaikatoacnz subscribe unsubscribe via world wide web visit httpslistscmswaikatoacnzmailmanlistinfowekalist via email send message subject body help ohwwlejspbfotbplistscmswaikatoacnz reach person managing list pobypuhshvhzjlistscmswaikatoacnz replying please edit subject line specific contents wekalist digest todays topics 1 preserving order split leads inferior results bernhard pfahringer 2 question gridsearch peter reutemann 3 association rules keywords large text usingwekajava benoît depaire 4 setting delimiters wordtokenizer peter reutemann 5 setting delimiters wordtokenizer peter reutemann 6 cobweb clustering mark hall 7 attribute selection test set benoît depaire 8 preserving order split leads inferior results mark hall plqebtmylhuiyuzyplgmailcom sat apr 26 154908 2008 message 1 date mon 25 feb 2008 091445 1300 bernhard pfahringer subject wekalist preserving order split leads inferior results weka machine learning workbench list messageid 61d5fe0o3996155761d598uo39bcp5t761359eu2r99mailgmailcom contenttype textplain charsetiso88591 training set test set 10fold cross validation gave accuracy around 90 running dataset 10times using 66 training sample using different seed values gave average around 91 tested preserve order option accuracy test set dropped 75 discovered datasets order matters different explanations eg maybe timeseries sorted timestamps likely guess data sorted classvalue cheers bernhard dlnirllcwaikatoacnz sat apr 26 154908 2008 message 2 date mon 25 feb 2008 092730 1300 peter reutemann subject wekalist question gridsearch weka machine learning workbench list messageid 47c1d3324040507waikatoacnz contenttype textplain charsetiso88591 formatflowed would like know gridsearch libsvm 10fold cv test option implements following algorithm 1 split dataset 90 train 10 test set 2 split train set 10 folds 10 pairs traintest sets 3 try cg pair 10 pairs traintest sets created step 2 calculate accuracies 1 accuracy pair 10 accuracies total 4 find average 10 accuracies found step 3 5 repeat steps 3 4 cg pairs 6 pick cg pair yielding highest average accuracy 7 train 90 set created step 1 using cg pair step 6 8 find testing accuracy 10 set created step 1 9 repeat steps 1 8 10 times 10 display average accuracy heres happens running gridsearch explorer applies commandline 1 explorer randomizes data 2 explorer splits data traintest set crossvalidation cv case 10fold cv 9010 3 gridsearch presented training data parameter pair initial grid 2fold cv run setup filterclassifier b best pair selected metric user selected c directly adjacent points best pair run 10fold crossvalidation better pair found metric user selected among adjacent points done otherwise pair border grid go 3c using pair new center ii pair border grid grid extended done iii pair border grid grid extended extend grid go 3c using best point new center e setup filterclassifier best parameters found 4 explorer tests gridsearchs performs test set basically evaluating filterclassifier combination determine work best training data step 3e 5 go 2 next traintest split 6 output results averaged crossvalidation traintest splits notes internal crossvalidations within gridsearch run training set ie 90 explorer hands classifier filter always run data 3a 3c dont use filter set wekafiltersallfilter wont modify data case libsvm youre optimizing complexity gamma filter setup stay throughout search hth cheers peter peter reutemann dept computer science university waikato nz httpwwwcswaikatoacnzfracpete 64 7 8384466 ext 5174 zyylbmbsnsytkuhasseltbe sat apr 26 154908 2008 message 3 date sun 24 feb 2008 214335 0100 benoît depaire subject wekalist association rules keywords large text usingwekajava weka machine learning workbench list messageid 200802242042130eced1aae8calphauhasseltbe contenttype textplain charsetiso88591 hello basically approach seems ok however difficult say better approach since clear purpose analysis want find words often occur together sentence would need information want use association rules classification without knowing ultimate goal find difficult answer questions counter heapsize problem might want try predictiveapriori dont know solve problem algorithm set number rules want mine algorithm adjusts confidence support mining process might increase speed mining process dont know also solve memory problem actualy doubt might want try secondly splitting data several data sets afraid work since apriori uses minimum support treshold splitting data specific rule might fall minimum treshold one datasets allow recalculate support itemset datasets set minimum support zero avoid problem probably combine support values different data sets thus setting minimum support zero probably split data mine association rules data set combine results end allows solve memory problem however minimum support zero apriori algorithm becomes dumb algorithm simply calculates support confidence possible itemsets data probably provide new problem ie processing powertime dividing keysets means splitting instances doesnt seem solution item dataset 1 item b data set 2 way find rule like ab final question actually means want use specific set keywords premise rule another set keywords consequence rule could change java code apriori algorithm kind regards benoît depaire oorspronkelijk bericht van mackmusdhbijzgslistscmswaikatoacnz mailtomackmusdhbijzgslistscmswaikatoacnz namens sasa verzonden zondag 24 februari 2008 053 aan iuxxdkqklistscmswaikatoacnz onderwerp wekalist association rules keywords large text usingwekajava hi using java weka try find association rules set keywords 35000 keywords large text around million sentences teh following algorithm used apriori number keywords n number sentences text k 1 create n attributes attribute corresponds one unique keyword attribute namekeyword attribute value0 1 2 create k sparse instances instance represents single sentence instance contains n number attributes representing keywords value 1 keyword present particular sentence 3 run buildassociations apriori following concerns correctly efficient way getting heap space error buildassociations use keywords sentences resolve already using xmx2024m option need divide text smaller groups sentences run code seperately guidelines combine association rules results still get heap space error way divide keywordsattributes sets manageable size run buildassociations apriori right algorithm use purpose 2 sets keywords want find associations keywords set1 keywords set2 using large text way help appreciated thank advance ____________________________________________________________________________ ________ looking last minute shopping deals find fast yahoo search httptoolssearchyahoocomnewsearchcategoryphpcategoryshopping _______________________________________________ wekalist mailing list iuxxdkqklistscmswaikatoacnz httpslistscmswaikatoacnzmailmanlistinfowekalist dlnirllcwaikatoacnz sat apr 26 154908 2008 message 4 date mon 25 feb 2008 094533 1300 peter reutemann subject wekalist setting delimiters wordtokenizer weka machine learning workbench list messageid 47c1d76d2040807waikatoacnz contenttype textplain charsetiso88591 formatflowed quick look xml option processing somethings working quite right requires lot time im currently swamped work im getting paid weka stuff hopefully ill time weekend look ok ive checked fix xml option processing options stored xml file grab cvs see faq get latest bugfixes details link faqs weka homepage cheers peter peter reutemann dept computer science university waikato nz httpwwwcswaikatoacnzfracpete 64 7 8384466 ext 5174 dlnirllcwaikatoacnz sat apr 26 154908 2008 message 5 date mon 25 feb 2008 094630 1300 peter reutemann subject wekalist setting delimiters wordtokenizer weka machine learning workbench list messageid 47c1d7a69010100waikatoacnz contenttype textplain charsetiso88591 formatflowed youre im mistaken second person using xml option ive heard count fact recommended colleague uses metaclassifiers metaclassifiers far easier commandline options span several lines using copypaste functionality explorer developer version isnt really problem anymore implemented copypaste feature sick long deeply nested commandlines ive submitted fix options xml file see following post httpslistscmswaikatoacnzmailmanhtdigwekalist2008february012465html hth cheers peter peter reutemann dept computer science university waikato nz httpwwwcswaikatoacnzfracpete 64 7 8384466 ext 5174 vafzspentahoorg sat apr 26 154908 2008 message 6 date mon 25 feb 2008 094745 1300 mark hall subject wekalist cobweb clustering weka machine learning workbench list messageid 20080224204744ga20320cygnuslocal contenttype textplain charsetusascii would say youd modify wekacoreattribute order support matrix values nominal type far cobweb goes hard give advice without details hoping achieve cheers mark sun feb 24 2008 121750pm 0500 priyanka sharma wrote hi hi trying modify cobweb algorithm clustering extending accept uncertain data nominal attribute type matrix possibly extend nominal attribute type priyanka sharma united states _______________________________________________ wekalist mailing list iuxxdkqklistscmswaikatoacnz httpslistscmswaikatoacnzmailmanlistinfowekalist mark hall senior developerconsultant pentaho open source business intelligence company citadel international suite 340 5950 hazeltine national dr orlando fl 32822 usa 5 grace avenue hamilton 3210 new zealand 64 7 8473537 office 64 21 399132 mobile 1 815 5508637 fax skype markandrewhall yahoo mark_andrew_hall download latest release today thu 07 aug 2008 181644 0500 wekalist digest vol 60 issue 34