haavard skinnemoen tfvatuqekpkatmelno patch aims improve support git buildbot specifically use git cogito support checking updating specific branch support checking updating specific revision sha1 implement parsegotrevision slave side implement computesourcerevision master side implement gitextractor class tryclient add testcases add postreceive hook contribgit_buildbotpy update documentation signedoffby haavard skinnemoen ive run test cases tried practice seems work nicely please let know spot something stupid tryclient stuff probably implemented best way possible work youre careful keeping local repository sync upstream repository regards håvard buildbotscriptstryclientpy 28 buildbotslavecommandspy 81 buildbotstepssourcepy 18 buildbottesttest_vcpy 147 contribgit_buildbotpy 254 docsbuildbottexinfo 93 6 files changed 579 insertions 42 deletions create mode 100755 contribgit_buildbotpy diff git abuildbotscriptstryclientpy bbuildbotscriptstryclientpy index 11f8e66dc398f7 100644 abuildbotscriptstryclientpy bbuildbotscriptstryclientpy 2036 20332 class darcsextractorsourcestampextractor daddcallbackselfreadpatch selfpatchlevel return class gitextractorsourcestampextractor patchlevel 1 vcexe git def getbaserevisionself selfdovcbranch nocolor v noabbrev daddcallbackselfparsestatus return def parsestatusself res current branch marked start line followed branch name sha1 branch names may contain pretty much anything whitespace researchr ss09af40 res remultiline selfbranch mgroup1 selfbaserev mgroup2 return raise indexerrorcould find current git branch res def getpatchself res selfdovcdiff selfbaserev daddcallbackselfreadpatch selfpatchlevel return def getsourcestampvctype treetop branchnone vctype cvs e cvsextractortreetop branch 2186 2448 def getsourcestampvctype treetop branchnone e mercurialextractortreetop branch elif vctype darcs e darcsextractortreetop branch elif vctype git e gitextractortreetop branch else raise keyerrorunknown vctype vctype return eget diff git abuildbotslavecommandspy bbuildbotslavecommandspy index 4fd348e5a3d636 100644 abuildbotslavecommandspy bbuildbotslavecommandspy 157 157 buildbotslaveregistry import registerslavecommand used cvs style revision autoupdated keyword since moved darcs primary repository updated manually time file changed last cvs_ver 151 command_version 23 command_version 24 version history 117 commands interruptable 356 357 command_version 23 release 074 22 added monotone uploadfile downloadfile release 075 23 added bzr 24 git understands revision branches class commandinterruptedexception pass 17187 17199 class gitsourcebase git specific vc operation addition arguments handled sourcebase command reads following keys repourl required cogito repository string repourl required upstream git repository string branch optional version ie branch tag retrieve default master header git operation 172631 172975 class gitsourcebase def setupself args sourcebasesetupself args selfrepourl argsrepourl selfsourcedata todo selfbranch argsgetbranch selfbranch selfbranch master selfsourcedata sn selfrepourl selfbranch def _fullsrcdirself return ospathjoinselfbuilderbasedir selfsrcdir def _commitspecself selfrevision return selfrevision return selfbranch def sourcedirisupdateableself ospathexistsospathjoinselfbuilderbasedir selfsrcdir buildbotpatched ospathexistsospathjoinself_fullsrcdir buildbotpatched return false return ospathisdirospathjoinselfbuilderbasedir selfsrcdir git return ospathisdirospathjoinself_fullsrcdir git def _didfetchself res selfrevision head selfrevision else head fetch_head command gitreset hard head c shellcommandselfbuilder command self_fullsrcdir sendrcfalse timeoutselftimeout selfcommand c return cstart def dovcupdateself ospathjoinselfbuilderbasedir selfsrcdir command cgupdate c shellcommandselfbuilder command command gitfetch selfrepourl selfbranch selfsendstatusheader fetching branch sn selfbranch selfrepourl c shellcommandselfbuilder command self_fullsrcdir sendrcfalse timeoutselftimeout selfcommand c return cstart cstart daddcallbackself_abandononfailure daddcallbackself_didfetch return def _didinitself res return selfdovcupdate def dovcfullself ospathjoinselfbuilderbasedir selfsrcdir osmkdird command cgclone selfrepourl c shellcommandselfbuilder command osmkdirself_fullsrcdir c shellcommandselfbuilder gitinit self_fullsrcdir sendrcfalse timeoutselftimeout selfcommand c return cstart cstart daddcallbackself_abandononfailure daddcallbackself_didinit return def parsegotrevisionself command gitrevparse head c shellcommandselfbuilder command self_fullsrcdir sendrcfalse keepstdouttrue cusepty false cstart def _parseres hash cstdoutstrip lenhash 40 return none return hash daddcallback_parse return registerslavecommandgit git command_version 22007 22477 class p4sourcebase return false assume client spec still around say arent updateable dir doesnt exist dont get enoent checking sourcedata dont get enoent checking surlourcedata return ospathisdirospathjoinselfbuilderbasedir selfsrcdir diff git abuildbotstepssourcepy bbuildbotstepssourcepy index dac7dea1059138 100644 abuildbotstepssourcepy bbuildbotstepssourcepy 57115 57125 class gitsource name git def __init__self repourl kwargs def __init__self repourl branchmaster kwargs type repourl string param repourl url points git repository type branch string param branch branch tag check default build specifies different branch used instead selfbranch none todo source__init__self kwargs selfaddfactoryargumentsrepourlrepourl selfargsrepourl repourl selfaddfactoryargumentsrepourlrepourl branchbranch selfargsupdaterepourl repourl branch branch def computesourcerevisionself changes changes return none return changes1revision def startvcself branch revision patch selfargsbranch branch diff git abuildbottesttest_vcpy bbuildbottesttest_vcpy index 73eb9790177feb 100644 abuildbottesttest_vcpy bbuildbottesttest_vcpy 3427 3428 class basehelper selfbranchappendrev selfallrevsappendrev def runcommandself basedir command failureisokfalse stdinnone def runcommandself basedir command failureisokfalse stdinnone envnone commands passed strings lists strings none arguments may spaces makes commands less verbose expense restricting 3558 35610 class basehelper print basedir basedir stdin print stdinn stdin nstdin done env osenvironcopy envlc_all c env env osenvironcopy envlc_all c mygetprocessoutputandvaluecommand0 command1 envenv pathbasedir stdinstdin 37919 38219 class basehelper daddcallbackcheck return def doself basedir command failureisokfalse stdinnone def doself basedir command failureisokfalse stdinnone envnone selfruncommandbasedir command failureisokfailureisok stdinstdin stdinstdin envenv return waitfordeferredd def dovcself basedir command failureisokfalse stdinnone def dovcself basedir command failureisokfalse stdinnone envnone like vc binary prepended command isinstancecommand str unicode command selfvcexe command else command list command selfvcexe command return selfdobasedir command failureisok stdin return selfdobasedir command failureisok stdin env class vcbasesignalmixin metadir none 24666 2469136 class mercurialvcbase unittesttestcase vcsregistervcmercurialvc_name mercurialhelper class githelperbasehelper branchname branch try_branchname branch def capableself gitpaths whichgit gitpaths selfvcexe gitpaths0 return true none return false git installed def createrepositoryself selfcreatebasedir selfgitrepo ospathjoinselfrepbase gitrepository tmp ospathjoinselfrepbase gittmp env osenvironcopy envgit_dir selfgitrepo w selfdovcselfrepbase init envenv yield w wgetresult selfpopulatetmp w selfdovctmp init yield w wgetresult w selfdovctmp add yield w wgetresult w selfdovctmp commit initial_import yield w wgetresult w selfdovctmp checkout b selfbranchname yield w wgetresult selfpopulate_branchtmp w selfdovctmp commit commit_on_branch yield w wgetresult w selfdovctmp revparse master selfbranchname yield w wgetresult revs outsplitlines selfaddtrunkrevrevs0 selfaddbranchrevrevs1 w selfdovctmp push selfgitrepo master selfbranchname yield w wgetresult rmdirrecursivetmp createrepository deferredgeneratorcreaterepository def vc_reviseself tmp ospathjoinselfrepbase gittmp rmdirrecursivetmp logmsgvc_revise selfgitrepo w selfdovcselfrepbase clone selfgitrepo gittmp yield w wgetresult selfversion 1 version_c version_c selfversion openospathjointmp versionc wwriteversion_c w selfdovctmp commit revised_to_d selfversion versionc yield w wgetresult w selfdovctmp revparse master yield w wgetresult selfaddtrunkrevoutstrip w selfdovctmp push selfgitrepo master yield w wgetresult rmdirrecursivetmp vc_revise deferredgeneratorvc_revise def vc_try_checkoutself workdir rev branchnone assert ospathabspathworkdir workdir ospathexistsworkdir rmdirrecursiveworkdir w selfdovcselfrepbase clone selfgitrepo workdir yield w wgetresult branch none w selfdovcworkdir checkout b branch origins branch yield w wgetresult hmmwhy nobody else bother check correct revision w selfdovcworkdir reset hard rev yield w wgetresult try_c_filename ospathjoinworkdir subdir subdirc opentry_c_filename wwritetry_c vc_try_checkout deferredgeneratorvc_try_checkout def vc_try_finishself workdir rmdirrecursiveworkdir class gitvcbase unittesttestcase vc_name git export mode yet metadir git vctype sourcegit vctype_try git has_got_revision true def testcheckoutself selfhelpervcargs repourl selfhelpergitrepo selfdo_vctest return def testpatchself selfhelpervcargs repourl selfhelpergitrepo branch master selfdo_patch return def testcheckoutbranchself selfhelpervcargs repourl selfhelpergitrepo branch master selfdo_branch return def testtryself selfhelpervcargs repourl selfhelpergitrepo branch master selfdo_getpatch return vcsregistervcgitvc_name githelper class sourcesunittesttestcase todo needs serious rethink diff git acontribgit_buildbotpy bcontribgit_buildbotpy new file mode 100755 index 0000000513209c devnull bcontribgit_buildbotpy 00 1254 usrbinenv python script meant run hookspostreceive git repository expects one line new revision form example aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refsheadsmaster changes passed buildbot server along change information manage extract repository largely based contribhookspostreceiveemail git import commands logging os sys twistedspread import pb twistedcred import credentials twistedinternet import reactor buildbotscripts import runner optparse import optionparser modify fit setup master localhost9989 git_dir environment variable must set git commands executed operate repository installed changes def connectfailederror loggingerrorcould connect master errorgeterrormessage return error def addchangedummy remote changei loggingdebugaddchange reprremote reprchangei try c changeinext except stopiteration remotebrokertransportloseconnection return none logginginfonew revision crevision8 key value citeritems loggingdebug key value remotecallremoteaddchange c daddcallbackaddchange remote changei return def connectedremote return addchangenone remote changes__iter__ def grab_commit_infoc rev extract information committer files using gitshow f ospopengitshow raw prettyfull rev r files true line freadline line break rematchrmads line loggingdebuggot file mgroup1 filesappendmgroup1 continue rematchrcommits line loggingdebuggot committer mgroup1 cwho mgroup1 cfiles files status fclose status loggingwarninggitshow exited status status def gen_changesinput branch true line inputreadline line break loggingdebugchange line rematchr09af linestrip c revision mgroup1 comments mgroup2 branch branch grab_commit_infoc mgroup1 changesappendc def gen_create_branch_changesnewrev refname branch new branch created generate changes everything newrev exist branch refname note may inaccurate two new branches created time pointing commit commits exists common subset new branches logginginfobranch created branch f ospopengitrevparse branches grep v gitrevparse refname gitrevlist reverse prettyoneline stdin newrev r gen_changesf branch status fclose status loggingwarninggitrevlist exited status status def gen_update_branch_changesoldrev newrev refname branch branch updated fastforward update generate change events everything oldrev newrev case forced update first generate fake change event rewinding branch common ancestor oldrev newrev generate change events commit common ancestor newrev logginginfobranch updated branch oldrev8 newrev8 baserev commandsgetoutputgitmergebase oldrev newrev loggingdebugoldrevs newrevs baserevs oldrev newrev baserev baserev oldrev c revision baserev comments rewind branch branch branch dummy logginginfobranch rewound branch baserev8 files f ospopengitdiff raw ss oldrev baserev r true line freadline line break file rematchrmads linegroup1 loggingdebug rewound file file filesappendfile status fclose status loggingwarninggitdiff exited status status files cfiles files changesappendc newrev baserev pure rewind f ospopengitrevlist reverse prettyoneline ss baserev newrev r gen_changesf branch status fclose status loggingwarninggitrevlist exited status status def cleanupres reactorstop def process_changes read branch updates stdin generate change events true line sysstdinreadline line break oldrev newrev refname linesplitnone 2 care regular heads ie branches rematchrrefsheads refname logginginfoignoring refname branch refname continue branch mgroup1 find branch created deleted updated branches deleted arent really interesting rematchr0 newrev logginginfobranch deleted ignoring branch continue elif rematchr0 oldrev gen_create_branch_changesnewrev refname branch else gen_update_branch_changesoldrev newrev refname branch submit changes changes loggingwarningno changes found return host port mastersplit port intport f pbpbclientfactory flogincredentialsusernamepasswordchange changepw reactorconnecttcphost port f dadderrbackconnectfailed daddcallbackconnected daddbothcleanup reactorrun def parse_options parser optionparser parseradd_optionl logfile actionstore typestring helplog specified file parseradd_optionv verbose actioncount helpbe verbose ignored l specified options args parserparse_args return options log errors critical messages stderr optionally log information file well well set later stderr loggingstreamhandlersysstderr fmt loggingformattergit_buildbot levelnames messages stderrsetlevelloggingerror stderrsetformatterfmt logginggetloggeraddhandlerstderr logginggetloggersetlevelloggingdebug try options parse_options level loggingwarning optionsverbose level 10 optionsverbose level 0 level 0 optionslogfile logfile loggingfilehandleroptionslogfile logfilesetlevellevel fmt loggingformatterasctimes levelnames messages logfilesetformatterfmt logginggetloggeraddhandlerlogfile process_changes except loggingexceptionunhandled exception sysexit1 diff git adocsbuildbottexinfo bdocsbuildbottexinfo index 37c8411e48a098 100644 adocsbuildbottexinfo bdocsbuildbottexinfo 12689 12689 archbazbazaar labeled tag used cvsfootnotemany vc systems provide complexity particular local views p4 clearcase assemble various source directories complex prepared take advantage sha1 revision id used monotone mercurial also kind revision stamp specifies unique copy source tree darcs context file sha1 revision id used monotone mercurial git also kind revision stamp specifies unique copy source tree darcs context file arent intending make changes sources check least need committed back upstream two 14686 146816 branch stored separate repository coderepourl way darcs revision however hash identifier returned commandhg identify urefhttpgitorcz git also follows decentralized model repository several branches tags source step configured static coderepourl specifies location repository addition optional codebranch parameter specified check code specific branch instead default master branch revision specified sha1 hash returned eg commandgit revparse attempt made ensure specified revision actually subset specified branch node attributes changes different vc systems specify sources version control systems subsection attributes changes 15266 15369 consumed codecomputesourcerevision method appropriate coderevision full revision id ending patchd item p4 coderevision int transaction number item git coderevision short string sha1 hash output eg codegit revparse end table heading branches 153414 154715 change might also codebranch attribute indicates changes files named branch schedulers get decide whether branch built vc systems like cvs arch monotone codebranch name unrelated filename branch name filename inhabit unrelated namespaces svn branches expressed subdirectories repository files svnurl combination base url branch name filename within branch sense branch name filename inhabit namespace darcs branches subdirectories base url like svn mercurial branches darcs vc systems like cvs arch monotone git codebranch name unrelated filename branch name filename inhabit unrelated namespaces svn branches expressed subdirectories repository files svnurl combination base url branch name filename within branch sense branch name filename inhabit namespace darcs branches subdirectories base url like svn mercurial branches darcs table samp item cvs 15546 15688 branchwarnernewfeature filessrcfooc branchwarnernewfeature filessrcfooc item archbazaar branchbuildbotusebranches0 filesbuildbotmasterpy item git branchwarnernewfeature filessrcfooc end table heading links 27166 273212 hook codecontribarch_buildbotpy run commit hook end itemize item git itemize bullet item pbpbchangesource listening connections codecontribgit_buildbotpy run postreceive hook end itemize end table vc systems driven pbchangesource 29656 298712 httpopensourceperligdeensvnmailer httpwwwseleniccommercurialwikiindexcginotifyextension end table item git table samp item postreceiveemail httpgitkernelorgpgitgitgitablobfcontribhookspostreceiveemailhbhead end table end table 38016 38297 arguments described following pages bazaar bzr p4 git end menu node cvs svn source checkout source checkout 41347 41637 passed codebzr checkout command node p4 bzr source checkout node p4 git bzr source checkout subsubsection p4 cindex perforce update 41756 420429 replace slaves slave name builders builder name default buildbot_slaves_builds end table node git p4 source checkout subsubsection git cindex git checkout bsindex buildbotstepssourcegit codegit build step clones updates urefhttpgitorcz git repository checks specified branch revision git step takes following arguments table code item repourl required url upstream git repository item branch optional specifies name branch use build provide one parameter specified build provide branch master branch used end table node shellcommand simple shellcommand subclasses source checkout build steps subsection shellcommand 69147 69667 tree use may convenient use codetry_topfile approach instead vc systems work full projects instead individual directories tla baz darcs monotone mercurial require directories tla baz darcs monotone mercurial git require commandtry know top directory optiontrytopfile optiontrytopdir arguments ignored c true think currently require topdirs time 69937 704520 revision commandtry work working directory must patches available remotelyavailable repository build process codestepmercurial use c todo monotone git item git codegit branch v lists branches available local repository along revision id points short summary last commit line containing currently checked branch begins star space others start two spaces commandtry scans line extracts branch name revision generates diff base revision c todo im sure actually works way intended c since extracted base revision might actually exist c upstream repository perhaps need add remote option c specify remote tracking branch generate diff c todo monotone end table heading waiting results 1525 sfnet email sponsored splunk inc still grepping log files find problems stop search log events configuration files using ajax browser download free copy splunk httpgetsplunkcom _______________________________________________ buildbotdevel mailing list buildbotdevellistssourceforgenet httpslistssourceforgenetlistslistinfobuildbotdevel wed 06 aug 2008 121532 0200 buildbotdevel patch better support git