august end month close summary debrief august end month close behind us learned things made progress also issues work summary improvements end month close would like congratulate users application team dba team much improved end month close august gas would possible without significant commitment focus teamwork exhibited folks involved effort tangible measure progress july close able keep users system throughout final week one kicked system gas accounting close process finished time key users folks work significant overtime finish process improvements arose upgrading unify system sybase 12 extra server memory reduced system locking blocking made possible keep users system throughout week nightly settlements gas invoicing also ran much faster many thanks dbas jim ogg kishroe bharathi many late nights weekends many thanks also application team users lots testing production rollout conducting pre planning session identified new end month techniques improve efficiency users generated several new ideas including idea pre process pmas night running current mass draft meeting provided several new approaches clear plan cohesive communication throughout end month process many thanks users well bob superty bryce baxter melissa ratnala dave nommensen chris schomer making happen many ideas part standard process performance monitoring tuning team reacted quickly address changes system performance many thanks matt pena rest team keeping system running fast possible day release new tuned version accurals process runs much faster issues challenges action items made progress month still several significant issues opportunities improvement following list things went bump night action items identified 1 experienced several sybase 12 0 errors early week impacted gas business problems fixed following business impacts bridgeback deals sitara delayed deals bridged manually end month valuation also fixed code problem generated th original problem global systems replication unify monday problem fixed software patch sybase monday evening affected deals uploaded nominations completed deals monday manual estimation schedulers communication dba group application group delayed global systems replication problem delayed ability notify users problem communication problems improved future default assumption notify key application infrastructure personnel soon serious problem identified 2 experienced hardware issue bad server cpu 3 00 thursday result database backups ran morning server came back created slow system response delayed logins users 10 00 problem fixed action item kishore coordinate randy matson infrastructure group coordinate stress test production server components allow us see find marginal components completed september 18 3 service pmas ena hplc picked settlements process hplc pmas run manually users ena missing pmas immaterial users decided close results found fixed error caused problem action item chris schomer team investigate problem see sybase 12 0 issue whether error could impacted previous months determination available september 15 4 enpower deal load load deals referenced prices correctly deals successfully loaded currently completed mass draft process power identified fixed software error caused problem 5 power mass draft running slow compared prior months currently working complete power mass draft process completed user deadline close business wednesday sept 6 action item chris schomer team investigating power mass draft running slow present definitive answer hope develop definitive answer action plan september 15 current hypothesis problem includes data changes much higher volume hourly priced deals potential database index issues build power database indexes compile stored procedures sybase optimizer flakiness often see sybase suddenly start processing problems apparent reason update future tactical strategic improvements several pending tactical unify performance improvements process include 1 isolating unify gas server 2 delivery middle tier mass draft process reduce locking blocking 3 upgrade middle tier mass draft process allow drafting deal level rather counter party level 4 delivery nominations balancing process middle tier architecture ramesh rao team strategic projects progressing schedule seeing significant performance gains testing sql server sybase also identified couple hardware vendors could provide large scaleable computing environments type unify architecture plan critical evaluation testing completed end september planned present recommendations along high level strategic delivery plan early october let know questions concerns x 39694 713 503 4873 thanks