On Thu, 2 Mar 2000, Boris Okun wrote:>I don't quite understand the logic of lost_ticks in do_gettimeofday().When the timer irq runs, it increases lost_ticks so that if the bh areinibithed we take care of the additional ticks that passed after wesnapshotted the new TSC state during the timer irq.>Andrea, could you explain it to me? I think this code really belongs to>do_fast_gettimeoffset() for the following 2 reasons:>1) When using do_slow_gettimeoffset() it's erroneus.do_slow_gettimeoffset is never erroneus, but it's slower and less precise.So on TSC capable hardware you really prefer to not use it but to use theTSC at get-time time (instead of doing sloww I/O at each get-time call).>2) You want to take lost_ticks into account in do_settimeofday() (when>undoing do_gettimeoffset().We just do that as far I can tell. See do_settimeofday() that callsdo_gettimeoffset().>To solve my problem I made the following changes (thanks to Artur>Skawina):>1) Made tsc_quotient[NR_CPUS] a per processor valueSo far the rule is been that on IA32 SMP machines the TSC goes at the samespeed on all CPUs. If that's not longer true not only do_gettimeoffsetwill break. If the TSC runs at different speed userspace will break too.Things like fftw uses the TSC for doing faster runtime benchmarking todynamically tune the algorithms.And anyway that change is a noop for your problem (see below).>2) Move calls to calibrate_tsc() to the same places where we call>calibrate_delay Since you said you have per-cpu tsc_quotient you could as well skipcalibrate_tsc, so this change meaningless w.r.t. gettimeofday.>2) Made last_tsc[NR_CPUS] a full 64 bit, instead of just last_tsc_low.That's useless since if the timer irq is not run for more than 10msec it'sa bug and you would lose time anyway if that would happen.The CPU should run a 429ghz to overflow the low part of the tsc in 10msecso we still have a few years before we need to use the high part ;). On amean 500mhz cpu it takes around 8 seconds to overflow the low tsc a8seconds is more than enough for IA32 CPUs.>3) Made delay_at_last_interrupt[NR_CPUS]That seems wrong. delay_at_last_interrupt is a per-timer-chip thing andthere's _only_ one timer chip not one chip per CPU.>4) Deleted lost_ticks code from do_gettimeofday. Use instead the full>difference >between current_tsc and last_tsc in do_fast_gettimeoffset(), not just>LSB.That's wrong. lost_ticks accounts the time that is going to be added toxtime but that's not been yet added because of bh inibithed at timer irqtime. At irq time you overwritten last_tsc and with only last_tsc youcan't calculate lost_ticks. So you can get wrong result out ofgettimeofday this way (if the timer irq gets delayed a bit more).>Now I have a perfectly monotone time and my interactive problems are>gone -- perfectly usable SMP system with different processors (despite>all naysayers ;-). If there is any interest I can post a patch.I guess the fact you handle delay_at_last_interrupt[NR_CPUS] wrong, hadthe side effect of hiding the probelm.See my supposion in my other email:On Mon, 28 Feb 2000, Andrea Arcangeli wrote:>>Date: Mon, 28 Feb 2000 02:32:09 +0100 (CET)>From: Andrea Arcangeli <andrea@suse.de>>To: Boris Okun <bokun@home.com>>Cc: "linux-kernel@vger.rutgers.edu" <linux-kernel@vger.rutgers.edu>>Subject: Re: gettimeofday non-monotonic on SMP 2.3.47>>On Sun, 27 Feb 2000, Boris Okun wrote:>>>in synchronize_tsc_bp because of 0 divide. My understanding is that this>>happens since fast_gettimeoffset_quotient is 0 now. So I put >>fast_gettimeoffset_quotient = calibrate_tsc();>>>>before #ifdef 0 in your patch.>>Correct.>>>If you have any suggestions, please let me know.>>Hmm maybe your timer chip is not reporting the time passed since the last ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^>timeout correctly. ^^^^^^^^^^^^^^^^^>>Andrea>>We just verifyed with my early patch that the TSC is not the problem sinceyou wasn't using it at all and you was still getting erroneous values.From the information I got from you I think it's a timer chip issue andnot a TSC problem at all. If that's is true you can try this patch:--- 2.3.49pre2/arch/i386/kernel/time.c Sun Jan 30 15:43:27 2000+++ /tmp/time.c Thu Mar 2 18:37:30 2000@@ -106,7 +106,7 @@ "0" (eax)); /* our adjusted time offset in microseconds */- return delay_at_last_interrupt + edx;+ return edx; } #define TICK_SIZE tick@@ -448,6 +448,7 @@ rdtscl(last_tsc_low); +#if 0 outb_p(0x00, 0x43); /* latch the count ASAP */ count = inb_p(0x40); /* read the latched count */@@ -455,6 +456,7 @@ count = ((LATCH-1) - count) * TICK_SIZE; delay_at_last_interrupt = (count + LATCH/2) / LATCH;+#endif } do_timer_interrupt(irq, NULL, regs);This patch will remove from the gettimeofday path any calculation wherethe timer chip latch is involved. BUT the above patch inserts a bug andwith the above patch applyed gettimeofday can report wrong results if thetimer irq gets delayed for a few msec by some code running clied. Maybe inyour patch you did something like the above and now you think all isrunning fine. I believe you are not getting right thedelay_at_last_interrupt[NR_CPUS] (exactly because it make no sense to makeit per-cpu) and may have the side effect of acting just like what theabove patch will do.BTW, what does happen if you run an UP compiled kernel? If my supposion isright you'll still get drift also with an UP kernel. Otherwise it probablymeans the early test we did was not reliable.Hmm...Andrea-To unsubscribe from this list: send the line "unsubscribe linux-kernel" inthe body of a message to majordomo@vger.rutgers.eduPlease read the FAQ at