iMac beachballing for no reason — I suspect it might be faulty hardware. What can I do to work around it? The Problem:


A couple of weeks ago, the iMac started "beachballing" regularly in normal use
There is no discernible pattern to what triggered a beachball. It could be switching between windows, trying to type, using Spotlight, opening or closing an application, or even just whilst idle.
Over the next few days, the hangs became so frequent it rendered the iMac effectively unusable
I tried literally everything I could think of to fix it (detailed below), but to no avail
The only thing that finally solved it was wiping the disk and reinstalling everything from a Time Machine backup
But about a week later, the same thing began happening again. I still couldn't figure it out, so was forced to reinstall (again!)
And now, yesterday, it's come back once more


I am at my wit's end. I work in IT, and rely on the iMac to be able to work from home whilst my country is in lockdown. My local Apple Store is obviously closed. Each Time Machine restoration takes almost 18 hours to complete, meaning I'm unable to work during that time (never mind the time waiting on the SBBOD to disappear).

The iMac:


2017 27-inch Retina 5K iMac
macOS Catalina 10.15.4
32GB RAM
4.2GHz i7 processor
Fusion Drive (APFS, comprised of a 128GB SSD + 2TB HDD)
T2 security chip


What I've Already Tried:


Checking Activity Monitor during beachballing


CPU: load is usually ~20%
Memory: 0 swap used, ~16GB free memory remaining

Signing out of iCloud
Disabling Spotlight indexing
Creating a new user account
SMC reset
NVRAM/PRAM reset
Disk Utility First Aid
Boot into Safe Mode
Boot into Recovery Mode, and use diskutil verifyVolume and repairVolume
Repairing Home Folder permissions
Checking Console and reviewing system.log and other files


No correlation between beachballing and log entries

Running a spindump via Activity Monitor


Worryingly, spindumps don't show up — they just result in a Crash Report appearing in Console (see here)

Running EtreCheck


No major issues found
No noteworthy minor issues found

Running DriveDx


The results for the SSD were fine
The results for the HDD do look a bit concerning — see here.

Running Apple's hardware diagnostic test at startup


No faults found

Reinstalling macOS


My Suspicions:

I found it interesting that reinstalling macOS does NOT fix the problem, but wiping the Fusion drive, reinstalling macOS, AND restoring from a Time Machine backup in one go DOES fix it (at least temporarily).

My DriveDx results seem to indicate the HDD is in a "failing" state. I don't know if this is just precautionary or if this could actually be causing it. It's hard to pin the blame on this directly, as the beachballs do not seem to correlate with disk I/O, but I can't say that for certain as I don't know enough about this area. I also would've assumed that the Apple hardware diagnostic test you can run at startup might have identified this if it really was a problem...?

So my hunch right now, in the absence of any more obvious clues, is that maybe this problem is caused by a failing HDD in the Fusion drive...? That might explain how wiping and reformatting the drive lets things work smoothly for a week or so, but after enough new sectors have been written to and data shuffled around on the disk, it starts to slow down after time. That's just a hunch!

What Next?


Do you have any other suggestions of things to check or try?
Are there any other clues I might have missed?
Do you think my hypothesis about the failing HDD is right?


If so, I understand the HDD in a Fusion drive is not user-serviceable...
So I would assume the next best alternative would be to buy e.g. 2TB Thunderbolt drive and use that as the startup disk, instead of the Fusion drive?
Is this possible? Are there any drawbacks? Any brands you'd recommend?



Thanks for reading this lengthy post. Any advice or guidance is really appreciated!

Jack
I'm just going to echo Allan's comment: if DriveDx says your drive is "failing" then it needs to be replaced. The only workaround is to remove the drive and stop using it.

Disk health checks have a low sensitivity but high specificity. In other words, it might report that a failing drive still looks healthy, but it will never report a healthy drive as failing. If it reports failing, we must trust that result.

The reason that wiping the drive helps, at least for a little while, is that it does more work to avoid using the parts of the drive that are already known to be bad. But the drive is like a rotting fruit 
 -- even the good parts are going bad, so eating around the bad part doesn't help for long.