The dumbfuckery of storage drive data integrity

With all the complicated standards including data integrity, it is outrageous how useless it all still is.

I got a 4 TB WD Black harddisk that is probably 99.99999999% just fine, but it is telling me it will die in less than 24 hours and I better throw it in the trash.

If your harddisk suffers from a bad sector, the file occupying it is ruined. But you usually won’t even know that until you try to access it manually, directly, intentionally, and that fails.

Generally you might get a warning of some kind. So you know there’s a defect.
Now, to find out where it is, you need a special tool like GSmartControl that can show you the log and then the 24 or so latest entries shown and spammed with the same error are hopefully indicating only onw bad sector.

Then you would have to use a Windows command line tool nfi.exe to inquire what file if any is using that sector. And once you get that info you know which one is damaged and thus lost.

But now the real insanity starts: Unless repair attempts on the sector succeed, no dozens of failed and time-wasting read attempts can convince the firmware to reallocate the sector, to replace it with a spare or simply block access. I haven’t seen it happening so I cannot even confirm it, but AFAIU this only happens if you WRITE to that sector the next time. BUT… things like secure erase of the file won’t do, because the sector also needs to be unused before you attempt to write to it.
AND apparently even secure erase still leaves a filesystem remnant, basically a nameless undeletable garbage file, and it seems that this is still considered part of the filesystem for God knows how long. Because I verified that the bad sector is not used by any file, and yet chkdsk gets stuck for all eternity on the files check at the start, as if that sector was still part of the filesystem contents. (And if you try chkdsk /b, it might take far too long to get to the damaged spot and even there it is not guaranteed to succeed.)

So to clarify: Failing to read a single damn spot on the disk for half an hour that procudes tons of SMART log spam about the exact location does not convince the drive to replace that bit.

Also, DiskGenius can do targeted scans of disk surface to identify damaged regions, but its repair attempt will fail quite like chkdsk, endless freeze. Sadly it doesn’t offer to replace it without that futile attempts. (It did manage to repair a weak sector, but since SMART does not report a new reallocation event, I have to assume it merely repaired it through writes, so it could start making trouble again soon.)

HDDScan offers such a surface test, too, but its output seems to get stuck on reporting bad sectors long after the one that actually is bad.

But the most frustrating thing about all this pretend-SMARTness that is F.U.C.K.I.N.G.D.U.M.B. is that while I am being informed there is “Current Pending Sector Count = 1”, it does not inform me which sector is the pending one that I haven’t identified yet, even though it would have to know the position from knowing it is there.

More or less realistic options you have:

  • Go to Linux and use command line hackery with smartctl and hdparm to manually surgically replace the bad sector.
  • Alter the disk’s partitioning to exclude the area with the defective sector. (DiskGenius can tell you the megabyte-based region on the disk based on the sector.)
  • Slow-format or secure-erase the whole disk. (But AFAIR I have had issues with such in the past because the system pretended the problems had been repaired but then they reoccured later after I recopied all the data onto it. Not on those writes, apparently, but afterwards.)

So, since I have an installation of that relatively shitty Linux, I went there and after some basic hassle this command did the trick:
“sudo hdparm –write-sector 6960709 –yes-i-know-what-i-am-doing /dev/sdc”
It is described as an alias for “–repair-sector” and I thought ‘damn, not that again’. And it went so quick with no delay whatsoever and reading the sector only yielded zeroes that I was wondering whether it was the correct sector, but after checking back in Windows, the obstacle was gone!
So apparently it can be easy to accomplish such simple and intelligent things!
(But as I keep pointing out, Linux has its own infuriating shortcomings.)

I am still not over my skepticism, though. The reallocated sector count is still the same, so I have to wonder whether under Windows the repair attempt just failed for some reason or whether under Linux that method simply doesn’t trigger the counter.
Also, Offline Uncorrectable is still 1, and I read it is supposed to go back to 0 when everything is fine, not be a history-type statistic, although that’s unreliable info.

Now, what SHOULD happen in this data safety system, as an integral part, is that the OS warns and informs which file is affected by a defective sector and not in its original state anymore, and the drive should also reallocate sectors that are just weak, because in my view there is no such thing as a repaired sector, as my own experience with this drive has shown. If it is weak, it cannot be trusted anymore, and there should be plenty of spares. It is no drama to lose even a couple megabytes, although that will rarely be the case, and if so, the harddisk is probably finished. – But to cause such an obstacle out of a single sector, that is nuts. A late surprise of data corruption and a huge drama. Very unprofessional and sloppy, kinda defying the purpose of accurately tracking what is happening with the data.

The existing system looks like not designed primarily to protect your data but to convince you to buy a new product.

Shopping cart requirement – idiot-proof is always bad

Some supermarkets have implemented a rule that makes me as a smart, thoughtful person cringe. I went to one today that had implemented that rule that forces you to use a shopping cart – so that you keep distance to others!
This is one of those stupid-simplistic ideas that have potential to cause more harm than good.

Awareness of the COVID problem is widespread anyway. In other markets you can see people being thoughtful in their conduct anyway, but regardless of rule of not, people can still act in a stupid way.

Interacting with anything touched by many people is an issue and that made me turn around and not do business with said market. They offered some dispensers for gloves and disinfectant wipes, but you also have to interact with those, and the wipes might not be very effective. And even though in that case I could at least disinfect in my own thourough way, other people are likely less thorough and, as is so common in society, execute standard rituals that are expected from them and then believe they did enough. This is a tendency incentivized even more when a measure is mandatory.
And the coin slot of the shopping carts might be a problem zone. (Also, if you use physical money all high-level safety goes out the window anyway.) I wasn’t in the mood to do a detailed study, but my general behavior based on own thoughtfulness resisted a coersive measure to interact with something I do not need.

This is a frequent experience of mine: The pain of living in an idiot society, to say it bluntly. Whenever I can freely do things my way, it works well, because I think about what I do and focus on the outcome. But too often one is forced into idiocies like this. Rules that are designed to manage severe idiocy of the rabble are a really bad compromise, and smart people suffer from it.

This is probably in line with various other disinfection acts we can see in the world that very much look like only being done so governments can say they did something. Like disinfecting the pyramids of Gizeh, as if the burning sun and dry hot sand of the place gave the virus any chance. Or disinfecting a popular tourist site that has been closed for a while anyway and continues to do so, so any viral contamination there would be dead by now, or if there is interaction through animals and weather, would be restored quickly anyway.

There are also at least more thoughtful ways to implement the shopping cart system. Another supermarket had a cart ready at the entrance, no coin needed, and it is ready and thoroughly disinfected by an employee, including side railings and everything. But in that case I still wouldn’t trust it as much as my own process, if I was to do it, which I wouldn’t, because I don’t need a cart and naturally keep distance to people! (When I was there, apparently the employee was somewhere else, so it is not a rigidly enforced system, and that is good. It should be seen as a service provided for those who do not object. It was probably not driven by that stupid unreliable idea of forced distancing, but was a safer way to offer shopping carts for those who use them anyway.)

The safe-and-stupid way of steering the rabble instead of educating people into self-reliance is a constant issue that also leads to erosion of democracy. Even in such a crisis, governments try to avoid having to turn people into intelligent, self-reliant beings who can run society just fine without government meddling. Instead, early-on they were offered a buffet of pundit opinions, and then pestered with said questionable panic measures.

People Power cannot be avoided forever. But since it is the enemy of capitalism, there will be more suffering for humankind as long as capitalism dominates. Smart people (again) understood the inherent problems with it long ago. Others need a pandemic to see the obvious, in gross intensification.