Technology

The dumbfuckery of storage drive data integrity

With all the complicated standards including data integrity, it is outrageous how useless it all still is.

I got a 4 TB WD Black harddisk that is probably 99.99999999% just fine, but it is telling me it will die in less than 24 hours and I better throw it in the trash.

If your harddisk suffers from a bad sector, the file occupying it is ruined. But you usually won’t even know that until you try to access it manually, directly, intentionally, and that fails.

Generally you might get a warning of some kind. So you know there’s a defect.
Now, to find out where it is, you need a special tool like GSmartControl that can show you the log and then the 24 or so latest entries shown and spammed with the same error are hopefully indicating only onw bad sector.

Then you would have to use a Windows command line tool nfi.exe to inquire what file if any is using that sector. And once you get that info you know which one is damaged and thus lost.

But now the real insanity starts: Unless repair attempts on the sector succeed, no dozens of failed and time-wasting read attempts can convince the firmware to reallocate the sector, to replace it with a spare or simply block access. I haven’t seen it happening so I cannot even confirm it, but AFAIU this only happens if you WRITE to that sector the next time. BUT… things like secure erase of the file won’t do, because the sector also needs to be unused before you attempt to write to it.
AND apparently even secure erase still leaves a filesystem remnant, basically a nameless undeletable garbage file, and it seems that this is still considered part of the filesystem for God knows how long. Because I verified that the bad sector is not used by any file, and yet chkdsk gets stuck for all eternity on the files check at the start, as if that sector was still part of the filesystem contents. (And if you try chkdsk /b, it might take far too long to get to the damaged spot and even there it is not guaranteed to succeed.)

So to clarify: Failing to read a single damn spot on the disk for half an hour that procudes tons of SMART log spam about the exact location does not convince the drive to replace that bit.

Also, DiskGenius can do targeted scans of disk surface to identify damaged regions, but its repair attempt will fail quite like chkdsk, endless freeze. Sadly it doesn’t offer to replace it without that futile attempts. (It did manage to repair a weak sector, but since SMART does not report a new reallocation event, I have to assume it merely repaired it through writes, so it could start making trouble again soon.)

HDDScan offers such a surface test, too, but its output seems to get stuck on reporting bad sectors long after the one that actually is bad.

But the most frustrating thing about all this pretend-SMARTness that is F.U.C.K.I.N.G.D.U.M.B. is that while I am being informed there is “Current Pending Sector Count = 1”, it does not inform me which sector is the pending one that I haven’t identified yet, even though it would have to know the position from knowing it is there.

More or less realistic options you have:

  • Go to Linux and use command line hackery with smartctl and hdparm to manually surgically replace the bad sector.
  • Alter the disk’s partitioning to exclude the area with the defective sector. (DiskGenius can tell you the megabyte-based region on the disk based on the sector.)
  • Slow-format or secure-erase the whole disk. (But AFAIR I have had issues with such in the past because the system pretended the problems had been repaired but then they reoccured later after I recopied all the data onto it. Not on those writes, apparently, but afterwards.)

So, since I have an installation of that relatively shitty Linux, I went there and after some basic hassle this command did the trick:
“sudo hdparm –write-sector 6960709 –yes-i-know-what-i-am-doing /dev/sdc”
It is described as an alias for “–repair-sector” and I thought ‘damn, not that again’. And it went so quick with no delay whatsoever and reading the sector only yielded zeroes that I was wondering whether it was the correct sector, but after checking back in Windows, the obstacle was gone!
So apparently it can be easy to accomplish such simple and intelligent things!
(But as I keep pointing out, Linux has its own infuriating shortcomings.)

I am still not over my skepticism, though. The reallocated sector count is still the same, so I have to wonder whether under Windows the repair attempt just failed for some reason or whether under Linux that method simply doesn’t trigger the counter.
Also, Offline Uncorrectable is still 1, and I read it is supposed to go back to 0 when everything is fine, not be a history-type statistic, although that’s unreliable info.


Now, what SHOULD happen in this data safety system, as an integral part, is that the OS warns and informs which file is affected by a defective sector and not in its original state anymore, and the drive should also reallocate sectors that are just weak, because in my view there is no such thing as a repaired sector, as my own experience with this drive has shown. If it is weak, it cannot be trusted anymore, and there should be plenty of spares. It is no drama to lose even a couple megabytes, although that will rarely be the case, and if so, the harddisk is probably finished. – But to cause such an obstacle out of a single sector, that is nuts. A late surprise of data corruption and a huge drama. Very unprofessional and sloppy, kinda defying the purpose of accurately tracking what is happening with the data.

The existing system looks like not designed primarily to protect your data but to convince you to buy a new product.

This video’s message sounds great, but it is painfully naive/foolish

I’ll mostly write my Youtube comment here. It is important to not happily stumble into more of the same folly, and that video bothers me a lot. It’s like a bullshit indoctrination commercial for more of the same insanity. I guess that it was basically funded by Bill Gates speaks for itself.

0:59 Everything changed with the industrial revolution? LOL! Zero sum/positive sum is all in the mind and it never changed because the people with power were and are still craving more, more, MORE as always. And it’s still zero sum if there’s just a temporary growth, if the sum cannot grow indefinitely and sustainably. This is exactly the mind’s folly that keeps perpetuating the problems. The solution is not “positive sum”, but ‘doesnmattersum’. Transcending the thinking that creates the problem.
1:50 More of this BS, claiming the growth of prosperity is continuing today when we see violent conquest and mass theft of all kinds as a way of life for certain countries. (Also, I guess once someone dies, they fall out of the statistic of how many things people own, hah.)
3:03 And there we see that what’s called “positive sum world” is just a choice of what speed of change to label as such and what not. Again, it’s an attitude and perception thing. With the speed of change came more pressure by the limited ‘confines’ of this planet and more total power of greed, so guess what the bottom line is: same. No change to society if it’s all external. But back then we had the same problems, and since there were less people living on the planet, it was just as foolish and sick. Or did we forget the rule of monarchs in their lavish castles and the miserable poverty among the People? Without that crap you could go and call those times a positive sum world. But since we still have that crap today, I don’t see a distinction as valid.
Videos like this emerge from minds too enamoured by science and technology and not enough by spirituality. (And I don’t mean the label, the mass culture, I mean the looking inwards, the very personal, the part where you actually discover your own bullshit.)
4:30 Oh yeah of course, now the cancer cure clichée. – Billions of dollars have been flowing and are still flowing into developing new lucrative treatments while dodging the cures. Raising people’s financial power against cancer doesn’t drive cures, but treatments, unless the People stop being tools, which I don’t see happening because the masses are way too emotionally convenient and are sometimes rather willing to die than to give up a convenient belief that makes them sick.

Gist of the video: ‘We need more money for the existing system! More money for everybody will solve everything!’ – so frickin’ naive and insane. In part because it is imagining a reality that cannot even come to pass because of the very system that is being glorified in the video.

Do not fill black bars on vertical video!

The black bars fill where you duplicate the video content onto a lower layer and magnify and blur it is one of those very stupid effects that are used because to a toddler’s mind they seem like a great idea. But it provides zero benefit. It reduces compression efficiency by leeching off bitrate and generates a constant distraction for the watcher’s attention through pointless movements over the majority of the picture, which is especially bad because human vision is more sensitive to movements towards the periphery and because the zoom amplifies the actual movement energy of the source video. It also blurs the perceptual boundaries, making it more difficult to clearly and easily identify where the actual informational video content ends. This is even worse on videos that are intended to be fully informational and not artistically playful, like for example this one:


In short, it is added effort to make a video a PITA. (In German this is called Verschlimmbesserung. Could be translated as worseprovement.)
Here is a good video demonstrating how easily attention can be diverted: https://www.youtube.com/watch?v=Ahg6qcgoay4
(It is a somewhat different case, but it is about moving elements drawing attention.)

#BlackBarsMatter

UPDATE: I recently encountered a typical result of when people follow some trend thoughtlessly; a 16:9 video with mixed media content in which all 4:3 (so not even 9:16) aspected content had black bars fill, but cinemascope content did not have it for the black bars on the top and bottom. Suddenly black bars are totally fine. Why? Because everybody leaves them black.
This is just one example for many common inconsistent habits that show how people can be programmed into behaviors if they don’t question them.

UPDATE 2: Related issue with what Twitter has been doing for a while now: https://twitter.com/Dowlphin/status/973490387443769350

I also made a short clip pointing out the somewhat arbitrary, inconsistent behavior and how it has to do with herd following and how it’s not happening if no one starts it: https://www.youtube.com/watch?v=okkIOgqki50

UPDATE 3 – 2020-08-16: UNBE-FRICKIN-LIEVABLE! This is collective derangement!

Vertical video, put a fat frame around it to further reduce vertical image resolution/size, add the notorious blur fill to a high-motion scene, zoomed(!)… and then flipping the blur content horizontally!!! (Actually the video itself is flipped and the blur cloning didn’t follow.) … And the image isn’t even perfectly vertical, but rotated slightly to the right!

What will be next? No blur but earthquake effect? Maybe some strobe lights on negative image? Spinning motion would surely be fun, too.
Why even use the original footage? Just put some blurred lewd action there and then we can forget the actual video content altogether.
Now I also finally know that at least that linked Youtuber adds this all later, which is even more insane. The original video is nice and sane native vertical resolution smartphone video and that’s it. Ideal. (If we ignore that often – but not always – horizontal video is better.)

UPDATE 2021-02-04: Eventually a theory occured to me that maybe people are doing this in order to prevent the GooTube algorithm from labeling Fair Use content as ripped off and demonetizing the video, but this would only work on videos with large black bars anyway and also I have seen a lot of content like that that was uploader-original, so that would have to be a pointless habitual thing then. – And also, still, to this day, no one ever responded to my inquiries about why they are doing this stuff, so I have to assume they know they’re doing something stupid but want to continue. (I’m really working with an utter lack of external information here.)

New Youtube interface yet another example of the growing madness

Youtube did it again. “Polymer” looks very much for mo’bile, and naturally, LESS space-optimized, no visual section distinctions, and less info in that more space used.

“Clean, fresh, new, going with the times”, these are words marketing people always use to describe beauty-less shit. Just recently I encountered exactly this with Störtebeker beer bottle labels. Boring white rectangular fields introduced where there was a parchment style before, less theme-appropriate beauty, and those words used, while proudly bragging about what an eye-catcher their unique bottle-surrounding label format allegedly is and how everybody loves their new design.

Youtube, too, now did the step we always see with the Grey Gentlemen taking over more of the world. The pleasant red gradients I see in my bookmark bar and tabs has turned into an eye-burning intense monochrome red and the website itself is stuff thrown into a sea of white.

As it is reported, ‘plenty of white space allows you to focus on a site’s content instead of its interface’. The idiocy! Suddenly visual helpers for distinguishing content sections and types, those white boxes on a gentle light gray background, are seen as DISTRACTIONS!

It is very depressing to see this deranged shit happen more and more wherever you look, especially in web design. Verschlimmbesserung everywhere.

UPDATE 2017-09-07: And the bits of genius keep coming. Now Youtube informed me I got early beta access to the new Community tab, which replaces the Discussions tab. So my first thought is: Review and maybe archive all the old Discussions content before it’s gone. Where can I do that if the new tab replaces it. And those eternal numbnuts point me to my comment history, as if that helps in any way. The comment history can’t even be searched, let alone filtered by date, and it’s definitely not useful for finding people’s more or less old comments on my Discussions tab and my responses to it.
Youtube – the 24/7 fucking joke
If they offered me a paid job to bring some brains into their conduct, I might see hope, but I’m smart enough to know the problem lies elsewhere, not in intellectual resources. At least not only.

Are the mids the music’s heart?

I had this thought when I considered the strong mids that music has and had in times and areas that ‘have heart’. Look at where and when you find music that very much focuses on anything but mids. Powerfull bass, aggessive highs. Electronic music, often. Electronic being more based on machines, logic, technology. All digital. Zero and one.
And I thought how this could match the visual scale of frequencies and thus also relate to the chakra model. The heart is in the middle. Its color is green. Green is the color of that powerful, defining part of nature, the ego-less sun-drinking plant life, and it doesn’t seem to be very popular if I look at trends like car paint colors and such. All kinds of strife to focus on other phenomena, as long as it avoids the heart.
In music, you could say the mids are unspectacular. They even don’t work if you intensify them a lot. They are like the epitome of boredom. But without them, something is missing. Maybe not to the mind, but to the heart. They are not the show-stealer, but the crucial support to make it well-rounded. The slow, deep breath.
You can see how frequency always matches a spiritual state. Powerful bass puts emphasis on the primal, the foundation of life, the power from below, felt with the whole body. Powerful highs are all about the mind, perceived with the ears only, and only if that chakra is healthy.
You could say the heart is the most elusive force. Naturally, since its domain cannot be fathomed by the mind. But the mind can allow it to be, or not to, and that might not be the ideal situation.

I won’t make a refined essay out of this. Use it as food for meditation if you like.

Why do game designers do incredibly idiotic things?

You can watch my detailed video about it or read the summary below.

I recently got an SSD and only now realized the full magnitude to something that bothered me a while ago.

The game: The Witcher 3
This game featured plenty of eyebrow-furling things. Some of them got fixed later, but the really stupid stuff actually kept coming. I’ll only focus on one issue here though that has existed all the time since the game was released.

When you load a savegame, you get a simple 2D-animated sequence and a speaker recounts the story position at the time of the savestate. That animation looks a lot like scripted to eventually end in a loop state until the load process has finished.
When it does finish, for a brief moment a regular loading screen pops up, with a nice picture and a loading animation, which looks kinda sloppy, but shows that there actually is an alternative mode for this.

Well, here comes the kicker: To use one example, you watch a loading process where the speaker talks for 18 seconds and the whole process takes 43 seconds until the game has finished loading, with the animated sequence looping for the rest of the time.
Damn, this game has long load times!

No, actually not, it’s just trolling you by deliberately wasting your time. While the devs for some reason decided to later add subtitles to those sequences that you cannot turn off even if you want to because you’re not deaf and have seen that story bit plenty of times anyway, what they never did is AT LEAST add an info telling you when the loading process has finished and you can now enter the game if you press a key. That would be the MINIMUM. That could actually be an optional feature, but not really that important when the loading process isn’t of patience-testing length, which it wouldn’t be without this artificial delay.

Here is another minor example of bad design decisions: The game has super-beautiful graphics, but taking screenshots without UI requires (not-so-ideal) fan mods, while a low-budget indie game shows how it’s done.

Why it is OK to negatively criticize a free game

You’ve probably read it a couple of times. There’s a free game, say, on Steam, and people complain about stuff and then others talk down to them, claiming since it’s a free game, they are bad people.

So let me widen the perceptual horizon here.

Justified critique is totally fine, especially considering that a freely published game, especially on a highly-frequented platform like Steam, is often a promotion instrument, a skill demo, so it’s not exactly for the pure love of the world. You’de be surprised how many people are way more egoistically motivated than they pretend to be. But you’d only be surprised when you find out, and you only find out when you can manage to put them to the test. (I’m digressing slightly here.)
Furthermore, critique is free consulting, so they’re actually getting a good deal. I mean, there are game devs who are totally inept design-wise, publish a piece of crap of a game, then collect the tons of critique and get to work implementing the suggestions and thus don’t need to be anything but a hacker monkey who is being force-fed with game design training. That business scheme seems especially popular with early access. People will even pay for crap and on top of that give you free consulting services, all fueled by the hope that the game becomes great one day.
In the most extreme case, an early access game is little more than: “OK, I made an empty world space. Pay up, then I’ll allow you to tell me what you want in it and I’ll put it there. (Unless I don’t want to.) Isn’t that great? Your own personal ideas implemented in a totally not generic game. Your ego must be so st(r)oked right now. Buy your own little piece of immortality today!
And the next day there’s an empty space with a pickaxe and a zombie.

I was inspired to write this when I learned about a free Steam game (Serena) that had some issues related to Steam being crap and Valve not caring (the usual), and I read up about the game, and it’s advertised as “…the result of a massive collaborative effort between dozens of fans and designers of adventure games.” while when you press ESC that story-progression-type game immediately quits, without so much as a confirmation dialogue. Unbelievable? Well, I’ve seen some wacky things. Totally justified to go “Dude! What the hell?!

Speaking of incompetence: Is there any major internet business that hasn’t made it its mission to destroy any usability? As I am writing this, I have to switch between Visual and HTML view, because HTML view doesn’t have quick formatting, but requires me to add code manually, while Visual doesn’t know what a line break is (!), yet in HTML view it doesn’t even need/show the code for it.
I’m not even gonna explain how to do it more elegantly. They need to figure such basics out themselves. My consulting services aren’t for free, since my expertise is apparently scarce.

The silliness of consumers purchasing M-DISCs

Clever business move to offer archival-tech optical media (https://en.wikipedia.org/wiki/M-DISC) to the consumer market. But I have to chuckle/facepalm when I read people mentioning how their 10 years old burned CDs aren’t readable anymore, so now, in the year 2015 where optical media are close to suffering the same fate that the floppy disc once did, people are paying insane prices for optical discs that will reliably last hundreds of years. What a comedy. For much less they could have transferred their data to fresh media if it’s so important to them to not just store them on a flash drive or hadddisk. And who knows? Maybe their old discs would still be alright if they had bought quality ones back then.

BTW this isn’t even relevant to people who still play their music from CDs, because M-DISCs are available for DVD and BD only. And pressed CDs have good longevity anyway, so you’ll have your originals as archive for a long time.

Even in the area of professional archival storage there are good technology competitors to optical M-DISC media. This is a niche technology, very useful for a very small number of applications. And not a surprise that such things come towards the end of a technology’s life. Because it’s business – focused on profit, not on usefulness. Shows how well it works to sell people stuff they don’t need.

And not that it would be relevant due to what I just pointed out, but even their marketing is deceptive, too. (As marketing so often is.) They advertise more than 1000 years of data storage, but if you dig a little for details, you learn that that doesn’t mean error-free. The time span in which you can call it virtually guaranteed that all data is still safely readable is significantly lower. … Yeah, the horribly truth is that in 100 years from now you might have to go through the hassle of transferring all your M-DISC data to quantum crystal storage, haha. What a drag. … Oh wait, in 100 years you’re dead! … And in 1000 years your distant offspring won’t even remember there ever was such a thing as M-DISCs, because even the few organisations who once made use of them will have switched to a different technology long ago. If the organisations still exist.

The losing battle of combating crappy web development

I used to use Flash player 11.5 so I could watch up to 60 fps on Youtube, but then they changed several things. First, they prevented that feasible method from working. Then they enforced HTML5 (a player with various usability downsides that I also didn’t use because back then 60 fps worked with Flash 11.5) so that I had to cripple my browser to default to Flash. (That also disabled Vine playback.)

And now, since recently, the Flash player’s bottom bar that shows video length and such is not there unless the video plays. I use an addon for disabling autoplay of videos (since Youtube and such steadfastly refuse to allow it), so maybe that’s a factor, but maybe not. I thought it’s because of my ancient Flash player version, but I should have known better. Even with the newest I don’t have it, so if I want to check lengths of videos I have tabbed (which I do a lot), I gotta play-and-page-reload it.

One would think that with all these obstacles, they’re done because HTML5 now plays 60 fps on Firefox, but at least when they set default to HTML5 it didn’t. I could reconfigure some stuff and check whether it does now, but why go through the effort if I know in advance that it very likely still doesn’t? I’m tired of this!

Youtube prevented the one means for watching 60 fps for my browser, so that kinda speaks for itself.

Same with IE by the way. That one I left to default, without modifications or stuff, and it uses Flash and thus can’t play 60 fps.

You know why all this shit is happening this way, right? Do I have to spell it out?

Google Chrome.

When that browser entered the scene, I knew exactly how it would go. That the power-hungry Google corporation would use it in order to conquer ‘market share’ and thus gain control and then focus on their browser so that others would be at a disadvantage. That’s pretty much what Microsoft tried in the past with the IE, too.

And now the (gradually less) intelligently designed Opera browser is basically Chrome, too, and even Steam has been based on Chrome.

Chrome is the corporate browser. Firefox might go through some bullshit dev ideologies, too, but THAT should be the free, independent standard used by most.

Back when I used Opera (up to version 12), I was the rare-case odd power user. When I finally saw no choice but to change to Firefox (actually Waterfox), that one had already become a minor figure.

If developments of this kind continue, we might eventually be using crappy ports of mobile operating systems on PCs, and “PC” then means gaming-console derivative multimedia computer. Also, private Linux users will be classified as potential domestic terrorists by the CIA, since they won’t be able/willing to imagine that anyone would use it unless they’re a hacker, while, in typical double standard, for business purposes it is considered a smart way to reduce expenses.

Yeah, you’ll have noticed, I’m really fed up with this shit.

People should have woken up years ago when Google dropped their motto/guideline “Do no evil.”. I mean, how much clearer do you need it?

My thesis is that people tolerate so much corporate bullshit because they are afraid of having their own personal bullshit be stripped away.