Question to IT people: Do you remember Novell Netware?
If I’m not mistaken, didn’t it have that cool feature of keeping a partition’s master file table in RAM, thus massively speeding up enormous small file operations?
After all, a harddisk always has to alternate between writing data and filesystem, and that’s why copying (i.e. writing) a large number of small files is so much slower than the same data size in one file.
Hybrid hard drives reminded me of this. But hybrid hard drives use the integrated SSD merely as a sector cache for frequently-read-accessed data, and due to this they also have certain drawbacks, like performance drop for a while after a defragmentation.
Now imagine this: Operating systems like Windows 7 can already create striped partitions, spanning several drives, in a flexible way. Now imagine the OS’s filesystem driver was able to create a partition with the data on a harddisk and the filesystem files like the MFT/GPT on an SSD!
This would be a great solution especially for huge drives. Small file operations happen a lot especially for power users.
Not only would the file system operations happen in SSD speed, but they’d also be virtually not happening for the harddrive. You could copy 10000 files of 10 kb size and all that would slow it down is the jumping from one file to the next, but not the write operation on the filesystem after every single file.
This technology could even be expanded to also store directories (and maybe the small files that are stored in a special file on NTFS partitions) on the SSD. It would allow a very flexible solution with much choice for the user.
I had this crazy thought that maybe this could even be realized with an extended symlink feature. On NTFS you can hard-link directories and files to different partitions and drives. Imagine you symlinked $MFT onto an SSD. ^^
Yeah, this is probably not gonna happen in our world where computer system design has to consider backwards compatibility, but it would be really cool. Probably ideal to offer it as one package, since the SSD for that kind of storage would only have to be very small, even with ample failsafe reserve. I don’t know exactly how big a filesystem is, but maybe it could even use high-reliability SSDs like SLC. To make it even cooler and increase long-term realibility, the SSD could be a connected module that you can remove from the harddisk case like a SIM card from a cell phone. Hell, if flash cards match the requirements of conventional SSD drives, you could make that a flash card slot.
UPDATE: A good comment about related solutions on Linux: http://www.reddit.com/r/Dowlphin/comments/29f0pc/softwarebased_hybrid_harddisks/
Dowlphin's Blog 