Four storage technologies that could be revived as the economy recovers

With the current recession arguably showing signs of an end, now is a good time to review some of the promising technologies that were killed, shelved, or delayed because of the economic downturn. In many cases, some of these technologies were simply delayed; in others, development was halted and roadmaps were modified. Whether these technologies were storage or storage-related, if vendors could get them out there at the prices they were supposed to be, the world would be very different. "Dead" technologies may be resurrected when the economy recovers, or maybe the world will take a different approach.

　　1. Object storage device disk drive

　　If you are a long-time reader, you may remember that I was a staunch advocate of the T10 OSD (Object Storage Device) standard for a while and hoped it would take off in the market. Back in 2004, I wrote about the T10 standard, which I had been looking forward to for many years. As a result, the standard never came to fruition. I guess the problem was that no disk drive vendor had released a disk drive that supported the T10 OSD standard.

　　Now the disk drive market is down to two major players (Seagate and Western Digital) and one with a small presence (Toshiba). Since the market has become so uncompetitive, we are not sure we will ever see OSD disk drives again. OSD would have changed the landscape in many ways. For example, the file system could virtually remove many allocations of space, as if it were a disk drive. Some RAID concepts as we know them would have changed - RAID and small objects could be RAID-1, large objects could be another RAID, and objects could be migrated between small and large allocations.

　　A lot of things would have been possible with the T10 OSD disk, but none of them are happening now. The question is whether we really need T10 OSD, as at least one vendor has developed a hybrid disk drive that combines NAND and spinning disk. How are you supposed to decide what should be in the flash cache and what should be in the hard drive partition? With a standard file system, you know nothing about the data at the disk drive level. All you know is that a block was written or read, and you can track how many times each block was written or read. Maybe you can track it by time after power-up. However, this doesn't help you in your efforts to boot quickly, because if you are moving a lot of data, it is difficult to determine which files must be placed in the flash portion to speed up the boot. I'm sure there are some specialized algorithms that can help, but something like a large patch set will make things messy, and the algorithm will have to determine what the device will access after power-up again.

　　This is extremely inefficient. All of this could be solved with the OSD disk. The file system could talk to the OSD so that the important data could be put on flash. The higher the expectations, the greater the disappointment.

　　2. PCIe 3.0 is coming, but it’s late

　　PCIe 1.0a was introduced in 2003 with 250MB/sec per lane. In 2005, the standard was updated to address some compatibility issues. In 2007, version 2.0 was released. Manufacturers introduced 2.0 products in 2007 with double the performance. In November 2010, PCI-SIG mentioned PCIe 3. The final specification was scheduled for release in 2011. As a result, the specification was not released until near the end of 2011. PCIe is very important for network and storage performance. For example, with PCIe 2.0 and 8 lanes, you can only get 4GB/sec bandwidth at best. With a 6.0Gb/sec SAS interface, you can only keep a little more than 5 SAS lanes at full speed. With a 10Gb/sec Ethernet interface, you can have a 4-port card, but you can't keep all 4 ports at full speed.

　　PCIe 3.0 is important for storage and network architectures because it is the path for information to get in and out of the system. As data size increases, we have to move more and more information out of the system, so the sooner the new PCIe comes out, the better. In my opinion, the late arrival of PCIe 3.0 has delayed the development of 40/100Gb/sec Ethernet, 12Gb/sec SAS, 16Gb/sec Fibre Channel, and FDR InfiniBand connections. Because PCIe 3.0 was late, PCIe 4.0 was also delayed. The latest news says that PCIe 4.0 will not be available until 2015. I don't really believe this because it is already mid-2012. Intel just announced PCIe 3.0 products in March of this year, and they are only now entering the market. This delay has slowed down the speed at which we can process data.

　　3.10Gb/s Ethernet Market Acceptance

　　Many of us predicted in 2008 that 10Gb Ethernet would dominate the market. I thought big price drops and broad market acceptance were promising. That is only beginning to happen. The delay of 10Gb Ethernet combined with the delay of PCIe 3.0 meant that the development of 40Gb Ethernet and 100Gb Ethernet was delayed. Vendors needed volume to gain economies of scale before they could reduce chipset prices. That has not happened.

　　4. Migrate to 2.5-inch disk drives

　　Small drives are used in notebooks, on the other hand, enterprises have been slow to migrate to nearline drives for the RAID market and workstations. These drives have very good power per IOP (input output operation) and power per bandwidth performance compared to 3.5" drives. Like others, I had predicted that the market acceptance of these drives would increase significantly, but it turns out that the RAID vendors were reluctant to redesign their disk drive bays during the recession, and you can't blame them. For them, such an investment can only get a priority return in the short term, which is a big cost. Of course, this means that the capacity per drive has to be reduced, and disk drive vendors will not be motivated to produce many drives when there is no market, and the rest is you know. I don't think the market will transition to 2.5" drives anytime soon, but I hope it will happen in the next few years.

　　Final Thoughts

　　This is a non-overlapping list. The reasons for each of the examples I mentioned are different. The lack of support for OSD disk drives from disk drive vendors means that the rest of the stack has no incentive to work on OSD. What a shame, such a high-profile standard has to go this way. As for PCIe 3.0, it's hard to say what the reasons are. Intel has a lot of influence over the PCIe 3.0 standard. Maybe Intel's PCIe 3.0 release was timed to coincide with the release of new chips? Only Henry Newman, who loves cynicism, would think so.

　　As for 10Gb/s Ethernet, it was delayed because the market didn't buy enough units, so we can only blame it on the fact that prices didn't come down. Early adopters didn't buy enough to drive prices down. As for the 2.5-inch disk drive market, the cause clearly lies with the RAID controller vendors. For profit reasons, they didn't redesign the back-end disk bays to be compatible with the new technology.

　　While there is nothing to blame for what has happened, there are long-term consequences. Right now, data path performance has not scaled as fast as CPU performance, or even as fast as memory bandwidth. We are already behind. These are just some of the major impacts on data processing and information generation that will have long-term consequences. We are still moving forward, but no one can skip this delay.