Analysis of file virtualization and four SAN virtual architectures

Storage virtualization creates an abstraction layer between the host and physical storage. When a SAN is implemented, it provides a single point of management for all block-level storage. In simple terms, multiple heterogeneous network storage devices form a virtual storage pool, which appears on the host as a set of virtual storage volumes.

In just a few years, storage virtualization, also known as virtual blocks, has proven its worth in large enterprises, transforming from an expensive solution to an affordable commodity, as described in the article "Improving Storage Efficiency with Symmetric Virtual Storage." As a standard feature, especially in the most conservative midrange disk arrays, storage virtualization has brought new developments in storage management to small and medium-sized enterprises. At the same time, centralized data management using proprietary solutions from top vendors has brought the greatest return on investment to large enterprises.

Storage virtualization creates an abstraction layer between the host and physical storage. When a SAN is implemented, it provides a single point of management for all block-level storage. In simple terms, multiple heterogeneous network storage devices form a virtual storage pool, which appears on the host as a set of virtual storage volumes.

In addition to creating storage pools from physical disks in different arrays, storage virtualization also provides a very wide range of services delivered in a unified manner. These extensions start from basic volume management, including LUN mapping, concatenation, volume grouping and striping, automatic thin provisioning, automatic volume expansion, and automatic data migration, to data protection and disaster recovery functions, including snapshots and mirroring. In short, virtualization solutions can serve as a centralized control point for managing storage and achieve higher SLAs.

Perhaps the most important service of block-level virtualization is non-disruptive data migration. For large enterprises, moving data is a common occurrence in life, and storage virtualization can move block-level data from one device to another without interruption as old devices are retired and new devices are brought online. Storage administrators can perform routine maintenance on their own or replace aging arrays without interfering with applications and users.

Four SAN virtual architectures

There are four approaches to providing storage virtualization services in the SAN virtualization architecture: in-band application, out-of-band application, hybrid approach, or separation virtual architecture, and controller-based virtualization. All storage virtualization must have three basic elements: maintaining a virtual disk and physical storage, as well as other configuration metadata mapping; execution commands for configuration changes and storage management tasks; and data transfer between hosts and storage. These four architectures differ in how they handle these three separate paths - metadata, controller, and data path - in the I/O architecture. The differences come mainly from performance and scalability.

In-band devices have metadata, controller, and data path information all in a single device. In other words, metadata management and control functions share the data path. This can represent a bottleneck in busy SANs, as all host requests must be fulfilled through a single control point. In-band device vendors have addressed this problem by adding advanced clustering and caching features, which have the potential for scalability. Examples include DataCore's SANsymphony, FalconStor's IPStor, and IBM's SAN Volume Controller.

Out-of-band applications pull metadata management and control applications out of the data path and offload them to a separate compute engine. A software agent must be installed on each host. The agent's job is to collect metadata and control requests from the data stream and forward them to the out-of-band application for processing, freeing the host to focus on data transfer. The only vendor of out-of-band applications is LSI Logic, whose StorageAge product can be used both out-of-band and in a separate path.

A split path system leverages the processing power of intelligent switch ports to offload metadata and control information from the data path. Unlike out-of-band devices, where the paths are split at the host, split path systems split the data and control paths at the intelligent device. Split path systems pass metadata and control information to an out-of-band compute engine and pass processing and transport data path information to storage devices. Thus, split path systems eliminate the need for host agents.

Typically, split-path virtualization software will be implemented in smart switches or built-in applications. Split-path virtualization controller vendors include EMC (Invista), Incipient, and LSI (StoreAge SVM).

Array controllers have been the most common location for virtualization services to be deployed. However, virtualized controllers typically virtualized internal physical disk storage systems. This is changing. The new approach is to deploy intelligent virtualization devices that virtualize both internal and external storage on the controller. Like an in-band device, the controller handles all three paths: data, control and metadata. This new type of controller-based virtualization is primarily Hitachi's Universal Storage Platform.

File-level virtualization overview

Just as block-level virtualization simplifies SAN management, file virtualization eliminates the complexity and limitations of enterprise-class NAS systems. We all recognize that unstructured data is growing rapidly and it is difficult for IT to control that data, and file virtualization can help a lot at this time.

File virtualization abstracts the basic details of physical file servers and NAS devices and creates a unified namespace across those physical devices. A namespace refers to directories and files and their corresponding data structures. Usually a standard file system, such as the NTFS file system, is associated with a computer or file system. By unifying multiple file systems and devices under a single namespace, file virtualization provides a single file and directory and provides administrators with a single control point for easier data management.

Similar to the benefits of storage virtualization, file virtualization can also migrate file data to another without interruption. Storage administrators can perform routine maintenance of NAS devices and discard old devices without affecting users and applications.

When file virtualization and cluster technology are combined, its scalability and performance are significantly improved. A NAS cluster can provide orders of magnitude throughput (Mbps) and IOPS. High-performance computing (HPC) applications, such as seismic processing, video rendering, and scientific research simulation, rely heavily on file virtualization technology to provide users with scalable data access.

Of the three architectural approaches, only file virtualization is still in its infancy. Different vendors' approaches are suitable for different usage patterns, and there is no one-size-fits-all approach. Generally speaking, you will find three different file virtualization approaches on the market today: platform-integrated namespaces, cluster storage-derived namespaces, and network virtualized namespaces.

Platform-integrated namespaces are extensions of the host file system. They provide a platform-specific namespace. This type of namespace is well suited for multi-site collaboration, but they often lack file control, so they are limited to a single file system or operating system. Examples include Brocade StorageX, NFS v4, and Microsoft Distributed File System (DFS).

Clustered storage systems combine clustering and advanced file system technology to create a modular, scalable system that can support an unlimited number of NFS and CIFS requests. These clustered systems are a unified, shared namespace across all cluster elements. Clustered storage systems are ideal for high-performance applications, consolidating multiple file servers into a single high-availability system. These vendors include Exanet, Isilon, NatApp (Data ONTAP GX), and HP (PolyServe).

A network virtual namespace is created by a network device, often called a network filer. A network filer exists between the client and the NAS device. Essentially acting as a router or switch, these devices present a virtual namespace between the user and the storage. Network virtual namespaces are ideal for tiered storage deployments and other non-disruptive data migration scenarios.

Summary: File and block storage virtualization may be the best way to mitigate data storms today. By virtualizing block and file storage environments, IT can achieve greater economic management as well as centralized execution and control of heterogeneous storage systems. The road to these solutions has been long and difficult, but these technologies are finally catching up to our needs.