Enterprise IT is going through some significant growth right now. Virtualization has enabled a shift in how compute tasks are performed, and storage is seeing incredible performance increases with SSDs. However, the network needs to catch up in terms of adoption for the next major revision: 10Gb.

10Gb networking is available, but very expensive — which inhibits widespread adoption. However, many new products are coming that require 10Gb to maximize performance (ex: Tintri VMstore). What is it going to take for widespread penetration of 10Gb networking into mass enterprise markets?

Here are a few factors common to most enterprise IT departments:

1. Expiring hardware: Unless enterprise environments are running modular switching systems, moving to 10Gb networking requires replacing the existing switching environment at significant cost. On the plus side, the number of physical ports should decrease, which means fewer switches are needed.
2. Cabling changes: 10Gb networking requires Cat6A cabling (for copper) or Fibre (for optical). Copper is reaching the limits of its transmission speed. So, may as well bite the bullet for optical if you’re going to need to change cabling from Cat5 or Cat6.
3. Availability of 10Gb NIC ports: This is really a two-pronged issue. Servers need the 10Gb NICs as well as the storage devices. New storage technologies are including these as standard, which is nice.
4. Equipment cost: Another two-pronged issue. Network vendors will need to reduce the cost of access to the 10Gb network.

Networking is really the key to unlocking everything that virtualization and SSD storage have to offer the enterprise compute world. Until 10Gb is adopted, the network will remain a bottleneck for performance.

As you continue reading, please keep your environment in mind and think about how you feel regarding security. It could be a show stopper, an acceptable risk, or a nonissue for you. Regardless, it’s a fine conversation-starter to ensure your journey to the cloud is orderly and secure.

Denial of Service

The dreaded DoS can occur in a cloud environment. However, denial of service is not always malicious.

Recall that the cloud is just a multitenant datacenter environment. Multitenant environments can suffer from the noisy neighbor effect: while you’re operating just fine, other tenants may be out of control and disrupt your performance and availability.

Think about your virtualization environment. Do you over-commit? This could include storage, network, CPU, or memory. Virtual environments tend not to use all of the available resources at once. The cloud is no different. Cloud providers still rely upon some level of over-commit to ensure availability and performance and allow for expansion when necessary.

Additionally, a zero-day vulnerability may hit a number of hosts inside the cloud environment. BPDU packets may be unprotected and crash the hypervisor, for example. Any number of situations can arise by accident (or on-purpose) that degrade or remove service availability.

Trust

Trusting an external entity is difficult. We trust the guy we just hired off the street because we need to. We trust consultants to properly design solutions that we don’t understand. But why is it we cannot trust service providers?

There must be some level of trust in service providers and some way to ensure the trust is realistic. Once my data leaves my network, I need to be able to trust that:

• My network is encrypted
• My VMDKs are truly isolated from access by admins and other tenants
• The servers hosting the environment are properly maintained

People are flocking to the larger cloud providers because they are established. The theory is, if a provider had a high rate of failure, they would not still be around.

What should we expect a less established service provider to do to earn our trust? Some suggestions:

• A new cloud provider certification (such as ISO:9001) that shows the provider has passed audits and best-practice analysis
• Knowing their employees run a gauntlet, pass background checks, and achieve top-secret clearance
• Passing the Security+ test
• Knowing that armed security guards protect the NOC and datacenter facility (think SuperNAP)
• Take their word for it

I guess that trust, in this case, takes a little leap of faith.

How You Get Your Data In — and Out

Getting your data to the cloud is the easy part. Open an agreement with a provider and upload data over an encrypted connection or send an external hard disk somewhere to be sucked into the environment.

The bigger issue becomes how to get your data out. It may be the case that the cloud is not what you thought it was going to be. So, you need to pull your data back to your private environment or to another cloud provider. Did you just enter the Hotel California?

The key is reading the contract. Look for sections pertaining to termination of the contract, procedures and processes around termination, and what happens to your data when that cloud provider goes away.

Development or Production — Which Is More Sensitive

The immediate reaction may be that your production environment is more sensitive. Therefore, production is the last to make the move to the cloud. Development environments are usually the first to make the jump, because they do not have the critical sensitivity production does.

However, consider for a moment that your data and intellectual property is what makes up your business. In many instances intellectual property is designed and refined in a development environment. Production ends up being a subset of the IP that your company provides. Pushing development to the cloud first can actually put your IP at risk.

Cloud provider failures may result in the cloud provider losing data, making your data available to other individuals, or even closing up shop. Your IP is what makes you stand out from the crowd. Losing it for any reason could be the downfall of your company. Be sure to have a good backup.

Conclusion

The IT industry is moving toward XaaS solutions hosted in the cloud. It’s  tempting to enter into an agreement and start throwing content into a cloud. But taking a moment to perform some due diligence and ensure this is not a shotgun decision is a must. Ask the questions, evaluate the risk, and make the jump if it fits your business.

Ensure connections are properly encrypted to protect you from outside influence; you may have the keys to the house, but determine who has a key to the backdoor. Look for SLAs and protection against availability (or lack thereof). Where encryption is concerned, ensure you know who has the keys and whether they can reset your access in the event you lose your encryption key (hint: if they can reset encryption, run the other way as fast as possible).

However, almost every IT vendor seems to have some cloud-focused solution to offer, with cloud-washed marketing to make it sound indispensible. Hosted email is no longer email, but email in the cloud. To back up your files, throw them in the cloud.

IT professionals have to get down to brass tacks, and think about what the cloud really is. An IaaS cloud is nothing more than a multitenant compute environment with infrastructure resources shared dynamically across multiple workloads. An SaaS cloud is a multitenant application environment where a single application platform serves multiple customers. But both exist in a datacenter somewhere. These solutions may provide some level of replication and redundancy, that like the cloud, appears to be everywhere you need it to be.

As we walk down the cloud path looking for cloud solutions, one component is too often neglected — the security of the cloud provider environment. This shouldn’t be an overly paranoid discussion; however, it should create some fodder for conversations while making the jump.

Security discussions need to consider more than just hackers breaking into your systems, but also access, availability, stability, and a host of other components, such as:

• Data ownership: This can seem like a funny concern. You would think you own your data. But, in reality, once it leaves your control, it is hard to ensure that you retain control of it. Your data may be mined, VMDKs copied, and unencrypted network connections sniffed.

• Geographic location: The location of your data is critical to ensuring it is secure. Depending on your business model, having the data close to your users is important. Take email, for example: If you select a cloud provider with presence in the U.S. only, performance may suffer for users in Europe, Asia, South America, Africa, and anywhere in between. No one wants that. So an offering with a global reach is more appropriate.

In addition, the locations in which cloud providers place their services and your data can have an impact on its direct security. Other countries have regulations and policies regarding access to data. By placing your data with a cloud provider, you may inadvertently make your data subject to the regulations of various countries and entities inside the country.

Take the U.S., for example. The Patriot Act enables law enforcement to perform searches of your cloud-based data without your knowledge (check out info on FISA Orders and National Security Letters). Obviously, you would need to make your users aware of this. European-based cloud service providers are using this as marketing ammunition to drive service away from U.S.-based providers (such as Office 365, Salesforce.com, Amazon, etc.) and to their own services.

China’s situation is interesting as well. The well-known Great Firewall of China blocks and filters traffic to ensure the interests of China are protected. As cloud solutions develop, China is requiring non-Chinese cloud providers to team with China-based cloud providers. Ultimately, this does not bode well for cloud providers or the users.

• Accessibility: Your data is your business. To function, you need to access your data. The cloud service provider ends up being responsible for ensuring your business is up and running by providing the data and connectivity to the data.

In April 2011, Amazon cloud services suffered a 12-hour outage. The outage was caused by a network misconfiguration in one of the U.S. sites. Countless sites and services were unavailable. Think about the impact of not being able to conduct business for 12 hours. What would the impact be to your environment?

Summary

The ownership, location, and accessibility of your data are critical to ensure your environments are secure. But, they aren’t the only concerns. Tune in for a second post about other issues to consider.

The hard problem with SSD

SSDs behave very differently from hard disks. The main complexity lies in the Flash Translation Layer (FTL), which provides the magic that makes a bunch of flash chips usable in enterprise storage: wear leveling, ECC for data retention, page remapping, garbage collection (GC), write caching, managing internal mapping tables and so on. However, these internal tasks conflict with user requests and manifest as two main issues: latency spikes and limited durability.

It’s all about latency

The main appeal of SSD is its low latency; however, it is not available consistently. And while write latency can be masked with write-back caching, read latency cannot be hidden. Typical SSD latencies are a couple of hundred microseconds but some accesses can be interrupted by device internal tasks, and their latency can exceed tens of milliseconds or even seconds. That’s slower than a hard disk.

There are myriad flash internal tasks that can contribute to latency, such as GC at inopportune times or stalling user IO to periodically persist internal metadata. What further complicates the situation is the lack of coordination across an array of devices. The most common way to use SSD is to configure a group of devices, typically in RAID-6. But since each device is its own eco-system completely unaware of others, the resultant performance of IOs to this array can become even more unpredictable since their internal tasks are not coordinated.

Unless the storage subsystem understands the circumstances under which latency spikes occur and can manage or proactively schedule them across the entire array, the end result will be inconsistent and have widely varying latency characteristics.

Figure 1:  This graphs latency plotted against time for a widely used SSD on the market. The workload is a combination of sequential writes and random read reads, common with many log-structured file systems today. You can see the periodic latency spike that disrupts normal access.

Endurance is the enemy

Another issue is Endurance. Although flash is great for IOPS, it has limited write cycles compared with a hard disk. And while SLC flash drives have higher endurance compared with MLC, they are too expensive for mainstream applications, and may still require over-provisioning to control write amplification. MLC flash is much more cost-effective, but if used naively will quickly wear out. Its lifetime is proportional to the amount of data written to it by both user as well as data produced by internal drive activity such as GC, page remapping, wear leveling, or data movement for retention.

The additional data written internally for each user write is referred to as write amplification and is usually highly dependent on device usage patterns. It is possible to nearly eliminate write amplification by using the device in way that hardly ever triggers GC, but the techniques are not widely understood and may be drive-specific. Techniques for total data reduction such as dedupe and compression are more widely known, but hard to implement efficiently with low latency. Similarly, building a file system that has a low metadata footprint and IO overhead per user byte are also challenging but yields high benefit.

Everything you want to know is hidden

A big challenge to designing a storage system that runs efficiently on flash is understanding drive geometry. Traditional file systems are highly tuned to the geometry of hard drives; this was possible because of the wealth of information available about hard disks. For SSD, however, the device geometry is highly virtualized and proprietary. This makes the task of reducing write amplification hard. For example, writing a program erase block instead of random pages within a block can reduce amplification, but what are the boundaries of these program erase blocks? How big are they, where do they start?

Reliability is paramount

Finally, one of the key requirements for enterprise storage is reliability. Given that write endurance is a challenge for SSD and the fact that suboptimal use patterns can further affect it, it is important to predict when data is at risk. SSD has failure modes that are different from hard disks. As SSDs wear out, they begin to experience more and more program failures, resulting in additional latency spikes. Furthermore, because of efficient wear-leveling, SSDs can wear out very quickly as they near the end of their useful life.

Figure 2: This is a graph of write cycles vs. use of SSD spare pages. You can see the use of spares rising very slowly in the beginning and then exponentially toward the end.

It is important to observe and deduce when a device is vulnerable. This is not a simple matter of counting the number of bytes written to the device and comparing it with its rating. It means observing the device for various signs of failure and errors and taking action.

The bottom line is that SSD is a game changer, but needs to be implemented correctly in any storage system and won’t be as effective if it’s bolted on to an existing system.  If you’re evaluating any SSD storage products, make sure you understand how the file system uses flash and manages performance, latency spikes, write endurance and reliability.

The Tintri OS has solved these problems with simple but efficient solutions. It allows you to use SSD as an integral part of your storage without compromising consistent performance or data reliability.

“Virtualizing lets us do things like snapshot or clone VMs easily, which should make managing our environment much simpler, but the storage layer trips us up when we try to scale.”

Snapshots and clones are closely related — in fact, clones are a form of snapshots — but have different attributes and uses:

• Snapshots are point-in-time read-only copies of data that are useful for data protection and replication.
• Clones, or writable snapshots, can be used to create multiple working copies of data that can be independently modified. This is extremely useful for creating test or development copies of production data and for deploying multiple copies of VMs in a virtual desktop infrastructure (VDI) environment.

Being a Clone Isn’t Easy

Many storage systems provide some form of snapshots and clones, but they differ greatly in terms of efficiency and performance. Some track changes between snapshots using very large blocks. This means that even small changes between snapshots consume large amounts of space. Others efficiently track changes to data but create lots of metadata, increasing management overhead. Finally, many implementations impose significant performance penalties for clones, which result in slow reads and writes.

In every instance, storage systems take snapshots at the storage object layer — usually a volume or LUN. Although these can be space-efficient, managing VM snapshots quickly becomes complicated as the volume of metadata explodes.

Rethinking Snapshots and Clones

We approached this problem from the standpoint of a virtualized environment. Our customers wanted to take snapshots of VMs, but had only two options — use the native hypervisor cloning capability, which consumes host resources and is inefficient at large scale, or use array snapshots.

Tintri snapshots and clones are operations on the VM itself (and just the VM!), and make very efficient use of data and metadata and impose very little, if any, performance overhead. This is achieved by sharing both data and metadata at a fine granularity and by designing the core data paths from the ground up to work efficiently with snapshots and clones. Snapshots and clones can be created instantaneously regardless of the size of the VM, and use no additional space until they are modified. Furthermore, because snapshots and clones can be created and managed on a per-VM basis, it gives the user more flexibility in managing the data protection policies for different VMs.

Tintri Snapshots

Snapshots can be created and deleted manually or automatically according to a default schedule but can also be customized for individual VMs (Figure 1).

Figure 1: Snapshot management policy

You can also choose to create either crash-consistent or VM-consistent snapshots (Figure 2). Tintri snapshots are integrated with vSphere to allow the creation of VM-consistent snapshots: i.e., before the snapshot is created on the storage array, the VM is first quiesced and stabilized. With crash-consistent snapshots, however, the VM is not quiesced. As a result, creation of these snapshots is significantly faster, but there is no guarantee that in-flight IO for the VM will be captured in the snapshot.

Figure 2: Snapshot creation

Users can view various snapshot statistics such as total snapshot space usage (Figure 3) and per-VM snapshot details including the change rate (Figure 4) to decide how to tune the snapshot schedule.

Figure 3: Total snapshot usage

Figure 4: Per VM snapshot change rate

Tintri Clones

Tintri cloning allows users to instantaneously create multiple space-efficient, high-performance clones or copies of a VM. Clones are treated as first-class citizens just like all other VMs. A single cloning operation can simultaneously create hundreds of clones, and can be customized based on templates created in VMware vCenter (Figure 5).

Figure 5: Clone creation

It’s a very easy process for the user, but a lot of work happens behind the curtain to clone a live VM. The Tintri OS first takes a snapshot of the VM. The clone is then configured to share the data and metadata in the snapshot. Since all of the clone’s data and metadata is initially stored in the shared snapshot, the initial space consumption of the clone is zero. The space consumption of the clone increases only as new data is written to the clone. Clones can also be created from an existing snapshot of a VM. As with all VMs on a system, clones benefit from Tintri’s deduplication and compression in flash. As a result, the clones are both space-efficient and benefit from flash performance.

Figure 6 below illustrates the benefit of Tintri clones. We have a VM called linux-vm with a provisioned size of 500GB and a total space consumption of 200GB. Suppose the user creates five copies of this VM. The brute-force solution would copy this VM’s virtual disk files a total of five times. This is very time-consuming and space-inefficient: the total amount of space consumed by these five copies would be 1TB, while five Tintri clones wouldn’t use any additional space; per-VM Tintri clones only consume additional space as they are modified.

Figure 6: Copies vs. clones

You can create snapshots of Tintri clones, which inherit the default snapshot creation and deletion policy of the base VM. You can even create clones of clones and snapshots of clones of clones and so forth. Upon creation, clones are automatically registered with vCenter. Any specified customization specs are applied, and the clones are ready to be powered on within seconds. The cloning capability is also available through the NFS VMware APIs for Array Integration (VAAI) plug-in, so array-side cloning can be initiated directly from vCenter.

We’re very excited about the new snapshot and cloning capability. As one customer told us last week, “This will revolutionize how you look at storage in your virtual environments. Cloning is so fast I didn’t have time to refill my coffee.”

• IBM PCIe card as a performance disk tier
• HP mezzanine card for blade solutions

Until the end of 2011, the solutions were presented as local storage. With the astronomical rise of virtualization into enterprise environments, the idea of local storage became blasphemy. Many of advantages of virtualization are realized through shared storage. Without going deep into the details, Fusion-io has developed a technique to turn server-side NAND-based storage devices into caching devices. Conceptually, the cache is a read-cache with write-passthrough. Using a filter driver in the guest OS, VMs can benefit from shared storage while taking advantage of locally available cache. Now storage vendors, like EMC and its Project Lightning/VFCache, are trying to play catch up.

Moving data from the storage device to the local server is packed full of benefits: microsecond access times, tens of thousands of IOPS, and so on. But, the bigger question becomes: How do storage vendors approach this new storage technique?

Storage design continues to be an extremely important component of enterprise architecture, but server-side caching may mean rethinking configuration. At first glance, it seems there should be fewer higher-performing disks in the storage infrastructure, as all of the performance is local to the server. Suddenly, populating storage arrays with high-capacity SATA disks sounds like a great cost savings. The storage array becomes truly purpose-built for capacity and performance is an afterthought.

However, hopefully before the storage admin pulls the trigger on replacing disks, they will realize some important truths about server cache technology:

• Cache warm up: Cache devices start out without data. Everything comes from the array.
• Cache over-commit: Cache devices can be over-committed. In this instance, an application may have questionable performance as the cache is consistently unloading and replacing data.
• Write-heavy applications: These cache devices benefit environments with significant read profiles, or aspects of applications that read often. However, as caches only function with read operations, heavily write-based applications will not see much of a performance benefit.
• VM migrations: This is, perhaps, one of the worst offenders — in a perfect world, virtual environments are balanced and stable. However, in the event that a VM must migrate to another host, performance will degrade significantly. Users expect an application to perform well even with thousands of IOPS. The move to another host introduces a cold cache to an unsuspecting user base. What once was an unnoticeable feature of virtualization becomes a carefully coordinated and calculated event.
• Selected candidate VMs: Administrators may configure a VM to use the cache, or not to. Depending on the OS, the VM may not be a candidate. These noncandidate or nonselected VMs rely on the storage array.

The fact of the matter is that with server-side caching solutions, a properly designed storage environment will be critical to support instances where the cache cannot be used. The utopian dream of inexpensive high-capacity SATA disks in the array is far from a reality. Storage vendors should be able to take advantage of the server-side cache to enable their solutions as well as a well-balanced storage solution.

At the end of the day, storage vendors do not appear to have much to worry about with the server-side cache technology. A cache still requires an underlying storage environment to provide the actual data and functionality, and that needs to perform at peak levels to support noncache situations. Some design decisions may change slightly, but modern solutions are still necessary.

What is missing in this discussion, however, are the broader ramifications of flash technology on the evolution of storage products.  Is flash just a better performance, higher density replacement for disk or is there more?  What will happen when everyone has more IOPS than they need?

With flash, we can finally remove a key mechanical barrier to scaling the performance of not only storage systems, but computing systems in general.  Going forward, CPU, network and storage can now all scale with improvements in semiconductor technology.  When transistors replaced vacuum tubes, we got more than just compact radios; we got simpler, more powerful systems.  Similarly, flash is a catalyst that will enable far greater levels of automation and functionality from storage and computing systems than is possible today.

I tend to think of the value of a new technology as the product of its simplicity times its functionality:

value = simplicity * functionality

We’ve been thinking about the same questions here at Tintri as we continue to build out our VM-aware storage appliance. In fact, we recently wrote about different approaches to using flash storage in a 3 part blog series.

An obvious way flash promotes simplicity is by eliminating performance bottlenecks, but as flash enables more dense storage systems many of those gains will be converted to problems in quality-of-service.  A more significant way flash promotes value is by providing a better building block for constructing storage systems: flash promotes simplicity by enabling higher levels of automation and allows the implementation of more powerful functionality.  That leads to much higher value, but only if we think outside the storage box and treat flash as more than just a faster, denser disk.

Figure 1: Intelligent flash-based products use a combination of flash and hard disk to achieve a 99% flash hit rate, providing sub-millisecond performance more efficiently than flash only arrays.

Intelligent flash-based products use a combination of flash and hard disk, but apply techniques such as inline deduplication, compression and working set analysis to service nearly all IO from flash (Figure 1). Most data evicted from flash is snapshots, replicas, unused applications, powered-off VMs and other very cold data. Unlike flash-only products, you can fill 100 percent of the useable flash without worrying about running out of space and having your applications come to a screeching halt. Intelligent flash-based products achieve sub-millisecond flash latencies, and are operationally far simpler and more cost-effective than flash-only products (Figure 2).

Figure 2: In intelligent flash-based storage systems, all writes are committed to flash, and 99% or more of reads are from flash. Disk writes only occur when cold data is evicted to disk and don't impact application performance.

Given that many application management problems today originate from storage, flash combined with application-awareness allows intelligent storage systems to not only simplify storage management, but applications and the overall IT infrastructure. So why hasn’t this been done before? Prior to the advent of flash, mechanical disk-based systems were too complex to support a high level of intelligence. It would be like trying to build a personal computer using vacuum tubes. The huge leap in flash performance, at last puts intelligent storage within reach.

Figure 3: Storage systems often incorporate flash in to an existing disk-based architecture, using it as a cache. This creates additional complexity and still has performance limitations. Flash-only architectures deliver performance but still require a disk tier for cold data. Intelligent flash architectures are built around flash, but can use disk as an integral part of the system to provide cost-effective data and performance management.

Future of Enterprise Storage

Although it is easy to think of flash as simply faster storage, it can offer far more (Figure 3). We have eliminated a key mechanical barrier to scaling computing systems. Computation, communication and — finally — storage will now scale with improvements in semiconductor technology. Consider that when transistors replaced vacuum tubes, we got much more than merely compact radios. We got more powerful and more intelligent systems. Similarly, flash is a technology with potentially profound impact when properly harnessed: intelligent products that are far simpler and far more powerful. It automates many of the tough but tedious problems such as configuration, management, efficiency and performance barriers that waste enormous amounts of system administrator effort.

Simple, intelligent, fast: This is the future of enterprise storage.

In contrast, flash-based products are designed specifically for flash rather than mechanical disk drives, delivering dramatically lower latency. The key distinction is that their basic data paths do not require accessing disk. Often using low-cost MLC technology, flash-based products sometimes incorporate new techniques such as inline dedupe and compression to reduce the high $/GB of flash. Most flash-based products are flash-only, while some integrate hard disks to expand capacity and simplify management.

Initial flash-only products are basic arrays.  Focused on getting the highest possible IOPS, they generally have very high $/GB, and are missing enterprise features such as HA, snapshots, and clones. Even with inline dedupe and compression, flash-only arrays are currently too expensive for running the vast majority of applications in an enterprise. Even very aggressive estimates for these advanced techniques cannot overcome the >15x cost/GB gap between low-cost SATA HDDs and MLC flash devices. Consequently, flash-only arrays require separate low-cost disk-based storage systems for storing snapshots, replicas, infrequently accessed data, and the data of less IO-intensive applications (Figure 1).

Figure 1: Flash-only products are incomplete solutions.

As a result, flash-only arrays require significant additional work to stage and de-stage data and applications between flash and disk.  Combined with their high $/GB and lack of enterprise features, this means that flash-only arrays are much better suited for very high-performance applications without significant data management requirements. Using flash-only products for enterprise applications will require extensive planning, monitoring and additional supporting infrastructure.

Next-up: Intelligent flash-based products.

While flash provides extraordinarily high IOPS, it brings a whole new set of problems: write amplification, latency spikes, limited write endurance, and – last but not least – very high $/GB. Today, commodity MLC SSDs cost about $2/GB – approximately twenty times more than SATA hard disks. This is too expensive to run many mainstream applications on SSD. To leverage the high IOPS but compensate for the high $/GB of flash, flash storage systems are employing a variety of techniques such as caching, tiering, and inline compression and dedupe.

Existing storage vendors and new entrants have attempted to exploit flash in different ways. These products can be grouped into two broad categories, based on their impact on latency:

• Disk-based products with flash as a cache.
• Flash-based products (with or without HDDs for expanded capacity).

Figure 1: Flash as Cache architectures commit writes to disk.

As seen in Figure 1, disk-based products are fundamentally designed to optimize the use of hard disk drives, with flash bolted-on as a cache to accelerate read performance. Flash as a cache is relatively easy to implement, so it is not surprising that existing legacy storage vendors have taken this path. Many flash-as-cache implementations are non-persistent and non-redundant, so performance plummets after crashes and/or component failures. Since the “master” copy remains on hard disk, reads benefit from flash, but writes do not. Therefore, overall performance will not scale directly with improvements in flash technology.

Figure 2: Disk-based architectures suffer from high latency even with flash added to them.

Figure 2 shows the impact of flash hit rate on latency. Hit rates of 50% are typical for disk-based products with flash as a cache.  Even with hit rates as high as 67%, average read latencies are ten times higher than flash-based products.

Next-up: Flash-based products.