I mentioned recently that I have been playing with VMware’s Virtual SAN (vSAN) in beta. VMware is really pushing the software defined datacenter. Since they already have software defined compute and memory it really only leaves storage and network. This year at VMworld they announced products to fill both these needs. I will discuss network virtualization in another post (NSX). The key item on my mind is if vsan is for small businesses or if they expect it to take over enterprise. Everything in this article is mearly my opinion I do not have any insider knowledge so enjoy it with a little salt. Let me be the first to say I am very excited about vSAN and in order to understand my thoughts it requires a little history lesson:
VMware Virtual Storage Appliance
A few years ago VMware saw a large hole in there market adoption: small business. For years small business was buying servers and shoving them into a closet. This was a model they could afford and works. As VMware saw enterprise adoption fill up they needed new customers. Small business would not mind getting the advantages of virtualization in fact in a lot of cases it would really fit them better: flexibility, vmotion etc.. There was one critical issue in the way.. Central shared storage. Small business could run a single node of ESXi and gain some of virtualization benefits but this did not provide VMware with any profit or software support. So how to we get around the entrance cost for storage? The simple process had already been provided by white box and home labs NFS datastores provided by a virtual machine. (In other words fake shared storage) VSA was VMware’s first and second attempt to address this issue. It worked ok but was slow compared to traditional SAN’s it also suffered from lack of scalability.
Slow Adoption by SAN providers
VMware desperately wants to provide new storage features via their software offerings. As such they created VASA (vSphere Storage API’s for Storage AwarenessVsphere 5.0) a set of extensions which allowed storage array’s to integrate into Vsphere via plug-ins. VASA allows the storage provider to serve up capabilities around availablity, provisioning and performance. When combined with VAAI VMware finally was able to design software around the arrays. But very few vendors adopted the standards. Their implementations were bare bones to certify to customers support for these API’s / Standards. This slow and poor implementation really hurt VMware’s ability to move quickly on software defined storage solutions (like replication, some forms of FT and stretch cluster solutions).
How does cache fit into the discussion
Cache has always been a part if high end arrays. They use ram to store reads and writes to speed up common operations. In my environment we have lots if arrays for example
- 1 Hp eva 4400 which has 126 1tb 7200 rpm drives with 1gb of total cache per controller
- 2 Hp eva 4400 which has 126 300 gb 15000 rpm drives with 1gb of total cache per controller
- 3 Hp p9500 which has 50 300 gb 15000 rpm drives with 96gb of total cache per controller
- 4 Hitachi hus vm which has 600 1tb rpm drives with 64gb of total cache per controller
It should be noted that these arrays can be defined two different ways
- Evas are aula arrays that stripes all data across all disks creating some artifical performance
- P9500 is really a hitachi vsp meaning it is the big brother ofa hus vm so they are essentially the same minus the cache differences. They are true active active arrays with 8+2 or in my case 14+2. They also support front ending other arrays.
In order to propose my thoughts around cache I will give you the following scenarios
- Virtual machine running a heavy sybase server. It has the ability to completely trash storage with a single badly written query using temp tables …. so it is a mixed load of reads and writes
- The operating system metrics will be observed and completion time of the query was recorded.
So here are the basic results this is just an illustration of my thoughts
- Query on eva 1 operating system shows waiting on storage takes 45 seconds
- Query on eva 2 operating system shows waiting on storage takes 42 seconds
- Query on p9500 3 operating system shows waiting on processor takes 22 seconds
- Query on hus vm 4 operating system shows waiting on processor takes 23 seconds
- Query on eva 1 while presented though p9500 3 operating system shows waiting on processor takes 23 seconds
- Query on eva 2 while presented through p9500 3 operating system shows waiting on processor takes 23 seconds
Full disclosure during all these tests the p9500 and hus vm were servicing other production vmware and non vmware loads. So the take away cache is everything until it is full. Disk speed dies not equal performance .. cache does. So we all need a lot more cache.
What is VMware VSan
Well now we are down to the nuts and bolts. Vsan combines local disks and SSD on servers into a single distributed (Think RAIN) VMFS datastore over a VSAN network. It can scale very large and includes a ton of features. Including:
- Striping across disks for performance on a vmdk level
- Truely VMFS aware storage system
- Ability to set the number of copies you want of your vm (0 copies, 1 copy or mirror, or additional copies)
- Storage build into the kernel to create shortest path for I/O
- Thin provisioning for speed of copy and deployment
- Use SSD for 30% write / 70% read cache
- Adjustment of striping or replication copies on the fly
In short Vsan uses SSD (remember that SSD for servers comes in 400GB / 800Gb / 1.2 TB models) for cache then uses back end slower disk for sequential work and offload. This allows us to run a vmware cluster without any shared storage.
Future or just for small business
Everyone I ask answers with it depends which is really true. But if you want my two cents it’s the future. I think companies like Nutanix have been in a similar market for a few years and done very well. VMware’s vSAN is only in beta and needs to grow up a little… and we will see new features but I consider it to be the future of virtualization and datacenter. I think it will start with the poorly prices EUC (VDI) market as a inexpensive entrance into the market… but as it proves it’s ability (Just think about it 800GB of cache per server not 256GB’s shared on the array) it will get adoption.
Understand thy place VSAN
Do not get me wrong I am not suggesting that traditional SAN’s are dead. I have been working with SAN’s for over ten years and they have their place. They will continue to exist but VMware would like to have full control over the stack and have additional products to sell. In trade they promise increased flexability features and speed of deployment… all good things. SAN makers will counter with smaller arrays and new features and the battle will begin.