VMware vSAN vExperts 2018

I’ve just found out that I’ve been selected to be a vSAN vExpert again this year which was great news indeed. The complete list of vSAN vExperts 2018 can be found at https://blogs.vmware.com/vmtn/2018/06/vexpert-vsan-2018-announcement.html

vSAN vExpert programme is a sub programme of the wider VMware vExpert programme where out of those already selected vExperts, people who have shown specific speciality and thought leadership around vSAN & related Hyper-Converged technologies are being recognised specifically for their efforts. vSAN vExpert programme only started back in 2016 and while I missed out during the first year, I was also a vSAN vExpert in 2017 too so it’s quite nice to have been selected again for 2018.

As a part of the vSAN vExpert program, selected members typically are entitled to a number of benefits such as NFR license keys for full vSAN suite for lab and demo purposes, access to vSAN product management team at VMware, exclusive webinars & NDA meetings, access to preview builds of the new software and also get a chance to provide feedback to the product management team on behalf of our clients which is great for me as technologist working in the channel.

I have been a big advocate of Software Defined everything for about 15 years now as, they way I saw it, the power in most technologies are often derived from software. Public cloud is the biggest testament for this we can see today. So when HCI became a “thing”, I was naturally a big promoter of the concept and realistically, the Software Defined Storage (SDS) which made HCI what it is, was something I’ve always seen the value in. While there are many other SDS tech have started to appear since then, vSAN was always something unique in that it’s more tightly coupled to the underlying hypervisor like no other HCI / SDS solution and this architectural difference was the main reason why I’ve always liked and therefore promoted the vSAN technology from beta days. Well, vSAN revenue numbers have grown massively for VMware since its first launch with vSAN 5.5 and now, the vSAN business unit within VMware is a self sufficient business in its own right. Since I am fortunate to be working for a VMware solutions provider partner here in the UK, I have seen first hand the number of vSAN solutions we’ve sold to our own customers have grown over 900% year on year between 2016 and 2017 which fully aligns with wider industry adoption of vSAN as a preferred storage option for most vSphere solutions.

This will only likely going to increase and some of the hardware innovation coming down the line such as Storage Class Memory integration and NVMe over Fabric technologies will further enhance the performance and reliability of genuinely distributed software defined storage technologies such as vSAN. So being recognised as a thought leader and a community evangelist for vSAN by VMware is a great honour as I can continue to share my thoughts, updates on the product development with the wider community for other people to benefit from.

So thank you VMware for the honour again this year, and congratulations for all the others who have also been selected to be vSAN vExperts 2018. Keep sharing your knowledge and thought leadership content…!

Chan

VMworld 2017 – vSAN New Announcements & Updates

During VMworld 2017 Vegas, a number of vSAN related product announcements will have been made and I was privy to some of those a little earlier than the rest of the general public, due being a vSAN vExpert. I’ve summerised those below. The embargo on disclosing the details lifts at 3pm PST which is when this blog post is sheduled to go live automatically. So enjoy! 🙂

vSAN Customer Adoption

As some of you may know, popularity of vSAN has been growing for a while now as a preferred alternative to legacy SAN vendors when it comes to storing vSphere workloads. The below stats somewhat confirms this growth. I too can testify to this personally as I’ve seen a similar increase to the number of our own customers that consider vSAN as the default choice for storage now.

Key new Announcements

New vSAN based HCI Acceleration kit availability

This is a new ready node program being announced with some OEM HW vendors to provide distributed data center services for data centers to keep edge computing platforms. Consider this to be somewhat in between vSAN RoBo solution and the full blown main data center vSAN solution. Highlights of the offering are as follows

  • 3 x Single socket servers
  • Include vSphere STD + vSAN STD (vCenter is excluded)
  • Launch HW partners limited to Fujitsu, Lenovo, Dell & Super Micro only
  • 25% default discount on list price (on both HW & SW)
  • $25K starting price

           

 

  • My thoughts: Potentially a good move an interesting option for those customers who have a main DC elsewhere or are primarily cloud based (included VMware Cloud on AWS). The practicality of vSAN RoBo was always hampered by the fact that its limited to 25 VMs on 2 nodes. This should slightly increase that market adoption, however the key decision would be the pricing. Noticeably HPe are absent from the initial launch but I’m guessing they will eventually sign up. Note you have to have an existing vCenter license elsewhere as its not included by default.

vSAN Native Snapshots Announced

Tech preview of the native vSAN data protection capabilities through snapshots have been announced and will likely be generally available in FY18. vSAN native snapshots will have the following characteristics.

  • Snapshots are all policy driven
  • 5 mins RPO
  • 100 snapshots per VM
  • Support data efficiency services such as dedupe as well as protection services such as encryption
  • Archival of snapshots will be available to secondary object or NAS storage (no specific vendor support required) or even Cloud (S3?)
  • Replication of snapshots will be available to a DR site.

  • My thoughts: This was a hot request and something that was long time coming. Most vSAN solutions need a 3rd party data center back up product today and often, SAN vendors used to provide this type of snapshot based backup solution from the array (NetApp Snap Manager suite for example) that vSAN couldn’t match. Well, it can now, and since its done at the SW layer, its array independent and you can replicate or archive that anywhere, even on cloud and this would be more than sufficient for lots of customers with a smaller or a point use case to not bother buying backup licenses elsewhere to protect that vSphere workload. This is likely going to be popular. I will be testing this out in our lab as soon as the beta code is available to ensure the snaps don’t have a performance penalty.

 

vSAN on VMware Cloud on AWS Announced

Well, this is not massively new but vSAN is a key part of VMware Cloud on AWS and the vSAN storage layer provide all the on premise vSAN goodness while also providing DR to VMware Cloud capability (using snap replication) and orchestration via SRM.

 

vSAN Storage Platform for Containers Announced

Similar to the NSX-T annoucement with K8 (Kubernetes) support, vSAN also provide persistent storage presentation to both K8 as well as Docker container instances in order to run stateful containers.

 
This capability came from the vmware OpenSource project code named project Hatchway and its freely available via GitHub https://vmware.github.io/hatchway/ now.

  • My thoughts: I really like this one and the approach VMware are taking with the product set to be more and more microservices (container based application) friendly. This capability came from an opensource VMware project called Project hatchway and will likely be popular with many. This code was supposed to be available on GitHub as this is an opensource project but I have not been able to see anything within the VMware repo’s on GitHub yet.

 

So, all in all, not very many large or significant announcements for vSAN from VMworld 2017 Vegas (yet), but this is to be expected as the latest version of vSAN 6.6.1 was only recently released with a ton of updates. The key take aways for me is that the popularity of vSAN is obviously growing (well I knew this already anyways) and the current and future announcements are going to be making vSAN a fully fledged SAN / NAS replacement for vSphere storage with more and more native security, efficiency and availability services which is great for the customers.

Cheers

Chan

 

VMware vSAN 6.6 Release – Whats New

VMware has just annouced the general availability of the latest version of vSAN which is the backbone of their native Hyper Converged Infrastructure offering with vSphere. vSAN has had a number of significant upgrades since its very first launch back in 2014 as version 5.5 (with vSphere 5.5) and each upgrade has added some very cool, innovative features to the solution which has driven the customer adoption of vSAN significantly. The latest version vSAN 6.6 is no different and by far it appears to be have the highest number of new features announced during an upgrade release.

Given below is a simple list of some of the key features of vSAN 6.6 which is the 6th generation of the products

Additional native security features

  • HW independent data at rest encryption (Software Defined Encryption)
    • Software Defined AES 256 encryption
    • Supported on all flash and hybrid
    • Data written already encrypted
    • KMS works with 3rd party KMS systems
  • Built-in compliance with dual factor authentication (RSA secure ID and Smart-card authentication)

Stretched clusters with local failure protection

With vSAN 6.6, if a site fails, surviving site will have local host and disk group protection still (not the case with the previous versions)

  • RAID 1 over RAID 1/5/6 is supported on All Flash vSAN only.
  • RAID 1 over RAID 1 is supported on Hybrid vSAN only

Proactive cloud analytics

This sounds like its kind of similar to Nimble’s cloud analytics platform which is popular with customers. With proactive cloud analytics, it uses data collected from VSAN support data globally to provide analytics through the vSAN health UI, along with some performance optimization advice for resolving performance issues.

Intelligent & Simpler operations

Simpler setup and post set up operations are achieved through a number of new features and capabilities. Some of the key features include,

  • Automated setup with 1 click installer & lifecycle management
  • Automated configuration & compliance checks for vSAN cluster (this was somewhat already available through vSAN health UI). Additions include,
    • Networking & cluster configurations assistance
    • New health checks for encryption, networking, iSCSI, re-sync operations
  • Automated controller firmware & driver upgrades
    • This automates the download and install of VMware supported drivers for various hard drives and RAID controllers (for the entire cluster) which is significantly important.
    • I think this is pretty key as the number of vSAN performance issues due to firmware mismatch (especially on Dell server HW) has been an issue for a while now.
  • Proactive data evacuation from failing drives
  • Rapid recovery with smart, efficient rebuild
  • Expanded Automation through vSAN SDK and PowerCLI

High availability

vSAN 6.6 now includes a highly available control plane which means the resilient management is now possible independent of vCenter.

Other key features

  • Increased performance
    • Optimized for latest flash technologies involving 1.6TB flash (Intel Optane drives anyone??)
    • Optimize performance with actionable insights
    • 30% faster sequential write performance
    • Optimized checksum and dedupe for flash
  • Certified file service and data protection (through 3rd party partners)
  • Native vRealize Operations integrations
  • Simple networking with Unicast
  • Real time support notification and recommendations
  • Simple vCenter install and upgrade
  • Support for Photon 1.1
  • Expanded caching tier choices

There you go. Another key set of features added to vSAN with the 6.6 upgrade which is great to see. If you are a VMware vSphere customer who’s looking at a storage refresh for your vSphere cluster or a new vSphere / Photon / VIC requirement, it would be silly not to look in to vSAN as opposed to looking at legacy hardware SAN technologies from a legacy vendor (unless you have non VMware requirements in the data center).

If you have any questions or thoughts, please feel free to comment / reach out

Additional details of whats new with VMware vSAN 6.6 is avaiable at https://blogs.vmware.com/virtualblocks/2017/04/11/whats-new-vmware-vsan-6-6/

Cheers

Chan

 

New Dedicated VSAN Management Plugin For vROps Released

Some of you may have seen the tweets and the article from legendary Duncan Epping here about the release of the new VMware VSAN plugin for vROPS (vRealize Operations Management Pack for vSAN version 1.0)

If you’ve ever had the previous VSAN plugin for vROps deployed, you might know that it was not a dedicated plugin for VSAN alone, but was a vRealize Operations Management Pack for Storage Devices as a whole which included not just the visibility in to VSAN but also legacy storage stats such as FC, iSCSI and NFS for legacy storage units (that used to connect to Cisco DCNM or Brocade Fabric switches).

This vROps plugin for vSAN  however is the first dedicated plugin for VSAN (hence the version 1.0) on vROps. According to the documentation it has the following features

  • Discovers vSAN disk groups in a vSAN datastore.
  • Identifies the vSAN-enabled cluster compute resource, host system, and datastore objects in a vCenter Server system.
  • Automatically adds related vCenter Server components that are in the monitoring state.

How to Install / Upgrade from the previous MPSD plugin

  1. Download the management pack (.pak file)
    1. https://solutionexchange.vmware.com/store/products/vmware-vrealize-operations-management-pack-for-vsan
  2. Login to the vROps instance as the administrator / with administrative privileges and go to Administration -> Solutions
  3. Click add (plus sign) and select the .Pak file and select the 2 check boxes to replace if already installed and reset default content. Accept any warnings and click upload.
  4. Once the upload is complete and staged, verify the signature validity and click next to proceed               
  5. Click next and accept the EULA and proceed. The management plugin will start to install.
  6. Now select the newly installed management plugin for VSAN and click configure. Within this window, connect to the vCenter server (cannot use previously configured credentials for MPSD). When creating the credentials, you need to specify an admin account for the vCenter instance. Connection can be verified using the test button.  
  7. Once connected, wait for the data collection from VSAN cluster to complete and verify collection is showing
  8. Go to Home and verify that the VSAN dedicated dashboard items are now available on vROps               
  9. By Default there will be 3 VSAN specific dashboard available now as follows under default dashboards
    1. vSAN Environment Overview – This section provide some vital high level information on the vSAN cluster including its type, total capacity, used, any congestion if available, and average latency figures along with any active alerts on the VSAN cluster. As you can see I have a number of alerts due to using non-compliant hardware in my VSAN cluster.   
    2. vSAN Performance
      1. This default dashboard provide various performance related information / stats for the vSAN cluster rand datastores as well as the VM’s residing on it. You can also check performance such as VM latency and IOPS levels based on the VM’s you select on the tile view and the trend forecast which is think is going to be real handy.    
      2. Similarly, you can see performance at vSAN disk group level also which shows information such as Write buffer performance or Reach cache performance levels, current as well as future forecasted levels which are new and were not previously accessible easily.
      3. You can also view the performance at ESXi host level which shows the basic information such as current CPU utilisation as well as RAM including current and future (forecast) trend lines in true vROps style which are going to be really well received. Expect the content available on this ppage to be significantly extended in the future iterations of this mgmt. pack.  
    3. Optimize vSAN Deployments – This page provide a high level comparison of vSAN and non vSAN enviorments which would be especially handy if you have vSAN datastores alongside traditional iSCSI or NFS data stores to see how for example, IOPS and latency compares between VM’s on VSAN and an NFS datastore presented to the same ESXi server (I have both)    
  10. Under Environment -> vSAN and Storage Devices, additional vSAN hierarchy information such as vSAN enabled clusters, Fault domains (if relevant), Disk groups and Witness hosts (if applicable) are now visible for monitoring which is real handy.                                                                        
  11. In the inventory explorer, you can see the list of vSAN inventory items that the data are being collected for.   

All in all, this is a welcome addition and will only continue to be improved and new monitoring features added as we go up the versions. I realy like the dedicated plugin factor as well as the nice default dashboards included with this version which no doubt will help customers truly use vROps as a single pane of glass for all things monitoring on the SDDC including VSAN.

Cheers

Chan

VMware Storage and Availability Technical Documents Hub

homepage

This was something I came across accidentally so thought it may be worth a very brief post about as I found some useful content there.

VMware Storage and Availability Technical Documents Hub, is an online repository of technical documents and “how to” guides including video documents for all storage and availability products within VMware. Namely, it has some very useful contents for 4 VMware product categories (as of now)

  • VSAN
  • SRM
  • Virtual Volumes
  • vSphere Replication

For example, under the VSAN section, there are a whole heap of VSAN 6.5 contents such as technical information on what’s new with VSAN 6.5, how to design and deploy VSAN 6.5…etc as well as some handy video’s on how to configure some of those too. There also seem to be some advanced technical documentation around VSAN caching algorithms…etc & some deployment guides which I though was quite handy.

vsan

Similarly there are some good technical documentation around vVols including overview, how to set up and implement VVols…etc.. However in comparison, the content is a little light for the others compared to VSAN, but I’m sure more content will be added as the portal gets developed further.

All the information are presented in HTML5 interface which is easy to navigate with handy option to print to PDF option on all pages if you wanna download the content as a PDF for offline reading which is cool.

I’d recommend you to check this documentation hub, especially if you use any storage solution from VMware like VSAN and would like to see most of the relevant technical documentation all in a single place.

Cheers

Chan

VSAN, NSX on Cisco Nexus, vSphere Containers, NSX Future & a chat with VMware CEO – Highlights Of My Day 2 at VMworld 2016 US

In this post,  I will aim to highlight the various breakout sessions I’ve attended during the day 2 at VMworld 2016 US, key items / notes / points learnt and few other interesting things I was privy to  during the day that is worth mentioning, along with my thoughts on them…!!

Day 2 – Breakout Session 1 – Understanding the availability features of VSAN

vsan-net-deploy-support

  • Session ID: STO8179R
  • Presenters:
    • GS Khalsa – Sr. Technical Marketing manager – VMware (@gurusimran)
    • Jeff Hunter – Staff Technical Marketing Architect – VMware (@Jhuntervmware)

In all honesty, I wasn’t quite sure why I signed up to this breakout session as I know VSAN fairly well, including its various availability features as I’ve been working with testing & analysing its architecture and performance when VSAN was first launched to then designing and deploying VSAN solutions on behalf of my customers for a while. However, having attended the session it reminded me of a key fact that I normally try to never forget which is “you always learn something new” even when you think you know most of it.

Anyways, about the session itself, it was good and was mainly aimed at the beginners to VSAN but I did manage to learn few new things as well as refresh my memory on few other facts, regarding VSAN architecture. The key new ones I learnt are as follows

  • VSAN component statuses (as shown within vSphere Web Client) and their meanings
    • Absent
      • This means VSAN things the said component will probably return. Examples are,
        • Host rebooted
        • Disk pulled
        • NW partition
        • Rebuild starts after 60 mins
      • When an item is detected / marked as absent, VSNA typically wait for 60 minutes before a rebuild is started in order to allow temporary failure to rectify itself
        • This means for example, pulling disks out of VSAN will NOT trigger an instant rebuild / secondary copy…etc. so it wont be an accurate test of VSAN
    • Degraded
      • This typically means the device / component is unlikely to return. Examples include,
        • A permeant Device Loss (PDL) or a failed disk
      • When a degraded item is noted, a rebuild started immediately
    • Active-Stale
      • This means the device is back online from a failure (i.e. was absent) but the data residing on it are NOT up to date.
  • VSAN drive degradation monitoring is proactively logged in the following log files
    • vmkernel.log indicating LSOM errors
  • Dedupe and Compression during drive failures
    • During a drive failure, de-duplication and compression (al flash only) is automatically disabled – I didn’t know this before

 

Day 2 – Breakout Session 2 – How to deploy VMware NSX with Cisco Nexus / UCS Infrastructure

  • Session ID: NET8364R
  • Presenters:
    • Paul Mancuso – Technical Product Manager (VMware)
    • Ron Fuller – Staff System Engineer (VMware)

This session was about a deployment architecture for NSX which is becoming increasingly popular, which is about how to design & deploy NSX on top of Cisco Nexus switches with ACI as the underlay network and Cisco UCS hardware. Pretty awesome session and a really popular combination too. (FYI – I’ve been touting that both these solutions are better together since about 2 years back and its really good to see both companies recognising this and now working together on providing guidance stuff like these). Outside of this session I also found out that the Nexus 9k switches will soon have the OVS DB support so that they can be used as TOR switches too with NSX (hardware VTEP to bridge VXLANs to VLANs to communication with physical world), much like the Arista switches with NSX – great great news for the customers indeed.

ACI&NSX-2

I’m not going to summarise the content of this session but wold instead like to point people at the following 2 documentation sets from VMware which covers everything that this session was based on, its content and pretty simply, everything you need to know when designing NSX solutions together with Cisco ACI using Nexus 9K switches and Cisco UCS server hardware (blades & rack mounts)

One important thing to keep in mind for all Cisco folks though: Cisco N1K is NOT supported for NSX. All NSX prepped clusters must use vDS. I’m guessing this is very much expected and probably only a commercial decision rather than a technical one.

Personally I am super excited to see VMware ands Cisco are working together again (at least on the outset) when it comes to networking and both companies finally have realised the use cases of ACI and NSX are somewhat complementary to each other (i.e. ACI cannot do most of the clever features NSX is able to deliver in the virtual world, including public clouds and NSX cannot do any of the clever features ACI can offer to a physical fabric). So watch this space for more key joint announcements from both companies…!!

Day 2 – Breakout Session 3 – Containers for the vSphere admin

Capture

  • Session ID: CNA7522
  • Presenters:
    • Ryan Kelly – Staff System Engineer (VMware)

A session about how VMware approaches the massive buzz around containerisation through their own vSphere integrated solution (VIC) as well as a brand new hypervisor system designed from ground up with containerisation in mind (Photon platform). This was more of a refresher session for than anything else and I’m not going to summarise all of it but instead, will point you to the dedicated post I’ve written about VMware’s container approach here.

Day 2 – Breakout Session 4 – The architectural future of Network Virtualisation

the-vision-for-the-future-of-network-virtualization-with-vmware-nsx-27-638

  • Session ID: NET8193R
    Presenters: Bruce Davie – CTO, Networking (VMware)

Probably the most inspiring session of the day 2 as Bruce went through the architectural future of NSX where he described what the NSX team within VMware are focusing on as key improvements & advancements of the NSX platform. The summary of the session is as follows

  • NSX is the bridge from solving today’s requirement to solving tomorrow’s IT requirements
    • Brings remote networking closer easily (i.e. Stretched L2)
    • Programtically (read automatically) provisoned on application demand
    • Security ingrained at a kernel level and every hop outwards from the applications
  • Challenges NSX is trying address (future)
    • Developers – Need to rapidly provision and destroy complex networks as a pre-reqs for applications demanded by developers
    • Micro services – Container networking ands security
    • Containers
    • Unseen future requirements
  • Current NSX Architecture
    • Cloud consumption plane
    • Management plane
    • Control plane
    • Data plane
  • Future Architecture – This is what the NSX team is currently looking at for NSX’s future.
    • Management plane scale out
      • Management plane now needs to be highly available in order to constantly keep taking large number of API calls for action from cloud consumption systems such as OpenStack, vRA..etc – Developer and agile development driven workflows….etc.
      • Using & scaling persistent memory for the NSX management layer is also being considered – This is to keep API requests in persistent memory in a scalable way providing write and read scalability & Durability
      • Being able to take consistent NSX snapshots – Point in time backups
      • Distributed log capability is going to be key in providing this management plane scale out whereby distributed logs that store all the API requests coming from Cloud Consumption Systems will be synchronously stored across multiple nodes providing up to date visibility of the complete state across to all nodes, while also increasing performance due to management node scale out
    • Control plane evolution
      • Heterogeneity
        • Currently vSphere & KVM
        • Hyper-V support coming
        • Control plane will be split in to 2 layers
          • Central control plane
          • Local control plane
            • Data plane (Hyper-V, vSphere, KVM) specific intelligence
    • High performance data plane
      • Use the Intel DPDK – A technology that optimize packet processing in Intel CPU
        • Packet switching using x86 chips will be the main focus going forward and new technologies such as DPDK will only make this better and better
        • DPDK capacities are best placed to optimise iterative processing rather than too many context switching
        • NSX has these optimisation code built in to its components
          • Use DPDK CPUs in the NSX Edge rack ESXi servers is  a potentially good design decision?
  • Possible additional NSX use cases being considered
    • NSX for public clouds
      • NSX OVS and an agent is deployed to in guest – a technical preview of this solution was demoed by Pat Gelsinger during the opening key note on day 1 of VMworld.
    • NSX for containers
      • 2 vSwitches
        • 1 in guest
        • 1 in Hypervisor

 

My thoughts

I like what I heard from the Bruce about the key development focus areas for NSX and looks like all of us, partners & customers of VMware NSX alike, are in for some really cool, business enabling treats from NSX going forward, which kind of reminds me of when vSphere first came out about 20 years ago :-). I am extremely excited about the opportunities NSX present to remove what is often the biggest bottleneck enterprise or corporate IT teams have to overcome to simply get things done quickly and that is the legacy network they have. Networks in most organisations  are still very much managed by an old school minded, networking team that do not necessarily understand the convergence of networking with other silos in the data center such as storage and compute, and most importantly when it comes to convergence with modern day applications. It is a fact that software defined networking will bring the efficiency to the networking the way vSphere brought efficiency to compute (want examples how this SDN efficiency is playing today? Look at AWS and Azure as the 2 biggest use cases) where the ability to spin up infrastructure, along with a “virtual” networking layer significantly increases the convenience for the businesses to consume IT (no waiting around for weeks for your networking team to set up new switches with some new VLANs…etc.) as well as significantly decreasing the go to market time for those businesses when it comes to launching new products / money making opportunities. All in all, NSX will act as a key enabler for any business, regardless of the size to have an agile approach to IT and even embrace cloud platforms.

From my perspective, NSX will provide the same, public cloud inspired advantages to customers own data center and not only that but it will go a step further by effectively converting your WAN to an extended LAN by bridging your LAN with a remote network / data center / Public cloud platform to create something like a LAN/WAN (Read LAN over WAN – Trade mark belongs to me :-))which can automatically get deployed, secured (encryption) while also being very application centric (read “App developers can request networking configuration through an API as a part of the app provisioning stage which can automatically apply all the networking settings including creating various networking segments, routing in between & the firewall requirements…etc. Such networking can be provisioned all the way from a container instance where part of the app is running (i.e. DB server instance as a container service) to a public cloud platform which host the other parts (i.e. Web servers).

I’ve always believed that the NSX solution offering is going to be hugely powerful given its various applications and use cases and natural evolution of the NSX platform through the focus areas like those mentioned above will only make it an absolute must have for all customers, in my humble view.

 

Day 2 – Meeting with Pat Gelsinger and Q&A’s during the exclusive vExpert gathering

vExpert IMG_5750

As interesting as the breakout sessions during the day have been, this was by far the most significant couple of hours for me on the day. As a #vExpert, I was invited to an off site, vExpert only gathering held at Vegas Mob Museum which happened to include VMware CEO, Pat Gelsinger as the guest of honour. Big thanks to the VMware community team lead by Corey Romero (@vCommunityGuy) for organising this event.

This was an intimate gathering for about 80-100 VMware vExperts who were present at VMworld to meet up at an off site venue and discuss things and also to give everyone a chance to meet with VMware CEO and ask him direct questions, which is something you wouldn’t normally get as an ordinary person so it was pretty good. Pat was pretty awesome as he gave a quick speech about the importance of vExpert community to VMware followed up by a Q&A session where we all had a chance to ask him questions on various fronts. I myself started the Q&A session by asking him the obvious question, “What would be the real impact on VMware once the Dell-EMC merger completes” and Pats answer was pretty straight forward. As Michael Dell (who happened to come on stage during the opening day key note speech said it himself), Dell is pretty impressed with the large ecosystem of VMware partners (most of whom are Dell competitors) and will keep that ecosystem intact going forward and Pat echoed the same  message, while also hinting that Dell hardware will play a key role in all VMware product integrations, including using Dell HW by default in most pre-validated and hyper-converged solution offerings going forward, such as using Dell rack mount servers in VCE solutions….etc. (in Pat’s view, Cisco will still play a big role in blade based VCE solution offerings and they are unlikely to walk away from it all just because of Dell integration given the substantial size of revenue that business brings to Cisco).

If I read in between the lines correctly (may be incorrect interpretations from my end here),  he also alluded that the real catch of the EMC acquisition as far as Dell was concerned was VMware. Pat explained that most of the financing charges behind the capital raised by Dell will need to be paid through EMC business’s annual run rate revenue (which by the way is roughly the same as the financing interest) so in a way, Dell received VMware for free and given their large ecosystem of partners all contributing towards VMware’s revenue, it is very likely Dell will continue to let VMware run as an independent entity.

There were other interesting questions from the audience and some of the key points made by Pat in answering those questions were,

  • VMware are fully committed to increasing NSX adoption by customers and sees NSX as a key revenue generator due to what it brings to the table – I agree 100%
  • VMware are working on the ability to provide networking customers through NSX, a capability similar to VMotion for compute as one of their (NSX business units) key goals. Pat mentioned that engineering in fact have this figured out already and testing internally but not quite production ready.
  • In relation to VMware’s Cross Cloud Services as a service offering (announced by Pat during the event opening keynote speech), VMware are also working on offering NSX as a service – Though the detail were not discussed, I’m guessing this would be through the IBM and vCAN partners
  • Hinted that a major announcement on the VMware Photon platform  (One of the VMware vSphere container solutions) will be taking place during VMworld Barcelona – I’ve heard the same from the BU’s engineers too and look forward to Barcelona announcements
  • VMware’s own cloud platform, vCloud air WILL continue to stay focused on targeted use cases while the future scale of VMware’s cloud business will be expected to come from the vCAN partners (hosting providers that use VMware technologies and as a result are part of the VMware vCloud Air Network…i.e IBM)
  • Pat also mentioned about the focus VMware will have on IOT and to this effect, he mentioned about the custom IOT solution VMware have already built or working on (I cannot quite remember which was it) for monitoring health devices through the Android platform – I’m guessing this is through their project ICE and LIOTA (Little IOT Agent) platform which already had similar device monitoring solutions being demoed in the solutions exchange during VMworld 2016. I mentioned about that during my previous post here

It was really good to have had the chance to listen to Pat up close and be able to ask direct questions and get frank answers which was a fine way to end a productive and an education day for me at VMworld 2016 US

Image credit goes to VMware..!!

Cheers

Chan

 

 

VVDs, Project Ice, vRNI & NSX – Summary Of My Breakout Sessions From Day 1 at VMworld 2016 US –

Capture

Quick post to summerise the sessions I’ve attended on day 1 at @VMworld 2016 and few interesting things I’ve noted. First up are the 3 sessions I had planned to attend + the additional session I managed to walk in to.

Breakout Session 1 – Software Defined Networking in VMware validated Designs

  • Session ID: SDDC7578R
  • Presenter: Mike Brown – SDDC Integration Architect (VMware)

This was a quick look at the VMware Validated Designs (VVD) in general and the NSX design elements within the SDDC stack design in the VVD. If you are new to VVD’s and are typically involved in designing any solutions using the VMware software stack, it is genuinely worth reading up on and should try to replicate the same design principles (within your solution design constraints) where possible. The diea being this will enable customers to deploy robust solutions that have been pre-validated by experts at VMware in order to ensure the ighest level of cross solution integrity for maximum availability and agility required for a private cloud deployment. Based on typical VMware PSO best practices, the design guide (Ref architecture doc) list out each design decision applicable to each of the solution components along with the justification for that decision (through an explanation) as well as the implication of that design decision. An example is given below

NSX VVD

I first found out about the VVDs during last VMworld in 2015 and mentioned in my VMworld 2015 blog post here. At the time, despite the annoucement of availability, not much content were actually avaialble as design documents but its now come a long way. The current set of VVD documents discuss every design, planning, deployment and operational aspect of the following VMware products & versions, integrated as a single solution stack based on VMware PSO best practises. It is based on a multi site (2 sites) production solution that customers can replicate in order to build similar private cloud solutions in their environments. These documentation set fill a great big hole that VMware have had for a long time in that, while their product documentation cover the design and deployment detail for individual products, no such documentaiton were available for when integrating multiple products and with VVD’s, they do now. In a way they are similar to CVD documents (Cisco Validated Designs) that have been in use for the likes of FlexPod for VMware…etc.

VVD Products -1

VVD Products -2

VVD’s generally cover the entire solution in the following 4 stages. Note that not all the content are fully available yet but the key design documents (Ref Architecture docs) are available now to download.

  1. Reference Architecture guide
    1. Architecture Overview
    2. Detailed Design
  2. Planning and preperation guide
  3. Deployment Guide
    1. Deployment guide for region A (primary site) is now available
  4. Operation Guide
    1. Monitoring and alerting guide
    2. backup and restore guide
    3. Operation verification guide

If you want to find out more about VVDs, I’d have a look at the following links. Just keep in mind that the current VVD documents are based on a fairly large, no cost barred type of design and for those of you who are looking at much smaller deployments, you will need to exercise caution and common sense to adopt some of the recommended design decisions to be within the appplicable cost constraints (for example, current NSX design include deploying 2 NSX managers, 1 integrated with the management cluster vCenter and the other with the compute cluster vCenter, meaning you need NSX licenses on the management clutser too. This may be an over kill for most as typically, for most deployments, you’d only deploy a single NSX manager integrated to the compute cluster)

As for the Vmworld session itself, the presenter went over all the NSX related design decisions and explained them which was a bit of a waste of time for me as most people would be able to read the document and understand most of those themselves. As a result I decided the leave the session early, but have downloaded the VVD documents in order to read throughly at leisure. 🙂

Breakout Session 2 – vRA, API, Ci Oh My!

  • Session ID: DEVOP7674
  • Presenters

vRA Jenkins Plugin

As I managd to leave the previous session early, I manage to just walk in to this session which had just started next door and both Kris and Ryan were talking about the DevOps best practises with vRealize Automation and vrealize Code Stream. they were focusing on how developpers who are using agile development that want to invoke infrastructure services can use these products and invoke their capabilities through code, rather than through the GUI. One of the key focus areas was the vRA plugin for Jenkins and if you were a DevOps person of a developper, this session content would be great value. if you can gain access to the slides or the session recordings after VMworld (or planning to attend VMworld 2016 Europe), i’d highly encourage you to watch this session.

Breakout Session 3 – vRealize, Secure and extend your data center to the cloud suing NSX: A perspective for service providers and end users

  • Session ID: HBC7830
  • Presenters
    • Thomas Hobika – Director, America’s Service Provider solutions engineering & Field enablement, vCAN, vCloud Proviuder Software business unit (VMware)
    • John White – Vice president of product strategy (Expedient)

Hosted Firewall Failover

This session was about using NSX and other products (i.e. Zerto) to enable push button Disaster Recovery for VMware solutions presented by Thomas, and John was supposed to talk about their involvement in designing this solution.  I didn’t find this session content that relevent to the listed topic to be honest so left failrly early to go to the blogger desks and write up my earlier blog posts from the day which I thought was of better use of my time. If you would like more information on the content covered within this sesstion, I’d look here.

 

Breakout Session 4 – Practical NSX Distributed Firewall Policy Creation

  • Session ID: SEC7568
  • Presenters
    • Ron Fuller – Staff Systems Engineer (VMware)
    • Joseph Luboimirski – Lead virtualisation administrator (University of Michigan)

Fairly useful session focusing about NSX distributed firewall capability and how to effectively create a zero trust security policy on ditributed firewall using vairous tools. Ron was talking about various different options vailablle including manual modelling based on existing firewall rules and why that could potentially be inefficient and would not allow customers to benefit from the versatality available through the NSX platform. He then mentioned other approaches such as analysing traffic through the use of vRealize Network Insight (Arkin solution) that uses automated collection of IPFIX & NetFlow information from thre virtual Distributed Switches to capture traffic and how that capture data could potentialy be exported out and be manipulated to form the basis for the new firewall rules. He also mentioned the use of vRealize Infrastructure Navigator (vIN) to map out process and port utilisation as well as using the Flow monitor capability to capture exisitng communication channels to design the basis of the distributed firewall. The session also covered how to use vRealize Log Insight to capture syslogs as well.

All in all, a good session that was worth attending and I would keep an eye out, especially if you are using / thinking about using NSx for advanced security (using DFW) in your organisation network. vRealize Network Insight really caught my eye as I think the additional monitoring and analytics available through this platform as well as the graphical visualisation of the network activities appear to be truely remarkeble (explains why VMware integrated this to the Cross Cloud Services SaS platform as per this morning’s announcement) and I cannot wait to get my hands on this tool to get to the nitty gritty’s.

If you are considering large or complex deployment of NSX, I would seriously encourage you to explore the additional features and capabilities that this vRNI solution offers, though it’s important to note that it is licensed separately form NSX at present.

vNI         vNI 02

 

Outside of these breakout sessions I attended and the bloggin time in between, I’ve managed to walk around the VM Village to see whats out there and was really interested in the Internet Of Things area where VMware was showcasing their IOT related solutions currently in R&D. VMware are currently actively developing an heterogeneous IOT platform monitoring soluton (internal code name: project Ice). The current version of the project is about partnering up with relevent IOT device vendors to develop a common monitoring platform to monitor and manage the various IOT devices being manufacured by various vendors in various areas. If you have a customer looking at IOT projects, there are opportunities available now within project Ice to sign up with VMware as a beta tester and co-develop and co-test Ice platform to perform monitoring of these devices.

An example of this is what VMware has been doing with Coca Cola to monitor various IOT sensors deployed in drinks vending machines and a demo was available in the booth for eall to see

IOT - Coke

Below is a screenshot of Project Ice monitoring screen that was monitoring the IOT sensors of this vending machine.   IOT -

The solution relies on an Open-Source, vendor neutral SDK called LIOTA (Little IOT Agent) to develop a vendor neutral agent to monitor each IOT sensor / device and relay the information back to the Ice monitoring platform. I would keep and eye out on this as the use cases of such a solution is endless and can be applied on many fronts (Auto mobiles, ships, trucks, Air planes as well as general consumer devices). One can argue that the IOT sensor vendors themselves should be respornsible for developping these mo nitoring agents and platforms but most of these device vendors do not have the knowledge or the resources to build such intelligent back end platforms which is where VMware can fill that gap through a partship.

If you are in to IOT solutions, this is defo a one to keep your eyes on for further developments & product releases. This solution is not publicly available as of yet though having spoken to the product manager (Avanti Kenjalkar), they are expecting a big annoucement within 2 months time which is totally exciting.

Some additional details can be found in the links below

Cheers

Chan

#vRNI #vIN #VVD # DevOps #Push Button DR # Arkin Project Ice # IOT #LIOTA

VMware VSAN 6.2 Performance & Storage savings

Just a quick post to share some very interesting performance stats observed on my home lab VSAN cluster (Build details here). The VSAN datastore is in addition to a few NFS datastores also mounted on the same hosts using an external Synology SAN.

I had to build a number of Test VMs, a combination of Microsoft Windows 2012 R2 Datacenter and 2016 TP4 Datacenter VMs on this cluster and I placed all of them on the VSAN datastore to test the performance. See below the storage performance stats during the provisioning (cloning from template) time. Within the Red square are the SSD drive performance stats (where the new VM’s being created) Vs Synology’s NFS mount’s performance  stats (where templates resides) in the Yellow box.

Provisioning Performance

Pretty impressive from all Flash VSAN running on a bunch of white box servers with consumer grade SSD drives (officially unsupported of course but works!), especially relative to the performance of the Synology NFS mounts (RAID1/0 setup for high performance), right??

Imagine what the performance would have been if this was on enterprise grade hardware in your datacentre?

Also caught my eye was the actual inline deduplication and compression savings immediately available on the VSAN datastore after the VM’s were provisioned.

Dedupe & Compression Savings

As you can see, to store 437GB of raw data, with a FTT=1 (where VSAN keeping redundant copies of each vmdk file), its only consuming 156GB of actual storage on the VSAN cluster, saving me 281GB of precious SSD storage capacity. Note that this is WITHOUT Erasure Coding RAID 5 or RAID 6 that’s also available with VSAN 6.2 which, had that been enabled, would have further reduced the actual consumed space more.

The point of this all is the performance and the storage savings available in VSAN, especially all flash VSAN is epic and I’ve seen this in my own environment. In an enterprise datacenter, All Flash VSAN can drastically improve your storage performance but at the same time, significantly cut down on your infrastructure costs for all of your vSphere storage environments. I personally know a number of clients who have achieved such savings in their production environments and each and every day, there seem to be more and more demand from customers for VSAN as their preferred storage / Hyper-Converged technology of choice for all their vSphere use cases.

I would strongly encourage you to have a look at this wonderful technology and realise these technical and business benefits (summary available here) for yourself.

Share your thoughts via comments below or feel free to reach out to discuss what you think via email or social media

Thanks

Chan

New VMware Product Availabilities – Now available to download

Update

VMware have just made a number of new product versions (mostly maintenance releases on few different products, including that of the much hyped VSAN 6.2) so a quick post to summarise the content that was released last night (15.03.2016)

  • VMware VSAN 6.2 – VMware VSAN 6.2 was officially announced in early February with a number of cool new features such as Erasure coding but unless you were a techie trying to download the software, you may have not known that it was not available for download despite being announced. That was until yesterday and the product is now available to download for every customer.

 

  • VMware vRealize Automation 7.0.1 now released and available for download
    • Release notes here
    • Product binaries here
    • Documentation here

 

  • VMware vRealize Orchestrator 7.0.1 is released and available to download
    • Release notes here
    • Product binaries here
    • Documentation here

 

  • vRealize Business for Cloud (Old ITBMS offering) is also released and available for grabs now
    • Release notes here
    • Product binaries here
    • Documentation here

 

  • vRealize Log Insight 3.3.1 is released and available to download
    • Release notes here
    • Product binaries here
    • Documentation here

 

  • vCloud Suite 7.0 is also released and available to download (here) – This includes all of the above new versions of products plus the exiting versions for vSphere Replication 6.1 + vSphere Data Protection 6.1.2 + vROPS 6.2.0a + vRealize Infrastructure Navigator 5.8.5

VMware All Flash VSAN Implementation (Home Lab)

I’ve been waiting for a while to be able to implement an all flash VSAN in my lab and now that VSAN 6.2 has been announced, I thought it would be time to upgrade my capacity disks from HDD’s to SSD’s and get cracking..! (note: despite the announcement, VSAN 6.2 binaries are NOT YET available to download. I’m hearing it would be available in a week or two on My VMware though so until then, mine is based on VSAN 6.1 – ESXi 6.0U1 binaries)

As I already had a normal (Hybrid) VSAN implementation using SSD+HDD in my management vSphere cluster, the plan was to keep the existing SSD’s as caching tier and replace the current HDD’s with high capacity SSD drives. So I bought 3 new Samsung 850 EVO 256GB drives from Amazon (here)                                       Capture

All Flash VSAN Setup

Given below are the typical steps involved in the processes to implement All Flash VSAN within the VMware cluster (I’m using the 3 node management cluster within my lab for the illustration below)

  1. Install the SSD drives in the server – This should be easy enough. If you are doing this in a production environment, you need to ensure that the capacity SSD’s (similar to all other components in your VSAN ready nodes)  are in the VMware HCL
  2. Enable VSAN on the cluster – Need to be done on the web client      1 - Enable VSAN
  3. Verify the new SSDs are available & recognised within the web client – All SSD’s are recognised as caching disks by default.              0 - Default disk assignment  2 - all caching
  4. Manually tag the required SSD drives as capacity disks VIA COMMANDLINE for them to be recognised as capacity disks within VSAN configuration – This step MUST be carried out using one the ways explained below and until then, SSD disks WILL NOT be available to be used as capacity disks within an all flash VSAN otherwise. (There currently is no GUI option on the web client to achieve this and cli must be used)
    1. Use esxcli command on each ESXi server
      1. SSH in to the ESXi server shell
      2. Use the vdq -q command to get the T10 SCSI name for the capacity SSD drive (Also verify “IsCapacityFlash” option is set to 0) 3 SSH
      3. Use the “esxcli vsan storage tag add -d <SCSI T10 name of the disk> -t capacityFlash” command to mark the disk as capacity SSD.   4 ESXCLI
      4. Use the vdq -q command to query the disk status and ensure the disk is now marked as “1” for “IsCapacityFlash” 5 esxcli verify
      5. If you now look at the Web client UI, the capacity SSD disk will now have been correctly identified as capacity (note the drive type changed to HDD which is somewhat misleading as the drive type is still SSD) 8.1 GUI
    2. Use the “VMware Virtual SAN All-Flash Configuration Utility” software – This is a 3rd party tool and not an officially supported VMware tool but if you do not want to manually SSH in to the ESXi servers 1 by 1, this software could be quite handy as you can bulk tag on many ESXi servers all at the same time. I’ve used this tool to tag the SSD’s in the next 2 servers of my lab in the illustration below. xx - Use VMware Virtual SAN all-flash configuration utility
  5. Verify capacity SSD across all hosts – Now that all the capacity SSD’s have been tagged as capacity disks, verify that the web client sees all capacity SSD’s across all hosts                                                9 Disk group manual
  6. Create the disk groups on each host – I’m opting to create this manually as shown below 9 Disk group manual 10 - Verify disk groups
  7. Verify the VSAN datastore now being available and accessible 11 - VSAN datastore active

There you have it. Implementing all flash VSAN requires manually tagging the SSDs as capacity SSDs for the time being and this is how you do it. I may also add that since the all flash VSAN, my storage performance has gone through the roof in my home lab which is great too. However this is all done on Whitebox hardware and not all of them are fully on VMware HCL….etc which makes those performance figures far from optimal. It would be really good to see performance statistics if you have deployed all flash VSAN in your production environment.

Cheers

Chan