The VMware NSX Platform – Healthcare Series Part 3 – Micro-Segmentation Concept

When using an application-based policy approach, security is a critical part to the application workload.  Security is just as important as how much CPU or RAM you give an application workload.  The VMware NSX Platform introduces 3 primary use cases when it comes to security for application workloads.  We’re going to focus on the first use case:  Micro-segmentation and how it relates to Healthcare organizations.


A quick background on why Micro-segmentation is important and security trends in modern data centers. In most modern data centers, there has been a large uptick in the amount of traffic that occurs between systems rather than inbound-and-outbound of systems.  This is referred to as East-West traffic within the data center, versus North-South traffic in and out of the data center.  In the hardware-base world, security for East-West traffic is sometimes done either by sending the traffic from their applications to their perimeter firewalls or by purchasing hardware appliances to put inside the data center between the applications. This form of security is through isolation and segmentation of the applications.  You can do this at the entire application level through concepts such as Trust Zones, achieving what some call macro-segmentation, but when we place security at the workload level we achieve what’s called, micro-segmentation.  Micro-segmentation facilitates a Zero-Trust security model. Zero-Trust means that unless communications between systems are explicitly trusted, it’s implicitly untrusted.  The application of Micro-segmentation using a hardware-based approach creates a two-fold problem:

Lack of interior controls and security – If an organization simply does nothing, no use of micro-segmentation, to secure the East-West traffic within their data center, the perimeter firewall becomes the single point of entry and security to their environment from North-South.  This type of defense still provides protection, however once an attacker is able to break through that perimeter, unfettered access around the inside of the environment is rather easy.  With no lateral controls, the attack surface is enormous for attackers to work with.


Operationally infeasible and lack of scale – If an organization puts in hardware appliances or use the external firewall to facilitate Micro-segmentation for both East-West and North-South traffic firewalling, those systems become operationally difficult to manage.  Hardware appliances are costly and only scale to a point.  Multiple user interfaces and policies that don’t scale as you add new workloads or even modify existing workloads into your data center and certainly don’t provide mobility in a virtualized environment.  As you add new applications you may need to add more firewalls.  If a workload needs to move, you may need to move or change the rules associated with that application.  And what happens when that application needs to talk to another application?  All those rules need to change as well.  While this can reduce the attack surface of the application, it’s operationally infeasible to support and lacks scaling as a long term option for customers.


How does the VMware NSX Platform provide a business value around these issues? The VMware NSX Platform uses a software-defined micro-segmentation approach applied at the Virtual Machine workload level to facilitate a Zero-Trust security model.  This security is built into the vSphere hypervisor creating a distributed and scale-out firewalling architecture.  This architecture provides kernel-level performance and scales as your organization and workload requirements increase without the need to add more specific hardware appliances to the environment.


Let’s focus on Healthcare customers specifically.  A recent study by the Ponemon Institute and IBM for 2016, shows that a security breach and exposure of a patient health record is now averaging $355 per record.  The average total cost to an organization in the US was $7.01 million and the average number of records breached was around 29,000.  Traditional methods of security, like those listed above, are no longer sufficient to prevent attacks.  While there is no ‘silver bullet’ to security, Healthcare organizations can provide a layered approach to security that helps reduce their attack surface overall and reduce the potential for exposure. The VMware NSX Platform, through the use of Micro-segmentation, allows Healthcare organizations to accomplish this.

The VMware NSX Platform can provide an application-based security policy around the critical and patient information sensitive applications within the data center.  This provides us the ability to effectively control all communication paths both in and out of the application, thus reducing the attack surface of that application immensely.

The EMR/EHR system for Healthcare organizations, represents a mission-critical application for the organization and houses the majority of patient record information.  For this example, we’re going to look at a typical installation of an Electronic Medical/Health Records system, EMR/EHR, and how traffic both in and out and between the servers within the application are secured. They can be comprised of several Windows/Linux and Appliance-based systems. Below is a typical example of the layout of an EMR/EHR system server architecture.  Most consist of a client application that connects to the Application Server which has a Database Server connection where the data is stored.


Let’s take a look at traditional security approaches to East-West and North-South traffic isolation, first.  You’ll see below that for North-South traffic, the end user workstation could traverse through either a perimeter firewall or an internal data center firewall before it gets to the presentation layer of the EMR/EHR.  Also from an East-West perspective, to secure communications between the servers within the application and Shared Services, the traffic patterns will need to traverse through those same firewalls to either allow or block the communications necessary for the application to function. This creates a hair pinning effect that is operationally inefficient.


With the Physical Firewall Policy, we’re now sending all the traffic through the external firewall to do the segmentation for the applications.  This firewall could also be an internal firewall between the applications.  Nevertheless, the premise stands.  Sending all the traffic through that firewall will not scale-out as your workloads increase and this is just one application in this environment.  Most Healthcare organizations have hundreds of applications they need to secure.

With the VMware NSX Platform, we instantiate a stateful, Layer 2-4, firewall at the Virtual Machine virtual network card level, which allows us to create security policies based on the application, that can secure the application in the host itself, rather than traversing to an external firewall.  This reduces the dependency on the external and internal physical firewalls for allowing and disallowing of traffic both in and out and between the EMR/EHR system and provides a much more operationally efficiency configuration for both network and operational resources.


As you can see, the VMware NSX Platform has Security Policies created for each of the different applications, in this case the EMR App Server, the EMR DB Server, and the Infrastructure Services Servers.  Through micro-segmentation, we can setup NSX Security Policies that only allow the traffic that needs to occur within the application, to actually occur.  This enforcement is done in the hypervisor with no need to traverse to a hardware Firewall device to secure the workloads.  What we see here is:

  • The EMR Client Application initiates a connection to the EMR App Server.
  • The EMR App Server allows inbound communications to occur with the EMR Client Application and also allows communication to the EMR DB server.
  • The EMR DB Server only allows connections inbound from the EMR App Server. This functionally secures the EMR application to only allow the communications that are necessary for the application to function, and the EMR Client Application to connect to the system securely.
  • The EMR App and DB Servers are also allowing both inbound-and-outbound, communications to the Infrastructure Services servers.

Using the VMware VMware NSX Platform, Healthcare organizations can implement security at a much more granular level that provides a simple way to secure Healthcare organizations application workloads and reduce their attack surface.  Security is now implemented at the virtual machine workload level using the Application-Based Policy control.  This new model, scales as the application workload scale in the data center environment while still providing the same security posture consistently.


The VMware NSX Platform – Healthcare Series Part 2 – Environment

To get started with the series, we need to establish the environment that we’re going to use to apply the NSX platform concepts on and the applications we’re going to use.  Access to real world applications is a great way to break out of the typical test application bubble and really apply these concepts.  One of the requirements I had for this environment was that all the software had to be in use somewhere in production and the applications needed to obviously be Healthcare applications.  Getting access to a full EMR, PACS, HR, or other Healthcare-related systems would be very difficult and costly.  However, after some serious digging I found open source alternatives that are in real-world usage today.  These applications can facilitate the points of how Healthcare application integrate with each other and are representative of typical applications in a Healthcare organization.

With that in mind, using a combination of open source software, VMware products, and 3rd party partner products, we’ll be able to showcase multiple use cases and how the NSX platform can help customers move toward a Software-Defined Datacenter and drive application-based policy through the stack.

Open Source Healthcare Applications:

  • OpenMRS – EMR system
    • Apache/PHP Web Server
    • MySQL Database Server
  • DCM4CHEE – PACS (Picture and Archiving Communication System)
    • Apache/PHP Web Server
    • MySQL Database Server
  • OrangeHRM – HRHIS (Human Resource for Health Information System)
    • Apache/PHP Web Server
    • MySQL Database Server
  • Weasis – DICOM (Digital Imaging and Communications in Medicine) system
    • Runs on a Windows-based system
  • DTKv – DICOM Emulator
    • Runs on a Windows-based system
    • Used to simulate a modality for the PACS system
      • Modality – MRI, CAT Scan, etc.
  • Mirth Connect – HL7 Interface Engine
    • Apache/Java
    • MySQL Database Server
  • DHIS – Health Information Management System/Data Warehouse
  • i2b2 – Informatics and Data Warehouse

VMware Applications:

  • vSphere
  • vCenter
  • NSX
  • vRealize Log Insight
  • vRealize Network Insight
  • vRealize Operations Manager
  • vRealize Automation
  • vCloud Air – Advanced Networking Services
  • Site Recovery Manager
  • vSphere Replication

3rd Party Applications:

  • Palo Alto
    • Panorama
    • V1000MH
  • Trend Micro
  • EMC Virtual Isilon

Infrastructure Servers:

  • Active Directory
  • DNS
  • NTP

This list subject to change and will grow over time, but provides the foundational basics we need to setup a true re-creation of what a Healthcare organization would be using in their own organizations. We’ll be using the new nested system I built to provide the environment to test in.

Starting with a conceptual design we have the following for our environment:


As we continue with the series, we’ll add in a second site for discussions around Business Continuity and Disaster Recovery, Metro-Pooling, Hybrid-Cloud use cases.

The VMware NSX Platform – Healthcare Series Part 1 – Intro

At VMware, I’m part of a dedicated team of professionals that are focused solely on Healthcare organizations.  My role is around providing NSX specialist support to my customers.  If you’re in the Healthcare field, you know that Healthcare has a vastly different approach when it comes to business goals than say Financial or Retail.  Healthcare customers are focused on patient care, driving down costs, security of patient information, and how fast they can innovate their services to their patients. Healthcare also brings up interesting and different challenges from a business and use case perspective.  We believe that the NSX platform can help organization with the business goals that matter to them, and more specifically we’re going to examine how Healthcare organizations can benefit.

The NSX platform is one of the three pillars in VMware’s Software-Defined Data Center (SDDC) approach.  When extended to our Compute virtualization platform, vSphere, and our Software Defined Storage platform, VSAN, the NSX platform provides the software networking portion of the infrastructure that allows an organization to drive Application-based policy through the entire infrastructure stack.


Similar to how vSphere disrupted the hardware compute business by removing dependency on specific hardware to run application workloads, the NSX platform is doing the same for the networking hardware layer. We’re not saying the hardware layer is not important and/or irrelevant, but driving application-based policies through hardware intelligence is something that can often times require specific and even proprietary hardware to accomplish.  With a software-defined approach, customers can choose the hardware of their choice and budget, and layer on the same hardware intelligence through software products and gain speed and agility, but not at the cost of security.  This means the focus of organizations moves to the application where it should be, and less on the hardware-based infrastructure.

When looking at a Healthcare organization and asking how the NSX platform can help achieve their goals, we need to examine the NSX use cases and find out how they apply to a Healthcare organization.  The NSX platform provides several outcomes for organizations:



As you can see, the NSX platform is truly a strategic platform, and not simply a tactical solution for one business goal a customer wishes to accomplish.  The NSX platform extends into multiple areas of the infrastructure and customers are seeing tremendous benefits.  While some customers often start with a single use case, we’re seeing our customers immediately find other use cases for the product.  Over the next series of blog posts, we will be digging into each of the use cases for the NSX platform listed above and how these use cases are applying to Healthcare organizations.

New Home Lab Build

In previous iterations of my home lab, I’ve built on the premise of keeping things as close to a ‘production’ system as I could.  This meant running physical equipment and setting up scenarios with that line of thinking. This grew from two physical hosts that I bought back in 2010 and then grew into the three vSAN nodes that I added in 2014 and Cisco 3750G switches that I added in as well.  I managed to build out a dual-DC type of configuration and it worked great for quite a while.

Since joining VMware in late 2014, my lab has been satisfactory to keep my skills current.  As I made the transition to the Networking and Security Business Unit, NSBU, at VMware in January of 2016, the lab has been in need of some updating.  Recently I made a decision to build out a new home lab, this time consisting of high-end pieces and going with nesting.  I sold off my vSAN nodes, moved one of my old servers to be my wife’s new desktop, it’s still a badass for what she needs, and I got rid of the loud and power-hungry 3750G’s.  The result has been a dramatic reduction in heat and noise in my office which is where I had to put my lab as I’m not fortunate to have basement where I can this gear.

As I started going down the path of deciding what I wanted to build, I knew it had to be powerful, quiet, and as power-efficient as possible.  I wanted maximum capabilities to run multiple topologies and basically be a on-premises version of VMware’s OneCloud environment we have to demo and testing.  With my new job in the NSBU and working with the VMware NSX platform, there are so many different use cases that I wanted to be able to setup and test out that my Healthcare customers may run into.  With some back and forth with my homeDC peer, Erik Bussink, I came up with a build that does just that.

The build consists of the following:

The best thing about this was I was able to sell all the other parts of my home lab and not have to add too much more cash to build this.  I got a really good deal on the NVMe drive off eBay as those can run as much as $2500 by themselves.

The rest of my lab consists of the following:

The result has been a new high-powered and fully capable lab system that meets all my requirements.




Managing Custom Services in NSX

I was working on some labs recently and creating custom services to block ports that may otherwise not exist in the NSX Service list. NSX allows you to create custom services in Firewall rules to block certain kinds of services. I was fumbling about in the interface trying to figure out where all you could create these custom services, and also where the heck do I edit them after I’ve created them? Just a bit of background on what I’m talking about.

Services – NSX has out of the box Services that can be used when creating Security Policies and Firewall rules for common types of services that may run in most datacenters. These can be used to either, Accept, Block, or Reject, those types of service traffic with NSX. Here’s just a glimpse of the list. It’s quite extensive.


Service Groups – NSX has out of the box Service Groups, which are groupings of common services that make up certain types of applications. You can see from this list, what I’m referring to. Let’s take a look at SharePoint 2010. It has several services that could be grouped together to easily create one rule that could encompass all of the services associated with that application. This means when I want to setup a rule for that type of application service, I only have to create one rule to ‘rule’ the entire application. Beautiful!


As I was working through my lab and practicing making Security Policies and Service Groups using the Service Composer, I noticed that when I went to put in a service I wanted to block, that service wasn’t in the list. Nobody likes CIFS and I wanted to try and block it for a VM. So I figured I would just create a custom service for CIFS to block it. What I found was that CIFS wasn’t in the list. 😦


What I also found out is that you can’t create a custom service from Service Composer through the Security Policies Wizard. You can see from the shot below there’s no ‘New Service…’ link.


However there is a ‘New Service…’ link when you edit a Firewall rule as you can see here. I figured there had to be a better way to get my custom Service input without having to go through a Firewall rule to do it.


Never one to consult the documentation, because what fun is that, I went digging through the interface to find out where the custom service would show up and I found it!

The list of Services and Service Groups can be found by selecting NSX Managers>Click on the NSX Manager in the list>Manage>Grouping Objects>Service/Service Groups. From this interface I can create new services and new service groups as well as edit them. You can also edit the out of the box policies as well.

According to this link, I can see what I need to help me in terms of ports and protocols.  Now I can create my custom service for CIFS. CIFS operates on both TCP and UDP 445, so I’m going to create a custom Service Group to encompass both of those into one easy group to make my rule with. But I first have to create both of the services I want to add to the group, one for TCP and one for UDP.



I can now add both of these to a new Service Group to be used in my Security Policy.


And now I can select it from the list when I create my Security Policy and all is well. Easy stuff.


Deploying F5 Virtual Edition for vSphere

During the rebuild of my home lab, I was bound and determined to do things as close to a production deployment as possible. This includes the introduction of a load balancer into my lab. I will preface this post with ‘I have no clue how to operate a load balancer at all’. That has never stopped me from trying to accomplish something and it certainly won’t now. There were some trials and tribulations when attempting to set this up so I wanted to talk about what I experienced during my deployment.

I’m going to be using the F5 BIG-IP-LTM Virtual Edition trial load balancer. I’m going to start off by using it to load balance my Platform Services Controllers (PSC) for my vCenter deployment at my primary site. VMware was gracious enough to include how to setup high availability of the PSC’s in this document. However the part that’s lacking is how to properly deploy the F5 and get it to the point where you can actually use it for load balancing the PSC’s.   I couldn’t find a definitive source of step-by-step to deploy the F5, so I thought I’d just do it myself.


  • vwilmo.local – – Deploy to Host
  • vwilmo.local – – Primary Node
  • vwilmo.local – – Secondary Node
  • vwilmo.local – – Virtual IP address of the HA pair

All entries have forward and reverse DNS entries.

Download the BIG-IP LTM VE 11.3 from here. You’ll need to create an account and login to download the trial. Generate license keys. You should be able to generate 4 keys of 90 days each for the trial.

The file you’re looking to download is – BIGIP-

Once downloaded you simply need to run through the deployment of the OVA.

  • Open the vSphere Client and connect to one of your hosts. Since we do not have vCenter setup because we’re trying to configure HA PSC prior to installing vCenter, you’re just going to have to pick one host to deploy this on
  • Select File > Deploy OVF Template
  • Browse for the BIGIP- file you downloaded


  • Verify that the details are what they say they are. You may notice an invalid publisher certificate. This is OK


  •  Accept the EULA


  • Name the appliance


  • Select the datastore to store it on


  • Select provisioning type


  • Map networks – You have to be careful here. What happened to me was putting the Management and the Internal interfaces of the appliance on the same VM Network and VLAN. This creates an issue when you put in a Self-IP for the appliance during configuration. Select two DIFFERENT networks for both Management and Internal. The others are inconsequential to us right now. This is an internal-only load balancer and I’m not doing an HA configuration of the F5.


  •  Confirm and Finish deployment.


Now that the F5 is deployed, we’ll go ahead and boot it up and run through the initial configuration for getting into the management interface.

The default login for the appliance is ‘root’ and ‘default’


Once you’re logged in, then type the word ‘config’ to take you through setting up the Management interface


You can either input your own IP address, or let the appliance pull from DHCP.


We can now browse back to the IP address of the appliance via HTTPS. The user and password here to login is ‘admin’ and ‘admin’.


We can now go ahead and start the initial configuration of the appliance from the GUI. The first thing we need to do is Activate a license


Copy and Paste one of the license keys you received from F5 into the ‘Base Key’ field. Check the interface to make sure that it has access to the Internet to activate the key.

At this point I don’t mess around with configuring any other sections using the wizards. I go through the regular interfaces to finish it up. The next thing we need to make sure that this thing will actually load balance is to configure the VLANs, Self-IPs, and network interfaces. You do this in the ‘Network’ tab to start.

Select VLAN > VLAN List > and click on the ‘+’


Fill in the information for the ‘Internal’ network you selected alternatively to the ‘Management’ network. These should be on different networks. This was the only way I could get this to work properly. Select the 1.1 interface, as that corresponds to the ‘Internal’ NIC of the VM.


Select Self IP > and click on the ‘+’.  This is the part, coupled with having the 1.1 Internal interface on the same network as the PSC’s where I screwed up.  I never did this step.


Make sure to select the VLAN name you created when you configured the previous setting. This is the IP that the load balancer will use to direct traffic to this network.

Now that those are configured, I finish up the configuration by adding in DNS and NTP settings to ensure proper time and resolution states for the appliance.

Select System > Configuration > Device > NTP/DNS



That’s the basic configuration necessary to use the F5 for load balancing. In the next post I’ll go through how to setup the PSC in HA and use the F5 to facilitate load balancing for the deployment.

When I attempted to load balance the PSC’s with the F5 with the following perfect storm:

  • Both Management and Internal NICs on the same VLAN
  • No Self-IP for VLAN2 because you can’t add another IP on the same VLAN if it matches the VLAN the management interface is on.

As soon as I made those changes, I was able to get the PSC’s to use the F5 properly.  Hopefully my pain is your gain.  Good luck.

What a year…

Yes I know there’s still a month left in the year.  To say this year was a great year would be a complete understatement. I made some hard promises to myself this year, and I’m really amazed where things have come and where they are going.

On the professional front –

I dug into Twitter and got involved. Apparently people think what I have to say is worthy enough to listen to. I do try and use Twitter to actually purvey worthy to read information and support for the community. There are the occasional ‘what beer I’m drinking’ tweets as well. Because beer is important.

I continued to blog as much as possible. Getting involved in Twitter and being inspired by the people around me caused my blog hits to skyrocket. They’re no top 50 blog types of numbers but I’m proud that my blog has been helpful, or least it appears that way, so I’m very happy that my efforts are finding their way to someone.

I decided to toss my hat into the Virtual Design Master competition. You can read more about that over in this thread, but it was nothing short of an amazing experience overall. It was well thought out, well executed, and just plain fun. I met some really great people in the competition and was fortunate enough to meet a few of them in person as well. I keep in touch with them frequently.

I started going to my VMUG meetings and treating them like normal business meetings. Doing this I made sure that VMUGs were used to learn and network with my peers and grow. Making them as important as a business meeting, it meant that I would make them mandatory to attend.

I was elected to the vExpert 2014 group. This was my first entry to the group and I was very surprised to see that I was brought in my first go around. It was great validation that what I was doing was recognized as helpful to a community that I wanted to be a part of. I’m proud to be involved in that program. It has led to talking on the vCommunity Roundtable about my VSAN homelab which was an awesome experience.

I made PernixPro this last go-around. This is a company that is doing some amazing things and if what I think they’re ultimate goal is comes to fruition, they’re going to floor the industry. Their product speaks for itself, and it works really really well. Really great people in all the areas of their organization. I’m glad they found my contributions useful and invited me. Looking forward to see what they’ll do next.

On the personal front –

My wife and I celebrated our 8th wedding anniversary this year. If I said it was easy, I would be lying, but she has supported me and stood by through everything and that’s saying a lot. I’m a lucky guy and I can’t wait to share another 8 (and then that’s enough I think) with her. Love ya baby!

My daughter celebrated her first birthday. No one can explain how much you’ll love a child until you actually have one. There’s really nothing like it and I wouldn’t change it ever. Every day it’s just amazing to watch what happens when she learns something new or does something and makes us laugh.

On December 30th, or before that maybe, my wife will be giving birth to our new daughter. We’re both really looking forward to meeting her and my oldest is ready as well. It’s yet another milestone for our little family and we’re both really excited, although I’m probably going to have to arm myself for when they start dating.

Last but definitely not least –

On December 15th I will be joining VMware as a Senior Systems Engineer for Healthcare. The decision to leave was very tough for me. I’ve been working with two of the same guys for 10 and 5 year respectively. You get used to how people work and when you find that great collaboration, you accomplish great things and it becomes difficult to move on.  These people are my family away from my family and not just people I know or work with. But just like anyone you know that’s genuinely concerned about you as a person, they embraced my decision and they know it’s the right move and support me. You can’t ask for a better way to resign than when they truly wish you well.

As a job, this is a change for me professionally to move to the pre-sales side of the house. I have lived on the customer side for my entire career.  Having talked with my family and friends about it, I’m ready to take that next career step and this is going to be a really fun ride. VMware is a great organization and I’m extremely lucky and humbled to be invited to become part of that. I’m really looking forward to working with our products more in depth and seeing the future of the products from the inside; all while helping solve my customer’s toughest business challenges.

This has been an amazing year on all fronts. Thanks to everyone that’s been a part of the journey.  I’m a very fortunate guy and while it’s nice to reminisce on 2014, I’m really looking forward to what’s in store for 2015.