NCDA Lab Configuration Extra: Expanding the ONTAP Simulator

As promised, this is the post that will show you how to expand the current 8.1.1 ONTAP simulator.  When I did this for the first time I probably built this thing 10 times because of the need to use multiple resources to get it done.  So while I can’t take credit for all of it, I felt it made sense to put it all together so hopefully someone else could do it easier.  With some help from these posts (link1, link2), I expanded both of my simulators to 42 disks each and they are all 4GB drives.  This gives me a little over 130GB of disk space to work with.  With the SSD that I’m currently using, this allows me to run both of the shelves on the same SSD and get the most out of the room used.  Below I’ve outlined the processes that I went through to expand the simulator.  
You shouldn’t have to expand the shelves or change out the drives with the ONTAP simulator for the NCDA.  Because I actually use these simulators to present storage to nested ESXi hosts, I expanded them to accommodate nested VMs within those nested hosts.  With this disclaimer in mind, I’ve noted the area in the configuration information that shows where I deviate from a normal configuration.  But I wanted to give you the option and show you how to expand the shelves at the same time if you want to use these simulators for more than this lab environment.  Below are the steps we need to take to expand the Simulator. 
  • Add two new 14 drive 4GB shelves
  • Create new  aggregate
  • Move root volume to new aggregate
  • Rename aggregate and root volume
  • Destroy the two 14 drive 1GB shelves
  • Add one new 14 drive 4GB shelf 

Most of the content is a rehash of Julian Wood’s post mixed with going back and forth from that to the second link.  However the newer simulators you don’t have to do all the stuff he does to expand them I found when I went through the process.   This is because NetApp has allocated a 250GB IDE drive with the VM for the disks.  So while some of the content may be the same, you’ll see where we deviate from each other’s configurations.  Again, all credit to link authors for the content.  I simply modified it a bit to accommodate my setup.  We’ll be skipping over the first half of his post because this is no longer required.  I’ll be using my ZOC SSH client to perform all the tasks.

Boot up the ONTAP VM and login 
Run the command – disk show – this shows all the disks in the VM
You’ll notice that no all the disks are assigned.  You can run the command ‘disk show –n’ to show the details of the unassigned disks.  Don’t assign them yet, we’re going to destroy this shelf so we can add 3 4GB shelves. 
Next we’re going to unlock the diag user account and create the disks on the VM
Now we need to launch the systemshell to add the shelves
We need to add the path to the simulator disk tools and then go to the simulated devices directory and take a look
Next we’re going to add two new 4GB shelves.  We can run the command below and see all the sizes of disks that the Simulator will allow us to create.  We can create up to 9GB drives in each shelf.  Since I don’t have that much room, I’m going with 4GB drives.   
We’re looking for the following disk type:
31  NETAPP__  VD-4000MB-FZ-520  4,194,304,000  4,215,799,808   Yes  520  15000
The reason is we want the 15K drives and not the ATA ones.  Other than how they appear, there aren’t any function differences that I’m aware of. 
Now that we know what drives we’re going to use, we can create the shelves
After the disks are created we can run the ls ,disks/ command to show that our new disks are now there. 
We’re ready to exit diag mode and reboot the filer so the new disks show up


Once the filer is back online, we’re going to login and then assign all the disks to the controller.  Then we’re going to create a new aggregate out of the 4GB drives and a new root volume to copy the current one to.  (I cut out a lot of the repetitious outputs)
To be able to get rid of the two 1GB shelves, we need to get the root volume off of vol0 which resides on the 1GB shelves aggregate.  We can ndmpcopy the contents to the new 850MB volume we just created and then set the root volume to the new vol1 on the larger aggregate. 
Per Julian’s post, something becomes corrupted with SSL certificates so you’ll need to re-setup SSL on the device.  We’re also going to turn off spare alerting as it’s an annoyance that continually pops up in the console. 
Since we’ve copied the root volume over to vol1 and set it as the root, we can now destroy aggr0 and vol0 so that we can remove the two 1GB shelves as well as renaming aggr1 and vol1 back to what they should be named. 
Now that we’re done, we can permanently remove the first two shelves and add our third shelf of 4GB


All done, reboot and assign all and we now have a simulator with 3 functional 4GB shelves. 
I now have a simulator that has plenty of space and I can play with multiple technologies and even present this storage to an ESXi host.  I’ll do another post showing that later and what other cool stuff we can setup for a lab to test. 

NCDA Lab Configuration Part 3: ONTAP Simulator VMs

As I previously discussed, to download the ONTAP Simulator you’ll need to have a NOW account.  Here’s hoping that they’ll open this up for all so that other people can take a look at the ONTAP software and try it out. It comes in both 7-mode and Cluster mode versions.  The ONTAP Simulator that we’ll be using is the ONTAP 8.1.1 7-mode simulator.  Each simulator comes with 28 drives in 2, 14 drive shelves with each drive equal to 1GB each with the ability to expand to 56 drives. 

I’m not going to go through how to download and add the simulator to Workstation.  That should be pretty straightforward.  Let’s list out the changes that need to take place to setup the ONTAP simulator.  
  • Boot up the ONTAP Simulator and login to the device
  • The ONTAP Simulator has one very clear flaw.  The System ID is always the same.  This comes into play when you want to use replication technologies like SnapMirror.  SnapMirror will not work if both of the Simulators have the same System ID.  So we’ll have to change it when we build the second simulator.  I just took the first simulators System ID and incremented it by 1.  Note:  you only need to perform this step on the second Simulator. 
  • Zero the current disks and configure the system
  • Verify connectivity and check disks
  • Configure appropriate network connectivity for lab
Simulator Configuration
Before starting the VM, change NIC1 and NIC2 to use the same VM network (most likely for VIF configurations in later labs)
  • Boot VM
  • When prompted to access the Boot menu press CTRL+C
  • Press 4 to Zeroize and clean drives (the link suggests that you need to assign ownership.  What I’ve found is that that is the case if you’re not setting up the simulator from scratch.  If you have an existing simulator that is already configured and usable, you need to perform the disk assign –o <simulator name> commands. 
  • Confirm with y
  • Confirm with y
  • System will reboot
Allow the wipe and disk assign process to take place.  When the system finishes zeroizing the disks and assigning ownership, it will then move into configuration mode. 
  • Enter the hostname of the simulator (I named mine ONTAP1/2)
  • Take default on IPv6 of no
  • Take default on interface groups of no
  • Enter the IP address of the e0a interface of
  • Enter the Subnet Mask of
  • Take default on media type of auto
  • Take default on flow control of full
  • Take default on jumbo frames of no
  • Enter the IP address of the e0b interface of
  • Enter the Subnet Mask of
  • Take default on media type of auto
  • Take default on flow control of full
  • Take default on jumbo frames of no
  • Don’t enter an IP for e0c
  • Don’t enter an IP for e0d
  • Take the default on web interface setup of no
  • Enter the IP address of the default gateway of router’s IP address)
  • Don’t enter the IP address for an administrative host
  • Enter the hours of the timezone from GMT of -6 (Chicago)
  • Enter the name of where the filer is located of Chicago
  • Take the default for root directory of /vol/vol0/home/http
  • Take the default of DNS resolver of no
  • Take the default of NIS client of no
  • Press Enter
  • Take the default for ACP of no
  • Allow VM to finish configuration
Once I’ve established an IP address that I can connect to on the VM, I switch from the VMware console window over to an SSH client.  It provides more screen room to work with when running commands.  
The second shelf of disks will be unassigned.  At this point there’s no real need to assign them to the aggregate that’s currently there.  When we get to disk assignment during labbing, this will come in handy as good practice to expand an aggregate from both the CLI and from OnCommand System Manager.  To configure the second VM, simply perform the same steps above and use IP addresses similar to the ones above but on the 10.10.10.x network.  You’ll need to make sure to follow the steps below for changing the System ID as well.  
System ID Change (Only necessary on the second simulator)
After you’ve added the ONTAP Simulator to VMware Workstation, boot the machine.  When the VM boots, you’ll be presented with a dialogue to boot or any key for a command prompt. Hit CTRL+C so we can boot into the loader and change the System ID.   Again, you only need to perform these steps on the second simulator.  The first simulator will always be System ID 4061490311.  Following this link, we change the System ID.  
  • Run the command – set SYS_SERIAL_NUM=4061490312
  • Run the command – set bootarg.nvram.sysid=4061490312
  • Run the command – boot
  • Proceed with the same steps as above for configuration

NCDA Lab Configuration Part 2: Network Configuration

As I continue on building out the NCDA lab, we need to start with our network configurations.  Following the same design that I originally posted, we’re going to start with building the three networks in VMware Workstation.  When you build a new network in Workstation, it automatically configured the IP addressing.  I went with whatever it came up with and only made a change on the 10.10.10.x network to make it completely disparate from any of the other networks.  I could have done it with another 192.168.x.x network but I wanted it to be visually different.

VMware Networks:
  • 192.168.175.x/24 – Site1
  • 192.168.112.x/24 – WAN
  • 10.10.10.x/24 – Site2
Vyatta Configuration:
The Vyatta configuration is pretty straightforward.  I’m using Vyatta Core 6.2 routers.  The Vyatta download is a live CD.  When you build the VM in which you’ll install the software onto, ensure that you add another NIC to the device.  Put each nice on the appropriate Workstation network that you configured above.  Example for Site1:  nic1 – 192.168.175.x network, nic2 – 1921.68.112.x network.  This will allow you to have connectivity to the WAN and Site1.  To install the image perform the following:
  • Boot the VM with the Vyatta Live CD
  • Press Enter when instructed to Boot
  • Log into the image
    • Username – vyatta
    • Password – vyatta
  • Run the command – install-system
  • Take the defaults
  • Reboot the router when complete (Be sure to disconnect the CDROM from the VM before you reboot or it’ll boot back into the Live CD)
Now that the install is complete we need to configure the IP addresses on the eth interfaces as well as setup RIP.  Vyatta is very similar to a Juniper router if you’ve configured one from a CLI perspective.  We’re going to configure eth0 as the local network, and eth1 as the WAN network.  Once the device boots back up perform the following:

Log into the router

Verify that the interfaces are online

NCDA Lab Configuration Part 2: Network Configuration

Set the IP address on eth0 and eth1

NCDA Lab Configuration Part 2: Network Configuration

Configure the RIP protocol and commit the changes to the router

NCDA Lab Configuration Part 2: Network Configuration

Repeat the previous process on the second router with the IP addressing for Site2:  Eth0 – 10.10.10.x/24 and Eth1 – 192.168.112.x/24

Confirm that RIP is running on each device


NCDA Lab Configuration Part 2: Network Configuration


NCDA Lab Configuration Part 2: Network Configuration
Now that we have established the networks in Workstation and built the Layer 3 connectivity, we’re ready to move onto building the VMs for the Lab.

NCDA Lab Configuration Part 1: The Design

The first section of the NCDA preparation involves configuring an appropriate lab for practicing.  Luckily NetApp provides us with a Lab template that helps us accomplish this task.  Looking over the ANCDA Lab Setup Guide Template, I built the following topology based on that guide in VMware Workstation.  I like making lab setups as close as I can to a possible production scenario so I took some liberties and you don’t have to do all of this.  I put in a couple Vyatta routers running the RIP routing protocol so that I could simulate a WAN connection between two disparate networks.  Vyatta is open source so you have OSPF, BGP and RIP routing protocols.  I went with RIP because it’s simple to configure and gets me what I need from a Layer 3 perspective. 
Lab Scenario:
 NCDA Lab Configuration Part 1: The Design
This configuration mimics the lab template very closely.  This will enable us to configure NetApp replication technologies in scenarios that could be as close to real world as possible.  As we continue forward I will go into the individual component configurations.  This includes the Vyatta router configuration and connecting them via RIP.  Then I will go into how to configure the ONTAP Simulators and how to expand their storage capabilities.  
Disclaimer:  It is worth noting that I am running the entire lab on a Crucial M4 512GB SSD.  To ensure that I have the proper performance of the lab environment, given that I’m going to be running two ONTAP simulators at times, this was essential.  I wouldn’t attempt to run it on a normal spinning drive.  When I get into how to expand the ONTAP simulators to allow more space on them, you don’t have to go with as large of drives as I did if you have a smaller SSD.  

New blog and new focus

I’ve tried doing this blog thing a few times.  I’d catch fire and put out stuff fairly quickly.  I’d suddenly change focus or find I didn’t have time to put in the effort that it took to keep a technology blog up and running with any currency.  Clearly I’m going to try and reverse that course and try and keep up. My current responsibilities include engineering VMware and NetApp solutions for our organization.  Given that I’m already certified VCP5, I’m shifting focus on fully learning all the technologies associated with NetApp.  With that in mind, I have decided to head down the certification path to help structure my learning of our NetApp purchases and all that they’re capable of.  The best structure that I’ve found for most technologies is to look at their certification path.  Certifications can be great at helping provide a blueprint of the best way to learn and apply technologies. 
NetApp has a few certificationsfor their products.  They range from Professional level to Specialist. 
  New blog and new focus
What this means to me is to look at the NetApp Certified Data Management Administrator, NCDA, NS0-154certification.  The exam covers several areas and they can each get very deep.  There is a study guide for the NS0-154 exam located here.  I plan on using it to help as my blueprint to structure the learning process on each topic.  Depending on workload and how well I can learn the material on my own, I may consider taking the NCDA training.  Once you purchase a NetApp product you’re provided quite a bit of free training materials on their web-based learning center.   You have to have a NOW account to access the site. 
Perhaps the biggest learning tool that I will be taking advantage of, and one that I’ve already been using quite a bit, is the ONTAP Simulator.   The simulator requires a NOW login as well and is only available to customers and partners.   This simulator allows you to run a virtual storage device that comes with a shelf of disk, the ONTAP software and nearly all the abilities of an actual piece of hardware with the exception of NVRAM and Fibre Channel.  I’ll be doing another post that goes into my lab configuration as well as getting the full benefit from the ONTAP simulator should you have access to it.