NCDA Study Part 1 – Data ONTAP: Overview

I know it’s been some time since I started, but some work items got in the way.  But I am resolute in my stand to keep posting so here is the next installment.  

Data ONTAP Software is composed of many different pieces.  This represents the technologies it encompasses.  This will be a significant post in terms of content and length.  It’s a compilation from numerous sources.  Also as I previously stated in the breakdown of the blueprint post, always refer back to that page.  I will be updating my PDF of the breakdown as a go through each section.  It should be treated as a living document.  You can get that document from here.  

Overview – Data ONTAP is the operating system that runs the NetApp appliance.  It’s a FreeBSD-based operating system that is capable of supporting both block and file based storage, multiple protocols, as well as many high-end applications like Advanced Single-Instance Storage (Deduplication) capabilities, high-availability, zero overhead RAID-DP as well as non-disruptive upgrade capabilities.    The Data ONTAP operating system can be managed via the CLI or by System Manager.   
WAFL – Write Anywhere File Layout – this is part of the software that runs on the Data ONTAP operating system.  The WAFL software is unique because when a write is performed, it’s done by using the next free block on the disk.  This helps eliminate disk seek times to place the write next to the other writes it’s associated with. 
Data ONTAP comes in two flavors:
7-Mode – Was built to extend the capabilities from the 7 series platform to a 64bit system.  Data ONTAP 8.0 and higher supports 64bit architectures and aggregate capabilities.  This allows the storage system to support larger size aggregates.  
Cluster-Mode – Was built to bring non-disruptive upgrades and the highest availability NetApp can provide to the storage system.  Cluster mode allows the storage controllers to work together to provide un-interrupted access to the storage system and data. 
We’ll be focusing on 7-Mode for Data ONTAP.  There are also two different versions of the NCDA, each covering 7-Mode or Cluster-Mode. 
Architecture – Data ONTAP is composed of several different underlying components.  Below we’ll take a look at each of those components and their associated components.  
Aggregate – An aggregate is the combined logical representation of the underlying physical disks.  It is composed of a plex which is made up of RAID groups.  It can contain 1 or more RAID Group and by doing so can take advantage of being able to abstract the underlying storage and gain the capabilities of having more disks to provide more performance for the aggregate. 
As of Data ONTAP 8.0, an aggregate can be either 32 or 64bit.  
32-bit aggregates– this is the only type that can be created on versions of Data ONTAP lower than 8.0.  They can grow only to a size of 16TB.  These types of aggregates can still be created on versions of Data ONTAP greater than 8.0 if necessary.
64-bit aggregates– this type of aggregate can only be created with versions of Data ONTAP 8.0 and greater.  They support sizes of up to 100TB based on the controller model being used.  
Unmirrored – All aggregates start as unmirrored aggregates.  They are singular copies of data and contain only one plex.  
Mirrored – The use of SyncMirror can create mirrored aggregates.  This is another copy of the aggregate on its own physical disks.  An aggregate that is being mirrored can contain one or more underlying RAID Groups as well.  Mirroring an aggregate is typically done to completely ensure that aggregate corruption or destruction doesn’t result in data loss.   You can tell that an aggregate is a mirror by looking at the plex number.  The initial aggregate should be plex0, and the copy is plex1.  
Aggregate Status– The Data ONTAP 8.0 7-Mode Storage Management Guide, page 122-123, has a great chart that shows all the status and state messages one might encounter. 
Read and Write access to volumes hosted on this aggregate are allowed
Some operations, such as parity reconstruction, are allowed, but data access is not allowed
No access to the aggregate is allowed
This aggregate is a 32-bit aggregate
This aggregate is a 64-bit aggregate
This aggregate is capable of containing FlexVol volumes
The aggregate is currently the target aggregate of an active copy operation
The aggregate contains at least one RAID Group with a single disk failure that is not being reconstructed
double degraded
The aggregate contains at least one RAID group with a double disk failure that is not being reconstructed (RAID-DP aggregates only)
Disks that the aggregate contains were moved to the current storage system from another storage system
Disks are in the process of being added to the aggregate
The aggregate is being initialized
The aggregate contains no volumes and none can be added
A WAFL consistency check is being performed on the aggregate
mirror degraded
The aggregate is mirrored and one of its plexes is offline or resynchronizing
The aggregate is mirrored
needs check
WAFL consistency check needs to be performed on the aggregate
The aggregate is unmirrored and all of its RAID Groups are functional
The aggregate is mirrored and needs to be resynchronized
At least one disk was found for the aggregate, but two or more disks are missing
The aggregate consists of RAID0 RAID groups
The aggregate consists of RAID 4 RAID groups
The aggregate consists of RAID-DP RAID groups
At least one RAID group in the aggregate is being reconstructed
Aggregate reallocation or file allocation with the –p option has been started on the aggregate.  Read performance on volumes in the aggregate might be degraded
One of the mirrored aggregate’s plexes is being resynchronized
The aggregate is a SnapMirror replica of another aggregate
The aggregate is a traditional volume and cannot contain FlexVol volumes
A mirror verification operation is currently running on the aggregate
wafl inconsistent
The aggregate has been marked corrupted; contact technical support
Plex – Each aggregate is composed of one plex called plex0.  They are built with RAID groups.  Typically there are one plex per aggregate, when SyncMirror is used, mirrored aggregates, then the second copy of the aggregate has plex1.  
RAID Group – RAID groups are composed of physical disks in a specific RAID configuration.  They can be RAID0 (V Series systems only), combined RAID 4/DP with RAID1 for SyncMirror but are typically RAID 4 or RAID-DP.  RAID groups can be certain sizes depending upon the disk type.  
RAID 4 – A standard RAID configuration that consists of a dedicated parity disk in the RAID set.  It uses horizontal parity to calculate the parity values stored on the parity drive.  
RAID-DP – This is a NetApp proprietary RAID configuration.  It is based on RAID 6 which is a double-parity style configuration.  NetApp combines RAID 6 protection with WAFL to eliminate the performance bottlenecks associated with RAID 6.  The DP in RAID-DP stands for diagonal parity.  RAID-DP uses both RAID 4 horizontal and diagonal parity to rebuild up to two failed drives.  
Obviously the more disks in the RAID group the more benefit you’ll get for spindle counts and speed from the RAID group.  The downside to RAID groups is that each one requires a parity drive.  Depending upon RAID 4 or DP, you could lose as many as 2 drives for parity.  What this means is that when you layout your RAID groups, you want to put as many drives into the group as possible while balancing the number of disks in each RAID group.  For example, if you have two shelves of 24 SAS disks and you’re going to use RAID-DP, you’ll want to create two RAID groups of 24 disks each so that you keep the groups balanced at the same number of disks and you lose the smallest amount of disks for parity.  This would net you a loss of 4 drives for parity in each group.  
Special note:  I didn’t factor in the loss of one other drive for a hot spare, but you’d actually have a group of 23 and a group of 24.  
Here is a list of the disk and RAID types and how many disks can be associated with each.  Each RAID group is fully configurable and capable of supporting fewer disks than the numbers below.  They key again is evening the amount in each group that will compose the aggregate. 
Disk Type
Number of Disks
26 disks plus 2 parity = 28
26 disks plus 2 parity = 28
13 disks plus 1 parity =14
6 disks plus 1 parity = 7
Disks – Data ONTAP supports a wide variety of disk types for use in NetApp arrays.  It supports FC, ATA, SATA, SAS and SSD disk types.  These disks come in speeds that range from 5.4K RPM – 15K RPM.
Disks are assigned ownership within the Data ONTAP OS to a controller.  That essentially presents the disks to the controller to be provisioned.  From that controller, new RAID groups are built and those RAID groups are then used to build new aggregates.  By default ONTAP 
Data ONTAP has the ability to use protective features within the shelves it manages via a private network that attaches to each shelf called the Alternate Control Path, or ACP.  This is a service that monitors the disk shelves.  This is done through a private Ethernet network that the shelves communicate across.  
Data ONTAP has the built-in ability to sanitize disks within its arrays.  This process allows an administrator to ‘clean’ the disks by writing irrelevant bit sets there-by destroying the underlying data and making recovery impossible.  
There are four types of RAID disks that exist in a NetApp array.  They are data, parity, double-parity and spares.  NetApp arrays can handle an assortment of disk types and as a general rule it’s not a good practice to mix drives in an aggregate.  Aggregates are typically composed of similar disks.  This is not to say that NetApp arrays don’t support mixing of drives however.  
Data Disks – These are the disks that actually contain the data within the aggregate.  Pretty self-explanatory
Parity Disks – These are the disks that are solely used as the disk that contains the parity data for the RAID group.  In RAID 4 configurations, there is 1 of these disks in the group. 
Double-Parity Disks – These disks are used to store the double-parity information from within a RAID-DP RAID group.  In a RAID-DP configuration, there is 1 parity disk and 1 double-parity disk. 
LUN – Stands for Logical Unit Number.  The LUN in a traditional storage sense is a SCSI logical carving of the underlying storage that is presented via iSCSI or Fibre Channel protocols.  When presented, they are a unit of storage in which to put data in.  In Data ONTAP, a LUN is created within a volume or qtree.  LUNs should never exist within the root volume (/vol/vol0).  They should always reside within their own volume and no other files should exist in the volume with the LUN.  LUNs are then assigned to an igroup (interface group) which are used to specify which hosts will be allowed access to the LUN.  
It should also be noted that LUNs have geometry size restrictions.  A LUN can never be more than 10x larger than it was originally created.  This means that if you create a LUN that is 5GB, it can only be grown to 50GB and no larger.  Keep this in mind when creating LUNs that may need to expand later.  This is because that a LUN is at its core, a SCSI disk representation. 
Volume – Data ONTAP supports different types of volumes, traditional, Flexible volumes and FlexCache volumes.  Data ONTAP also has a root volume, which is a special volume dedicated to the operating system.  Volumes can be presented to various types of protocols including, NFS, CIFS, FC, iSCSI, FTP and HTTP.  Volumes are tied to their underlying aggregates.  They exist much in the same way that aggregates do, they are either 32-bit or 64-bit and that bit value is determined by whether the underlying aggregate is 32-bit or 64-bit.  
32-bit volumes – these volumes can only be 16TB in size
64-bit volumes – these volumes can grow to 100TB in size depending on the underlying hardware
When a volume is either 32-bit or 64-bit there are interoperability concerns with certain features of Data ONTAP.  These features require interaction between aggregates or volumes that can be either 32-bit or 64-bit in nature.  Below is a table from the Data ONTAP 8.0 7-Mode Storage Management Guide, page 149, which shows how those features interact with the various bit flavors of volumes:
Data ONTAP feature
Interoperates between 32-bit and 64-bit
Qtree SnapMirror
Synchronous SnapMirror
vol copy
Volume SnapMirror
Volume attributes – Volumes have the following types of attributes
  • Name of the volume
  • Size of the volume
  • Security style – NTFS, UNIX, both
  • CIFS oplocks
  • Language of the volume – example English (US) = en_us as the language.  It is advised to not change the language of a volume if possible.  It can result in file access inability.
  • Space guarantee
  • Quotas (optional)
  • Snapshot scheduling (optional)
  • Is it a root volume?
Just like aggregates, volumes have specific state and status data that can be seen from within the Data ONTAP OS.  This is the chart from the Data ONTAP 8.0 7-Mode Storage Management Guide, pages 152-154.
Read and Write access to this volume is allowed
Some operations, such as parity reconstruction, are allowed, but data access is not allowed
No access to the volume is allowed
access denied
The origin system is not allowing access.  (FlexCache volumes only.)
active redirect
The volumes’s containing aggregate is undergoing reallocation. 
The caching system is trying to connect to the origin system.  (FlexCache volumes only.)
The volume is currently the target aggregate of an active copy operation
The volume contains at least one RAID Group with a single disk failure that is not being reconstructed
double degraded
The volume contains at least one RAID group with a double disk failure that is not being reconstructed (RAID-DP aggregates only)
The volume is a FlexVol volume
The volume is a FlexCache volume
Disks used by the volume’s containing aggregate were moved to the current storage system from another storage system
Disks are in the process of being added to the volume’s containing aggregate
The volume’s containing aggregate is being initialized
The volume does not contain a valid file system
A WAFL consistency check is being performed on the volume’s containing aggregate
Lang mismatch
The language setting of the origin volume was changed since the caching volume was created.  (FlexCache volumes only.)
mirror degraded
The volume’s containing aggregate is mirrored and one of its plexes is offline or resynchronizing
The volume’s containing aggregate is mirrored
needs check
WAFL consistency check needs to be performed on the volume’s containing aggregate
The volume’s containing aggregate is mirrored and needs to be resynchronized
At least one disk was found for the volume’s containing aggregate, but two or more disks are missing
The volume’s containing aggregate consists of RAID0 RAID groups
The volume’s containing aggregate consists of RAID 4 RAID groups
The volume’s containing aggregate consists of RAID-DP RAID groups
At least one RAID group in the volume’s containing aggregate is being reconstructed
The volume’s containing aggregate is undergoing aggregate reallocation or file allocation with the –p option has been started on the aggregate.  Read performance on volumes in the aggregate might be degraded
rem vol changed
The origin volume was deleted and re-created with the same name.  Re-create the FlexCache volume to reenable the FlexCache relationship. 
Rem vol unavail
The origin volume is offline or has been deleted
remote nvram err
The origin system is experiencing problems with its NVRAM
One of the volume’s containing mirrored aggregate’s plexes is being resynchronized
The volume is in a SnapMirror relationship with another volume
The volume is a traditional volume
A mirror verification operation is currently running on the volume’s containing aggregate
wafl inconsistent
The volume or its containing aggregate has been marked corrupted; contact technical support
Volumes have security styles for the files that are contained within them.  The types of security are UNIX, NTFS and mixed (both NTFS and UNIX).  New volumes created always take the default security of UNIX unless the volume is shared via CIFS and NFS together.  If a volume contains both UNIX data and NTFS data and a UNIX file is edited by a Windows client, the cifs.preserve_unix_security command needs to be used to ensure that the file permissions are not lost on the UNIX side of security.  
Root volume – While not a volume that you can have more than one of, the root volume is the most important volume to Data ONTAP.  It is the volume in which Data ONTAP resides, and it also contains other key configuration files for the storage system.  It is the /vol/vol0 volume on a NetApp array.  
Each controller has a root volume since each controller has a separate installation of Data ONTAP.  New volumes should never be created within the root volume.  Just like a UNIX type of file structure, a new file system branch should be used for each new volume.  Root volumes can reside on either RAID 4 or RAID-DP; however non-disruptive upgrades such as disk firmware upgrades cannot take place on root volumes that reside on RAID 4. 
In larger setups, the root volume can live on either the overall aggregate or by creating a separate aggregate composed of separate disks for the root volume to live on.  This helps segregate the root volume from the rest of the application data residing on the other aggregate.  This obviously means losing three disks to a RAID-DP aggregate composed only of the root volume.  In smaller deployments with minimal disks, this may not be a great option.  So it is possible to host the root volume on the larger aggregate using a FlexVol to contain it.  
Traditional volume – Traditional volumes are completely attached to their underlying aggregate and can only be 32-bit.  Since it can only be 32-bit, it can only be a maximum size of 16TB.  You cannot have more than one traditional volume in a single aggregate.  A traditional volume can also only be grown by adding more disks to the aggregate, and can never be shrunk.  
Flexible volume (FlexVol) – The flexible volume or FlexVol, is the ‘traditional’ type of volume that is created on a NetApp array.  This is because more than one FlexVol can exist on a single aggregate.  FlexVol’s are capable of being expanded and contracted at will.  FlexVol’s have several features that can be used to make sure that they are not overcommitted, grow with data growth and provide space savings.
FlexCache volume – These types of volumes are used to help speed up access to either local volumes with heavy traffic, or for faster access to data on a remove volume.  The FlexCache volume is best utilized with data that doesn’t change much.  Data that is read often is then put on a very fast volume which improves read performance.  In terms of remote data, the FlexCache volume only needs to obtain the date one time, and then subsequent requests for that same remote data are then pulled from the FlexCache volume.
Qtrees – Qtrees are the lowest level of granularity that can be achieved within the Data ONTAP operating system.  They are used to partition data sets.  They can have their own security, their own backup and restore strategies as well.  With the added benefit of having its own security, backup and restore methods, you can handle different data types within the same volume. 
RBAC – Role based access control is how Data ONTAP executes security with the operating system.  RBAC is divided up into different categories:
User/Domain User – Users are local to the storage system and are created, modified and deleted directly from the storage device.  A domain user is a user that comes from a Windows domain (requires CIFS be enabled).  Users do not have capabilities assigned directly to them.  They must be part of a group.  
Group – A group is simply a collection of users whether local or domain, that are situated together for a common purpose.  Roles are granted to groups.  
Role – A role is a collection of capabilities that are granted to groups on a storage system that allow the users within to perform functions on the filer.  
Capabilities – Capabilities are the privileges that are granted to a role to be performed on the filer.  Capabilities are not assigned to users; they are assigned to a role which is then assigned to a group. 
Networking – Data ONTAP supports multiple different network configurations.  They include static and multimode and multilevel interface groups with LACP, Jumbo Frames, Flow control, routing and VLAN membership capabilities.  Data ONTAP supports 1Gb and 10Gb Ethernet technologies depending upon model.  Array controllers can also contain e0M management network interfaces which when used allow access to the Baseboard Management Controller (BMC) and the Remote LAN Module (RLM) for out-of-band access.  The RLM is also referred to as the Service Processor.  
Interfaces – Typical Netapp controllers include 10/100/1000 Ethernet networking connections.  These interfaces are named based on their slot number within the chassis, i.e. e0a, e1b, etc.  Data ONTAP can support up to 1024 network interfaces depending on the model of array and the amount of system memory in the system.  
The e0M interface is a special interface for NetApp controllers.  Inside the management LAN NIC is an Ethernet switch that connects both the e0M and the RLM interfaces.  This allows for the separation of management and data traffic if the networking environment requires it.  Both the e0M interface and the RLM have two different IP addresses for connectivity.   The e0M interface is the typical interface for performing CLI functions within Data ONTAP.  
To connect, use SSH (it will not accept any other protocol) to the RLM or Service Processor which allows an administrator to remotely watch the system boot, get core dumps, manage the power status of the controller and view storage events from within the RLM logs.  The RLM doesn’t use the normal root account for access.  It uses a narootaccount.  The naroot password is the same as the root password.  
Interface Groups – Interface groups are simply a logical grouping of interfaces.  The main reasons behind using interface groups is for better throughput, fault tolerance and removing single points of failure.  Depending upon vendor, interface groups create link aggregates on the NetApp side.  They’re also called Etherchannels, Trunks, Portchannels, etc.  
There are different types of interface groups and each has different characteristics.  
Single-mode – The most basic interface group, this is because the interfaces do not require any special switches to connect to that support link aggregation.  One interface is online at a time and the other is in standby.  Upon failure of the active link, the standby takes over.  The array picks the interface that is active and which one is standby.  
Static Multimode – Better known as static etherchannel or 802.3ad static.  The static multimode connection does not support dynamic protocols such as PAgP or LACP.  All interfaces are active and share a single MAC address.  Since all interfaces are active, static multimode requires a switch that supports that type of configuration with multiple switch ports bonded together to form a single logical port.  There are certain types of switches, namely Cisco Catalyst switches with stacking technologies and Cisco Nexus switches with Virtual Port Channel, vPC,  technologies that allow the ability to span the physical interface links across multiple switching chassis to provide further redundancy.  
Dynamic Multimode – Better known as dynamic etherchannel or 802.3ad dynamic.  Data ONTAP support Link Aggregation Control Protocol, LACP, as its dynamic bonding method.  Dynamic multimode interface groups have all the best qualities, fault tolerance, flow control, jumbo frames, and link status detection.  Just as static multimode, dynamic supports cross chassis switching technologies such as vPC and Cisco stacking capabilities. 
Second-level – Second-level interface group configuration is the result of taking two interface groups and grouping them a second time together.  Only single-mode configurations are accepted.  This allows you to create a standby multimode group to take over in case the primary fails.   
Jumbo Frames – Jumbo frames are larger than normal Ethernet frames.  The theory behind them is that it takes less processing power to process large data flows when the default frame size is 1518 bytes and a Jumbo Frame is 9000 bytes.  Example if you had a piece of data that was 9000 bytes, it would take one Jumbo Frame to process the data, but it would take just over 6 frames in normal size.  Obviously this depends quite a bit on the size of the data being moved across the network.  If you have data that is small say 800 bytes, Jumbo frames might hurt your performance as the frames are very large and it would always send a 9000 byte frame.  Jumbo frames require interfaces over 1Gbe and are typically used in 10Gbe configurations.  
Flow Control – Flow control is quite simply the ability to control the rate at which data is sent or received by a device.  If one device sends too much data to be processed by a slower device, that slower device can get overrun by the faster device and result in dropped packets and increased retransmits.  Flow control is set to help temper this type of behavior.  
VLAN – VLANs in Data ONTAP are no different than with any other device.  They are a logical separating of broadcast domains.  Creating VLANs with the network interfaces allows segmentation of specific traffic types.  Typical configurations include segregating CIFS, NFS and iSCSI traffic types from each other.  By using VLANs, it cuts down on excessive broadcasts that may need to take place on that network.  It eases administrative needs by allowing you to focus on traffic types rather than the network as a whole if there are performance issues.  Switches must support 801.Q VLAN tagging to take advantage of VLANs within Data ONTAP.  
Routing – Data ONTAP has the ability to route packets, but only its own.  It does this using a couple of different methods.  
Fast path – This technology examines TCP packets and forwards responses back across the same interface that they came across.  This is used to avoid routing look ups which reduce CPU and resources usage on the controller.  Data ONTAP can automatically disable the service if it feels that fast path is not providing that functionality.
Routing Table – A table that contains current routing information or past information about where routes have been established.  
Routed daemon – This is the daemon that performs the routing functions on the NetApp controller.  By default when it’s on will listen to RIP packets.  If you perform vulnerability scanning that involves sending malformed RIP packets to a NetApp controller, you can kernel panic it.  I have seen this happen first hand.  
Deduplication – Deduplication is done at the volume level on a NetApp filer.  It’s done by comparing 4KB blocks from the data and putting a fingerprint on each block.  The filer then compares the fingerprints and any ones that it finds that match, one of the blocks is kept and a pointer is made for the duplicate block and the duplicate block is discarded.  Deduplication is a post-process effort.  This means that data is written to the disks and then deduplication happens on a schedule.  Deduplication consumes considerable system resources so it’s recommended to be performed off-hours.  There are several considerations to take into account when using certain NetApp products and how they interact with deduplication.  
Snapshots – Snapshots can lock metadata within them if they are run before Deduplication takes place.  The recommended setting is to run Deduplication and then run the Snapshot.  This cuts down on the amount of space consumed.  
Volume SnapMirror – When replicating volumes with SnapMirror, SnapMirror is aware of the Deduplication and only moves the deduplicated data.  Since SnapMirror also will only replicate changed data, when combined with deduplication, only the changed and deduplicated blocks will be moved.  This results in network resource savings.  
Qtree SnapMirror – Qtree SnapMirror does not automatically initiate a deduplication process when the transfer is complete.  This has to be setup independently from the transfer.  Deduplication should be run on the destination after a transfer.  Qtree SnapMirror sees all deduplicated blocks on the primary volume as changed blocks, and will replicate them.  This could result in longer transfer times.  Qtree SnapMirror should only keep the latest snapshot copies when coupled with deduplication.  
SnapVault – SnapVault takes full advantage of deduplication by providing that space saving prior to sending the data.  However, deduplication on a SnapVault destination is not controllable.  The process is automatically carried out once a transfer is complete. 
Synchronous SnapMirror – Deduplication is not supported if running this type of replication.  
SnapRestore – Deduplication retains the space savings on a restore and any new data will be deduplicated.  However, only new data will be deduplicated.  Sis start –s should be run on the volume to deduplicate the new data with the existing data.  This is because the metadata for deduplication is stored in the aggregate.  SnapRestore doesn’t restore metadata, therefore the volume has no idea about the fingerprints of the old data, but does still deduplicate the new data as it creates new fingerprint metadata and adds it to the aggregate.  Running a full deduplication on the entire restored volume will recreate the fingerprint metadata and space savings will be realized at the full volume level.  
MetroCluster – Deduplication is supported on stretched clusters.  
Data Fabric Manager – Deduplication is handled in all three areas of the Data Fabric Manager application:  Protection Manager, Provisioning Manager and Operations Manager.  This ranges from scheduled, on-demand and automated deduplication methods.  Only the ways in which deduplication is configured differ with each.   Deduplication does not behave any differently than normal constraints.  
Volume copy – Volume copy behaves much the same as SnapRestore.  Using the vol copy command, space savings is still realized on the data being moved.  However the fingerprint metadata will not be moved, thus a full volume deduplication must take place to repair that.  Otherwise only new data will be deduplicated.  
FlexClone volumes – FlexClone volumes are simply child volumes of the parent volume that is being cloned.  They inherit all the deduplication attributes that the parent volume has, including scheduling.  This also means that if a parent is not deduplicated, then neither will the clones.  You can deduplicate a clone, but that will not deduplicate the parent.  FlexClone volumes behave similar to Volume Copy and SnapRestore; you must run deduplication at the volume level to ensure that all data is deduplicated not just new data.  There is also one small caveat to FlexClone volumes, if you split the volume, all deduplication savings is lost and you must run a full volume deduplication to regenerate that savings.  If the parent has deduplication scheduled, the split volume will inherit that schedule and perform the deduplication anyway so manually running sis start isn’t mandatory.   
HA pair – Fully supported in an HA pair configuration with a limit of 8 concurrent deduplication operations.  
VMware – VMware VMDK files deduplicate very well, especially similar OS VMDKs.  NetApp recommends that OS VMDKs be stored on the same volume to get as high of a deduplication degree as possible.  Application VMDKs, page files, temp files, etc do not deduplicate well.  Application installations only deduplicate well if the application is the same.  
Multistore – Multistore vFilers support deduplication.  All attributes are inherited from the primary storage system.  The biggest concern is ensuring that the licenses are in place on both sides. 
Compression – Data ONTAP implements compression in two different ways, inline or post process scheduled.  Compression works very much in tandem with deduplication.  Being that it works closely with deduplication means that it also integrates with the other technologies as well.  It can be enabled via OnCommand System Manager or via the CLI.  Space savings when using compression is also realized when using replication technologies resulting in network resource savings.  Below are the considerations for using either method of compression:
Minimize Snapshot space
Inline compression will minimize the amount of space used by Snapshot copies
Minimize disk space usage on qtree SnapMirror or SnapVault destionations
Inline compression for immediate savings and minimal backup window impact.  Less space in snapshot reserve
Minmize disk I/O
Inline compression reduces the number of new blocks written to disk
Avoid performance impact for new writes
Postprocess compression writes the new data to disk uncompressed without any write impact and schedules for later processing to recover space
Minimize impact on CPU during peak hours
Postprocess compression allows off hour scheduling
As discussed, compression works in tandem with deduplication which means that other NetApp technologies work with it as well.  
Snapshot Copies – Compression reduces the amount of space consumed by snapshot copies.  Like deduplication, compression is recommended to be run before the snapshot is run.  This is because snapshots lock data within them.  
Volume SnapMirror – Compression savings, like deduplication savings, is realized on both the source and the destination when using Volume SnapMirror.  Compression and deduplication is only handled on the source machine, the destination volume inherits all compression and deduplication settings from the source.  Network link compression is not needed as Data ONTAP already compresses the data traffic.  
Qtree SnapMirror and SnapVault – These two products handled compression and deduplication differently on the source and destination.  You don’t have to run both or either on the source or destination.  Postprocess compression and deduplication of SnapVault transfers happens automatically anyway.  
Cloning – Cloned volumes inherit the parent’s settings.  You can modify the clone settings for compression without affecting the settings on the parent. 
Snapshots – Snapshots are point in time captures of the active file system.  They are zero footprint, with the exception of the 4KB inode (every file in WAFL has an inode that contains the files attributes) created when executed, and only grow if the data within changes.  Snapshots can be performed at the volume and the aggregate level.  Volumes can support up to 255 snapshots.  When data changes that a Snapshot references, the data block is changed and the active file system now references that block of data.  The Snapshot keeps the old block of data and references that data, however the old block is not available for any other use.  This is how Snapshots grow in size.  They reference old data blocks now unusable by the active file system.  
Inode – Every file within WAFL has an inode that is 192bytes in size and contains the following attributes about the file:  File type, Size, Owner/Group/Permissions, pointer to xinode (ACLs), if less than 64bytes will contain the entire contents of the file, pointers to data blocks.  The most important inode in Data ONTAP is the root inode.  Every volume has a root inode, and it’s located in a fixed disk location.  Within the root inode are the inodes for all the blocks that make up the volume.  Each inode contains the metadata for each block it represents.  
Snapshot reserve – Snapshot reserves are the space that’s allocated to Snapshots so they don’t consume active file system space.  Both volumes and aggregates have a default setting of 20% and 5% respectively.  These can be modified as necessary.  The reserve is setup to keep the Snapshots from consuming active file system space.  If the reserve is set to 0%, a Snapshot could potentially consume all the active file system space.  There is fine tuning involved in setting reserves.  The keys are to avoid wasting space and to avoid consuming all the space within the reserve.  Even with a reserve in place, a Snapshot can consume more than 100% of the reserve.  
Snapshot directory – The Snapshot directory is available for CIFS and NFS clients and it resides at the root of the volume.  Within the Snapshot directory are the previous versions of the files that were Snapshot.  A user can access this directory and look through previous Snapshots to take a look at what the file system used to look like.  They can then pull files from this folder and put them back into the current state folder.  This folder has to be exposed to CIFS and NFS clients.  It’s not enabled by default. 
Snapshot autodelete – Snapshot autodelete is just like it sounds.  It’s an autodelete function that automatically deletes the oldest Snapshot when the reserve gets close to full.  This option is set at the volume level. 
Thin Provisioning – Thin provisioning is very simple, it’s the allocation of an amount of disk space that only consumes what the application actually uses.  This means that if you allocate 100GB and the application only consumes 60GB, the volume will look like 100GB to the application but only consume the 60GB on the storage array.  However, since 100GB is thin provisioned the volume can expand as the space is used up to the 100GB limit imposed.  Thin provisioning is used to over provision the storage array.  Some applications require certain size disks but only consume very little of them.  Thin provisioning removes the wasted free space storage involved with allocating all the storage space up front.  Thin provisioning with Data ONTAP is used with FlexVol volumes.  
Space Management – Space management deals with knowing how your data will react to the underlying storage and considerations to take into effect in regards to out-of-space conditions and over commitment expectations.  
Space Guarantee – Space guarantees come in three flavors:  volume (default), file and none.  Each one guarantees that space will be available for a write no matter what.  The guarantee is taken from the aggregate free space and is no longer usable by any other volumes.  This also applies to things like Snapshot copies as well.  If there is no free space available, those operations will fail.  Space guarantees are only guaranteed to volumes that are online.  If a volume is offline, the unused space is up for grabs and other volumes could take it up.  If you attempt to bring the volume online and the free space is gone, the volume can’t be brought online unless you use the –f­ command which is not recommended.  
Space Guarantee (Volume) – The equivalent to making a traditional volume basically.  This means that whatever space is allocated is subtracted from the aggregate free space.  In other words, doing so effectively disables thin-provisioning.  
Space Guarantee (File) – Any file within the volume can be rewritten if it has a space reservation associated with it.  An example would be if you created a 50GB empty LUN within a 100GB volume, the volume would have 100GB of free space within it.  The 50GB LUN would be guaranteed the space to commit writes to and would consume space as the data grew.  
Space Guarantee (None) – This is the epitome of thin provisioning and over commitment.  None means that there are no guarantees that data can be written to the volume if the aggregate is full.  Not recommended for CIFS shares because of out-of-space errors.  
Space Reservation – Pertains to files or LUNs within a volume.  An example would be if you had a 50GB LUN within a 100GB Volume.  That would leave 50GB free within the volume even if the LUN had no data within it.  Space reservation is automatically set for newly created LUNs.  Space reservations also mean that the space is reserved to writes within the file of LUN.  If operations that require free space do not have the space outside of that reservation, they will fail however all writes to the reserved space will continue without issue.  
Auto-Grow – A FlexVol attribute that allows a volume to automatically grow if the volume reaches a certain point of capacity.  It is done by setting a maximum size setting and an incremental growth setting.  The maximum size determines the upper boundary of how large a volume can grow to.  The incremental growth is how amount the volume grows at one time.  When a volume reaches capacity, the volume is incrementally expanded based on the options configured and will grow only to the upper boundary limit.  
Fractional Reserve – This is probably one of the more difficult subjects to understand.  If you have a 100GB space reserved LUN and you set the fractional reserve to 100%, the volume would need to be 200GB and the volume would be full at this point because Fractional Reserve needs 100GB of space to guarantee that a Snapshot could be created of the entire LUN changing.  
Another example would be if you had a space reserved LUN of 20GB and a volume of 100GB with a fractional reserve of 20%.  This would result in the LUN consuming 20GB to start and when a snapshot that consumes 10GB was taken, the volume would show 70GB free space.  20GB LUN + 10GB Fractional Reserve = 30GB consumed space.  This assumes that you only have 1 LUN per volume.  If you have two LUNs in a volume, it’s possible that one LUN could consume all the Fractional Reserve space.  This is one reason for 1 LUN: 1 Volume configurations.  
AutoSupport – This is the automated phone home system used to automatically send support data to NetApp.  This also includes automatically sending in trouble tickets for hardware and software issues as well.  AutoSupport can use HTTP/HTTPS/SMTP to send messages.  Once enabled, messages begin being sent within 24 hours.  HTTPS is the default.  AutoSupport generates three messages per week on Sunday morning between 12 and 1 am.  One is a weekly performance snapshot of the unit; the other is the hard drive report (not always sent unless there’s an issue) and a default system information snapshot.  AutoSupport can be a pain sometimes because it will alert NetApp even if you’re doing testing with the array and it’s enabled.  It will auto-generate a support ticket and you’ll get a message from a technician.  If you’re performing testing on the NetApp array and you don’t want to send a ticket, disable AutoSupport by running the following command options autosupport.enable off.
References used in this post –
Data ONTAP 8.0 7-Mode Storage Management Guide
Data ONTAP 8.0 7-Mode System Administration Guide
Data ONTAP 8.0 7-Mode Network Management Guide
Data ONTAP 8.0 7-Mode Storage Efficiency Management Guide
Fractional Reserve Explanations Simplified – Refer to paleon’s post

NCDA Study: Blueprint Breakdown

The NCDA covers a ton of information needed to know about not only NetApp hardware, but NetApp software.  NetApp provides a great document that covers some highlights of the things you need to know to get started studying for the exam.  I’ll be using this as a template but digging deeper into each section than the NS0-154 study guide offers.  However the study guide does provide a very good structure to learning the material.  Below I’m going to list out each section and the deeper look at each subject.  

We’ll be using the product documentation from the NOW site.  Since the NS0-154 exam covers ONTAP 8.0, we’ll need to study that documentation.  You should be able to download the docs using a guest NOW account.  You’ll need to go into the archive section to get the 8.0 docs.  These are the documents that you’ll need. 
  • System Administrator Guide
  • Storage Administrator Guide
  • High Availability Configuration Guide
  • File Access and Protocols Management Guide
  • Block Access Management Guide
  • Data Protection Online Backup and Recovery Guide
  • Data Protection Tape Backup and Recovery Guide
  • Commands:  Manual Page Reference, Volume 1
  • Commands:  Manual Page Reference, Volume 2
  • Core Command Quick Reference

For the breakdown of the sections I put this into a PDF to download.  It was just too immense to put as a post on the site.  I’ve attempted to breakdown each section as much as possible for depth.  It’s probably overkill for the exam but we’re going to deep understanding and not ‘enough to pass’ situations. 

Edit:  As I’ve been going through the sections I may change this PDF as I find a better layout or find something I missed studying.  Please bear with me in that regard.  

Download the PDF here.  

NCDA Lab Configuration Part 4: Virtual Machines

This is going to be a short post.  I’m not going to go through how to create virtual machines and install operating systems.  I think that’s pretty straightforward and if there are any issues, there are tons of instructional videos and walkthroughs using Google.  Looking over the NCDA Lab Template, we need to build and configure 3 virtual machines with the following applications and software programs on each.  They consist of:
Windows Server 2008 R2

  • Putty
  • System Manager installation
  • Open System SnapVault installation
  • Windows MPIO installation (Device Specific Module)
  • Host Utilities Kit
  • Java 32bit JRE 1.6 installation
  • Adobe Flash 8 or greater
  • .NET 3.5.1 installed

Red Hat Enterprise Linux – CentOS 6.1

  • No real software needed, has NFS built-in and other capabilities as well  

Windows Server 2003

  • Active Directory installed with test user accounts

Building Virtual machines with VMware workstation
Installing Active Directory Windows 2003
CentOS download
Windows Trial Software
This completes the lab build and configuration.  The next post will cover the breakdown of the blueprint and setup how we will use the lab to coincide with learning all the topics.