Category Archives: Powershell

Powershell Script to Setup new NetApp clustered Data ONTAP system

Setting up a new NetApp clustered Data ONTAP system involves a number of steps. I have tried to automate these steps using NetApp Powershell toolkit. This saves time and reduces human errors while configuring new systems.
This script assumes that hardware is installed, “cluster setup” is run and all nodes joined in. I use a suffix of “-mgmt” with the cluster name when i configure a new clustered Data ONTAP system. This script has been tested to work with ONTAP 8.3 (simulators).

At present i have automated the following tasks:
    1.  Rename Nodes
2.  Rename root aggregates
3.  Create failover groups for cluster-mgmt and node-mgmt interfaces
4.  Add feature licenses
5.  Configure Storage Failover
6.  Unlock diag User
7.  Setup diag user password
8.  Create admin user for access to logs through http
9.  Setup Timezone and NTP server
10. Remove 10 Gbe ports from Default broadcast domain
11. Create ifgroups and add ports to ifgroups
12. Enable Cisco Discovery Protocol (cdpd) on all of the nodes
13. Setup disk auto assignment
14. Setup flexscale options
15. Disable flowcontrol on all the ports
16. Create data aggregates

The script displays the results on Powershell console as it iterates through the setup tasks. A transcript is also saved as a text file.

Cluster_config_screenshot

Source Code

 

<# .SYNOPSIS Automate Setup of a new NetApp clustered Data ONTAP cluster install .DESCRIPTION The script assumes the basic cluster setup is completed and all nodes joined. The script automates the following tasks: 1. Rename Nodes 2. Rename root aggregates 3. Create failover groups for cluster-mgmt and node-mgmt interfaces 4. Add feature licenses 5. Configure Storage Failover 6. Unlock diag User 7. Setup diag user password 8. Create admin user for access to logs through http 9. Setup Timezone and NTP server 10. Remove 10 Gbe ports from Default broadcast domain 11. Create ifgroups and add ports to ifgrps 12. Enable Cisco Discovery Protocol (cdpd) on all of the nodes 13. Setup disk auto assignment 14. Setup flexscale options 15. Disable flowcontrol on all the ports 16. Create data aggregates Example: PS C:\Users\vadmin\Documents\pshell-scripts> .\cluster_config_v1.5.ps1
.PARAMETER settingsFilePath
    Location of the File with User defined Parameters.
.EXAMPLE
    PS C:\Users\vadmin\Documents\pshell-scripts> .\cluster_config_v1.5.ps1
#>
#####################
# Declare Variables
#####################
$ClusterName             = "ntapclu1-mgmt"
$mgmtIP                  = "aa.bb.cc.dd"
$mgmtSubnet              = "aaa.bbb.ccc.ddd"
$mgmtGateway             = "aa.bb.cc.xx"
$ntpServer               = "ntp-server1"
$ClusterNameMgmtPort     = "e0d"
$NodeMgmtPort            = "e0c"
$timezone                = "Australia/Sydney"
[int]$maxraidsize        = 17 #raid group size for creating an aggregate
[int]$diskCount          = 51 
$TranscriptPath          = "c:\temp\cluster_setup_transcript_$(get-date -format "yyyyMMdd_hhmmtt").txt"
$licensesPath            = "c:\temp\licenses.txt" 
$ifgrp_a0a_port1         = "e0e"
$ifgrp_a0a_port2         = "e0f"
$timezone                = 'Australia/Sydney'

###########################
# Declare the functions
###########################
function Write-ErrMsg ($msg) {
    $fg_color = "White"
    $bg_color = "Red"
    Write-host " "
    Write-host $msg -ForegroundColor $fg_color -BackgroundColor $bg_color
    Write-host " "
}
#'------------------------------------------------------------------------------
function Write-Msg ($msg) {
    $color = "yellow"
    Write-host " "
    Write-host $msg -foregroundcolor $color
    Write-host " "
}
#'------------------------------------------------------------------------------
function Invoke-SshCmd ($cmd){
    try {
        Invoke-NcSsh $cmd -ErrorAction stop | out-null
        "The command completed successfully"
    }
    catch {
       Write-ErrMsg "The command did not complete successfully"
    }
}
#'------------------------------------------------------------------------------
function Check-LoadedModule {
  Param( 
    [parameter(Mandatory = $true)]
    [string]$ModuleName
  )
  $LoadedModules = Get-Module | Select Name
  if ($LoadedModules -notlike "*$ModuleName*") {
    try {
        Import-Module -Name $ModuleName -ErrorAction Stop
        Write-Msg ("The module DataONTAP is imported")
    }
    catch {
        Write-ErrMsg ("Could not find the Module DataONTAP on this system. Please download from NetApp Support")
        stop-transcript
        exit 
    }
  }
}
#'------------------------------------------------------------------------------
##############################
# Begin Cluster Setup Process
##############################
#'------------------------------------------------------------------------------
## Load Data ONTAP Module
start-transcript -path $TranscriptPath
#'------------------------------------------------------------------------------
Write-Msg  "##### Beginning Cluster Setup #####"
Check-LoadedModule -ModuleName DataONTAP
try {
    Connect-nccontroller $ClusterName -ErrorAction Stop | Out-Null   
    "connected to " + $ClusterName
    }
catch {
    Write-ErrMsg ("Failed connecting to Cluster " + $ClusterName + " : $_.")
    stop-transcript
    exit
}
#'------------------------------------------------------------------------------
## Get the nodes in the cluster
$nodes = (get-ncnode).node
#'------------------------------------------------------------------------------
## Rename the nodes (remove "-mgmt" string)
Write-Msg  "+++ Renaming Node SVMs +++"
foreach ($node in $nodes) { 
    Rename-NcNode -node $node -newname ($node -replace "-mgmt") -Confirm:$false |Out-Null
} 
Get-NcNode |select Node,NodeModel,IsEpsilonNode | Format-Table -AutoSize
$nodes = (get-ncnode).node
#'------------------------------------------------------------------------------
## Rename root aggregates
Write-Msg  "+++ Renaming root aggregates +++"
# get each of the nodes
Get-NcNode | %{ 
    $nodeName = $_.Node
    # determine the current root aggregate name
    $currentAggrName = (
        Get-NcAggr | ?{ 
             $_.AggrOwnershipAttributes.HomeName -eq $nodeName `
               -and $_.AggrRaidAttributes.HasLocalRoot -eq $true 
        }).Name
    # no dashes
    $newAggrName = $nodeName -replace "-", "_"
    # can't start with numbers
    $newAggrName = $newAggrName -replace "^\d+", " "
    # append the root identifier
    $newAggrName = "$($newAggrName)_root"
    if ($currentAggrName -ne $newAggrName) {
        Rename-NcAggr -Name $currentAggrName -NewName $newAggrName | Out-Null 
    }
    sleep -s 5
    Write-Host "Renamed aggregates containing node root volumes"
    (Get-NcAggr | ?{ $_.AggrOwnershipAttributes.HomeName -eq $node -and $_.AggrRaidAttributes.HasLocalRoot -eq $true }).Name 
}
#'------------------------------------------------------------------------------
## Create failover groups for cluster-mgmt and node-mgmt interfaces
Write-Msg  "+++ Create failover groups for cluster-mgmt and node-mgmt interfaces +++"
# get admin vserver name
$adminSVMTemplate = Get-NcVserver -Template
Initialize-NcObjectProperty -Object $adminSVMTemplate -Name VserverType | Out-Null
$adminSVMTemplate.VserverType = "admin"
$adminSVM         = (Get-NcVserver -Query $adminSVMTemplate).Vserver
# create cluster-mgmt failover group 
$clusterPorts     = ((get-ncnode).Node | % { $_,$ClusterNameMgmtPort -join ":" })
$nodePorts        = ((get-ncnode).Node | % { $_,$NodeMgmtPort -join ":" })
$firstClusterPort = $clusterPorts[0]
$allClusterPorts  = $clusterPorts[1..($clusterPorts.Length-1)]
New-NcNetFailoverGroup -Name cluster_mgmt -Vserver $adminSVM -Target $firstClusterPort | Out-Null
foreach ($cPort in $allClusterPorts) {
    Add-NcNetFailoverGroupTarget -Name cluster_mgmt -Vserver $adminSVM -Target $cPort | Out-Null
}
Set-NcNetInterface -Name cluster_mgmt -Vserver $adminSVM -FailoverPolicy broadcast_domain_wide -FailoverGroup cluster_mgmt | Out-Null
Write-Host "Created cluster-mgmt failover group"
Get-NcNetInterface -Name cluster_mgmt  | select InterfaceName,FailoverGroup,FailoverPolicy
# create node-mgmt failover-group for each node
foreach ($node in $nodes) {
    $prt1 = ($node,$NodeMgmtPort -join ":")
    $prt2 = ($node,$ClusterNameMgmtPort -join ":")
    New-NcNetFailoverGroup -Name $node"_mgmt" -Vserver $adminSVM -Target $prt1 | Out-Null
    Add-NcNetFailoverGroupTarget -Name $node"_mgmt" -Vserver $adminSVM -Target $prt2 | Out-Null
    $nodeMgmtLif = (Get-NcNetInterface -Role node-mgmt | Where-Object {$_.HomeNode -match "$node"}).InterfaceName
    Set-NcNetInterface -Name $nodeMgmtLif -Vserver $adminSVM -FailoverPolicy local-only -FailoverGroup $node"_mgmt" | Out-Null
    sleep -s 5
    Write-Host "Created node-mgmt failover group for node "$node
    Get-NcNetInterface -Role node-mgmt | Where-Object {$_.HomeNode -match "$node"} | select InterfaceName,FailoverGroup,FailoverPolicy
}
sleep -s 15
#'------------------------------------------------------------------------------
## Add licenses to cluster
Write-Msg "+++ Adding licenses +++"
$test_lic_path = Test-Path -Path $licensesPath
if ($test_lic_path -eq "True") {
    $count_licenses = (get-content $licensesPath).count
    if ($count_licenses -ne 0) {
        Get-Content $licensesPath |  foreach { Add-NcLicense -license $_ }
        Write-Host "Licenses successfully added"
        Write-Host " "
    }
    else {
        Write-ErrMsg ("License file is empty. Please add the licenses manually")
    }
}
else {
    Write-ErrMsg ("License file does not exist. Please add the licenses manually")       
}
sleep -s 15
#'------------------------------------------------------------------------------
## Configure storage failover
Write-Msg  "+++ Configure SFO +++"
Write-Host "SFO Does not work with Simulators"
if ($nodes.count -gt 2) {
    foreach ($node in $nodes) {
        $sfo_enabled = Invoke-NcSsh "storage failover modify -node " $node " -enabled true"
        if (($sfo_enabled.Value.ToString().Contains("Error")) -or ($sfo_enabled.Value.ToString().Contains("error"))) {
            Write-ErrMsg ($sfo_enabled.Value)
        }
        else {
            Write-Host ("Storage Failover is enabled on node " + $node)
        }

	    $sfo_autogive = Invoke-NcSsh "storage failover modify -node " $node " -auto-giveback true"
        if (($sfo_autogive.Value.ToString().Contains("Error")) -or ($sfo_autogive.Value.ToString().Contains("error"))) {
                Write-ErrMsg ($sfo_autogive.Value)
        }
        else {
            Write-Host ("Storage Failover option auto giveback is enabled on node " + $node)
            Write-Host " "
        }
        sleep -s 2
    }
}
elseif ($nodes.count -eq 2) {
    foreach ($node in $nodes) {
        $sfo_enabled = Invoke-NcSsh "cluster ha modify -configured true"
        if (($sfo_enabled.Value.ToString().Contains("Error")) -or ($sfo_enabled.Value.ToString().Contains("error"))) {
            Write-ErrMsg ($sfo_enabled.Value)
        }
        else {
            Write-Host ("Cluster ha is enabled on node " + $node)
            Write-Host
        }  
    }
}
else {
    Write-Host "No HA required for single node cluster. Continuing with the setup"
    Write-Host " "
}
sleep -s 15
#'------------------------------------------------------------------------------
## Unlock the diag user
Write-Msg "+++ Unlock the diag user +++"
try {
    Unlock-NcUser -username diag -vserver $ClusterName -ErrorAction stop |Out-Null
    Write-Host "Diag user is unlocked"
}
catch {
    Write-ErrMsg "Diag user is either unlocked or script could not unlock the diag user"
}
#'------------------------------------------------------------------------------
## Setup diag user password
Set-Ncuserpassword -UserName diag -password netapp123! -vserver $ClusterName | Out-Null
Write-Host "created diag user password"
sleep -s 15
#'------------------------------------------------------------------------------
## Create admin user for access to logs through http
Write-Msg "+++ create web log user +++"
Set-NcUser -UserName admin -Vserver $ClusterName -Application http -role admin -AuthMethod password | Out-Null
Write-Host "created admin user access for http log collection"
sleep -s 15
#'------------------------------------------------------------------------------
## Set Date and NTP on each node
Write-Msg  "+++ setting Timezones/NTP/Datetime +++"
foreach ($node in $nodes) {
    Set-NcTime -Node $node -Timezone $timeZone | Out-Null
    Set-NcTime -Node $node -DateTime (Get-Date) | Out-Null
}
New-NcNtpServer -ServerName $ntpServer -IsPreferred | Out-Null
Write-Host "NTP Sever setup complete"
sleep -s 15
#'------------------------------------------------------------------------------
## Remove 10 Gbe ports from Default broadcast domain
Write-Msg  "+++ Rmoving 10Gbe Ports from Default broadcast domain +++"
# remove ports from Default broadcast domain
$broadCastTemplate = Get-NcNetPortBroadcastDomain -Template
Initialize-NcObjectProperty -Object $broadCastTemplate -Name Ports | Out-Null
$broadCastTemplate.BroadcastDomain = "Default"
$defaultBroadCastPorts = ((Get-NcNetPortBroadcastDomain -Query $broadCastTemplate).Ports).Port
foreach ($bPort in $defaultBroadCastPorts) {
	if (($bPort -notlike "*$ClusterNameMgmtPort") -and ($bPort -notlike "*$NodeMgmtPort")) {
		Write-Host "Removing Port: " $bPort
		Set-NcNetPortBroadcastDomain -Name Default -RemovePort $bPort | Out-Null
	}	
}
sleep -s 15
#'------------------------------------------------------------------------------
## Create ifgroups and add ports to ifgrps
Write-Msg  "+++ starting ifgroup creation +++"
foreach ($node in $nodes) {
    try {
        New-NcNetPortIfgrp -Name a0a -Node $node -DistributionFunction port -Mode multimode_lacp -ErrorAction Stop | Out-Null
        Add-NcNetPortIfgrpPort -name a0a -node $node -port $ifgrp_a0a_port1 -ErrorAction Continue | Out-Null
        Add-NcNetPortIfgrpPort -name a0a -node $node -port $ifgrp_a0a_port2 -ErrorAction Continue | Out-Null
        Write-Host ("Successfully created ifgrp a0a on node " + $node)
    }
    catch {
        Write-ErrMsg ("Error exception in ifgrp a0a " + $node + " : $_.")
    }
}
sleep -s 15
#'------------------------------------------------------------------------------
## Enable cdpd on all of the nodes
Write-Msg  "+++ enable cdpd on nodes +++"
foreach ($node in $nodes) {
    $cdpd_cmd = Invoke-NcSsh "node run -node " $node " -command options cdpd.enable on"
    if (($cdpd_cmd.Value.ToString().Contains("Error")) -or ($cdpd_cmd.Value.ToString().Contains("error"))) {
        Write-ErrMsg ($cdpd_cmd.Value)
    }
    else {
        Write-Host ("Successfully modified cdpd options for " + $node)
    }
}
sleep -s 15
#'------------------------------------------------------------------------------
## Set option disk.auto_assign on
Write-Msg  "+++ Setting disk autoassign +++"
foreach ($node in $nodes) {
    $set_disk_auto = Invoke-NcSsh "node run -node " $node " -command options disk.auto_assign on"
    if (($set_disk_auto.Value.ToString().Contains("Error")) -or ($set_disk_auto.Value.ToString().Contains("error"))) {
        Write-ErrMsg ($set_disk_auto.Value)
    }
    else {
        Write-Host ("Successfully modified disk autoassign option on node " + $node)
    }   
}
sleep -s 15
#'------------------------------------------------------------------------------
## Set flexscale options
Write-Msg  "+++ Setting flexscale options +++"
foreach ($node in $nodes) {
	$flexscale_enable = Invoke-NcSsh "node run -node " $node " -command options flexscale.enable on" 
    if (($flexscale_enable.Value.ToString().Contains("Error")) -or ($flexscale_enable.Value.ToString().Contains("error"))) {
        Write-ErrMsg ($flexscale_enable.Value)
    }
    else {
        Write-Host ("options flexscale.enable set to on for node " + $node)
    } 

	$flexscale_lopri = Invoke-NcSsh "node run -node " $node " -command options flexscale.lopri_blocks on"
    if (($flexscale_lopri.Value.ToString().Contains("Error")) -or ($flexscale_lopri.Value.ToString().Contains("error"))) {
        Write-ErrMsg ($flexscale_lopri.Value)
    }
    else {
        Write-Host ("options flexscale.lopri_blocks set to on for node " + $node)
    } 

	$flexscale_data = Invoke-NcSsh "node run -node " $node " -command options flexscale.normal_data_blocks on"
    if (($flexscale_data.Value.ToString().Contains("Error")) -or ($flexscale_data.Value.ToString().Contains("error"))) {
        Write-ErrMsg ($flexscale_data.Value)
    }
    else {
        Write-Host ("options flexscale.normal_data_blocks set to on for node " + $node)
        Write-Host " "
    } 

}
sleep -s 15
#'------------------------------------------------------------------------------
## Disable flowcontrol on all of the ports
Write-Msg  "+++ Setting flowcontrol +++"
foreach ($node in $nodes) {
    try {
        Write-Host "Setting flowcontrol for ports on node: " $node
        get-ncnetport -Node $node | Where-Object {$_.Port -notlike "a0*"} | select-object -Property name, node | set-ncnetport -flowcontrol none -ErrorAction Stop | Out-Null
        sleep -s 15
        Get-NcNetPort -Node $node | Select-Object -Property Name,AdministrativeFlowcontrol | Format-Table -AutoSize
    }
    catch {
        Write-ErrMsg ("Error setting flowcontrol on node " + $node + ": $_.")
    }
}
sleep -s 15
#'------------------------------------------------------------------------------
## Create data aggregates
Write-Msg  "+++ Creating Data Aggregates +++"
# get each of the nodes
Get-NcNode | %{ 
    $nodeName = $_.Node
    # no dashes
    $newAggrName = $nodeName -replace "-", "_"
    # can't start with numbers
    $newAggrName = $newAggrName -replace "^\d+", " "
    # append the root identifier
    $newAggrName = "$($newAggrName)_data_01"
    # create an aggreagate
    $aggrProps = @{
        'Name' = $newAggrName;
        'Node' = $nodeName;
        'DiskCount' = $diskCount;
        'RaidSize' = $maxraidsize;
        'RaidType' = "raid_dp";
    }
    New-NcAggr @aggrProps | Out-Null
#
    sleep -s 15
    # enable free space reallocation
    Get-NcAggr $newAggrName | Set-NcAggrOption -Key free_space_realloc -Value on
}
#'------------------------------------------------------------------------------
Write-Host " "
Write-Host " "
stop-transcript
#'------------------------------------------------------------------------------

Get-NodePerfData – Powershell Script to query NetApp Oncommand Performance Manager (OPM)

<#
script : Get-NodePerfData.ps1
Example:
Get-NodePerfData.ps1

This script queries OPM(version 2.1/7.0) Server and extract following performance counters for each node in clusters
    Date, Time, avgProcessorBusy, cpuBusy, cifsOps, nfsOps, avgLatency

All data is saved in to "thismonth" directory. e.g. 1608 (YYMM)

#>
Function Get-TzDateTime{
   Return (Get-TzDate) + " " + (Get-TzTime)
}
Function Get-TzDate{
   Return Get-Date -uformat "%Y-%m-%d"
}
Function Get-TzTime{
   Return Get-Date -uformat "%H:%M:%S"
}
Function Log-Msg{
   <#
   .SYNOPSIS
   This function appends a message to log file based on the message type.
   .DESCRIPTION
   Appends a message to a log a file.
   .PARAMETER
   Accepts an integer representing the log file extension type
   .PARAMETER
   Accepts a string value containing the message to append to the log file.
   .EXAMPLE
   Log-Msg -logType 0 -message "Command completed succuessfully"
   .EXAMPLE
   Log-Msg -logType 2, -message "Application is not installed"
   #>
   [CmdletBinding()]
   Param(
      [Parameter(Position=0,
         Mandatory=$True,
         ValueFromPipeLine=$True,
         ValueFromPipeLineByPropertyName=$True)]
      [Int]$logType,
      [Parameter(Position=1,
         Mandatory=$True,
         ValueFromPipeLine=$True,
         ValueFromPipeLineByPropertyName=$True)]
      [String]$message
   )
   Switch($logType){
      0 {$extension = "log"; break}
      1 {$extension = "err"; break}
      2 {$extension = "err"; break}
      3 {$extension = "csv"; break}
      default {$extension = "log"}
   }
   If($logType -eq 1){
      $message = ("Error " + $error[0] + " " + $message)
   }
   $prefix = Get-TzDateTime
   ($prefix + "," + $message) | Out-File -filePath `
   ($scriptLogPath + "." + $extension) -encoding ASCII -append
}
function MySQLOPM {
    Param(
      [Parameter(
      Mandatory = $true,
      ParameterSetName = '',
      ValueFromPipeline = $true)]
      [string]$Switch,
      [string]$Query
      )

    if($switch -match 'performance') {
        $MySQLDatabase = 'netapp_performance'
    }
    elseif($switch -match 'model'){
        $MySQLDatabase = 'netapp_model_view'    
    }
    $MySQLAdminUserName = 'report'
    $MySQLAdminPassword = 'password123'
    $MySQLHost = 'opm-server'
    $ConnectionString = "server=" + $MySQLHost + ";port=3306;Integrated Security=False;uid=" + $MySQLAdminUserName + ";pwd=" + $MySQLAdminPassword + ";database="+$MySQLDatabase

    Try {
      [void][System.Reflection.Assembly]::LoadFrom("E:\ssh\L080898\MySql.Data.dll")
      $Connection = New-Object MySql.Data.MySqlClient.MySqlConnection
      $Connection.ConnectionString = $ConnectionString
      $Connection.Open()

      $Command = New-Object MySql.Data.MySqlClient.MySqlCommand($Query, $Connection)
      $DataAdapter = New-Object MySql.Data.MySqlClient.MySqlDataAdapter($Command)
      $DataSet = New-Object System.Data.DataSet
      $RecordCount = $dataAdapter.Fill($dataSet, "data")
      $DataSet.Tables[0]
      }

    Catch {
      Write-Host "ERROR : Unable to run query : $query `n$Error[0]"
     }

    Finally {
      $Connection.Close()
    }
}
#'------------------------------------------------------------------------------
#'Initialization Section. Define Global Variables.
#'------------------------------------------------------------------------------
##'Set Date and Time Variables
[String]$lastmonth      = (Get-Date).AddMonths(-1).ToString('yyMM')
[String]$thismonth      = (Get-Date).ToString('yyMM')
[String]$yesterday      = (Get-Date).AddDays(-1).ToString('yyMMdd')
[String]$today          = (Get-Date).ToString('yyMMdd')
[String]$fileTime       = (Get-Date).ToString('HHmm')
[String]$workDay        = (Get-Date).AddDays(-1).DayOfWeek
[String]$DOM            = (Get-Date).ToString('dd')
[String]$filedate       = (Get-Date).ToString('yyyyMMdd')
##'Set Path Variables
[String]$scriptPath     = Split-Path($MyInvocation.MyCommand.Path)
[String]$scriptSpec     = $MyInvocation.MyCommand.Definition
[String]$scriptBaseName = (Get-Item $scriptSpec).BaseName
[String]$scriptName     = (Get-Item $scriptSpec).Name
[String]$scriptLogPath  = $scriptPath + "\Logs\" + (Get-TzDate) + "-" + $scriptBaseName
[System.Object]$fso     = New-Object -ComObject "Scripting.FileSystemObject"
[String]$outputPath     = $scriptPath + "\Reports\" + $thismonth
[string]$logPath        = $scriptPath+ "\Logs"

# MySQL Query to get objectid, name of all nodes
$nodes = MySQLOPM -Switch model -Query "select objid,name from node"

# Create hash of nodename and objid
$hash =@{}

foreach ($line in $nodes) {
    $hash.add($line.name, $line.objid)
}
# Create Log Directory
if ( -not (Test-Path $logPath) ) { 
       Try{
          New-Item -Type directory -Path $logPath -ErrorAction Stop | Out-Null
          Log-Msg 0 "Created Folder ""logPath"""
       }
       Catch{
          Log-Msg 0 "Failed creating folder ""$logPath"" . Error " + $_.Exception.Message
          Exit -1;
       }
    }

# Check hash is not empty, then query OPM server to extract counters
if ($hash.count -gt 0) {

    # If Report directory does not exist then create
    if ( -not (Test-Path $outputPath) ) { 
       Try{
          New-Item -Type directory -Path $outputPath -ErrorAction Stop | Out-Null
          Log-Msg 0 "Created Folder ""$outputPath"""
       }
       Catch{
          Log-Msg 0 "Failed creating folder ""$outputPath"" . Error " + $_.Exception.Message
          Exit -1;
       }
    }
    # foreach node
    foreach ($h in $hash.GetEnumerator()) {
    
        $nodeperffilename  = "$($h.name)`_$filedate.csv"
        $nodePerfFile = Join-Path $outputPath $nodeperffilename

        # MySQL Query to query each object and save data to lastmonth directory
        MySQLOPM -Switch performance -Query "select objid,Date_Format(FROM_UNIXTIME(time/1000), '%Y:%m:%d') AS Date ,Date_Format(FROM_UNIXTIME(time/1000), '%H:%i') AS Time, round(avgProcessorBusy,1) AS cpuBusy,round(cifsOps,1) AS cifsOps,round(nfsOps,1) AS nfsOps,round((avgLatency/1000),1) As avgLatency from sample_node where objid=$($h.value)" | Export-Csv -Path $nodePerfFile -NoTypeInformation
        Log-Msg 0 "Exported Performance Logs for $($h.name)"
    }
} 

Error Handling in Powershell Scripts

Introduction

I have been writing powershell scripts to address various problems with utmost efficiency. I have been incorporating error handling in my scripts, however, i refreshed my knowledge and i am sharing this with fellow IT professionals. While running powershell cmdlets, you encounter two kinds of errors (Terminating and Non Terminating):

  • Terminating : These will halt the function or operation. e.g. syntax error, running out of memory. Can be caught and handled.

Terminating-Error

  • Non Terminating : These allow the function or operation to continue. e.g. file not found, permission issues, if the file is empty the operation continues to next peice of code. Difficult to capture.

So How do you capture non terminating errors in a funcion?

Powershell provides various Variables and Actions to handle errors and exceptions:

  • $ErrorActionPreference : environment variable which applies to all cmdlets in the shell or the script
  • -ErrorAction : applies to specific cmdlets where it is applied
  • $Error : whenever an exception occurs its added to $Error variable. By default the variable holds 256 errors. The $Error variable is an array where the first element is the most recent exception. As new exceptions occur, the new one pushes the others down the list.
  • -ErrorVariable: accepts the name of a variable and if the command generates and error, it’ll be placed in that variable.
  • Try .. Catch Constructs : Try part contains the command or commands that you think might cause an error. You have to set their -ErrorAction to Stop in order to catch the error. The catch part runs if an error occurs within the Try part.

-ErrorAction : Use ErrorAction parameter to treat non terminating errors as terminating. Every powershell cmdlet supports ErrorAction.Powershell halts execution on terminating errors. For non terminating errors we have the option to tell powershell how to handle these situations.

Available Choices

  • SilentlyContinue : error messages are supressed and execution continues
  • Stop : forces execution to stop, behaves like a terminating error
  • Continue : default option. Errors will display and execution will continue
  • Inquire : prompt the user for input to see if we should proceed
  • Ignore : error is ignored and not logged to the error stream

function Invoke-SshCmd ($cmd){
try {
Invoke-NcSsh $cmd -ErrorAction stop | out-null
"The command completed successfully"
}
catch {
Write-ErrMsg "The command did not complete successfully"
}
}

$ErrorActionPreference : It is also possible to treat all errors as terminating using the ErrorActionPreference variable.You can do this either for the script your are working with or for the whole PowerShell session.

-ErrorVariable : Below example captures error in variable “$x”

function Invoke-SshCmd ($cmd){
try {
Invoke-NcSsh $cmd -ErrorVariable x -ErrorAction SilentlyContinue | out-null
"The command completed successfully"
}
catch {
Write-ErrMsg "The command did not complete successfully : $x.exception"
}
}

$x.InvocationInfo : provides details about the context which the command was executed
$x.Exception : has the error message string
If there is a further underlying problem that is captured in $x.Exception.innerexception
The error message can be futher broken in:
$x.Exception.Message
and $x.Exception.ItemName
$($x.Exception.Message) another way of accessing the error message.

$Error : Below example captures error in default $error variable

function Invoke-SshCmd ($cmd){
try {
Invoke-NcSsh $cmd -ErrorAction stop | out-null
"The command completed successfully"
}
catch {
Write-ErrMsg "The command did not complete successfully : $error[0].exception"
}
}

Query Oncommand Performance Manager (OPM) Database using Powershell

Introduction

OnCommand Performance Manager (OPM) provides performance monitoring and event root-cause analysis for systems running clustered Data ONTAP software. It is the performance management part of OnCommand Unified Manager. OPM 2.1 is well integrated with Unified Manager 6.4. You can view and analyze events in the Performance Manager UI or view them in the Unified Manager Dashboard.

Performance Manager collects current performance data from all monitored clusters every five minutes (5, 10, 15). It analyzes this data to identify performance events and potential issues. It retains 30 days of five-minute historical performance data and 390 days of one-hour historical performance data. This enables you to view very granular performance details for the current month, and general performance trends for up to a year.

Accessing the Database

Using powershell you can query MySQL database and retrieve information to create performance charts in Microsoft Excel or other tools. In order to access OPM databse you’ll need a user created with “Database User” role.

OPM-User

The following databases are availbale in OPM 2.1

  • information_schema
  • netapp_model
  • netapp_model_view
  • netapp_performance
  • opm

Out of the above, the two databases that have more relevant information are “netapp_model_view” and “netapp_performance”Database “netapp_model_view” has tables that define the objects and relationships among the objects for which performance data is collected, such as aggregates, SVMs, clusters, volumes, etc.  Database netapp_performance has tables which contain the raw data collected as well as periodic rollups used to quickly generate the graphs OPM presents through its GUI.

Refer to MySQL function in my previous post on Querying OCUM Database using Powershell to connect to OPM database.

Understanding Database

OPM assigns each object (node, cluster, lif, port, aggregate, volumes etc.) a unique id. These id’s are independent of id’s in OCUM database. Theser id’s are stored in tables in “netapp_model_view” database. You can perform join on various tables through the object id’s.

Actual performance data is collected and stored in tables in “netapp_performance” database. All table have a suffix “sample_”. Each table row contains OPM object id for the object (node, cluster, lif, port, aggregate, volumes etc.), the timestamp of the collection and the raw data.

Few useful Database queries

Below example queries database to retrieve performance counter of a node.

Connect to “netapp_model_view” database and list the objid and name from table nodes

"MySQL -Query ""select objid,name from node"" | Format-Table -AutoSize"

Connect to “netapp_performance” database and export cpuBusy, cifsOps, avgLatency from table node

"MySQL -Query ""select objid,Date_Format(FROM_UNIXTIME(time/1000), '%Y:%m:%d %H:%i') AS Time,cpuBusy,cifsOps,avgLatency from sample_node where objid=2"" | Export-Csv -Path E:\snowy-01.csv -NoTypeInformation"

Querying OCUM Database using Powershell

Oncommand Unified Manager (OCUM) is the software to monitor and troubleshoot cluster or SVM issues relating to data storage capacity, availability, performance and protection. OCUM polls the clustered Data ONTAP stoage systmes and stores all inventory information in MySQL database. Using powershell we can query MySQL database and retrieve information to create reports.

All we need is MySQL .NET connector to query OCUM database and retrieve information from various tables. Another tool that is helpful is “HeidiSQL” client for MySQL. You can connect to OCUM Database using Heidi SQL and view all the tables and columns within the database.

Download and use version 2.0 MySQL Connector with OCUM 6.2

Donwload link to HeidiSQL

NetApp Communities Post

First of all you’ll need to create a “Database User” with Role “Report Schema” (OCUM GUI -> Administration -> ManagerUsers -> Add)

Use HeidiSQL to connect to OCUM database

Connect-OCUM

OCUM

Ocum_report

Sample Powershell Code to connect to OCUM Database and retrieve information

# Get-cDOTAggrVolReport.ps1
# Date : 2016_03_10 12:12 PM
# This script uses MySQL .net connector at location E:\ssh\MySql.Data.dll to query OCCUM 6.2 database

# Function MySQL queries OCUM database
# usage: MySQL -Query <sql-query>
function MySQL {
Param(
[Parameter(
Mandatory = $true,
ParameterSetName = '',
ValueFromPipeline = $true)]
[string]$Query
)

$MySQLAdminUserName = 'reportuser'
$MySQLAdminPassword = 'Netapp123'
$MySQLDatabase = 'ocum_report'
$MySQLHost = '192.168.0.71'
$ConnectionString = "server=" + $MySQLHost + ";port=3306;Integrated Security=False;uid=" + $MySQLAdminUserName + ";pwd=" + $MySQLAdminPassword + ";database="+$MySQLDatabase

Try {
[void][System.Reflection.Assembly]::LoadFrom("E:\ssh\MySql.Data.dll")
$Connection = New-Object MySql.Data.MySqlClient.MySqlConnection
$Connection.ConnectionString = $ConnectionString
$Connection.Open()

$Command = New-Object MySql.Data.MySqlClient.MySqlCommand($Query, $Connection)
$DataAdapter = New-Object MySql.Data.MySqlClient.MySqlDataAdapter($Command)
$DataSet = New-Object System.Data.DataSet
$RecordCount = $dataAdapter.Fill($dataSet, "data")
$DataSet.Tables[0]
}

Catch {
Write-Host "ERROR : Unable to run query : $query `n$Error[0]"
}

Finally {
$Connection.Close()
}
}

# Define disk location to store aggregate and volume size reports retrieved from OCUM
$rptdir = "E:\ssh\aggr-vol-space"
$rpt = "E:\ssh\aggr-vol-space"
$filedate = (Get-Date).ToString('yyyyMMdd')
$aggrrptFilename = "aggrSize`_$filedate.csv"
$aggrrptFile = Join-Path $rpt $aggrrptFilename
$volrptFilename = "volSize`_$filedate.csv"
$volrptFile = Join-Path $rpt $volrptFilename

# verify Report directory exists
if ( -not (Test-Path $rptDir) ) {
write-host "Error: Report directory $rptDir does not exist."
exit
}

# Produce aggregate report from OCUM
#$aggrs = MySQL -Query "select aggregate.name as 'Aggregate', aggregate.sizeTotal as 'TotalSize KB', aggregate.sizeUsed as 'UsedSize KB', aggregate.sizeUsedPercent as 'Used %', aggregate.sizeAvail as 'Available KB', aggregate.hasLocalRoot as 'HasRootVolume' from aggregate"
$aggrs = MySQL -Query "select aggregate.name as 'Aggregate', round(aggregate.sizeTotal/Power(1024,3),1) as 'TotalSize GB', round(aggregate.sizeUsed/Power(1024,3),1) as 'UsedSize GB', aggregate.sizeUsedPercent as 'Used %', round(aggregate.sizeAvail/Power(1024,3),1) as 'Available GB', aggregate.hasLocalRoot as 'HasRootVolume' from aggregate"
$aggrs | where {$_.HasRootVolume -eq $False} | export-csv -NoTypeInformation $aggrrptFile

# Produce volume report from OCUM
$vols = MySQL -Query "select volume.name as 'Volume', clusternode.name as 'Nodename', aggregate.name as 'Aggregate', round(volume.size/Power(1024,3),1) as 'TotalSize GB', round(volume.sizeUsed/Power(1024,3),1) as 'UsedSize GB', volume.sizeUsedPercent as 'Used %', round(volume.sizeAvail/Power(1024,3),1) as 'AvaliableSize GB', volume.isSvmRoot as 'isSvmRoot', volume.isLoadSharingMirror as 'isLSMirror' from volume,clusternode,aggregate where clusternode.id = volume.nodeId AND volume.aggregateId = aggregate.id"
$vols | where {$_.isSvmRoot -eq $False -and $_.isLSMirror -eq $False -and $_.Volume -notmatch "vol0$"} | export-csv -NoTypeInformation $volrptFile

How to clean Snapmirrors in clustered Data ONTAP

This Blog post lists the procedure used to clean up snapmirror relationships in clustered Data ONTAP.  In this scenario the destination cluster has large number of volumes that are snapmirror target

Drawing8

Source Cluster: SrcCluster
Destination Cluster: DstCluster
Source SVM: SrcVsv
Destination SVM: DstVsv

Use snapmirror show command to display source and destination paths

    DstCluster::*> snapmirror show -source-cluster SrcCluster
    Source Destination Mirror Relationship Total Last
    Path Type Path State Status Progress Healthy Updated
    ———– —- ———— ——- ————– ——— ——- ——–
    SrcVsv:src_vol_c0165 DP DstVsv:SrcCluster_src_vol_c0165r Snapmirrored Idle – false –
    SrcVsv:src_vol_c0180 DP DstVsv:SrcCluster_src_vol_c0180r Snapmirrored Idle – false –
    SrcVsv:src_vol_c0282 DP DstVsv:SrcCluster_src_vol_c0282r Snapmirrored Idle – false –
    SrcVsv:src_vol_c0284 DP DstVsv:SrcCluster_src_vol_c0284r Snapmirrored Idle – false –
    SrcVsv:src_vol_c0286 DP DstVsv:SrcCluster_src_vol_c0286r Snapmirrored Idle – true –
    SrcVsv:src_vol_c0313 DP DstVsv:SrcCluster_src_vol_c0313r Snapmirrored Idle – true –

Use snapmirror quiesce command to quiesce the snapmirror relations

    DstCluster::*> snapmirror quiesce -source-cluster SrcCluster -destination-path DstVsv:*
    Operation succeeded: snapmirror quiesce for destination “DstVsv:SrcCluster_src_vol_c0165r”.
    Operation succeeded: snapmirror quiesce for destination “DstVsv:SrcCluster_src_vol_c0180r”.
    Operation succeeded: snapmirror quiesce for destination “DstVsv:SrcCluster_src_vol_c0282r”.
    Operation succeeded: snapmirror quiesce for destination “DstVsv:SrcCluster_src_vol_c0284r”.
    Operation succeeded: snapmirror quiesce for destination “DstVsv:SrcCluster_src_vol_c0286r”.
    Operation succeeded: snapmirror quiesce for destination “DstVsv:SrcCluster_src_vol_c0313r”.

Break the snapmirror relations

    DstCluster::*> snapmirror break -source-cluster SrcCluster -destination-path DstVsv:*
    [Job 18594] Job succeeded: SnapMirror Break Succeeded
    [Job 18595] Job succeeded: SnapMirror Break Succeeded
    [Job 18596] Job succeeded: SnapMirror Break Succeeded
    [Job 18597] Job succeeded: SnapMirror Break Succeeded
    [Job 18598] Job succeeded: SnapMirror Break Succeeded
    [Job 18599] Job succeeded: SnapMirror Break Succeeded
    [Job 18600] Job succeeded: SnapMirror Break Succeeded

[ ON THE SOURCE CLUSTER ] : Use snapmirror list-destinations command to check valid  snapmirror relations

    SrcCluster::> snapmirror list-destinations -destination-vserver DstVsv
    Source Destination Transfer Last Relationship
    Path Type Path Status Progress Updated Id
    ———– —– ———— ——- ——— ———— —————
    SrcVsv:src_vol_c0165 DP DstVsv:SrcCluster_src_vol_c0165r – – – a58a0def-2c61-11e4-b66e-123478563412
    SrcVsv:src_vol_c0282 DP DstVsv:SrcCluster_src_vol_c0282r – – – a5cbb158-2c61-11e4-8fb5-123478563412
    SrcVsv:src_vol_c0284 DP DstVsv:SrcCluster_src_vol_c0284r Idle – – a908f9f0-2c61-11e4-a8c1-123478563412
    SrcVsv:src_vol_c0286 DP DstVsv:SrcCluster_src_vol_c0286r Idle – – a773c018-2c61-11e4-8fb5-123478563412
    SrcVsv:src_vol_c0313 DP DstVsv:SrcCluster_src_vol_c0313r Idle – – aaaa528e-2c61-11e4-8fb5-123478563412

Issue snapmirror release command from the source cluster

    SrcCluster::> snapmirror release -destination-vserver DstVsv -destination-volume *
    [Job 74850] Job succeeded: SnapMirror Release Succeeded
    [Job 74851] Job succeeded: SnapMirror Release Succeeded
    [Job 74852] Job succeeded: SnapMirror Release Succeeded
    [Job 74853] Job succeeded: SnapMirror Release Succeeded
    [Job 74854] Job succeeded: SnapMirror Release Succeeded
    [Job 74855] Job succeeded: SnapMirror Release Succeeded

Delete the snapmirror relations from Destination cluster

    DstCluster::*> snapmirror delete -source-vserver SrcVsv -destination-path DstVsv:*
    Operation succeeded: snapmirror delete for the relationship with destination “DstVsv:SrcCluster_src_vol_0043r”.
    Operation succeeded: snapmirror delete for the relationship with destination “DstVsv:SrcCluster_src_vol_0048r”.
    Operation succeeded: snapmirror delete for the relationship with destination “DstVsv:SrcCluster_src_vol_0070r”.
    Operation succeeded: snapmirror delete for the relationship with destination “DstVsv:SrcCluster_src_vol_0082r”.

NetApp Powershell toolkit can be very handy to automate large number of tasks

    Import Data ONTAP Module in Powershell command window
    Import-Module DataOnTap
    Connect to the Source cluster
    Connect-NcController SrcCluster -Credential admin
    Save a list of source volumes in a file called srcvolumes.txt. We will import the volumes in the variable $volumes and parse through each volume deleting snapshots with name “snapmirror*”
    $volumes = get-content C:\scripts\srcvolumes.txt
    foreach ($vol in $volumes) {Get-NcSnapshot $vol snapmi* | Remove-NcSnapshot -IgnoreOwners -Confirm:$false}
    Save a list of destination volumes in a file called dstvolumes.txt. we will import the volumes in a variable $volumes and we will offline and destroy each destination volume
    $volumes = get-content C:\scripts\dstvolumes.txt
    foreach ($vol in $volumes) {Dismount-NcVol $vol -VserverContext DstVsv -Confirm:$false}
    foreach ($vol in $volumes) {Set-NcVol $vol -Offline -VserverContext DstVsv -Confirm:$false}
    foreach ($vol in $volumes) {Remove-NcVol $vol -VserverContext DstVsv -Confirm:$false}