Redis as a data structures server makes it very easy to create, store and report on many aspects of an infrastructure. We'll take a look at how this could be used with one example from a Storage/SAN environment. I'm going to use Perl and the Redis module in the examples, but this could very easily be done in any other favorite scripting language. I'm also going to assume data has already been collected from the various sources. There are lots of ways to do this, CLI, REST, SMI-S, third party tools, vendor tools just to name a few options.

One of the first things we'll need to do is design a key scheme for all the data. We simply need to ensure the key is unique in the data. We'll use Redis hashes and sets to store this data based on the key. There is nothing to create ahead of time. We'll simply use these structures to store the data in Redis. It does all this for us.


This is how we will layout our data structure.

Redis Hashes


SAN Switches
This source will contain attributes of a switch.
switch:$sw
Store: name, model, firmware, ports, switchwwn


SAN Ports
This source will will contain SAN port attributes.
port:$sw:slot:$slot:port:$port
Store: idx, portspd, portstate, porttype, portphys


WWNs
This source will contain attributes associated to a wwn.
wwn:$wwn
Store: hostname, hba, port, type


WWN Connections
The switch/slot/port to which a given wwn is attached. This allows us to associate from a wwn to a SAN port location.
wwnconn:$wwn
Store: switch, slot, port


Redis Sets


SAN Connections
All the wwns connected to a given SAN port. This allows us to associate all the wwns connected to a given SAN port.
sannconn:$sw:slot:$slot:port:$port
Store: wwns


Host WWNs
All wwns in a server os instance. This will allow us to report SAN connections at a host level (multi-path).
host:$hostname
Store: wwns


We'll be using the following Redis commands via the Perl Redis module.

HMSET - Sets the specified fields to their respective values in the hash stored at key.
HMGET - Returns the values associated with the specified fields in the hash stored at key.
SMEMBERS - Returns all the members of the set value stored at key.
SADD - Add the specified members to the set stored at key.
KEYS - Returns all keys matching pattern.


Here's sample data we'll use in this exercise. We'll use this data to populate all our hashes and sets in this exercise.

#SAN Switches
switch,manu,model,fw,ports,wwn
sw121,Brocade,8510-8,v7.4.1,384,1122334455667788
sw122,Brocade,8510-8,v7.4.1,384,1122334455667799
 
#SAN Ports (colon separated wwns for NPIV SAN ports)
switch,slot,port,pidx,portspd,portstate,porttype,portphys,wwpns
sw121,1,0,0,N8Gbps,Online,F_Port,In_Sync,500601604460356C
sw121,1,1,1,N8Gbps,Online,F_Port,In_Sync,500601684460356C
sw121,1,2,2,N8Gbps,Online,F_Port,In_Sync,21000024FF218DDE:2101001B32AE17DB
sw121,1,3,3,N8Gbps,Online,F_Port,In_Sync,2100001B329BF124
sw122,1,0,0,N8Gbps,Online,F_Port,In_Sync,500601614460356C
sw122,1,1,1,N8Gbps,Online,F_Port,In_Sync,500601694460356C
sw122,1,2,2,N8Gbps,Online,F_Port,In_Sync,21000024FF218DDF:2101001B32AECBDB
sw122,1,3,3,N8Gbps,Online,F_Port,In_Sync,2100001B329DE91C
 
#Device WWNs (we'll use this format for both host and storage devices)
host,adapter,port,type,wwn
host0068,hba1,1,host,21000024FF218DDE
host0068,hba2,1,host,21000024FF218DDF
host0115,hba1,1,host,2100001B329BF124
host0115,hba2,1,host,2100001B329DE91C
host0014,hba3,1,host,2101001B32AE17DB
host0014,hba4,1,host,2101001B32AECBDB
stor124,A,0,storage,500601604460356C
stor124,B,0,storage,500601684460356C
stor124,A,1,storage,500601614460356C
stor124,B,1,storage,500601694460356C

Loading Data

Let's load some data. Each section below highlights key portions of perl code we use. You'll notice there is no error checking or other validations. This is bare bones to show essentials.

The switch data is a basic Redis hash where we will store the fields indicated in code (hmset). This could easily be expanded to many more fields as desired.

#Switch data
foreach my $line (<SWITCHES>){
   chomp $line;
   my ($switch,$manu,$model,$fw,$ports,$wwn)=split(/\,/, $line);
   my $swkey="switch".':'."$switch";
   #Add to Redis
   my $x = $r->hmset($swkey, manu, "$manu", model, "$model", fw, "$fw", wwn, "$wwn");
}

WWN data is handled very much like switch data, but we also create a Redis set to hold all the wwn 'members' for each host/device (sadd).

#WWN data
#We load the hash (hmset) and the set (sadd).
foreach my $line (<WWNS>){
   chomp $line;
   my ($host,$adapter,$port,$type,$wwn)=split(/\,/,$line);
 
   #Add wwn data to the hash
   my $wkey="wwn".':'."$wwn";
   #Add to Redis
   my $x = $r->hmset($wkey, host, "$host", adapter, "$adapter", port, "$port", type, "$type");
 
   #Add wwn to the set for this host
   my $hkey="host".':'."$host";
   my $x = $r->sadd($hkey, $wwn);
}

Finally we'll load all the SAN port data. We load records for each port, create records for wwn to port assignments and a set to hold all wwns attached to a port.

#SAN port data
#Port attributes hash
#WWN port connection information hash
#SAN Connection information set (all wwns connected to a given SAN port)
foreach my $line (<SANPORTS>){
   chomp $line;
   my ($switch,$slot,$port,$pidx,$portspd,$portstate,$porttype,$portphys,$wwpns)=split(/\,/,$line);
 
   #Add port attributes to the hash
   my $swportkey="swport".':'."$switch".':'."slot".':'."$slot".':'."port".':'."$port";
   #Add to Redis
   my $x = $r->hmset($swportkey, idx, "$pidx", portspd, "$portspd", portstate, "$portstate", porttype, "$porttype", portphys, "$portphys");
 
   #Add wwn(s) to the set for this port. Multiple are separated by colons. Each wwn will also be added to the wwnconn hash.
   my (@wwns)=split(/\:/, $wwpns);
   foreach my $wwn (@wwns){
      my $wwnconnkey="wwnconn".':'."$wwn";

      #Add to Redis
      my $x = $r->hmset($wwnconnkey, switch, "$switch", slot, "$slot", port, "$port");
 
      my $sanconnkey="sanconn".':'."$switch".':'."slot".':'."$slot".':'."port".':'."$port";
      #Add SAN connection information
      my $x = $r->sadd($sanconnkey, $wwn);
   }
}

Listing Redis Keys

Using the redis CLI tool, redis-cli, we can examine the keys created. Specifically we'll use the keys command.


Hash keys. Every SAN switch and attributes are here.

127.0.0.1:6379> keys switch:*
1) "switch:sw122"
2) "switch:sw121"

Hash keys. Every wwn is associated to a host, adapter and port as well as device type.

127.0.0.1:6379> keys wwn:*
1) "wwn:21000024FF218DDF"
2) "wwn:500601604460356C"
3) "wwn:2101001B32AE17DB"
4) "wwn:2100001B329BF124"
5) "wwn:500601694460356C"
6) "wwn:2100001B329DE91C"
7) "wwn:500601684460356C"
8) "wwn:500601614460356C"
9) "wwn:2101001B32AECBDB"
10) "wwn:21000024FF218DDE"

Set keys. Each host entry has one or more wwns associated to it.

127.0.0.1:6379> keys host:*
1) "host:host0068"
2) "host:stor124"
3) "host:host0014"
4) "host:host0115"

Hash keys. Every Port in SAN is captured here.

127.0.0.1:6379> keys swport:*
1) "swport:sw121:slot:1:port:1"
2) "swport:sw122:slot:1:port:3"
3) "swport:sw122:slot:1:port:0"
4) "swport:sw122:slot:1:port:2"
5) "swport:sw121:slot:1:port:2"
6) "swport:sw121:slot:1:port:3"
7) "swport:sw122:slot:1:port:1"
8) "swport:sw121:slot:1:port:0"

Hash keys. Each WWN connected to the SAN with switch/slot/port information.

127.0.0.1:6379> keys wwnconn:*
1) "wwnconn:500601684460356C"
2) "wwnconn:500601614460356C"
3) "wwnconn:500601604460356C"
4) "wwnconn:2101001B32AE17DB"
5) "wwnconn:500601694460356C"
6) "wwnconn:2101001B32AECBDB"
7) "wwnconn:21000024FF218DDE"
8) "wwnconn:21000024FF218DDF"
9) "wwnconn:2100001B329DE91C"
10) "wwnconn:2100001B329BF124"

Set keys. We can have one or more members (wwns) for a given SAN port.

127.0.0.1:6379> keys sanconn:*
1) "sanconn:sw122:slot:1:port:3"
2) "sanconn:sw121:slot:1:port:3"
3) "sanconn:sw122:slot:1:port:0"
4) "sanconn:sw122:slot:1:port:1"
5) "sanconn:sw121:slot:1:port:0"
6) "sanconn:sw122:slot:1:port:2"
7) "sanconn:sw121:slot:1:port:2"
8) "sanconn:sw121:slot:1:port:1"

Reporting Examples

The Switch Report.

Let's create some reports. We'll look at three examples of what can be done easily for reporting. We'll use the keys command in Redis to build our switch list first.

#Get all switch keys from Redis
my (@switches)=$r->keys("switch:*");
 
#We'll grab all the port/keys for a given switch and process.
foreach my $swkey (@switches){
   my (undef,$sw)=split(/\:/, $swkey);
   print "Switch: $sw\n";
 
   #Build the swport key
   my $swpkey="swport".':'."$sw";
 
  #Collect all switch ports from Redis
   my (@switchports)=$r->keys("$swpkey:*");
 
   foreach my $swportkey (@switchports){
      #Pull port details from Redis
      my ($idx,$spd,$state,$type,$phy)=$r->hmget($swportkey, idx, portspd, portstate, porttype, portphys);
      print "   Port: $swportkey - $idx,$spd,$state,$type,$phy\n";
   }
}

The result.

Switch: sw122
   Port: swport:sw122:slot:1:port:3 - 3,N8Gbps,Online,F_Port,In_Sync
   Port: swport:sw122:slot:1:port:0 - 0,N8Gbps,Online,F_Port,In_Sync
   Port: swport:sw122:slot:1:port:2 - 2,N8Gbps,Online,F_Port,In_Sync
   Port: swport:sw122:slot:1:port:1 - 1,N8Gbps,Online,F_Port,In_Sync
Switch: sw121
   Port: swport:sw121:slot:1:port:1 - 1,N8Gbps,Online,F_Port,In_Sync
   Port: swport:sw121:slot:1:port:2 - 2,N8Gbps,Online,F_Port,In_Sync
   Port: swport:sw121:slot:1:port:3 - 3,N8Gbps,Online,F_Port,In_Sync
   Port: swport:sw121:slot:1:port:0 - 0,N8Gbps,Online,F_Port,In_Sync

Now a WWN report.

#Sample WWN report
my (@sanwwns)=$r->keys("wwnconn:*");
foreach my $connkey (@sanwwns){
   my (undef,$wwn)=split(/\:/, $connkey);
   my $wwnkey="wwn".':'."$wwn";
 
   #pull the data from Redis
   my ($host,$adapter,$port,$type)=$r->hmget($wwnkey, host, adapter, port, type);
   my ($sw,$swslot,$swport)=$r->hmget($connkey, switch, slot, port);
 
   print "\nWWN $wwn Information:\n";
   print "  Connection: $sw,$swslot,$swport\n";
   print "     Details: $host,$adapter,$port,$type\n";
}

The result.

WWN 500601684460356C Information:
  Connection: sw121,1,1
     Details: stor124,B,0,storage
 
WWN 500601614460356C Information:
  Connection: sw122,1,0
     Details: stor124,A,1,storage
 
WWN 500601604460356C Information:
  Connection: sw121,1,0
     Details: stor124,A,0,storage

WWN 2101001B32AE17DB Information:
  Connection: sw121,1,2
     Details: host0014,hba3,1,host
 
WWN 500601694460356C Information:
  Connection: sw122,1,1
     Details: stor124,B,1,storage
 
WWN 2101001B32AECBDB Information:
  Connection: sw122,1,2
     Details: host0014,hba4,1,host
 
WWN 21000024FF218DDE Information:
  Connection: sw121,1,2
     Details: host0068,hba1,1,host
 
WWN 21000024FF218DDF Information:
  Connection: sw122,1,2
     Details: host0068,hba2,1,host
 
WWN 2100001B329DE91C Information:
  Connection: sw122,1,3
     Details: host0115,hba2,1,host
 
WWN 2100001B329BF124 Information:
  Connection: sw121,1,3
     Details: host0115,hba1,1,host
 

A Device Report.

#Sample device report
my (@hostkeys)=$r->keys("host:*");
foreach my $hostkey (@hostkeys){
   my(undef,$host)=split(/\:/,$hostkey);
   print "\nDEVICE $host Information:\n";
   my (@hostwwns)=$r->smembers($hostkey);
   foreach my $wwn (@hostwwns){
      my $wwnkey="wwn".':'."$wwn";
      my $connkey="wwnconn".':'."$wwn";
 
      #pull the data from Redis
      my ($host,$adapter,$port,$type)=$r->hmget($wwnkey, host, adapter, port, type);
      my ($sw,$swslot,$swport)=$r->hmget($connkey, switch, slot, port);
 
      $type=uc($type);
      print "   $type WWN:$wwn Adapter:$adapter PORT:$port connected to $sw slot:$swslot port:$swport\n";
   }
}

The result.

DEVICE host0068 Information:
   HOST WWN:21000024FF218DDE Adapter:hba1 PORT:1 connected to sw121 slot:1 port:2
   HOST WWN:21000024FF218DDF Adapter:hba2 PORT:1 connected to sw122 slot:1 port:2
 
DEVICE stor124 Information:
   STORAGE WWN:500601614460356C Adapter:A PORT:1 connected to sw122 slot:1 port:0
   STORAGE WWN:500601604460356C Adapter:A PORT:0 connected to sw121 slot:1 port:0
   STORAGE WWN:500601694460356C Adapter:B PORT:1 connected to sw122 slot:1 port:1
   STORAGE WWN:500601684460356C Adapter:B PORT:0 connected to sw121 slot:1 port:1
 
DEVICE host0014 Information:
   HOST WWN:2101001B32AECBDB Adapter:hba4 PORT:1 connected to sw122 slot:1 port:2
   HOST WWN:2101001B32AE17DB Adapter:hba3 PORT:1 connected to sw121 slot:1 port:2
 
DEVICE host0115 Information:
   HOST WWN:2100001B329DE91C Adapter:hba2 PORT:1 connected to sw122 slot:1 port:3
   HOST WWN:2100001B329BF124 Adapter:hba1 PORT:1 connected to sw121 slot:1 port:3

So with just a minimal amount of coding you can create very nice set of data and reports about your SAN environment. The sky is the limit.

Comments

There are no comments on this post.

Recent Posts

Archives


Contact Cecil