lördag 26 april 2014

Using IBM ServeRAID M1015 card in Linux

The FusionMPT SAS2 based cards by LSI are cheap RAID cards that perform well in RAID 0 and 1 under Linux with SSDs and ZFS arrays. They are used in many 1U servers with one or two disks. They are also a popular SAS 6.0 / SATA III controllers for enthusiasts. You can get them cheap on ebay. Before SATA III was common on motherboards, this was THE card to get. It can still be useful if you need SAS disks or additional disks or have a need for disk enclosures.

It is identical to the LSI MegaRAID SAS 9220-8i. The same hardware is also used in LSI 9240 and LSI 9211. It is possible to use the BIOS from another card (crossflashing) to change the features of the card. There are a few different BIOS files you can use.

Specifications: 


Official LSI information
  • 8-lane, 5 GT/s PCI Express 2.0
  • Identical to LSI 9240-8i card
  • 6 Gb/s per port
  • Two x4 internal SFF-8087 connectors
  • Controller LSISAS2008
  • Low cost SATA+SAS RAID solution
  • RAID levels 0, 1, 5, 10, 50 and JBOD mode
  • >2TB logical drive support
  • S.M.A.R.T. support
Low-profile, 8 internal port 6Gb/s SATA+SAS RAID card with x8 PCIe 2.0 host interface and a full complement of data protection features.

Crossflashing


The BIOS flashing procedure is well explained on the following page:
http://www.servethehome.com/ibm-m1015-part-1-started-lsi-92208i/

I chose to flash the card to LSI9211-IR. This way I can use pass through for single drives and software RAID, and additionally use the RAID features of the card (RAID 0, RAID 1, RAID 1e and RAID 10).

To flash the card, read the excellent article above, but in short you have to boot with an USB stick with the correct firmware and and issue:

megarec -writesbr 0 sbrempty.bin
megarec -cleanflash 0
<reboot, back to USB stick>
sas2flsh -o -f 2118ir.bin -b mptsas2.rom
sas2flsh -o -sasadd 500605b0xxxxxxxx (x= numbers for SAS address)
<reboot>

The SAS address is found on the back of the card, on the green sticker.

In Linux

The card works out of the box in Linux. It is using the mpt2sas kernel driver.

[root@localhost ~]# lspci|grep LSI
02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

Hard disks that use passthrough will be visible to smartmontools.

[root@localhost ~]# smartctl -a /dev/sdb

To show disks in RAID arrays you have to load the generic sg SCSI driver:

[root@localhost ~]# modprobe sg


Then you can add the disk to smartd:

echo "/dev/sg4 -d scsi -a -s L/../../3/02" >> /etc/smartmontools/smartd.conf

To keep the sg driver loaded over reboots, issue (on Fedora):

[root@localhost ~]# echo sq >> /etc/modules-load.d/sg.conf

Linux Tools

Because this is a low-end card, MegaCLI and similar tools will not work with the Fusion-MPT SAS-2. LSI provides a command line tool with basic funtionality, sas2ircu. The tool is hard to find, but at least supermicro provides a link for it. The tool is mostly useful for monitoring.

Simple usage:

[root@localhost ~]# sas2ircu LIST
 
LSI Corporation SAS2 IR Configuration Utility.
Version 16.00.00.00 (2013.03.01) 
Copyright (c) 2009-2013 LSI Corporation. All rights reserved. 


         Adapter      Vendor  Device                       SubSys  SubSys 
 Index    Type          ID      ID    Pci Address          Ven ID  Dev ID 
 -----  ------------  ------  ------  -----------------    ------  ------ 
   0     SAS2008     1000h    72h   00h:02h:00h:00h      1000h   3020h 
SAS2IRCU: Utility Completed Successfully.
 
 
[root@localhost ~]# sas2ircu 0 DISPLAY 
 
...
------------------------------------------------------------------------
IR Volume information
------------------------------------------------------------------------
IR volume 1
  Volume ID                               : 286
  Status of volume                        : Okay (OKY)
  Volume wwid                             : 0c620c8063bd337c
  RAID level                              : RAID1
  Size (in MB)                            : 428199
  Boot                                    : Primary
  Physical hard disks                     :
  PHY[0] Enclosure#/Slot#                 : 1:0
  PHY[1] Enclosure#/Slot#                 : 1:1
...

There is a nice script, sas2ircu-status, at HWRAID. The script can be used with nagios to report the status of the controller and RAID arrays.

I made a cron script to regularly check the status of the controller and send a mail if there is something wrong. It is used together with the sas2ircu-status script. You can download it here.