Tag Archives: Thunderbolt

Questioning my Xsan-ity

My employer has an extremely talented group of folks working on the company websites and videos. When I was designing our network infrastructure and storage upgrade, I wanted to make things easier and more performant for them if we could do so reasonably.

They are exclusively Apple users, including Final Cut Pro for video editing. Apple marches to the beat of their own drummer most of the time and has never been a “best practice” in the enterprise space. That has led to a lack of robust solutions and a dearth of supported functionality from other vendors. Apple’s own efforts have frequently been failures. Apple’s most promising start was abandoned after a few years leaving those who had invested in it with no good replacement options.

Final Cut Pro does NOT like video stored on network shares. In the past we’ve implemented workarounds and hacks in an attempt to get around that limitation but none of them worked well. It does seem to play well with Apple’s Xsan shared storage file system implementation.

There is little information to be found on teh webz regarding Xsan installations and most of what I did find was outdated or applicable to some other solution.

The SAN solution I was already building for my employer consisted of a disaster recovery filer from NetApp located physically near the video team’s offices. Offsite disaster recovery and video work very well together. The slower spindle, large capacity drives typically used for storing video work great for SnapMirror & SnapRestore too.  The filer utilization for video would primarily be during business hours while the backup operation would be after hours to minimize the impact on the WAN links. It certainly seemed like a good fit and we decided to try and make it work.

ATTO cards come highly recommended for Apple use so we installed FC-82EN cards in their Mac Pros and purchased a Mac Mini server with ATTO’s Thunderlink FC 1082 for use as a file system meta-data controller. Xsan is based on the StorNext file system from Quantum which allows for simultaneous volume read & write by multiple clients and requires a meta-data controller. If the tests were successful I was planning on getting a couple more mini’s to act as failover controllers.

Racked and cabled the NetApp FAS-2240 and disk shelf. Racked the 2 x Cisco Nexus 5548UP and installed the license for Fibre Channel on both. So far so good.

I decided to work out the bugs with the Xsan installation that I assumed would crop up before doing anything else with the filer:

We ran pre-molded 30 meter LC patch cables from each of the Mac Pro offices to the rack room and hooked it up to the fabric.

Configured a vsan, zone and zoneset and tested all interfaces with fcping. From the Nexus fcping worked great, all ports showed up and the fcns showed all of the WWPNs.

Created a test LUN on the filer and added all the HBA WWPNs into the initiator group that was bound to that LUN.

Nothing. Nada. Zilch.

OK, that’s not unexpected. I probably overlooked something obvious.

First thing I found was the filer’s FC interfaces were in initiator mode instead of target mode. Fixed that. Still nothing.


Tried fcp ping from the filer console… ah hah! Nothing. That’s certainly a problem. I could still fcping all WWPNs successfully from the Nexus console though.

Double-checked everything. Rebuilt everything from scratch. Rebuilt it from scratch again. Triple-checked everything.

Nothing at all.

I’m not an NX-OS guru. This was also my first foray into Fibre Channel [because $$$]. So I assumed I just missed something fundamental in the documentation and started opening tickets.

Now, I know this is NOT a supported configuration for any of the vendors and I don’t expect them to replace hardware until the problem is fixed. Cisco has been helpful as have the techs from NetApp and ATTO. For 3 weeks!

A NetApp technician suggested I plug one of the HBAs directly into the NetApp fiber interface: LUN pops up instantly on the Mac. [How did I NOT think of that?]

None of the diagnostics or logging that we tried with the Cisco Nexus helped.

I purchased a [relatively] inexpensive QLogic SANbox 3810 bundled with 8 8Gbps SFPs and cabled it up in place of the Cisco Nexus 5548UP and everything worked flawlessly.

So what are the final results?

Speed and usability is significantly improved.