I've got my hands on some old SC gigabit fibre kit:
2x Netgear GA620 64bit PCI
1x Allied Telesyn 9006SX/SC
1x Brocade Silkworm 2400
The Brocade is technically a fibre channel switch for disk arrays, SANs.
It's all about 7 years old now, but worth playing with. I've now got a machine to machine fibre link between two of my servers, and will be looking on eBay for another fibre card to extend that network with the 9006 switch.
The cards use the acenic driver in the Linux kernel. This has apparently been removed in 2.6.23, so I may be stuck at .22... I have found one flaw: with flow control enabled in the cards, they will eventually lock up and stop transmitting data. I don't know if this is the bug which led to driver removal, or a hardware issue.
The fix is to switch off flow control, module options:
options acenic link=0x0140
If compiled as a module and using Gentoo, create a file /etc/modprobe.d/acenic containing that line and run 'update-modules force'. rmmod then 'modprobe -v acenic' to confirm that it's read the line correctly (it'll show the "link" bit on the screen).
The cards support jumbo frames, so it your switch does as well you should turn that on. In Gentoo, /etc/conf.d/net:
mtu_eth4="9000"
So far, performance isn't great - 120Mbit/s is the record so far. This may be due to the vastly overloaded machines I use and one of them running in 32bit PCI mode. Flow control on/off hasn't made a difference. I have some tweaking left to do.
Of course, having put these cards in has disturbed the machines and the recent perfect uptime has disappeared... multiple hangs over the last few days, very frustrating when the machines should be perfect. Most traced so far to bad disk power connections causing disk dropouts, except for one happening while dumping data to SCSI tape which ruined a backup, which has happened a number of times in the distant past. sigh.
