[xcat-user] Jumbo frames & NFS

[Peter McLachlan]

Thanks for the note - I tried the aliasing, as it turns out an aliased 
interface can't have a different MTU however a VLAN'd interface using the 
802.1q code can have a different MTU. 

One possible workaround would be to set up 2 VLAN's on the switch, and 
change the interface from one VLAN to another when it was time to perform 
the imaging operation.  This would be cumbersome to perform manually, 
might be able to automate it with some expect scripts. 

Adding another NIC would be a lot easier and might be the simplest 
solution.  I've confirmed that dropping the MTU on the management node to 
1500 allows a successful node install to take place. 

Thanks,

Peter McLachlan
I/T Specialist - Redhat Certified Engineer (RHCE)
Linux Solutions
IBM Global Services
Office: (905) 316-8646
e-mail:  pmclachl at ca.ibm.com




"Egan Ford" <egan at sense.net>
04/24/2003 11:15 AM
Please respond to xcat-user
 
        To:     <xcat-user at lists.xcat.org>
        cc: 
        Subject:        RE: [xcat-user] Jumbo frames & NFS

 

I haven't tried this, but can an aliased NIC have a different mtu?
 
Use 9000 on eth0 and 1500 on eth0.1?
 
Other option is to add another NIC to the management node, just for 
installs.
-----Original Message-----
From: Peter McLachlan [mailto:pmclachl at ca.ibm.com] 
Sent: Wednesday, April 23, 2003 8:47 AM
To: xcat-user at lists.xcat.org
Subject: RE: [xcat-user] Jumbo frames & NFS


Thanks for the response - this one is still stumping me.  The redhat 
install image initrd does not have a /etc/modules.conf - so I created one, 
with the right parameters for mtu 9000 on the bcm5700 driver with no luck. 
 I also tried passing the dhcp server option to set the mtu 9000 with no 
luck, chances are the dhcp client that comes with the redhat install 
initrd doesn't implement this option.   

So an open question:  Anyone out there who is running jumbo frames, how do 
you re-image your nodes?   

I'm currently looking at dropping the mtu on the management node interface 
back to 1500 while re-imaging is taking place and then boosting it back up 
to 9000  when the imaging is done while leaving  the switch set to mtu 
9000.  (Its a big pain to change mtu settings on this switch - it requires 
a reboot which of course would disrupt any other cluster activities just 
to re-image a few nodes.  Yuck.)  Although cludge-y this will hopefully 
work.   

Regards, 

Peter McLachlan
I/T Specialist - Redhat Certified Engineer (RHCE)
Linux Solutions
IBM Global Services
Office: (905) 316-8646
e-mail:  pmclachl at ca.ibm.com 



Pellegrini Nicholas S Contr ASC/HPTI <Nicholas.Pellegrini at wpafb.af.mil> 
04/16/2003 04:23 PM 
Please respond to xcat-user 
        
        To:        "'xcat-user at lists.xcat.org'" <xcat-user at lists.xcat.org> 

        cc:         
        Subject:        RE: [xcat-user] Jumbo frames & NFS 

 


We've also experienced weirdness with NFS using jumbo frames if the MTU is 
not set properly.  I looked up some docs on the Broadcom GigE card driver 
(bcm5700).  In the parameters section, "mtu" is obviously used for the 
parameter.  However, it continues to say:  "The valid range is from 1500 
to 8184."  If this is an old doc, it may not be entirely accurate... but 
something to keep in mind if problems persist. 
  
According to 'man mkinitrd', one should be able to put the option for mtu 
9000 in /etc/modules.conf.  When you make a new initrd image, it says "Any 
module options specified in /etc/modules.conf are passed to the modules as 
they are loaded by the initial ramdisk."  Could also use the 
"--preload=module" if needed. 
  
-Nick 
  
 -----Original Message-----
From: Peter McLachlan [mailto:pmclachl at ca.ibm.com]
Sent: Wednesday, April 16, 2003 3:52 PM
To: xcat-user at lists.xcat.org
Subject: [xcat-user] Jumbo frames & NFS


The "short" question: 

How can I pass module parameters to modules in the initrd image that is 
loaded on pxe boot when imaging a node?  In particular I need to pass mtu 
9000 specification to the bcm5700 module.   

I suspect (if it can be done) it is done either by passing parameters to 
the kernel image in the bootloader, or by modifying the file module-info 
file on the modules/ subdirectory of the redhat initrd. 

The "detailed" background: 

I've run into a "small" problem involving jumbo frames and NFS.  NFS seems 
to be more sensitive to MTU mismatches than other services.  Here's the 
summary: 

1) The switch is configured for MTU 9000 on all ports. 
2) The masternode (tftp server, and nfs server for both kickstart and 
installation rpms) is configured for mtu 9000 
3) The nodes were initially imaged when the system was set to mtu 1500 
4) The node will now PXE boot correctly, download the kernel image and 
initrd over tftp, and boot 
5) The problem: NFS will timeout trying to access the ks73.ks file 

Reasons I believe this to be an MTU issue: 
1) We are trunking between 2 switches, the trunks were initially not 
configured correctly and some of the slave connections were at MTU 1500. 
NFS would timeout from nodes that had to traverse the trunk even though 
other services (ssh, tftp, dhcp etc.)  worked fine.  Fixing the MTU on the 
trunk eliminated the exact same NFS timeout message as we are now 
experiencing.   
2) This worked fine before we raised MTU to 9000 on the switches.   

Any thoughts, suggestions etc. are much appreciated, 

Peter McLachlan
I/T Specialist - Redhat Certified Engineer (RHCE)
Linux Solutions
IBM Global Services
Office: (905) 316-8646
e-mail:  pmclachl at ca.ibm.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.xcat.org/pipermail/xcat-user/attachments/20030424/da78acf2/attachment.htm