Only the first run of boards had a ksz switch on it, so if building for a
newer silicon rev or SPI is disabled, don't bother checking for the ksz.

Signed-off-by: Mike Frysinger <vapier@gentoo.org>