IPXE

What is PXE?

Abbreviation Means

Preboot Execution Environment (PXE)

Alternate Resources

A good resource to use to gain a great understand of exactly what PXE is would be to click Understanding PXE Booting

Summary

The PXE protocol is, approximately, a combination of DHCP and TFTP working together to provide a boot environment over networking means. Subtle modifications to both DHCP and TFTP during the PXE Boot environment are made. DHCP is used to locate the appropriate boot server. TFTP is used to download the initial bootstrap program and/or additional files as needed.

To initiate a PXE bootstrap session the PXE firmware broadcasts a DHCPDISCOVER packet extended with PXE-specific options (extended DHCPDISCOVER) to port 67/UDP (DHCP server port). The PXE options identify the firmware as capable of PXE, but they will be ignored by standard DHCP servers. If the firmware receives DHCPOFFERs from such servers, it may configure itself by requesting one of the offered configurations.

These are normally portrayed with:

Linux DHCP (ISC|DHCP|DHCP3)

Server Location

next-server

File to Download

filename "<FILENAME TO RECIEVE>"

Windows DHCP/AD

Server Location

Option 66

File to Download

Option 67

DNSMasq/proxyDHCP

Item's in reference are bolded to help standout. The 3 fields are not needed, just used here to represent typical usage. You can set dhcp-boot with only one field (the boot filename) or two (bootfilename and server).

Server Location

dhcp-boot=pxelinux.0,fogserver,10.0.0.10

File to Download

dhcp-boot=pxelinux.0,fogserver,10.0.0.10

What is iPXE?

Formerly gPXE project, iPXE is an open source PXE implementation and bootloader. It can be used to enable computers without built-in PXE support to boot from the network, or to extend an existing PXE implementation with support for additional protocols. While traditional PXE clients use TFTP to transfer data, iPXE adds the ability to retrieve data through other protocols like HTTP, iSCSI, ATA over Ethernet (AoE), and Fibre Channel over Ethernet (FCoE), and can work with Wi-Fi rather than requiring a wired connection.

• YES, (technically) wireless imaging is supported but largely untested due to the absence in most BIOS setups.

What are the differences between the different PXE files?

Filenames

• ipxe has drivers native to the ipxe project. Those drivers are handled from the iPXE developers.
• undionly uses the "undi" stack made by the manufacturer of the NIC.

Universal Network Device Interface (UNDI) is an application programming interface (API) for network interface cards (NIC) used by the Preboot Execution Environment (PXE) protocol.

When chainloading iPXE from PXE, iPXE can use this API (instead of loading a hardware driver). This way, you're getting support for network controllers that are not natively supported by iPXE. Some network controllers have improved performance when using the UNDI driver over the vendor specific iPXE driver.

To use the UNDI driver, select the UNDI driver (undionly) when generating the iPXE ROM. (e.g. make bin/undionly.kpxe EMBED=embedscriptname) Reference without edits here.

Extensions

More info can be referenced here: [1]

1. .pxe is an image designed to be chain loaded, unloading both the underlying PXE and UNDI code sections. This is ultimately the goal, but there's not enough information to allow this to actually work flawlessly every time. It uses, purely, the drivers from the iPXE information. One of the benefits is the codebase for the drivers are handled by the iPXE developers. So, in theory and given enough time, all NICs could potentially be supported.
   • .pxe is an image designed to be chain loaded, unloading both the underlying PXE and UNDI code sections. etherboot.org
2. .kpxe unloads just the pxe stack and is the normal file we want in use as it seems to be the best between pxe/chaining I can find without flashing roms.
   • .kpxe is a PXE image that keeps UNDI loaded and unloads PXE etherboot.org
3. .kkpxe keeps both undi and pxe stacks in place. kkpxe works best for buggy hardware. Only recommended if you're having weird issues with the undionly.kpxe
   • .kkpxe is a PXE image that keeps PXE+UNDI loaded and return to PXE (instead of int 18h). From here etherboot.org
4. .kkkpxe is only used to generate the ipxelinux.0 file. This is only used in conjunction with the syslinux project. When gpxe was the developed software of this type the file was called gpxelinux.0 which can usually be built with modern syslinux.

More information on this can be found on the ipxe forum thread located here.

Undi and iPXE Stack

More information on differences and when to use are located here. The UNDI driver is a generic driver that works on network cards that have a vendor UNDI ROM. The ROM contains driver code that is supposed to conform to the PXE/UNDI specification. iPXE can load the UNDI driver and use it instead of a native driver.

Depending on the iPXE image type, UNDI support works as follows:

• undionly.kpxe is loaded from a vendor PXE stack and uses UNDI on the network card that it was booted from.

• All-driver (ipxe) or undi images can load the UNDI for PCI network cards. The network boot ROM must be enabled in the BIOS in order for the UNDI ROM to be visible to iPXE. Note that only the first network card is supported with UNDI since multiple instances of UNDI is unreliable and cannot be supported.

Why write native drivers if UNDI works with every network card?

• iPXE is an open source PXE stack and provides UNDI services. iPXE cannot be used as an option ROM without a native driver.

• UNDI is slow because iPXE must switch CPU modes when calling it.

• UNDI ROMs can be buggy or violate the PXE specification. Native drivers are known to work with iPXE and can be fixed if there is a bug since they are part of the iPXE codebase.

• Enabling the network boot ROM in the BIOS is not always possible or desirable.

---

STP/Portfast/RSTP/MSTP To Enable or Disable?

STP

• What is STP?
  • The Spanning Tree Protocol (STP) is a network protocol that ensures a loop-free topology for any bridged Ethernet local area network. The basic function of STP is to prevent bridge loops and the broadcast radiation that results from them. Spanning tree also allows a network design to include spare (redundant) links to provide automatic backup paths if an active link fails, without the danger of bridge loops, or the need for manual enabling/disabling of these backup links.

Port Fast

• What is Portfast?
  • The time Spanning Tree Protocol (STP) takes to transition ports over to the Forwarding state can cause problems. PortFast is a Cisco network function which can be configured to resolve this problem. This factor of time is not an issue for many people, but it can cause problems for some. (i.e. Fog imaging) You may see this issue is with Pre-Boot Execution (PXE) devices, such as Windows Deployment Services. PortFast is the solution to this problem of delays when client computers are connecting to switches. PortFast is not enabled by default. With PortFast enabled on a port, you effectively take the port and tell spanning tree not to implement STP on that port.

RSTP

• What is Rapid STP(RSTP)?
  • The 802.1D Spanning Tree Protocol (STP) standard was designed at a time when the recovery of connectivity after an outage within a minute or so was considered adequate performance. With the advent of Layer 3 switching in LAN environments, bridging now competes with routed solutions where protocols, such as Open Shortest Path First (OSPF) and Enhanced Interior Gateway Routing Protocol (EIGRP), are able to provide an alternate path in less time. Cisco enhanced the original 802.1D specification with features such as Uplink Fast, Backbone Fast, and Port Fast to speed up the convergence time of a bridged network. The drawback is that these mechanisms are proprietary and need additional configuration. Rapid Spanning Tree Protocol (RSTP; IEEE 802.1w) can be seen as an evolution of the 802.1D standard more than a revolution. The 802.1D terminology remains primarily the same. Most parameters have been left unchanged so users familiar with 802.1D can rapidly configure the new protocol comfortably. In most cases, RSTP performs better than proprietary extensions of Cisco without any additional configuration. 802.1w can also revert back to 802.1D in order to interoperate with legacy bridges on a per-port basis. This drops the benefits it introduces.

MSTP

• What is Multiple STP (MSTP)?
  • The Multiple Spanning Tree Protocol (MSTP), originally defined in IEEE 802.1s and later merged into IEEE 802.1Q-2005, defines an extension to RSTP to further develop the usefulness of virtual LANs (VLANs). This Multiple Spanning Tree Protocol configures a separate Spanning Tree for each VLAN group and blocks all but one of the possible alternate paths within each Spanning Tree. If there is only one Virtual LAN (VLAN) in the network, single (traditional) STP works appropriately. If the network contains more than one VLAN, the logical network configured by single STP would work, but it is possible to make better use of the alternate paths available by using an alternate spanning tree for different VLANs or groups of VLANs.

What do I enable and disable?

• If you don't need STP all these options should be disabled already and nothing should need to be done. (DISABLE ALL)
• If you have to use STP, to get (ipxe/dhcp) Fog (v1.x.x) working correctly you will need to ENABLE PORTFAST OR ENABLE RSTP.
• Currently MSTP is untested with Fog but may be useful for networks with multiple VLANS.

More information on Spanning Tree Protocol

http://en.wikipedia.org/wiki/Spanning_Tree_Protocol#Multiple_Spanning_Tree_Protocol

Compile

FOG is using the most current iPXE source code to build many different PXE binaries, some being undionly and others specific for NICs made by intel or realtek - BIOS and UEFI compatible. But still you might want to build your own binary to suit your needs (be it a custom script or debugging enabled). Here you'll find some hints on how to build your own iPXE binary.

Prerequisites

To be able to build iPXE from source you need tools to checkout and compile source code.

debian/ubuntu# sudo apt-get install git build-essential zlib1g-dev binutils-dev
fedora/centos# sudo yum install git gcc gcc-c++ make zlib-devel binutils-devel    # this is untested so far as I don't have a system handy

Then checkout the code and download Tom's config header files from FOG SVN. The header files need to be a little different for BIOS and UEFI and therefore I usually checkout the source twice to have one ready for each platform.

mkdir ~/projects/ipxe
cd ~/projects/ipxe
git clone git://git.ipxe.org/ipxe.git ipxe-bios
cd ipxe-bios/src/config
rm console.h general.h settings.h
wget -O console.h "http://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe/src/config/console.h?format=raw"
wget -O general.h "http://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe/src/config/general.h?format=raw"
wget -O settings.h "http://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe/src/config/settings.h?format=raw"
cd ..
wget -O ipxescript "http://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe/src/ipxescript?format=raw"

cd ~/projects/ipxe
git clone git://git.ipxe.org/ipxe.git ipxe-efi
cd ipxe-efi/src/config
rm console.h general.h settings.h
wget -O console.h "http://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe-efi/src/config/console.h?format=raw"
wget -O general.h "http://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe-efi/src/config/general.h?format=raw"
wget -O settings.h "http://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe-efi/src/config/settings.h?format=raw"
cd ..
wget -O ipxescript "http://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe-efi/src/ipxescript?format=raw"

Bake the cake

Now you are ready to build your (first) binary from source. But how do you do that? One simple call but it can be heavily customized with parameters.

# Build a simple BIOS binaries including an embedded script (executed right when iPXE comes up)
cd ~/projects/ipxe/ipxe-bios/src
make bin/undionly.kpxe EMBED=ipxescript
make bin/ipxe.pxe EMBED=ipxescript
make bin/undionly.kkpxe EMBED=ipxescript
make bin/intel.pxe EMBED=ipxescript
...

# simple 32 bit EFI binaries with embedded script
cd ~/projects/ipxe/ipxe-efi/src
make bin-i386-efi/ipxe.efi EMBED=ipxescript
make bin-i386-efi/snponly.efi EMBED=ipxescript
make bin-i386-efi/intel.efi EMBED=ipxescript
...

# simple 64 bit EFI binaries
cd ~/projects/ipxe/ipxe-efi/src
make bin-x86_64-efi/ipxe.efi EMBED=ipxescript
make bin-x86_64-efi/snponly.efi EMBED=ipxescript
make bin-x86_64-efi/intel.efi EMBED=ipxescript
...

Build whichever binary you want to have. Now we are getting to the interesting part of adding debug output to iPXE to be able to better find issues. Each and every c-file in the iPXE source can be compiled with debug enabled. Here is an example:

make bin/realtek.kpxe EMBED=ipxescript DEBUG=realtek

Most of the native drivers consist of just one source file. Have a look at src/drivers/net to see them - 3c509, bnx2, forcedeth, intel, pcnet32, realtek, rhine and many more.

The most commonly used binaries ipxe.pxe and ipxe.efi include UNDI interface as well as all the native drivers. You can add debugging selectively. Check out the source code. Here are some more examples:

make ... DEBUG=dhcp
make ... DEBUG=device,efi_driver,efi_init,efi_pci,efi_snp
make ... DEBUG=snp,snponly,snpnet,netdevice
make ... DEBUG=intel:4
make ... DEBUG=undi
...