Exadata Security: the joy of PXE and UEFI: Secure boot violation

As usual, Andy was right. Trying to boot one of the cells on the pxe “the old method” resulted in:

“ Secure boot violation: Invalid signature detected. Check Secure boot policy in setup ”

So … we now knew we had some work to do.

Step 1, find your sole source of truth … my oracle support. This time, close but no cigar. The MOS-note “How to setup a PXE Boot Server to Re-Image an Exadata Compute Node (Doc ID 1577323.1)” was not complete. It is lacking the information on how to do a PXE boot with an UEFI system.

It is not difficult, at least once you know it.

A very valuable source of information, is actually the online documentation on how to setup a PXE server for a Oracle linux 7 system. You can find that one here: https://docs.oracle.com/cd/E52668_01/E54695/html/ol7-install-pxe-dhcp-tftp.html

Because it’s a bit long, I’ll highlight the steps we had to take to convert our PXE server, which we normally use and is built using the mos-note guidelines, into one who can support the UEFI boot.

Packages

First of all, you need some more required packages: BOOTX64.efi, grubx64.efi, shim.efi

and also, there are some optional ones as well:MokManager.efi

These rpms can be found in the online yum repo from oracle, or in the ISO files from oracle linux.

# cd /tempdir # rpm2cpio grub2-efi-version.rpm | cpio -idmv # rpm2cpio shim-version.rpm | cpio -idmv

Of course, in the datacenter you don’t have internet access. No problem for that, these files are in the PXE images you have downloaded from edelivery as well.

The grubx64, BOOTX64.efi and MokManager.efi are located in the nfsimg-18.xxxx.tar file.

You can get them out like this:

# tar xf nfsimg-18.<fill in your version>-cell.tar EFI # cd EFI/BOOT # cp BOOTX64.efi grubx64.efi MokManager.efi grub.cfg /tftpboot/efi

the shim is a little trickier, you get it from the cellbits:

# tar xf nfsimg-<fill in your version>-cell.tar cellbits/commonos.tbz # cd cellbits # tar xf commonos.tbz shim-0.9-2.0.6.2.el6.x86_64.rpm # rpm2cpioshim-0.9-2.0.6.2.el6.x86_64.rpm | cpio -idmv # cp ./boot/efi/EFI/redhat/shim.efi /tftpboot/efi/

that’s all for the extra package requirements.

dhcpd.conf

Yes, this one needs some love and attention as well.

In the general section on top following info must be added:

set vendorclass = option vendor-class-identifier; option pxe-system-type code 93 = unsigned integer 16; set pxetype = option pxe-system-type;

And in the subnet section you need an if-clause:

# Enable UEFI Netboot if substring(vendorclass, 0, 9)="PXEClient" { if pxetype=00:06 or pxetype=00:07 { filename "shim.efi"; } else { filename "pxelinux.0"; }

take into account, that this is the way on my system. This is a relative path. Just keep that in mind it can be different for you.

Grub-files

In a normal pxe linux boot you need to create the files in pxelinux.cfg/01-<mac address lower case and instead of : >

Well, this is a little different here as well.The name of the file it’s grub.cfg-01-“MAC ADDRESS with separator” and in my case, it was also expecting an at the end.

So we have two components, components and storage cells.

Compute nodes

They get this as content for the 01-<mac address> file for the compute node:

set default 0 set timeout=3 menuentry 'Compute Node' { echo "Loading vmlinux" linuxefi efi/vmlinux-nfs-<fill in the correct version for this file>-compute stit dhcp pxe boot-from=uefi factory reboot-on-success notests=diskgroup sk=<change to the pxe servers ip>:/tftpboot/ preconf=<change to the pxe servers ip>:/tftpboot/customerconfigs/preconf.csv console=ttyS0,115200n8 echo "Loading initrd" initrdefi efi/initrd-nfs-<fill in the correct version for this file>-compute.img echo "Booting install kernel" } Storage cell nodes

They get this as content for the 01-<mac address> file for the storage cell node:

set default 0 set timeout=10 menuentry 'Storage cell' { echo "Loading vmlinux" linuxefi efi/vmlinux-nfs-<fill in the correct version for this file>-cell stit dhcp pxe boot-from=uefi factory reboot-on-success notests=diskgroup sk=<change to the pxe servers ip>:/u01/exadata_nfs preconf=<change to the pxe servers ip>:/tftpboot/customerconfigs/preconf.csv console=ttyS0,115200n8 echo "Loading initrd" initrdefi efi/initrd-nfs-<fill in the correct version for this file>-cell.img echo "Booting install kernel" } Software location

As this is a first version, I will definitely change it to clean it up, the system expects the files in /tftboot. So move the files vmlinux-nfs* and initrd-nfs-* to /tftpboot/ and make sure to respect the proper permissions.

Restart services

As the configuration from the dhcp server has been changed, this service must be restarted. This can be done using

# service dhcpd restart

And basically, that’s it folks. No other changes are needed. Set the bootdevice from the cell/compute to pxe and restart the components. They will be happy to boot from your pxe server.

Disclaimer: this works for me, and this is not intended as a copy/paste procedure. Also, make sure that you are comfortable with knowing what you are doing or changing. If in doubt about anything, seek some help and if necessary ask Oracle or an installation partner to assist you in this.

As always, questions, remarks? find me on twitter @vanpupi