Thursday, June 11, 2009

How to create a bootable USB flash drive using grub

from http://www.marwanshaher.com/USB/index.html

Requirements

# A computer with a BIOS that allows booting from a USB port.
The computer MUST have an option to boot off of a USB device. I can't help you identify whether your computer has this capability or not. Please refer to your system documentation for this
I used a Dell Precision Workstation 450 Series that has a Phoenix ROM BIOS Plus version 1.10 revision A03.
# A running Linux installation
This linux installation should have programs and utilities to partition your USB flash drive and create file systems, like fdisk, mkfs, etc.. . These programs usually are part of any linux distribution. I used Fedora 9
# The Boot Loader GNU GRUB
This is the boot loader that loads my linux and Windows installations on my computer. I kind of went about doing this the hard way by chosing GNU GRUB to be the boot loader on my USB flash drive, since almost all of the live CD's out there use isolinux as the boot loader. I like using GRUB but I'll also have similar instructions using isolinux shortly.
# The distro that you want to put on and boot from your USB flash drive
In this document, I'll be using the terms "distro', "Linux distribution" or "operating system" interchangeably. I'll use the terms to describe what you are trying to boot into from your USB thumb drive. This can be a copy of a CD/DVD (iso) image if you don't have a CD/DVD ROM that contains the installation files of your favorite distro, or a working OS that runs either from your flash drive or from memory (ramdisk). There is no need to reinvent the wheel here, so the easiest way is to download any of the Linux Live CD's that are available for the different distros and work with those. I used Slax Linux and BackTrack 3, which is based on Slax, and GParted, which is based on Unbuntu. I'm working on getting Helix, which is based on ubuntu, to work and I'm almost there. I'll include the instructions on how to get these distros to boot off of the flash drive in this document
# Of course, the USB drive that you want to make bootable
This drive needs to be big enough to save the distros/operating systems that you want to boot to plus the boot loader files. I used a Patriot Xporter XT 32GB Flash Drive

Summary
# Basically, creating the bootable USB flash drive consists of the following tasks: Partitioning and creating file system(s) on the USB flash drive.
# Copying the boot loader to the master boot record of the USB flash drive.
# Downloading and copying the distro of your choice to the USB flash drive.
# Configuring the boot loader to load the distro of your choice.
Directions

1. Partitioning and creating file system(s) on the USB flash drive.
My Patriot Xporter XT 32GB Flash Drive came with a single 32GB FAT32 partition. For this project, I wanted to change the way the Flash Drive was carved out to the following:
* - One small active partition where the boot loader files are kept. This is formatted with ext3
* - Another patition where I would keep all my linux OS's that I wanted to boot up. This is formatted with ext3
* - A third partition that I can use to put files that I can access from Linux and Windows based computers. This has to be a FAT32 partition since it's the only file system type supported by both operating systems.
A visual representation of this setup looks something like this

--------------------------------------------------
| 5MB | 16GB | 15GB |
| ext3 | ext3 | FAT32 |
--------------------------------------------------

After I created and formatted the partitions the way they are shown above, I couldn't get Windows XP to recognize, assign a letter, or access the FAT32 partition. This is an issue I'm currently working on resolving. I'll post more details on this as soon as I figure out the solution.
It's up to you how you want your flash drive partitioned. The partition where I planned on saving the boot loader files to didn't have to be on its own. The whole drive could be one big partition. I wanted the boot loader files to be on its own partition because I thought that was a much cleaner setup. Besides, depending on the distro used, the boot loader's directory and files names may conflict with the distro's directory names and files. Therefore it's better to have them in differnet partitions

So, on with carving out the USB flash drive.
* Plug in your USB flash drive to your computer running linux and note the device that the operating system assigns to the drive.
You may want check your systems log files to figure out what the OS assigned drive is. On my system, this was /dev/sdc . Since the drive already had one big FAT32 partition, /dev/sdc1 was also created by the OS when the flash drive was plugged in. This of course may be different on your system, so check the system log files.
I'll be using /dev/sdc or /dev/sdcx to refer to my usb flash drive throughout this document. Make sure you use the correct device name that refers to yours
* Use fdisk to setup the paritions on the USB flash drive
The commands I ran are in bold and my comments are in bold and italics below:

# fdisk /dev/sdc

The number of cylinders for this disk is set to 391577.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)

Command (m for help): p

Disk /dev/sdc: 32.0 GB, 32078036992 bytes
5 heads, 32 sectors/track, 391577 cylinders
Units = cylinders of 160 * 512 = 81920 bytes
Disk identifier: 0xc0620719

Device Boot Start End Blocks Id System
/dev/sdc1 51 391578 31322176 c W95 FAT32 (LBA)

---Begin: Deleted the existing partition ------
Command (m for help): d
Selected partition 1
---End: Deleted the existing partition ------

---Begin: Creating the first primary partition of 5MB ------
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-391577, default 1): # hit [Enter] here
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-391577, default 391577): +5M
---End: Creating the first primary partition of 5MB ------

Command (m for help): p

Disk /dev/sdc: 32.0 GB, 32078036992 bytes
5 heads, 32 sectors/track, 391577 cylinders
Units = cylinders of 160 * 512 = 81920 bytes
Disk identifier: 0xc0620719

Device Boot Start End Blocks Id System
/dev/sdc1 1 62 4944 83 Linux

---Begin: Making the first partition bootable ------
Command (m for help): a
Partition number (1-4): 1
---End: Making the first partition bootable ------

Command (m for help): p

Disk /dev/sdc: 32.0 GB, 32078036992 bytes
5 heads, 32 sectors/track, 391577 cylinders
Units = cylinders of 160 * 512 = 81920 bytes
Disk identifier: 0xc0620719

Device Boot Start End Blocks Id System
/dev/sdc1 * 1 62 4944 83 Linux

---Begin: Creating the second partition ------
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (63-391577, default 63): # Hit [Enter] here
Using default value 63
Last cylinder or +size or +sizeM or +sizeK (63-391577, default 391577): +16384M
---End: Creating the second partition ------

Command (m for help): p

Disk /dev/sdc: 32.0 GB, 32078036992 bytes
5 heads, 32 sectors/track, 391577 cylinders
Units = cylinders of 160 * 512 = 81920 bytes
Disk identifier: 0xc0620719

Device Boot Start End Blocks Id System
/dev/sdc1 * 1 62 4944 83 Linux
/dev/sdc2 63 200063 16000080 83 Linux

---Begin: Creating the third partition ------
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 3
First cylinder (200064-391577, default 200064): # Hit [Enter] here
Using default value 200064
Last cylinder or +size or +sizeM or +sizeK (200064-391577, default 391577): # Hit [Enter] here
Using default value 391577
---End: Creating the third partition ------

Command (m for help): p

Disk /dev/sdc: 32.0 GB, 32078036992 bytes
5 heads, 32 sectors/track, 391577 cylinders
Units = cylinders of 160 * 512 = 81920 bytes
Disk identifier: 0xc0620719

Device Boot Start End Blocks Id System
/dev/sdc1 * 1 62 4944 83 Linux
/dev/sdc2 63 200063 16000080 83 Linux
/dev/sdc3 200064 391577 15321120 83 Linux

---Begin: Changing the 3rd partition's type to FAT32 ------
Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): c
Changed system type of partition 3 to c (W95 FAT32 (LBA))
---End: Changing the 3rd partition's type to FAT32 ------

Command (m for help): p

Disk /dev/sdc: 32.0 GB, 32078036992 bytes
5 heads, 32 sectors/track, 391577 cylinders
Units = cylinders of 160 * 512 = 81920 bytes
Disk identifier: 0xc0620719

Device Boot Start End Blocks Id System
/dev/sdc1 * 1 62 4944 83 Linux
/dev/sdc2 63 200063 16000080 83 Linux
/dev/sdc3 200064 391577 15321120 c W95 FAT32 (LBA)

---Begin: Saving the new partition information ------
Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.

WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.
---End: Saving the new partition information ------

#

* Next, the file systems were created.

---Begin: Creating ext3 file systems on the first and 2nd partitions ------
# mkfs -t ext3 /dev/sdc1
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
1240 inodes, 4944 blocks
247 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=5242880
1 block group
8192 blocks per group, 8192 fragments per group
1240 inodes per group

Writing inode tables: done
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.


# mkfs -t ext3 /dev/sdc2
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1001712 inodes, 4000020 blocks
200001 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4097835008
123 block groups
32768 blocks per group, 32768 fragments per group
8144 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 30 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
---End: Creating ext3 file systems on the first and 2nd partitions ------

---Begin: Creating FAT32 file system on the 3rd partitions ------
# mkfs.msdos -v -F 32 /dev/sdc3
mkfs.msdos 2.11 (12 Mar 2005)
/dev/sdc3 has 5 heads and 32 sectors per track,
logical sector size is 512,
using 0xf8 media descriptor, with 30642240 sectors;
file system has 2 32-bit FATs and 16 sectors per cluster.
FAT size is 14948 sectors, and provides 1913269 clusters.
Volume ID is 498de6ce, no volume label.
---End: Creating FAT32 file system on the 3rd partitions ------

* Creating mount points
With partitioning and file systems creation done, I ended up with /dev/sdc , /dev/sdc1, /dev/sdc2, and /dev/sdc3.
I created two directories that will be used as mount points for the first and second partitions of the USB flash drive.

# mkdir -p /mnt/usb1 --- This will be the mount point for the first partition
# mkdir -p /mnt/usb2 --- This will be the mount point for the second partition

2. Copying the boot loader to the master boot record of the USB flash drive.
This is the most important step in this document. Here, the boot loader files are copied from your current running linux system, which is booted using GNU GRUB, to your USB flash drive
Then GRUB is used to install itself on the master boot record (MBR) of the USB flash drive
The commands I ran are in bold and my comments are in bold and italics below:

# mount /dev/sdc1 /mnt/usb1
# mkdir -p /mnt/usb1/boot
# cp -r /boot/grub /mnt/usb1/boot
# grub
Probing devices to guess BIOS drives. This may take a long time.


GNU GRUB version 0.97 (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename.]
grub> find /boot/grub/stage1
find /boot/grub/stage1
(hd1,0)
The command above is try to find out all the drives that have /boot/grub/stage1 installed.
Since we copied the GRUB boot loader files before, in theory, the output of this command
should list both my hard drive where liunx is already installed AND my USB flash drive
where I copied the files to.
In reality, the output may list one, all, or none of the drives that has /boot/grub/stage1 installed.
I was lucky to have grub find the files on my USB flash drive's first partition (hd1,0). Another
computer I tried listed my hard drive's first partition (hd0,0) and not my USB flash drive . Your
output may also differ. In general, your hard drive will be (hd0) and unless your have another
hard drive installed, your USB flash drive should be (hd1). The "geometry" command below should
give us more details as to which drive we are dealing with.

grub> geometry (hd1)
geometry (hd1)
drive 0x81: C/H/S = 391577/5/32, The number of sectors = 62652416, /dev/sdc
Partition num: 0, Filesystem type is ext2fs, partition type 0x83
Partition num: 1, Filesystem type is ext2fs, partition type 0x83
Partition num: 2, Filesystem type is fat, partition type 0xc
grub> geometry (hd0)
geometry (hd0)
drive 0x80: C/H/S = 4863/255/63, The number of sectors = 78125000, /dev/sda
Partition num: 0, Filesystem type unknown, partition type 0x7
Partition num: 4, Filesystem type unknown, partition type 0x7
Partition num: 5, Filesystem type is ext2fs, partition type 0x83
Partition num: 6, Filesystem type unknown, partition type 0x8e
grub's "geometry" commands prints out information about the drive specified. As shown above, I ran this
command against both (hd1) and (hd0) just to make sure which drive I'm dealing with. My "geometry (hd1)"
output, shows me that I have 3 partitions, so it's likely that hd1 is my USB flash drive. The end of the
line that begins with "drive 0x....." leaves no doubt that it is, since /dev/sdc is infact my USB flash
drive.
So, hd1 is my USB flash drive. Now what?

First, I need to set GRUB's root device to be the partition on my USB flash drive that contains the boot directory.
This is done with grub's "root" command.
grub> root (hd1,0)
root (hd1,0)
Filesystem type is ext2fs, partition type 0x83

Then, I need to install GRUB's boot loader on the master boot record of that drive.
This is done with grub's "setup" command.
grub> setup (hd1)
setup (hd1)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd1)"... 15 sectors are embedded.
succeeded
Running "install /boot/grub/stage1 (hd1) (hd1)1+15 p (hd1,0)/boot/grub/stage2 /boot/grub/grub.conf"... succeeded
Done.

If the previous commands were successful, then quit gurb
grub> quit
quit

#

Optional: Change the permissions on the grub directory on the USB flash drive so that everyone has full permissions
# chmod -R 777 /mnt/usb1/boot/grub
Optional: Make a copy of your grub.conf file
# cd /mnt/usb1/boot/grub
# cp -p grub.conf grub.conf.orig


3. Test1: Booting off of your USB flash drive
This will test whether the previous step has made the USB flash drive bootable.
We will not attempt to boot a distro off of the USB flash drive here. At least not just yet.
We will, however, attempt to boot off of the USB flash drive and display a test GRUB boot menu.

# cd /mnt/usb1/boot/grub
# cat grub.conf

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,5)
# kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00
# initrd /initrd-version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,5)/grub/splash.xpm.gz
hiddenmenu
title Fedora (2.6.27.12-78.2.8.fc9.i686)
root (hd0,5)
kernel /vmlinuz-2.6.27.12-78.2.8.fc9.i686 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.27.12-78.2.8.fc9.i686.img
.....
.....

The above is my original truncated grub.conf file on my USB flash drive. This of course is the same as the
grub.conf on the computer I copied the GRUB boot loader files from. We are going to make some changes to
this file to make it different enough that we know we are booting off of the USB flash drive and not the
computer's hard drive.
The lines starting with # are all comments and are ignore by grub.
We are going to change any mention of (hd0,5), which is where the GRUB files are located on my hard drive
to refer to the first partition of the USB flash drive, which is where the USB flash drive's boot loader
files are located. When booting off of the USB flash drive, it becomes first drive and the computer's hard
drive becomes the second drive. In grub, (hd0,0) means the first partition of the first drive. We will also
change the title and timeout options so that there is not doubt that we are booting off of the USB flash drive.
So, using your favorit text editor, edit and save the grub.conf file so that it looks like the following.
The changes I made are in blue below:

# cat /mnt/usb1/boot/grub/grub.conf

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,5)
# kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00
# initrd /initrd-version.img
#boot=/dev/sda
default=0
timeout=30
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
hiddenmenu
title USB Flash Drive First Option
root (hd0,5)
kernel /vmlinuz-2.6.27.12-78.2.8.fc9.i686 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.27.12-78.2.8.fc9.i686.img
.....
.....


Now restart your computer with the USB flash drive plugged in and try to boot off of it.
You should see a grub menu with the first option reading "USB Flash Drive First Option".
If that's the case, then your USB flash drive is bootable!
Reboot your computer off of your hard drive back into your linux operating system.
The next steps will make it possible to boot into a distro saved on your USB flash drive.
This requires saving the distro's files and directories to the correct location and modifying
the grub.conf file.

4. Booting off of your USB flash drive into a distro stored on it
On your computer running linux, mount the partition that contains your GRUB boot loader files and the partition where you're going to be saving the distro's files
In my case, these were /dev/sdc1 and /dev/sdc2 respectively.

# mkdir /tmp/usb1
# mkdir /tmp/usb2
# mount /dev/sdc1 /tmp/usb1
# mount /dev/sdc2 /tmp/usb2

Booting into a distro stored on USB flash drive entails saving the distro's files and directories in the right location on the drive's 2nd partition and modifying the grub.conf file in the first partition. All of the linux installation and Live CD's that I downloaded and used here, used isolinux as their boot loader. The structure of the directories and files on the distro's CD/DVD, with the exception of the directories that contain the kernel and initrd files, has to be exact on the USB flash drive's root directory.
So for example, if a distro's CD/DVD root directory structure looked like the following:

boot/ directory1/ directory2/ file1 file2

Let's assume that boot/ contained the kernel and initrd files. Then the root of the USB flash drive's
2nd partition must have directory1/ , directory2/ , file1 and file2 . boot/ can be located elsewehere
on the USB flash drive's 2nd partition, as long as the grub.conf in the 1st partition has the path to its
files mentioned correctly. I'll explain this more in detail later in this step.

The issue with having to maintain these files and directories structures is that you run into the possibility of having conflicting file names if you have more than one distro saved on the 2nd partition that you want to boot into. Luckily, some of the distro providers did a great job in organizing the files and directories into one top level directory, as is in the case of Slax and Back Track. Unfortunately, it's not as neatly structured in other distros.
GRUB and isolinux menus use different syntax when loading the kernel and passing it parameters. Luckily, changing the syntax from one to the other is relativley simple.

Here is an example of an isolinux menu syntax for a boot menu item:

LABEL menu1
MENU LABEL My Test Menu Item
KERNEL /boot/vmlinuz
APPEND initrd=/boot/initrd.gz ramdisk_size=6666 root=/dev/ram0 rw autoexec=xconf;telinit~4

This is the exact same configuration in grub's syntax:

title My Test Menu Item
root (hd0,1)
kernel /boot/vmlinuz ramdisk_size=6666 root=/dev/ram0 rw autoexec=xconf;telinit~4
initrd /boot/initrd.gz


Basically, the "initrd=/some/path" in isolinux becomes a line by itself in grub.conf and changed to "initrd /some/path"
Everything else on the "APPEND" line in isolinux is copied and pasted after "kernel /boot/vmlinuz" in grub.conf
The "LABEL" and "MENU" lines in isolinux are combined into one "title" line in grub.conf
The all important "root (hdx,y)" must exist in grub.conf . This always refer to the partition containing the distro files

So where is the isolinux menu configuration file kept?
This is usually in the file isolinux.cfg. Use find to locate where the file is.
Sometimes isolinux.cfg would contain a line that references another file. This looks something like:

INCLUDE /somepath/somefile.cfg

In this case, /somepath/somefile.cfg will contain the isolinux menu configuration parameters.

So, Let's put this all together:
* Create a directory in the root of the USB flash drive's 2nd partition. Let's call it "myboot"
This directory will be used to store the other directories that contain the kernel and initrd files

# cd /tmp/usb2
# mkdir myboot

* Download and/or mount the CD/DVD image of the distro.
Using a BackTrack ISO image as an example:

# mkdir /tmp/ISO
# mount -o loop bt3final_usb.iso /tmp/ISO

Luckily, BackTrack neatly organizes all the of its files into two directories, BT3/ and boot/. boot/ is where the kernel and initrd files are kept, so it can be moved and renamed. BT3/ , has to be copied over into the root of the USB flash drives 2nd partition:

# rsync -ravz /tmp/ISO/BT3 /tmp/usb2/
# cd /tmp/usb2/myboot/
# mkdir bt3
# rsync -ravz /tmp/ISO/boot/* /tmp/usb2/myboot/bt3/

* Locate the isolinux.cfg file and change the syntax in your grub.conf according to the guidelines mentioned above.
Using the BackTrack example, the isolinux.cfg file is in the directory /tmp/usb2/myboot/bt3/isolinux . Let's take the following entry from isolinux.cfg:

LABEL xconf1
MENU LABEL BT3 Graphics mode (KDE)
KERNEL /boot/vmlinuz
APPEND vga=0x317 initrd=/boot/initrd.gz ramdisk_size=6666 root=/dev/ram0 rw autoexec=xconf;kdm

Open your grub.conf file in /tmp/usb1/boot/grub/ using your favorite text editor.
In grub's syntax, the above entry becomes the following. Please note how we changed
the path to the kernel and initrd files to reflect our changes from the previous step:

title BT3 Graphics mode (KDE)
root (hd0,1)
kernel /myboot/bt3/vmlinuz ramdisk_size=6666 root=/dev/ram0 rw autoexec=xconf;kdm
initrd /myboot/bt3/initrd.gz

Save your changes to grub.conf . Reboot and try booting off of your USB flash drive into this new distro.

No comments:

Post a Comment