![]() | ![]() |
|
Linux Partition HOWTOAnthony LissotKristian Koehntopp
This Linux Mini-HOWTO teaches you how to plan and create partitions on IDE and SCSI hard drives. It discusses partitioning terminology and considers size and location issues. Use of the fdisk partitioning utility for creating and recovering of partition tables is covered. The most recent version of this document is here.
1. Introduction1.1. What is a partition?Partitioning is a means to divide a single hard drive into many logical drives. A partition is a contiguous set of blocks on a drive that are treated as an independant disk. A partition table (the creation of which is the topic of this HOWTO) is an index that relates sections of the hard drive to partitions. Why have multiple partitions?
1.2. Constraints
1.3. Other Partitioning Software:
1.4. Related HOWTOs
Table 1. Related HOWTOs
2. DevicesThere is a special nomenclature that linux uses to refer to hard drive partitions that must be understood in order to follow the discussion on the following pages. In Linux, partitions are represented by device files. These are phoney files located in /dev. Here are a few entries:
2.1. Device names2.1.1. Naming ConventionBy convention, IDE drives will be given device names /dev/hda to /dev/hdd. Hard Drive A(/dev/hda) is the first drive and Hard Drive C /dev/hdc) is the third. Table 2. IDE controller naming convention
A typical PC has two IDE controllers, each of which can have two drives connected to it. For example, /dev/hda is the first drive (master) on the first IDE controller and /dev/hdd is the second (slave) drive on the second controller (the fourth IDE drive in the computer). You can write to these devices directly (using cat or dd). However, since these devices represent the entire disk, starting at the first block, you can mistakenly overwrite the master boot record and the partition table, which will render the drive unusable.
Table 3. partition names
Once a drive has been partitioned, the partitions will represented as numbers on the end of the names. For example, the second partition on the second drive will be /dev/hdb2. The partition type primary (Section 3.3 is listed in the table above for clarity, although the concept is not explained until Section 3Section 3.3.
Table 4. SCSI Drives
SCSI drives follow a similar pattern; They are represented by 'sd' instead of 'hd'. The first partition of the second SCSI drive would therefore be /dev/sdb1. In the table above, the drive number is arbitraily chosen to be 6 to introduce the idea that SCSI ID numbers do not map onto device names under linux. 2.1.2. Name AssignmentUnder (Sun) Solaris and (SGI) IRIX, the device name given to a SCSI drive has some relationship to where you plug it in. Under linux, there is only wailing and gnashing of teeth. Lower SCSI ID numbers are assigned lower-order letters, so if you remove one drive from the chain, the names of the higher ID number drives will change. If you have two SCSI controllers in your linux box, you will need to examine the output of /bin/dmesg in order to see what name each drive was assigned. If you remove one of two controllers, the remaining controller might have all its drives renamed. Grrr... There are two work-arounds; both involve using a program to put a label on each partition. You then refer to the partition directly or indirectly by label. 2.1.3. Logical Partitions
Table 5. Logical Partitions
The table above illustrates a mysterious jump in the name assignments. This is due to the use of logical Section 3.4 partitions, which always start with 5, for reasons explained later Section 5.1.3. This is all you have to know to deal with linux disk devices. For the sake of completeness, see Kristian's discussion of device numbers below. 2.2. Device numbersThe only important thing with a device file are its major and minor device numbers, which are shown instead of the file size:
Table 6. Device file attributes
When accessing a device file, the major number selects which device driver is being called to perform the input/output operation. This call is being done with the minor number as a parameter and it is entirely up to the driver how the minor number is being interpreted. The driver documentation usually describes how the driver uses minor numbers. For IDE disks, this documentation is in /usr/src/linux/Documentation/ide.txt. For SCSI disks, one would expect such documentation in /usr/src/linux/Documentation/scsi.txt, but it isn't there. One has to look at the driver source to be sure ( /usr/src/linux/driver/scsi/sd.c:184-196). Fortunately, there is Peter Anvin's list of device numbers and names in /usr/src/linux/Documentation/devices.txt; see the entries for block devices, major 3, 22, 33, 34 for IDE and major 8 for SCSI disks. The major and minor numbers are a byte each and that is why the number of partitions per disk is limited. 3. Partition Types3.1. Partition TypesA partition is labeled to host a certain kind of file system. Such a file system could be the linux standard ext2 file system or linux swap space, or even foreign file systems like (Microsoft) NTFS or (Sun) UFS. There is a numerical code associated with each partition type. For example, the code for ext2 is 0x83 and linux swap is 0x82. 3.2. Foreign Partition TypesThe partition type codes have been arbitrarily chosen (you can't figure out what they should be) and they are particular to a given operating system. Therefore, it is theoretically possible that if you use two operating systems with the same hard drive, the same code might be used to designate two different partition types. OS/2 marks its partitions with a 0x07 type and so does Windows NT's NTFS. MS-DOS allocates several type codes for its various flavors of FAT file systems: 0x01, 0x04 and 0x06 are known. DR-DOS used 0x81 to indicate protected FAT partitions, creating a type clash with Linux/Minix at that time, but neither Linux/Minix nor DR-DOS are widely used any more. 3.3. Primary PartitionsThe number of partitions on an Intel-based system was limited from the very beginning: The original partition table was installed as part of the boot sector and held space for only four partition entries. These partitions are now called primary partitions. 3.4. Logical PartitionsOne primary partition of a hard drive may be subpartitioned. These are logical partitions. This effectively allows us to skirt the historical four partition limitation. The primary partition used to house the logical partitions is called an extended partition and it has its own file system type (0x05). Unlike primary partitions, logical partitions must be contiguous. Each logical partition contains a pointer to the next logical partition, which implies that the number of logical partitions is unlimited. However, linux imposes limits on the total number of any type of partition on a drive, so this effectively limits the number of logical partitions. This is at most 15 partitions total on an SCSI disk and 63 total on an IDE disk. 3.5. Swap PartitionsEvery process running on your computer is allocated a number of blocks of RAM. These blocks are called pages. The set of in-memory pages which will be referenced by the processor in the very near future is called a "working set." Linux tries to predict these memory accesses (assuming that recently used pages will be used again in the near future) and keeps these pages in RAM if possible. If you have too many processes running on a machine, the kernel will try to free up RAM by writing pages to disk. This is what swap space is for. It effectively increases the amount of memory you have available. However, disk I/O is very slow compared to reading from and writing to RAM. Expect performance to drop by approximately the ratio between memory access speed and disk access speed. If memory becomes so scarce that the kernel pages out from the working set of one process in order to page in for another, the machine is said to be thrashing. Some readers might have inadvertenly experienced this: the hard drive is grinding away like crazy, but the computer is slow to the point of being unusable. Swap space is something you need to have, but it is no substitute for sufficient RAM. See the discussion in Section 4 Section 4.4.1 for tips on determining the size of swap space you need. 4. Partitioning requirements4.1. What Partitions do I need?Boot Drive: If you want to boot your operating system from the drive you are about to partition, you will need:
4.2. Discussion:
4.3. File Systems4.3.1. Which file systems need their own partitions?Everything in your linux file system can go in the same (single) partition. However, there are circumstances when you may want to restrict the growth of certain file systems. For example, if your mail spool was in the same partition as your root fs and it filled the remaining space in the partition, your computer would basically hang.
4.3.2. File lifetimes and backup cycles as partitioning criteriaWith ext2, partitioning decisions should be governed by backup considerations and to avoid external fragmentation Section 7.3 from different file lifetimes. Files have lifetimes. After a file has been created, it will remain some time on the system and then be removed. File lifetime varies greatly throughout the system and is partly dependent on the pathname of the file. For example, files in /bin, /sbin, /usr/sbin, /usr/bin and similar directories are likely to have a very long lifetime: many months and above. Files in /home are likely to have a medium lifetime: several weeks or so. File in /var are usually short lived: Almost no file in /var/spool/news will remain longer than a few days, files in /var/spool/lpd measure their lifetime in minutes or less. For backup it is useful if the amount of daily backup is smaller than the capacity of a single backup medium. A daily backup can be a complete backup or an incremental backup. You can decide to keep your partition sizes small enough that they fit completely onto one backup medium (choose daily full backups). In any case a partition should be small enough that its daily delta (all modified files) fits onto one backup medium (choose incremental backup and expect to change backup media for the weekly/monthly full dump - no unattended operation possible). Your backup strategy depends on that decision. When planning and buying disk space, remember to set aside a sufficient amount of money for backup! Unbackuped data is worthless! Data reproduction costs are much higher than backup costs for virtually everyone! For performance it is useful to keep files of different lifetimes on different partitions. This way the short lived files on the news partition may be fragmented very heavily. This has no impact on the performance of the / or /home partition. 4.4. Swap Partitions4.4.1. How large should my swap space be?A general rule of thumb is the same amount as your RAM, but there is no technical reason for this. Other techical considerations are:
But keep in mind that this is just a rule of thumb. It is easily possible to create scenarios where programs have extremely large or extremely small working sets. For example, a simulation program with a large data set that is accessed in a very random fashion would have almost no noticeable locality of reference in its data segment, so its working set would be quite large. On the other hand, an xv with many simultaneously opened JPEGs, all but one iconified, would have a very large data segment. But image transformations are all done on one single image, most of the memory occupied by xv is never touched. The same is true for an editor with many editor windows where only one window is being modified at a time. These programs have - if they are designed properly - a very high locality of reference and large parts of them can be kept swapped out without too severe performance impact. One could suspect that the 25% number from the age of the command line is no longer true for modern GUI programs editing multiple documents, but I know of no newer papers that try to verify these numbers. So for a configuration with 16 MB RAM, no swap is needed for a minimal configuration and more than 48 MB of swap are probably useless. The exact amount of memory needed depends on the application mix on the machine (what did you expect?). 4.4.2. Where should I put my swap space?The short answer is anywhere is fine. However, if you are interested in extracting as much speed as possible, there are two basic strategies (other than buying more RAM.
Here are the considerations:
Summary: Put your swap on a fast disk with many heads that is not busy doing other things. If you have multiple disks: Split swap and scatter it over all your disks or even different controllers. 5. Partitioning with fdisk5.1. Partitioning with fdiskThis section shows you how to actually partition your hard drive with the fdisk utility. Linux allows only 4 primary partitions. You can have a much larger number of logical partitions by sub-dividing one of the primary partitions. Only one of the primary partitions can be sub-divided. Examples:
5.1.1. Notes about fdisk:fdisk is started by typing (as root) fdisk device at the command prompt. Section 2.1.1device might be something like /dev/hda or /dev/sda. The basic fdisk commands you need are:
Changes you make to the partition table do not take effect until you issue the write (w) command. Here is a sample partition table:
5.1.2. Four primary partitionsThe overview:Decide on the size (Section 4.4.1) of your swap space and where (Section 4.4.2) it ought to go. Divide up the remaining space for the three other partitions. Example: I start fdisk from the shell prompt:
Side topics: 5.1.3. Mixed primary and logical partitionsThe overview: create one use one of the primary partitions to house all the extra partitions. Then create logical partitions within it. Create the other primary partitions before or after creating the logical partitions. Example: I start fdisk from the shell prompt:
First I figure out how many partitions I want. I know my drive has a 183Gb capacity and I want 26Gb partitions (because I happen to have back-up tapes that are about that size). 183Gb / 26Gb = ~7 so I will need 7 partitions. Even though fdisk accepts partition sizes expressed in Mb and Kb, I decide to calculate the number of cylinders that will end up in each partition because fdisk reports start and stop points in cylinders. I see when I enter fdisk that I have 22800 cylinders.
Since I have 4 primary partitions, 3 of them can be 3258 long. The extended partition will have to be (4 * 3258), or 13032, cylinders long in order to contain the 4 logical partitions. I enter the following commands to set up the first of the 3 primary partitions (stuff I type is bold ):
5.1.4. Submitted ExamplesI'd like to submit my partition layout, because it works well with any distribution of Linux (even big RPM based ones). I have one hard drive that ... is 10 gigs, exactly. Windows can't see above 9.3 gigs of it, but Linux can see it all, and use it all. It also has much more than 1024 cylenders. Table 7. Partition layout example
I also noticed that you don't have any REAL examples of partition tables, and for newbies I HIGHLY suggest putting quite a few up. I'm freshly out of the newbie stage, and partitioning was what messed me up the most. 6. Recovering a Deleted Partition TableBelow are instructions for manually recovering a deleted partition table. There are utilities such as gpart or TestDisk which can make this task considerably easier. If you are reading this, however, because you have run out of luck, this is what you will have to do:
Credit goes to:
7. Formating PartitionsAt the shell prompt, I begin making the file systems on my partitions. Continuing with the example in Section 5.1.3, this is:
The most common file systems can be made with programs in /sbin that start with "mk" like mkfs.msdos and mke2fs. 7.1. Activating Swap SpaceTo set up a swap partition:
7.2. Mounting PartitionsMounting a partition means attaching it to the linux file system. To mount a linux partition:
7.3. Some facts about file systems and fragmentationDisk space is administered by the operating system in units of blocks and fragments of blocks. In ext2, fragments and blocks have to be of the same size, so we can limit our discussion to blocks. Files come in any size. They don't end on block boundaries. So with every file a part of the last block of every file is wasted. Assuming that file sizes are random, there is approximately a half block of waste for each file on your disk. Tanenbaum calls this "internal fragmentation" in his book "Operating Systems". You can guess the number of files on your disk by the number of allocated inodes on a disk. On my disk
Data transfer is faster for large contiguous chunks of data, though. That's why ext2 tries to preallocate space in units of 8 contigous blocks for growing files. Unused preallocation is released when the file is closed, so no space is wasted. Noncontiguous placement of blocks in a file is bad for performance, since files are often accessed in a sequential manner. It forces the operating system to split a disk access and the disk to move the head. This is called "external fragmentation" or simply "fragmentation" and is a common problem with MS-DOS file systems. In conjunction with the abysmal buffer cache used by MS-DOS, the effects of file fragmentation on performance are very noticeable. DOS users are accustomed to defragging their disks every few weeks and some have even developed some ritualistic beliefs regarding defragmentation. None of these habits should be carried over to Linux and ext2. Linux native file systems do not need defragmentation under normal use and this includes any condition with at least 5% of free space on a disk. There is a defragmentation tool for ext2 called defrag, but users are cautioned against casual use. A power outage during such an operation can trash your file system. Since you need to back up your data anyway, simply writing back from your copy will do the job. The MS-DOS file system is also known to lose large amounts of disk space due to internal fragmentation. For partitions larger than 256 MB, DOS block sizes grow so large that they are no longer useful (This has been corrected to some extent with FAT32). Ext2 does not force you to choose large blocks for large file systems, except for very large file systems in the 0.5 TB range (that's terabytes with 1 TB equaling 1024 GB) and above, where small block sizes become inefficient. So unlike DOS there is no need to split up large disks into multiple partitions to keep block size down. Use a 1Kb block size if you have many small files. For large partitions, 4Kb blocks are fine. |