Defragmentation

Fragmentation (fragment = fragment ) is defined as the scattered storage of logically related data blocks of the file system on a disk, which can be considered as a special case of the general memory fragmentation. For storage media with relatively high access times such as hard drives, fragmentation can lead to a significant slowdown of the read and write operations, as these technical reasons reach full speed only for sequential access.

See Defragmentation means the feasible with special programs reordering of fragmented data blocks on the storage medium so that logically related data blocks are stored sequentially on the disk as possible. This can speed up the sequential access, thereby increasing the operating speed of the entire system.

• 6.1 Disk Defragmenter in Windows 6.1.1 MS- DOS -based operating systems ( historically )
• 6.1.2 NT -based operating systems

Emergence of fragmentation

A volume ( is allowed on the reading and writing ) is not a static entity, but find it constantly read, write and erase operations instead. Some operations frees up space again, some require new space. This continuous release and re- allocating memory blocks means that on the one hand the unused memory blocks on the disk distribute ( free memory fragmentation ) and on the other, logically contiguous data blocks of files and metafiles no longer behind the other on the disk stored ( Dateifragmentation ). Another cause of fragmentation is the lengthening of existing files.

In simple terms: If the operating system wants to put a file on the storage medium, it is possible that this is not contiguous but scattered saved on the disk. Some file systems do not check whether the data can be stored contiguously within the free space, but start the save operation easy in the first free space region. However, it is not uncommon that this storage area is not large enough to receive the entire file. In this case, the file system describes the first free space, until it is full, and puts the rest of the file again from the next free space.

In the diagram of this process is illustrated:

Types of fragmentation

There are several types of fragmentation (although usually meant by fragmentation intra file fragmentation ):

• Intra - file Fragmentation is distributing Nutzdateien on the disk, so that small additional delays may occur during sequential reading of files.
• Inter - File fragmentation is the distribution of files and metafiles that are usually read sequentially. It plays a role (for example directories with many small images ), especially when correlated reading many small files.
• Metafile fragmentation is the fragmentation of directories, block allocation tables and related metadata.
• Fragmentation of unused space; it represents in itself no disadvantage for accessing files, but is one of the causes for the emergence of fragmentation ( creating new files).

Effects of fragmentation

When reading and writing from fragmented data can cause slowdowns depending on the nature of the storage medium. In modern file systems ( ReiserFS, XFS, NTFS ) it comes in addition to an increase in the metadata describing where the data is located on the storage medium.

The example of a hard disk can be the negative impact of fragmentation illustrated particularly well: When reading a fragmented file that is stored scattered everywhere on the magnetic disk of the hard disk, the read head of the disk must be in short distances very often and over long stretches on the magnetic disk be repositioned. This causes a lot of small delays that are within the range of a few milliseconds. Depending on the file size, degree of fragmentation and access time of the hard disk these delays over the entire read operation can add up to a noticeable slowdown.

In contrast, there are also other types of storage media, without moving parts and are therefore not affected by mechanical justified delays in read and write operations. These include media that are based on memory chips (mainly flash memory ), such as USB sticks, memory cards and solid state disks. Only partially affected by this problem, hybrid hard drives that use both magnetic disks and memory chips to permanently store data.

However, such storage media may be slowed by a fragmented file system. Due to the different internal organization of the memory, the effects of fragmentation are dependent on the model and manufacturer, however, very different and can not be compared to a hard drive. The controller of the storage media stores the data according to the model in some cases the physical order on the memory chip from being mapped by the file system. This allows no reliable statement about the impact of fragmentation on flash -based storage devices meet. It is certain, however, that performing a defragmentation negative effect on the life of flash memory media as they are not indefinitely writable.

Avoiding and reducing fragmentation

There are various strategies to reduce the fragmentation of a file system. These strategies are primarily based on hard disks as used disk and could not, or only slightly beneficial in other storage media impact.

With all kinds of file systems, a releasing effected by 5 to 20 percent of the storage space, reducing the fragmentation. Each complete a file system is, the more likely occurs even fragmentation, since the free areas become smaller and smaller. Once a file is to be stored, which is larger than the largest "gap", a continuous data is no longer possible and they inevitably fragmented.

Avoiding fragmentation

A proven method is the separation of programs and user data. The file system makes no difference to, but arise fragments with user data continuously and in programs with each update. A separation thus requires the distribution on different partitions. There is also a suitable tool, in particular for the displacement of the user data. As a result, the defragmentation of partitions will require less time.

Parameterization of the partitions

Simple measures provide less fragmentation according to the volume of the different file classes.

• Nifty distributing operating system, directory trees, and user data on the hard disk using appropriate software

• Using a higher block size of the filesystem ( this is, however, sometimes wasted space)

Parameterizing the applications and system services

Other measures relate to the parameterization of the applications:

• Preallocation ( to reserve advance provision blocks, although these are not yet required )
• Late Setting -to-use memory blocks (late allocation ) instead of immediate fixed laying (early allocation )
• Multistage Allozierungssysteme ( decomposition of a hard disk in the cluster, block groups and blocks).

Fragmentation level of a file system

There are various ways to indicate the fragmentation of a file system:

• Ratio of the read or write speed of the fragmented (real) file system over a nichtfragmentierten ( optimal ) file system
• Ratio of the number of fragmented files to total number of files
• Ratio of the occupied space by fragmented files to total used space
• Ratio of the occupied space by fragmented files to total available space
• Number of contiguous blocks to the total number of occupied blocks
• Number of contiguous blocks to the total number of available blocks.

The determined by different methods are not comparable degrees of fragmentation; some not even by the same method fragmentations are comparable, because the real effects in many processes continue to depend on block size, average file size, speed and internal memory organization of the medium.

In addition, the degree of fragmentation alone is not a reliable measure of the performance of a file system. Firstly, the reason is that in some file systems, the metadata can be fragmented, but others do not. ( MFT to NTFS, for example, affected by it. ) On the other hand have different file systems via various mechanisms to reduce the performance decline in existing fragmentation.

The file system Ext2 implements several mechanisms to reduce fragmentation. This includes the grouping of blocks to block groups. Files are then distributed as evenly as possible on block groups. This produces a uniform distribution of files (and free areas ) on the storage medium. Unlike FAT files are now always put in as ill-fitting free areas, ie small files in large gaps in order to react to size changes of the relevant files without losses. Behind each file tries Ext2 to let enough space, so not promptly perform minor size modifications to fragmentation of files. By distributing the data over the entire usable range falls in the server mode, the fragmentation of less importance than during reading in the " burst" mode. To arrange modern disk scheduler, the read or write requests in such a way (for example, the read-write head of the disk) that search times are minimal.

A further reduction of fragments obtained inter alia by the file system XFS, which write operations delayed ( delayed write ) and as completely as possible caches in RAM. For small files, so the entire file size is known to the file system before the file is written to disk. Thus XFS can find an optimal location for the file. Fits the file is not completely in the RAM of course is started earlier to write the buffer to disk.

In assessing the fragmentation of file systems must distinguish between throughput and performance: Performance is the data rate that can reach a user with a single maximum process of file system ( in MB / s). Throughput is the data rate, the file system for all users and processes (with multiple competing IO requests ) can provide summed. In multi-tasking or multi-user systems, the throughput is often much larger than the performance. The Linux operating system has algorithms that do not increase the throughput, the performance of the system. Strong fragmentation has a greater impact on performance. For systems with many competing requests to the file system one (larger) file is not already read in one piece, it will be simultaneously made ​​other requests.

Defragmentation in operating systems

Depending on the file system, there are different methods of defragmentation. While there really is no way to defragment your house especially for older file systems, Defragmentierprogramme are available for modern file systems.

The following methods are common:

• Copying the data to a blank partition: This method is more expensive, but more reliable. As each file is copied individually, all parts of the file to be assembled on the source partition, which are stored on the target partition in successive memory blocks. Alternatively, the files can be stored in an archive on the same partition; the original files are then deleted and unpacked the archive. The disadvantage is the double the storage space in the form of an additional partition or of free space on the partition. For this possibility is applicable to any file system.
• Terms of Defragmentierprogrammen: One must distinguish between offline and online defragmentation. In the former, the file system must not be mounted. Thus it is not possible for example to defragment the system partition of a running system. Online defragmentation is free of this problem usually. An example of a file system that supports online defragmentation, XFS, which can be defragmented xfs_fsr with the program.

Defragmentation in Windows

New fragmentation occurs among the different autonomous system services on an ongoing basis. Each system Update produced many new fragments. Defragmenting should be repeated if the system performance for updates or for new installations of large application packages, such as office suites, drops considerably.

MS- DOS -based operating systems ( historically )

Under MS- DOS file directory and file contents were not separated. Therefore, a defragment was achieved only by writing in another place. The program defrag.exe was introduced with MS- DOS version 6 and was a limited licensed version of Norton Speed ​​Disk. It was - as the full version of Speed ​​Disk - in certain cases to a significant increase in speed on the then available computers ( 386/486 ) lead. The computer had to no longer gather together individual fragments of a file across the entire hard disk after defragmentation. This phenomenon could even be " followed " on these older computers by the hard drive noises were analyzed ( rattling from side to side, later continuous chatter ).

Defrag.exe under DOS allowed up to Windows ME ( also still DOS - type) an accurate visual tracking of this process. For example, blocks of files were first highlighted in read green, looking for an empty spot on the hard drive and backed for writing in red. If no sufficiently large vacancy found within the processing segment, the data migrated mostly at the end of the disk. After the open spaces just this block has been moved again.

Under DOS and third-party programs ( Norton, Digital Research ) were common ( DISKOPT.EXE / DSKSPEED.EXE ). Together is the defragmentation programs that they offer an option to sort by file name, size or date, but which have little or no effect on the efficiency of the file system. Depending on the selected option ( eg sorting by modification date or by file name extensions ) can be certain system files to the beginning of the partition move, so this area is hardly affected by renewed fragmentation. Thus, the process of future fragmentation can slow down and the system performance remains stable for longer.

NT -based operating systems

With the new file system NTFS is the file directory and the file contents are written separately. Defragmenting therefore proceeds in several phases for the directories (folders ) and files ( files). One noticeable time savings occurs first, if the file directories to be defragmented. This is possible with third-party products.

1995 Executive Software licensed the Windows NT source code and published on its base in April Disk Defragmenter Diskeeper for Windows NT 3.5, and later also for its successor Windows NT 3.51. However, this caused major problems when the user a service pack installed, after Microsoft decided to work with Executive Software to integrate a standardized API for Defragmentation in Windows NT 4.0. Although this was undocumented and Windows NT 4.0 had no defragmentation tool included, but many third-party published defragmentation utilities for Windows NT.

Windows 2000 includes a defragmentation program from Microsoft. A detailed procedure of the program, however, is visible only 9x flat rate for the entire hard drive compared to Windows. Individual edited files will be displayed in the status bar, but only one timer is used internally, which shows the progress. With Windows Vista, the graphic display was replaced by a percentage.

Windows 7 defragmented automatically in the event of low system utilization from time to time. According to measurements, this results in a noticeable improvement in performance during start-up and during operation.

Defragmentation on Mac OS X

The file systems used by Mac OS X HFS and HFS are designed so that they are looking for the largest free block of memory to the hard disk to which a file is to be saved. Only when a file does not fit into the largest free memory block, the file is divided ( fragmented) and the unwritten part is stored in another block.

Defragmentation on Unix and Linux

It is sometimes claimed that Unix-like operating systems would not need defragmentation. The result is this claim from the contents of the thesis A Fast File System for UNIX by Bill Joy, Samuel Leffler and Kirk McKusick. However, the statements made ​​therein refer only to the UFS, which does not noticeably lose performance, in fact, even in extremely adverse use, and are not generally transferable to other file systems.

As mentioned in the "Avoiding and reducing fragmentation " section, counter-measures are implemented in the file systems used do, but in practice occurs fragmentation. Also, there are various programs to defragment on Unix and similar operating systems.

Tools for defragmentation

Under Windows NT operating systems (Windows 2000) is only the administrators supplied defragmenter available to normal users to access denied. However, compared to foreign products based on Diskeeper defragmenter has only limited functionality.

As of Windows 7, the Windows own defragmenter service runs by default once a week in the background. This can also be disabled if the use of other programs is preferred.

In the freeware area, for example, Ultimate Defrag, MyDefrag and Defraggler are known. They support a fragmentation according to various criteria, which saves significant running time.

The file systems used in Linux and Unix variants bend the fragmentation mostly prefer therefore less emphasis is placed on the defrag. However, such file systems are not resistant to fragmentation. Ext2 can be defragmented by defrag, XFS using xfs_fsr.

Other popular file systems such as ReiserFS and Ext3 provide such programs to defragment not available. The only solution is to defragment copying is (see above ) to an empty partition.

To defragment the used by Apple in Mac OS X HFS runs in the background a defragmentation program that the user is presented with no output and the defragmentation automatically interrupts in case of disk accesses, as Microsoft has realized also in Windows Vista.