add directory study
@@ -0,0 +1,73 @@
|
||||
<html><head>
|
||||
<title>HPFS: Application Programs and the HPFS</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>Application Programs and the HPFS</h1>
|
||||
</center>
|
||||
|
||||
Each of the OS/2 releases thus far have carried with them a major
|
||||
discontinuity for application programmers who teamed their trade in the
|
||||
MS-DOS environment. In OS/2 1.0, such programmers were faced for the first
|
||||
time with virtual memory, multitasking, inter-process communications, and
|
||||
the protected mode restrictions on addressing and direct control of the
|
||||
hardware and were challenged to master powerful new concepts such as
|
||||
threading and dynamic linking. In OS/2 Version 1.1, the stakes were raised
|
||||
even fufiher. Programmers were offered a powerful hardware-independent
|
||||
graphical user interface but had to restructure their applications
|
||||
drastically for an event-driven environment based on objects and message
|
||||
passing. In OS/2 Version 1.2, it is time for many of the file- oriented
|
||||
programming habits and assumptions carried forward from the MS-DOS
|
||||
environment to fall by the wayside. An application that wishes to take
|
||||
full advantage of the HPFS must allow for long, free-form, mixed-case
|
||||
filenames and paths with few restrictions on punctuation and must be
|
||||
sensitive to the presence of EAs and ACLs.
|
||||
<p>
|
||||
After all, if EAs are to be of any use, it won't suffice for applications
|
||||
to update a file by renaming the old file and creating a new one without
|
||||
also copying the EAs. But the necessary changes for OS/2 Version 1.2 are
|
||||
not tricky to make. A new API function, DosCopy, helps applications create
|
||||
backups--it essentially duplicates an existing file together with its EAs.
|
||||
EAs can also be manipulated explicitly with DosQFileInfo DosSetFileInfo
|
||||
DosQPathlnfo and DosSetPathInfo. A program should call DosQSysInfo at run
|
||||
time to find the maximum possible path length for the system and ensure
|
||||
that all buffers used by DosChDir DosQCurDir and related functions are
|
||||
sufficiently large. Similarly the buffers used by DosOpen DosMove
|
||||
DosGetModName, DosFindFirst D DosFindNext and like functions must allow for
|
||||
longer filenames. Any logic that folds cases in filenames or tests for the
|
||||
occurrence of only one dot delimiter in a filename must be rethought or
|
||||
eliminated. The other changes in the API will not affect the average
|
||||
application. The functions DosQFileInfo DosFindFirst and DosFindNext now
|
||||
retain all three sets of times and dates (created last accessed last
|
||||
motified) for a file on an HPFS volume but few programs are concerned with
|
||||
time and date stamps anyway. DosQFslnfo is used to obtain volume labels or
|
||||
disk characteristics just as before and the use of DosSetFsInfo for volume
|
||||
labels is unchanged. There are a few totally new API functions such as
|
||||
DosFsCtl (analogous to DosDevlOCtl but used for communication between an
|
||||
application and an FSD) DosFsAttach (a sort of explicit mount call) and
|
||||
DosQFsAttach (determines which FSD owns a volume) these are intended mainly
|
||||
for use by disk utility program. In order to prevent old OS/2 applications
|
||||
and MS-DOS applications running in the DOS box from inadvertently damaging
|
||||
HPFS files a new flag bit has been defined in the EXE file header that
|
||||
indicates whether an application is HPFS-aware. If this bit is not set the
|
||||
application will only be able to search for open or create files on HPFS
|
||||
volumes that are compatible with the FAT' file system's 8.3 naming conventions.
|
||||
lf the bit is set OS/2 allows access to all files on an HPFS volume because
|
||||
it assumes that the program knows how to handle long free-form filenames and
|
||||
will take the responsibility of conserving a file's EAs and ACLs.
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="faultol.html">[Fault Tolerance]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="sum.html">[Summary]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
43
study/sabre/os/files/FileSystems/DesignGoalsHPFS/design.html
Normal file
@@ -0,0 +1,43 @@
|
||||
<html><head>
|
||||
<title>HPFS: Design</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>
|
||||
Design Goals and Implementation of the<br>
|
||||
New High Performance File System
|
||||
</h1>
|
||||
</center>
|
||||
|
||||
The High Performance File System (hereafter HPFS), which is making its first
|
||||
appearance in the OS/2 operating system Version 1.2, had its genesis in the
|
||||
network division of Microsoft and was designed by Gordon Letwin, the chief
|
||||
architect of the OS/2 operating system. The HPFS has been designed to meet
|
||||
the demands of increasingly powerful PC's, fixed disks, and networks for many
|
||||
years to come and to serve as a suitable platform for object-oriented languages,
|
||||
applications, and user interfaces.
|
||||
The HPFS is a complex topic because it incorporates three distinct yet
|
||||
interrelated file system issues. First, the HPFS is a way of organizing data
|
||||
on a random access block storage device. Second, it is a software module that
|
||||
translates file-oriented requests from an application program into more
|
||||
primitive requests that a device driver can understand, using a variety of
|
||||
creative techniques to maximize performance. Third, the HPFS is a practical
|
||||
illustration of an important new OS/2 feature known as Installable File Systems.
|
||||
This article introduces the three aspects of the HPFS. But first, it puts the
|
||||
HPFS in perspective by reviewing some of the problems that led to the system's
|
||||
existence.
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="fat.html">[FAT File System]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
91
study/sabre/os/files/FileSystems/DesignGoalsHPFS/dirs.html
Normal file
@@ -0,0 +1,91 @@
|
||||
<html><head>
|
||||
<title>HPFS: Directories</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>Directories</h1>
|
||||
</center>
|
||||
|
||||
Directories like files are anchored on Fnodes. A pointer to the Fnode for
|
||||
the root directory is found in the Super Block The Anodes for directories
|
||||
other than the root are reached through subdirectory entries in their parent
|
||||
directories. Directories can grow to any size and are built up from 2Kb
|
||||
directory blocks, which are allocated as four consecutive sectors on the disk.
|
||||
The file system attempts to allocate directory blocks in the directory band,
|
||||
which is located at or near the seek center of the disk. Once the directory
|
||||
band is full, the directory blocks are allocated wherever space is available.
|
||||
Each 2Kb directory block contains from one to many directory entries.
|
||||
A directory entry contains several fields, including time and date stamps,
|
||||
an Fnode pointer, a usage count for use by disk maintenance programs, the
|
||||
length of the file or directory name, the name itself, and a B-Tree pointer.
|
||||
Each entry begins with a word that contains the length of the entry. This
|
||||
provides for a variable amount of flex space at the end of each entry, which
|
||||
can be used by special versions of the file system and allows the directory
|
||||
block to be traversed very quickly (<a href="#fig5.html">Figure 5</a>).
|
||||
The number of entries in a directory block varies with the length of names.
|
||||
If the average filename length is 13 characters, an average directory block
|
||||
will hold about 40 entries.
|
||||
<p>
|
||||
|
||||
The entries in a directory block are sorted by the binary lexical order of
|
||||
their name fields (this happens to put them in alphabetical order for the US.
|
||||
alphabet). The last entry in a directory block is a dummy record that marks
|
||||
the end of the block. When a directory gets too large to be stored in one
|
||||
block, it increases in size by the addition of 2Kb blocks that are organized
|
||||
as a B-Tree (see B-T tees and B+ Trees ). When searching for a specific name,
|
||||
the file system traverses a directory block until it either finds a match or
|
||||
finds a name that is lexically greater than the target. In the latter case,
|
||||
the file system extracts the Tree pointer from the entry. If there is no
|
||||
pointer, the search failed otherwise the file system follows the pointer to
|
||||
the next directory block in the tree and continues the search. A little
|
||||
back-of-the-envelope arithmetic yields some impressive statistics. Assuming
|
||||
40 entries per block, a two-level tree of directory blocks can hold 1640
|
||||
directory entries and a three-level tree can hold an astonishing 65,640 entries.
|
||||
In other words, a particular file can be found (or shown not to exist) in a
|
||||
typical directory of 65,640 files with a maximum of three disk hits--the
|
||||
actual number of disk accesses depending on cache contents and the location
|
||||
of the file's name in the directory blockB-Tree.That's quite a contrast to
|
||||
the FAT file system, where in the worst case more than 4000 sectors would
|
||||
have to be read to establish that a filewas or was not present in a directory
|
||||
containing the same number of files. The B-Tree directory structure has
|
||||
interesting implications beyond its effect on open and find operations.
|
||||
A file creation, renaming, or deletion may result in a cascade of complex
|
||||
operations, as directory blocks are added or freed or names are moved from
|
||||
one block to the other to keep the tree balanced. In fact, a rename
|
||||
operation could theoretically fail for lack of disk space even though the
|
||||
file itself is not growing. In order to avoid this sort of disaster, the
|
||||
HPFS maintains a small pool of free blocks that can be drawn from in a
|
||||
directory emergency; a pointer to this pool of free blocks is stored in the
|
||||
Spare Block.
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig5.gif" name="fig5">
|
||||
<img src="fig5.gif" alt="[Fig. 5]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 5</b>:
|
||||
Here directories are anchored on an Fnode and are built up from 2Kb directory
|
||||
blocks. The number of entries in a directory block varies because the length
|
||||
of the entries depends on the filename. When a directory requires more than
|
||||
one block the blocks are organized as a B-Tree. This allows a filename to be
|
||||
located very quickly with a small number of disk accesses even when the
|
||||
directory grows very large.
|
||||
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="fnodes.html">[Files and Fnodes]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="ea.html">[Extended Attributes]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
65
study/sabre/os/files/FileSystems/DesignGoalsHPFS/ea.html
Normal file
@@ -0,0 +1,65 @@
|
||||
<html><head>
|
||||
<title>HPFS: Extended Attributes</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>Extended Attributes</h1>
|
||||
</center>
|
||||
|
||||
File attributes are information about a file that is maintained by the
|
||||
operating system outside the file's overt storage area. The FAT file
|
||||
system supports only a few simple attributes (read only, system, hidden,
|
||||
and archive) that are actually stored as bit flags in the file's directory
|
||||
entry these attributes are inspected or modified by special function calls
|
||||
and are not accessible through the normal file open, read, and write calls.
|
||||
The HPF'S supports the same attributes as the FAT file system for historical
|
||||
reasons, but it also supports a new form of file- associated, highly
|
||||
generalized information called Extended Attributes (EAs). Each EA is
|
||||
conceptually similar to an environment variable, taking the form
|
||||
(name=value) except that the value portion can be either a null- tenninated
|
||||
(ASCIIZ) string or binary data. In OS/2 1.2, each file or direc-tory can
|
||||
have a maximum of 64Kb of EAs attached to it. This limit may be lifted in
|
||||
a later release of OS/2. The storage method for EAs can vary. If the EAs
|
||||
associated with a given file or directory are small enough, they will be
|
||||
stored right in the Fnode. If the total size of the EAs is too large, they
|
||||
are stored outside the Fnode in sector runs, and a B+ Tree of allocation
|
||||
sectors can be created to describe the runs. If a single EA gets too large,
|
||||
it can be pushed outside the Fnode into a B+ Tree of its own.
|
||||
<p>
|
||||
The kernel API functions DosQFileInfo and DosSetFileInfo have been expanded
|
||||
with new information levels that allow application programs to manipulate
|
||||
extended attributes for files. The new functions DosQPathInfo and
|
||||
DosSetPathInfo are used to read or write the EAs associated with arbitrary
|
||||
path names. An application program can either ask for the value of a
|
||||
specific EA (supplying a name to be matched) or can obtain all of the EAs
|
||||
for the file or directory at once. Although application programs can begin
|
||||
to take advantage of EAs as soon as the HPFS is released, support for EAs
|
||||
is an essential component in Microsoft's long-range plans for object-oriented
|
||||
file systems. Information of almost any type can be stored in EAs, ranging
|
||||
from the name of the application that owns the file to names of dependent
|
||||
files to icons to executable code. As the HPFS evolves, its facilities for
|
||||
manipulating EAs are likely to become much more sophisticated. It's easy to
|
||||
imagine, for example, that in future versions the API might be extended with
|
||||
EA functions that are analogous to DosFindFirst and DosFindNext and EA data
|
||||
might get organized into B-Trees. I should note here that in addition to EAs,
|
||||
the LAN Manager version of HPFS will support another class of fil-associated
|
||||
information called Access Control Lists (ACLs). ACLs have the same general
|
||||
appearance as EAs and are manipulated in a similar manner, but they are used
|
||||
to store access rights, passwords, and other information of interest in a
|
||||
networking multi user environment.
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="dirs.html">[Directories]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="ifs.html">[Installable File Systems]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
94
study/sabre/os/files/FileSystems/DesignGoalsHPFS/fat.html
Normal file
@@ -0,0 +1,94 @@
|
||||
<html><head>
|
||||
<title>HPFS: FAT File System</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>FAT File System</h1>
|
||||
</center>
|
||||
|
||||
The so-called FAT file system ,which is the file system used in all versions of the
|
||||
MS-DOS operating system to date and in the first two releases of OS/2' (Versions 1.0
|
||||
and 1.1), has a dual heritage in Microsoft's earliest programming language products
|
||||
and the Digital Research CP/M operating system--software originally written for
|
||||
8080-based and Z-80-based microcomputers. It inherited characteristics from both
|
||||
ancestors that have progressively turned into handicaps in this new era of
|
||||
multitasking, protected mode, virtual memory, and huge fixed disks.
|
||||
<p>
|
||||
|
||||
The FAT file system revolves around the File Allocation Table for which it is
|
||||
named. Each logical volume has its own FAT, which serves two important functions:
|
||||
it contains the allocation information for each file on the volume in the fonn
|
||||
of linked lists of allocation units (clusters, which are power-of-2 multiples
|
||||
of sectors) and it indicates which allocation units are free for assignment to
|
||||
a file that is being created or extended.
|
||||
<p>
|
||||
|
||||
The FAT was invented by Bill Gates and Marc McDonald in 1977 as
|
||||
a method of managing disk space in the NCR version of standalone Microsoft's Disk
|
||||
BASIC. Tim Paterson, at that time an employee of Seattle Computer Products (SCP), was
|
||||
introduced to the FAT concept when his company shared a booth with Microsoft at the
|
||||
National Computer Conference in 1979. Paterson subsequently incorporated FATs into
|
||||
the file system of 86-DOS, an operating system for SCP s S-100 bus 8086 CPU boards.
|
||||
86-DOS was eventually purchased by Micro-soft and became the starting point for
|
||||
MS-DOS Version 1.0, which was released for the original lBM PC in August 1981.
|
||||
<p>
|
||||
When the FAT was conceived, it was an excellent solution to disk management,
|
||||
mainly because the floppy disks on which it was used were rarely larger than
|
||||
1 Mb.
|
||||
On such disks, the FAT was small enough to be held in memory at all times,
|
||||
allowing very fast random access to any part of any file. This proved far
|
||||
superior to the CP/M method of tracking disk space, in which the information
|
||||
about the sectors assigned to a file might be spread across many directory
|
||||
entries, which were in turn scattered randomly throughout the disk directory.
|
||||
When applied to fixed disks, however, the FAT began to look more like a bug
|
||||
than a feature. it became too large to be held entirely resident and had to
|
||||
be paged into memory in pieces: this paging resulted in many superfluous disk
|
||||
head movements as a program was reading through a file and degraded system
|
||||
throughput. in addition, because the information about free disk space was
|
||||
dispersed across many sectors of FAT, it was impractical to allocate file
|
||||
space contiguously, and file fragmentation became another obstacle to good
|
||||
performance. Moreover, the use of relatively large clusters on fixed disks
|
||||
resulted in a lot of dead space, since an average of one- half cluster was
|
||||
wasted for each file. (Some network servers use clusters as large as 64Kb.)
|
||||
<p>
|
||||
|
||||
The FAT file system 's restrictions on naming files and directories are
|
||||
inherited from CP/M. When Paterson was writing 86-DOS one of his primary
|
||||
objectives was to make programs easy to port from CP/M to his new operating
|
||||
system. He therefore adopted CP/M's limits on filenames and extensions so the
|
||||
critical fields of 86-DOS File Control Blocks (FCBs) would look almost exactly
|
||||
like those of CP/M. The sizes of the FCB filename and extension fields were
|
||||
also propagated into the structure of disk directory entries. In due time 86-DOS
|
||||
became MS- DOS and application programs for MS-DOS proliferated beyond anyone's
|
||||
wildest dreams. Since most of the early programs depended on the structure of
|
||||
FCBs the 8.3 format for filenames became irrevocably locked into the system.
|
||||
<p>
|
||||
|
||||
During the last couple of years Microsoft and IBM have made valiant attempts
|
||||
to prolong the useful life of the FAT file system by lifting the restrictions
|
||||
on volume sizes improving allocation strategies caching path names and moving
|
||||
tables and buffers into expanded memory. But these can only be regarded as
|
||||
temporizing measures because the fundamental data structures used by the FAT
|
||||
file system are simply not well suited to large random access devices.
|
||||
The HPFS solves the FAT file system problems mentioned here and many others
|
||||
but it is not derived in any way from the FAT file system. The architect of
|
||||
the HPFS started with a clean sheet of paper and designed a file system that
|
||||
can take full advantage of a multitasking environment and that will be able to
|
||||
cope with any sort of disk device likely to arrive on microcomputers during
|
||||
the next decade.
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="design.html">[HPFS Design]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="hpfs_vol.html">[HPFS Volumes]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
@@ -0,0 +1,84 @@
|
||||
<html><head>
|
||||
<title>HPFS: Fault Tolerance</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>Fault Tolerance</h1>
|
||||
</center>
|
||||
|
||||
The HPFS's extensive use of lazy writes makes it imperative for the HPFS to
|
||||
be able to recover gracefully from write errors under any but the most dire
|
||||
circumstances. After all, by the time a write is known to have failed, the
|
||||
application has long since gone on its way under the illusion that it has
|
||||
safely shipped the data into disk storage. The errors may be detected by the
|
||||
hardware (such as a "sector not found" error returned by the disk adapter),
|
||||
or they may be detected by the disk driver in spite of the hardware during a
|
||||
read-after-write verification of the data. The primary mechanism for handling
|
||||
write errors is called a hot fix. When an error is detected, the file system
|
||||
takes a free block out of a reserved hot fix pool, writes the data to that
|
||||
block, and updates the hot fix map. (The hot fix map is simply a series of
|
||||
pairs of double words, with each pair containing the number of a bad sector
|
||||
associated with the number of its hot fix replacement. A pointer to the hot
|
||||
fix map is maintained in the Spare Block.) A copy of the hot fix map is then
|
||||
written to disk, and a warning message is displayed to let the user know that
|
||||
all is not well with the disk device.
|
||||
<p>
|
||||
|
||||
Each time the file system requests a sector read or write from the disk
|
||||
driver, it scans the hot fix map and replaces any bad sector members with the
|
||||
corresponding good sector holding the actual data. This look aside translation
|
||||
of sector numbers is not as expensive as it sounds, since the hot fix list
|
||||
need only be scanned when a sector is physically read or written, not each
|
||||
time it is accessed in the cache. One of CHKDSK's duties is to empty the hot
|
||||
fix map. For each replacement block on the hot fix map, it allocates a new
|
||||
sector that is in a favorable location for the file that owns the data, moves
|
||||
the data from the hot fix block to tile newly allocated sector, and updates
|
||||
the file's allocation information which may involve rebalancing allocation
|
||||
trees and other elaborate operations). It then adds the bad sector to the bad
|
||||
block list, releases the replacement sector back to the hot fix pool, deletes
|
||||
the hot fix entry from the hot fix map, and writes the updated hot fix map to
|
||||
disk. of course, write errors that can be detected and fixed on the fly are
|
||||
not the only calamity that can befall a file system. The HPFS designers also
|
||||
had to consider the inevitable damage to be wreaked by power failures, program
|
||||
crashes, malicious viruses and Trojan horses, and those users who turn off
|
||||
the machine without selecting Shut-down in the Presentation Manager Shell.
|
||||
(Shutdown notifies the file system to flush the disk cache, update directories,
|
||||
and do whatever else is necessary to bring the disk to a consistent state.)
|
||||
<p>
|
||||
|
||||
The HPFS defends itself against the user who is too abrupt with the Big Red
|
||||
Switch by maintaining a Dirty FS flag in the Spare Block of each HPFS volume.
|
||||
The flag is only cleared when all files on the volume have been closed and
|
||||
all dirty buffers in the cache have been written out or, in the case of the
|
||||
boot volume since OS2.INI and the swap file are never closed), when Shutdown
|
||||
has been selected and has completed its work. During the OS/2 boot sequence,
|
||||
the file system inspects the Dirty FS flag on each HPFS volume and, if the
|
||||
flag is set, will not allow further access to that volume until CHKDSK has
|
||||
been run. If the Dirty FS flag is set on the boot volume, the system will
|
||||
refuse to boot the user must boot OS/2 in maintenance mode from a diskette
|
||||
and run CHKDSK to check and possibly repair the boot volume. In the event
|
||||
of a truly major catastrophe, such as loss of the Super Block or the root
|
||||
directory, the HPFS is designed to give data recovery the best possible
|
||||
chance of success. Every type of crucial file objects including Fnodes,
|
||||
allocation sectors, and directory blocks is doubly linked to both its parent
|
||||
and its children and contains a unique 32-bit signature. Fnodes also contain
|
||||
the initial pofiion of the name of their file or directory. Consequently,
|
||||
CHKDSK can rebuild an entire volume by methodically scanning the disk for
|
||||
Fnodes, allocation sectors, and directory blocks, using them to reconstruct
|
||||
the files and directories and finally regenerating the freespace bitmaps.
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="perform.html">[Performance Issues]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="app_hpfs.html">[Application Programs and the HPFS]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/fig1.gif
Normal file
|
After Width: | Height: | Size: 7.0 KiB |
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/fig2.gif
Normal file
|
After Width: | Height: | Size: 2.5 KiB |
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/fig3.gif
Normal file
|
After Width: | Height: | Size: 8.3 KiB |
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/fig4.gif
Normal file
|
After Width: | Height: | Size: 5.1 KiB |
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/fig5.gif
Normal file
|
After Width: | Height: | Size: 9.7 KiB |
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/fig6.gif
Normal file
|
After Width: | Height: | Size: 4.5 KiB |
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/figa.gif
Normal file
|
After Width: | Height: | Size: 2.2 KiB |
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/figb.gif
Normal file
|
After Width: | Height: | Size: 2.5 KiB |
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/figc.gif
Normal file
|
After Width: | Height: | Size: 3.4 KiB |
156
study/sabre/os/files/FileSystems/DesignGoalsHPFS/figs.html
Normal file
@@ -0,0 +1,156 @@
|
||||
<html><head>
|
||||
<title>HPFS: Illustrations</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>HPFS: Illustrations</h1>
|
||||
</center>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig1.gif" name="fig1">
|
||||
<img src="fig1.gif" alt="[Fig. 1]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 1</b>:
|
||||
This figure shows the overall structure of an HPFS volume. The most important
|
||||
fixed objects in such a volume are the Bootblock the Super Block, and the
|
||||
Spare Block. The remainder of the volume is divided into 8Mb bands. There is
|
||||
a freespace bitmap for each band and the bitmaps are located between alternate
|
||||
bands consequently, the maximum contiguous space which can be allocated to a
|
||||
file is 16Mb.
|
||||
<li><a href="hpfs_vol.html">HPFS Volume Structure</a>
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig2.gif" name="fig2">
|
||||
<img src="fig2.gif" alt="[Fig. 2]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 2</b>:
|
||||
This figure shows the overall structure of an Fnode. The Fnode is the
|
||||
fundamental object in an HPFS volume and is the first sector allocated to a
|
||||
file or directory. it contains control and access history information used
|
||||
by the file system, cached EAs and ACLs or pointers to same, a truncated
|
||||
copy of the file or directory name (to aid disk repair programs, and an
|
||||
allocation structure which defines the size and location of the file's storage.
|
||||
<li><a href="fnodes.html">Files and FNodes</a>
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig3.gif" name="fig3">
|
||||
<img src="fig3.gif" alt="[Fig. 3]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 3</b>:
|
||||
The simplest form of tracking for the sectors owned by a file is shown. The
|
||||
Fnode s allocation structure points directly to as many as eight sector runs.
|
||||
Each run pointer consists of a pair of 32-bit doublewords: a starting sector
|
||||
number and a length !n sectors.
|
||||
<li><a href="fnodes.html">Files and FNodes</a>
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig4.gif" name="fig4">
|
||||
<img src="fig4.gif" alt="[Fig. 4]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 4</b>:
|
||||
This figure demonstrates the technique used to track the sectors owned by a
|
||||
file with 9-480 sector runs. The allocation structure in the Fnode holds the
|
||||
roots for a B+ Tree of allocation sectors. Each allocation sector can describe
|
||||
as many as 40 sector runs. lf the file requires more than 480 sector runs,
|
||||
additional intermediate levels are added to the B+ Tree, which increases the
|
||||
number of possible sector runs by a factor of sixty for each new !evel.
|
||||
<li><a href="fnodes.html">Files and FNodes</a>
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig5.gif" name="fig5">
|
||||
<img src="fig5.gif" alt="[Fig. 5]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 5</b>:
|
||||
Here directories are anchored on an Fnode and are built up from 2Kb directory
|
||||
blocks. The number of entries in a directory block varies because the length
|
||||
of the entries depends on the filename. When a directory requires more than
|
||||
one block the blocks are organized as a B-Tree. This allows a filename to be
|
||||
located very quickly with a small number of disk accesses even when the
|
||||
directory grows very large.
|
||||
<li><a href="dirs.html">Directories</a>
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig6.gif" name="fig6">
|
||||
<img src="fig6.gif" alt="[Fig. 6]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 6</b>:
|
||||
A simplified sketch of the relationship between an application program, the
|
||||
OS/2 kernel, an installable file system, a disk drlver, and the physical disk
|
||||
device. The applicatIon issues logical file requests to the OS/2 kernel by
|
||||
calling the entry points for DosOpen, DosRead, DosWrlte, DosChgFilePtr, and
|
||||
so on. The kernel passes these requests to the appropriate installable file
|
||||
system for the volume holding the file. The installable file system translates
|
||||
the logical file requests into requests for reads or writes of logical sectors
|
||||
and calls a kernel File System Helper (FsHlp) to pass these requests to the
|
||||
appropriate disk drlver. The disk driver transforms the logical sector
|
||||
requests into requests for specific physical units, cylinders heads, and
|
||||
sectors, and issues commands to the disk adapter to transfer data between the
|
||||
disk and memory.
|
||||
<li><a href="ifs.html">Installable File Systems</a>
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="figa.gif" name="figa">
|
||||
<img src="figa.gif" alt="[Fig. A]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE A</b>:
|
||||
To find a piece of data, the binary tree is traversed from the root until
|
||||
the data is found or an empty subtree is encountered.
|
||||
<li><a href="sum.html">Summary</a>
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="figb.gif" name="figb">
|
||||
<img src="figb.gif" alt="[Fig. B]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE B</b>:
|
||||
In a balanced B-Tree, data is stored in nodes, more than one data item can
|
||||
be stored in a node, and all branches of the tree are the same length.
|
||||
<li><a href="sum.html">Summary</a>
|
||||
<p>
|
||||
|
||||
<center>
|
||||
<a href="figc.gif" name="figc">
|
||||
<img src="figc.gif" alt="[Fig. C]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE C</b>:
|
||||
A B+ Tree has internal nodes that point to other nodes and external nodes
|
||||
that contain actual data.
|
||||
<li><a href="sum.html">Summary</a>
|
||||
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
<a href="hpfs.html">[HPFS Home]</a>
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
122
study/sabre/os/files/FileSystems/DesignGoalsHPFS/fnodes.html
Normal file
@@ -0,0 +1,122 @@
|
||||
<html><head>
|
||||
<title>HPFS: Files and FNodes</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>Files and FNodes</h1>
|
||||
</center>
|
||||
|
||||
Every file or directory on an HPFS volume is anchored on a fundamental file
|
||||
system object called an Fnode (pronounced "eff node"). Each Fnode occupies a
|
||||
single sector and contains control and access history information used
|
||||
internally by the file system extended attributes and access control lists
|
||||
(more about this later) the length and the first 15 characters of the name
|
||||
of the associated file or directory and an allocation structure
|
||||
(<a href="#fig2">Figure 2</a>).
|
||||
<p>
|
||||
|
||||
An Fnode is always stored near the file or directory that it represents.
|
||||
The allocation structure in the Fnode can take several forms depending on the
|
||||
size and degree of contiguity of the file or directory.
|
||||
The HPFS views a file as a collection of one or more runs or extents of one
|
||||
or more contiguous sectors. Each run is symbolized by a pair of
|
||||
double-words--a 32-bit starting sector number and a 32-bit length in sectors
|
||||
(this is referred to as run length encoding).
|
||||
From an application program's point of view the extents are invisible; the
|
||||
file appears as a seamless stream of bytes. The space reserved for allocation
|
||||
information in an Fnode can hold pointers to as many aseight runs of sectors
|
||||
of up to 16Mb each . (This maximum run size is a result of the band size and
|
||||
free space bitmap placement only; it is not an inherent limitation of the file
|
||||
system.)
|
||||
Reasonably small files or highly contiguous files can therefore be described
|
||||
completely within the Fnode (<a href="#fig3">Figure 3</a>).
|
||||
<p>
|
||||
|
||||
HPFS uses a new method to represent the location of files that are too large
|
||||
or too frag-mented for the Fnode and consist of more than eight runs.
|
||||
The Fnode's allocation structure becomes the root for a B+ Tree of allocation
|
||||
sectors which in turn contain the actual pointers to the file's sector runs
|
||||
(see <a href="#fig4">Figure 4</a> and the sidebar, "B-Trees and B+ Trees").
|
||||
The Fnode's root has room for 12 elements. Each allocation sector can contain,
|
||||
in addition to various control information, as many as 40 pointers to sector
|
||||
runs. Therefore a two-level allocation B+ Tree can describe a file of 480
|
||||
(12x40) sector runs with a theoretical maximum size of 7.68Gb (12x40x16Mb)
|
||||
in the current implementation (although the 32-bit signed offset parameter
|
||||
for DosChgFilePtr effectively limits the sizes to 2Gb).
|
||||
In the unlikely event that a two-level B+ Tree is not sufficient to describe
|
||||
the highly fragmented file the file system will introduce additional levels
|
||||
in the tree as needed. Allocation sectors in the intermediate levels can hold
|
||||
as many as 60 intemal (nonterminal) B+ Tree nodes which means that the
|
||||
descriptive ability of this structure rapidly grows to numbers that are nearly
|
||||
beyond comprehension. For example a three level allocation B+ tree can describe
|
||||
a file with as many as 28 800 (12x60x40) sector runs. Run-length encoding and
|
||||
B+ Trees of allocation sectors are a memory-efficient way to specify a file's
|
||||
size and location but they have other s significant advantages.
|
||||
Translating a logical file offset into a sector number is extremely fast:
|
||||
the file system just needs to traverse the list (or B+ Tree of lists)
|
||||
of run pointers until it finds the correct range. It can then identify the
|
||||
sector within the run with a simple calculation.
|
||||
Run-length encoding also makes it trivial to extend the file logically if
|
||||
the newly assigned sector iscontiguous with the file's previous last sector
|
||||
the file system merely needs to increment the size double word of the file's
|
||||
last run pointer and clear the sector's bit in the appropriate freespace bitmap.
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig2.gif" name="fig2">
|
||||
<img src="fig2.gif" alt="[Fig. 2]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 2</b>:
|
||||
This figure shows the overall structure of an Fnode. The Fnode is the
|
||||
fundamental object in an HPFS volume and is the first sector allocated to a
|
||||
file or directory. it contains control and access history information used
|
||||
by the file system, cached EAs and ACLs or pointers to same, a truncated
|
||||
copy of the file or directory name (to aid disk repair programs, and an
|
||||
allocation structure which defines the size and location of the file's storage.
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig3.gif" name="fig3">
|
||||
<img src="fig3.gif" alt="[Fig. 3]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 3</b>:
|
||||
The simplest form of tracking for the sectors owned by a file is shown. The
|
||||
Fnode s allocation structure points directly to as many as eight sector runs.
|
||||
Each run pointer consists of a pair of 32-bit doublewords: a starting sector
|
||||
number and a length !n sectors.
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig4.gif" name="fig4">
|
||||
<img src="fig4.gif" alt="[Fig. 4]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 4</b>:
|
||||
This figure demonstrates the technique used to track the sectors owned by a
|
||||
file with 9-480 sector runs. The allocation structure in the Fnode holds the
|
||||
roots for a B+ Tree of allocation sectors. Each allocation sector can describe
|
||||
as many as 40 sector runs. If the file requires more than 480 sector runs,
|
||||
additional intermediate levels are added to the B+ Tree, which increases the
|
||||
number of possible sector runs by a factor of sixty for each new level.
|
||||
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="hpfs_vol.html">[HPFS Volume Structure]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="dirs.html">[Directories]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
BIN
study/sabre/os/files/FileSystems/DesignGoalsHPFS/hpfs.inf
Normal file
@@ -0,0 +1,70 @@
|
||||
<html><head>
|
||||
<title>HPFS: HPFS Volume Structure</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>HPFS Volume Structure</h1>
|
||||
</center>
|
||||
|
||||
HPFS volumes are a new partition type--type 7--and can exist on a fixed disk
|
||||
alongside of the several previously defined FAT partition types.
|
||||
IBM-compatible HPFS volumes use a sector size of 512 bytes and have a maximum
|
||||
size of 2199Gb (232 sectors).
|
||||
Although there is no particular reason why floppy disks can't be formatted
|
||||
as HPFS volumes Microsoft plans to stick with FAT file systems on floppy disks
|
||||
for the foreseeable future.
|
||||
(This ensures that users will be able to transport files easily between MS-DOS
|
||||
and OS/2 systems.)
|
||||
An HPFS volume has very few fixed structures (<a href="#fig1">Figure 1</a>).
|
||||
Sectors 0-15 of a volume (8Kb) are the Bootblock and contain a volume name,
|
||||
32-bit volume ID, and a disk bootstrap program. The bootstrap is relatively
|
||||
sophisticated (by MS-DOS standards) and can use the HPFS in a restricted
|
||||
mode to locate and read the operating system files wherever they might be found.
|
||||
Sectors 16 and 17 are known as the Super Block and the Spare Block respectively.
|
||||
The Super Block is only modified by disk maintenance utilities.
|
||||
It contains pointers to the free space bitmaps the bad block list the directory
|
||||
block band and the root directory.
|
||||
It also contains the date that the volume was last checked out and repaired
|
||||
with CHKDSK /F. The Spare Block contains various flags and pointers that
|
||||
will be discussed later it is modified although infrequently as the system
|
||||
executes. The remainder of the disk is divided into 8Mb bands.
|
||||
Each band has its own free space bitmap in which a bit represents each sector.
|
||||
A bit is 0 if the sector is in use and 1 if the sector is available.
|
||||
The bitmaps are located at the head or tail of a band so that two bitmaps are
|
||||
adjacent between alternate bands. This allows the maximum contiguous free space
|
||||
that can be allocated to a file to be 16Mb. One band located at or toward the
|
||||
seek center of the disk is called the directory block band and receives
|
||||
special treatment (more about this later). Note that the band size is a
|
||||
characteristic of the current implementation and may be changed in later
|
||||
versions of the file system.
|
||||
<p>
|
||||
|
||||
<center>
|
||||
<a href="fig1.gif" name="fig1">
|
||||
<img src="fig1.gif" alt="[Fig. 1]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 1</b>.
|
||||
This figure shows the overall structure of an HPFS volume.
|
||||
The most important fixed objects in such a volume are the Bootblock the Super
|
||||
Block, and the Spare Block.
|
||||
The remainder of the volume is divided into 8Mb bands.
|
||||
There is a freespace bitmap for each band and the bitmaps are located between
|
||||
alternate bands consequently, the maximum contiguous space which can be
|
||||
allocated to a file is 16Mb.
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="fat.html">[FAT File System]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="fnodes.html">[Files and Fnodes]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
81
study/sabre/os/files/FileSystems/DesignGoalsHPFS/ifs.html
Normal file
@@ -0,0 +1,81 @@
|
||||
<html><head>
|
||||
<title>HPFS: Installable File Systems</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>Installable File Systems</h1>
|
||||
</center>
|
||||
|
||||
Support for installable file systems has been one of the most eagerly
|
||||
anticipated features of OS/2 Version 1.2. it will make it possible to
|
||||
access multiple incompatible volume structures--FAT, HPFS, CD ROM, and
|
||||
perhaps even UNIX on the same OS/2 system at the same time, will
|
||||
simplify the life of network implementors, and will open the door to
|
||||
rapid file system evolution and innovation. Installable file systems
|
||||
are, however, only relevant to the HPFS insofar as they make use of
|
||||
the HPFS optional. The FAT file system is still embedded in the OS/2
|
||||
kernel, as it was in OS/2 1.0 and 1.1, and will remain there as the
|
||||
compatibility file system for some time to come. An installable file
|
||||
system driver (FSD) is analogous in many ways to a device driver. An
|
||||
FSD resides on the disk in a file that is structured like a
|
||||
dynamic-link library (DLL), typically with a SYS or IFS extension,
|
||||
and is loaded during system initialization by <tt>IFS=</tt> statements
|
||||
in the <tt>CONFIG.SYS</tt> file. <tt>IFS=</tt> directives are processed
|
||||
in the order they are encountered and are also sensitive to the order of
|
||||
<tt>DEVlCE=</tt> statements for device drivers. This lets you load a
|
||||
device driver for a nonstandard device, load a file system driver from
|
||||
a volume on that device, and so on. Once an FSD is installed and
|
||||
initialized, the kernel communicates with it in terms of logical requests
|
||||
for file opens, reads, writes, seeks, closes, and so on.
|
||||
The FSD translates these requests--using control structures and tables
|
||||
found on the volume itself--into requests for sector reads and writes for
|
||||
which it can call special kernel entry points called File System Helpers
|
||||
(FsHlps). The kernel passes the demands for sector I/O to the appropriate
|
||||
device driver and returns the results to the FSD
|
||||
(<a href="#fig6">Figure 6</a>).
|
||||
The procedure used by the operating system to associate volumes with
|
||||
FSDs is called dynamic mounting and works as follows. Whenever a volume
|
||||
is first accessed, or after it has been locked for direct access and then
|
||||
unlocked (for example, by a FORMAT operation), OS/2 presents identifying
|
||||
information from the volume to each of the FSDs in turn until one of them
|
||||
recognizes the information. When an FSD claims the volume, the volume is
|
||||
mounted and all subsequent file I/O requests for the volume are routed to
|
||||
that FSD.
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="fig6.gif" name="fig6">
|
||||
<img src="fig6.gif" alt="[Fig. 6]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE 6</b>:
|
||||
A simplified sketch of the relationship between an application program, the
|
||||
OS/2 kernel, an installable file system, a disk drlver, and the physical
|
||||
disk device. The applicatIon issues logical file requests to the OS/2 kernel
|
||||
by callng the entry points for DosOpen, DosRead, DosWrlte, DosChgFilePtr, and
|
||||
so on. The kernel passes these requests to the appropriate installable file
|
||||
system for the volume holding the file. The installable file system translates
|
||||
the logical file requests into requests for reads or writes of logical sectors
|
||||
and calls a kernel File System Helper (FsHlp) to pass these requests to the
|
||||
appropriate disk drlver. The disk driver transforms the logical sector requests
|
||||
into requests for specific physical units, cylinders heads, and sectors, and
|
||||
issues commands to the disk adapter to transfer data between the disk and
|
||||
memory.
|
||||
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="ea.html">[Extended Attributes]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="perform.html">[Performance Issues]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
38
study/sabre/os/files/FileSystems/DesignGoalsHPFS/index.html
Normal file
@@ -0,0 +1,38 @@
|
||||
<html><head>
|
||||
<title>HPFS</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>HPFS</h1>
|
||||
<b>High Performance File System</b>
|
||||
</center>
|
||||
|
||||
<p>
|
||||
Also available in OS/2 IPF format: <a href="hpfs.inf">hpfs.inf</a>.
|
||||
|
||||
<ul>
|
||||
<li><a href="design.html">Design Goals and implementation of the new
|
||||
High Performance File System</a>
|
||||
<li><a href="fat.html">FAT File System</a>
|
||||
<li><a href="hpfs_vol.html">HPFS Volume Structure</a>
|
||||
<li><a href="fnodes.html">Files and Fnodes</a>
|
||||
<li><a href="dirs.html">Directories</a>
|
||||
<li><a href="ea.html">Extended Attributes</a>
|
||||
<li><a href="ifs.html">Installable File Systems</a>
|
||||
<li><a href="perform.html">Performance issues</a>
|
||||
<li><a href="faultol.html">Fault Tolerance</a>
|
||||
<li><a href="app_hpfs.html">Application Programs and the HPFS</a>
|
||||
<li><a href="sum.html">Summary</a>
|
||||
<p>
|
||||
<li><a href="figs.html">Collection of Illustrations</a>
|
||||
</ul>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
@@ -0,0 +1,92 @@
|
||||
<html><head>
|
||||
<title>HPFS: Performance issues</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>Performance issues</h1>
|
||||
</center>
|
||||
|
||||
The HPFS attacks potential bofflenecks in disk throughput at multiple levels.
|
||||
It uses advanced data structures contiguous sector allocation, intelligent
|
||||
caching, read-ahead, and deffered writes in order to boost performance.
|
||||
First, the HPFS matches its data structures to the task at hand:
|
||||
sophisticated data structures (B-Trees and B+ Trees) for fast random access
|
||||
to filenames, directory names, and lists of sectors allocated to files or
|
||||
directories, and simple compact data structures (bitmaps) for locating chunks
|
||||
of free space of the appropriate size. The routines that manipulate these
|
||||
data structures are written in assembly language and have been painstakingly
|
||||
tuned, with special focus on the routines that search the freespace bitmaps
|
||||
for patterns of set bits (unused sectors). Next, the HPFS's main goal --its
|
||||
prime directive, if you will -- is to assign consecutive sectors to files
|
||||
whenever possible. The time required to move the disk's readowrite head from
|
||||
one track to another far out-weighs the other possible delays, so the HPFS
|
||||
works hard to avoid or minimize such head movements by allocating file space
|
||||
contiguously and by keeping control structures such as Fnodes and freespace
|
||||
bitmaps near the things they control.
|
||||
<p>
|
||||
|
||||
Highly contiguous files also help the file system make fewer requests of the
|
||||
disk driver for more sectors at a time, allow the disk driver to exploit the
|
||||
multisector transfer capabilities of the disk controller, and reduce the
|
||||
number of disk completion interrupts that must be serviced. Of course, trying
|
||||
to keep files from becoming fragmented in amultitasking system in which many
|
||||
files are being updated concurrently is no easy chore. One strategy the HPFS
|
||||
uses is to scatter newly created files across the disk--in separate bands,
|
||||
if poosible-so that the sectors allocated to the files as they are extended
|
||||
will not be interleaved. Another strategy is to reallocate approximately 4Kb
|
||||
of contiguous space to the file each time it must be extended and give back
|
||||
any excess when the file is closed. If an application knows the ultimate size
|
||||
of a new file in advance, it can assist the file system by specifying an
|
||||
initial file allocation when it creates the file. The system will then search
|
||||
all the free space bitmaps to find a run of consecutive sectors large enough
|
||||
to hold the file. That failing, it will search for two runs that are half
|
||||
the size of the file, and so on.
|
||||
<p>
|
||||
|
||||
The HPFS relies on several different kinds of caching to minimize the number
|
||||
of physical disk transfers it must request. Naturally, it caches sectors, as
|
||||
did the FAT file system. But unlike the FAT file system, the HPFS can manage
|
||||
very large caches efficiently and adjusts sector caching on a per handle basis
|
||||
to the manner in which a file is used. The HPFS also caches path names and
|
||||
directories, transforming disk directory entries into an even more compact and
|
||||
efficient in-memory representation. Another technique that the HPFS uses to
|
||||
improve performance is to preread data it believes the program is likely to
|
||||
need. For example, when a file is opened, the file system will pre-read and
|
||||
cache the Fnode and the first few sectors of the file's contents. If the file
|
||||
is an executable program or the history information in the file's Fnode shows
|
||||
that an open operation has typically been followed by an immediate sequential
|
||||
read of the entire file, the file system will preread and cache much more of
|
||||
the file's contents. When a program issues relatively small read requests, the
|
||||
file system always fetches data from the file in 2Kb chunks and caches the
|
||||
excess, allowing most read operations to be satisfied from the cache. Finally,
|
||||
the OS/2 operating system's support for multitasking makes it possible for the
|
||||
HPFS to rely heavily on lazy writes (sometimes called deferred writes or write
|
||||
behind) to improve performance. When a program requests a disk write, the data
|
||||
is placed in the cache and the cache buffer is flagged as dirty (that is,
|
||||
inconsistent with the state of the data on disk). When the disk becomes idle
|
||||
or the cache becomes saturated with dirty buffers, the file system uses a
|
||||
captive thread from a daemon process to write the buffers to disk, starting
|
||||
with the oldest data. In general, lazy writes mean that programs run faster
|
||||
because their read requests will almost never be stalled waiting for a write
|
||||
request to complete. For programs that repeatedly read, modify, and write a
|
||||
small working set of records, it also means that many unnecessary or redundant
|
||||
physical disk writes may be avoided. Lazy writes have their dangers, of course,
|
||||
so a program can defeat them on a per-handle basis by setting the write-through
|
||||
flag in the Open Mode parameter for DosOpen or it can commit data to disk on a
|
||||
per-handle basis with the DosBufReset function.
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="ifs.html">[Installable File Systems]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a> |
|
||||
<a href="faultol.html">[Fault Tolerance]</a> >
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||
147
study/sabre/os/files/FileSystems/DesignGoalsHPFS/sum.html
Normal file
@@ -0,0 +1,147 @@
|
||||
<html><head>
|
||||
<title>HPFS: Summary</title>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<center>
|
||||
<h1>Summary</h1>
|
||||
</center>
|
||||
|
||||
The HPFS solves all of the historical problems of the FAT file system. it
|
||||
achieves excellent throughput even in extreme casses--many very small files
|
||||
or a few very large files--by means of advanced data structures and techniques
|
||||
such as intelligent caching read-ahead and write-behind. Disk space is used
|
||||
economically because it is managed on a sector basis. Existing application
|
||||
programs will need modification to take advantage of the HPF'S's support for
|
||||
extended attributes and long filenames but these changes will not be difficult.
|
||||
All application programs will benefit from the HPFS's improved performance and
|
||||
decreased CPU use whether they are modified or not. This article is based on a
|
||||
prerelease version of the HPFS that was still undergoing modification and
|
||||
tuning. Therefore the final release of the HPFS may differ in some details
|
||||
from the description given here.
|
||||
<p>
|
||||
Most programmers are at at least passingly famiIiar with the data structure
|
||||
known as a binary tree. Binary trees are a technique for imposing a logical
|
||||
ordering on a collection of data items by means of pointers, without regard
|
||||
to the physical order of the data. In a simple binary tree, each node contains
|
||||
some data, including a key value that determines the node's logical position
|
||||
in the tree, as well as pointers to the node's left and right sub trees. The
|
||||
node that begins the tree is known as the root: the nodes that sit at the
|
||||
ends of the tree's branches are sometimes called the leaves. To find a
|
||||
particular piece of data, the binary tree is traversed from the root. At each
|
||||
node, the desired key is compared with the node's key: if they don't match,
|
||||
one branch of the node's sub tree or another is selected based on whether the
|
||||
desired key is less than or greater than the node's key. This process
|
||||
continues until a match is found or an empty sub tree is encountered
|
||||
(see <a href="#figa">Figure A</a>).
|
||||
Such simple binary trees, although easy to understand and implement, have
|
||||
disadvantages in practice. If keys are not well distributed or are added to
|
||||
the tree in a non-random fashion, the tree can become quite asymmetric,
|
||||
leading to wide variations in tree traversal times. In order to make access
|
||||
times uniform, many programmers prefer a particular type of balanced tree
|
||||
known as a B- Tree. For the purposes of this discussion, the important
|
||||
points about a B-Tree are that data is stored in all nodes, more than one
|
||||
data item might be stored in a node, and all of the branches of the tree
|
||||
are of identical length (see <a href="#figb">Figure B</a>).
|
||||
The worst-case behavior of a B-Tree is predictable and much better than that
|
||||
of a simple binary tree, but the maintenance of a B-Tree is correspondingly
|
||||
more complex. Adding a new data item, changing a key value, or deleting a
|
||||
data item may result in the splitting or merging of a node, which in turm
|
||||
forces a cascade of other operations on the tree to rebalance it. A B+ Tree
|
||||
is a specialized form of B-Tree that has two types of nodes: internal, which
|
||||
only point to other nodes, and external, which contain the actual data
|
||||
(see <a href="#figc">Figure C</a>).
|
||||
The advantage of a B+ Tree over a B- Tree is that the internal nodes of the
|
||||
B+Tree can hold many more decision values than the intenmediate-level nodes
|
||||
of a B-Tree, so the fan out of the tree is faster and the average length of
|
||||
a branch is shorter. This makes up for the fact that yell must always follow
|
||||
a B+ Tree branch to its end to get the data for which you are looking, whereas
|
||||
in a B-Tree you may discover the data at an interme-diate code or even at the
|
||||
root.
|
||||
|
||||
<table><tr>
|
||||
<th> </th><th align=left>FAT File System </th><th align=left>High Performance File System</th>
|
||||
</tr><tr>
|
||||
<td>Maximum filename length </td><td>11(in 8.3 format) </td><td>254</td>
|
||||
</tr><tr>
|
||||
<td>Number of dot (.) delimeters allowed </td><td>One </td><td>Multiple</td>
|
||||
</tr><tr>
|
||||
<td>File Attributes </td><td>Bit flags </td><td>Bit flags plus up to 64Kb of free-form ASCll of binary information</td>
|
||||
</tr><tr>
|
||||
<td>Maximium Path Length </td><td>64 </td><td>260</td>
|
||||
</tr><tr>
|
||||
<td>Miniumum disk space overhead per file </td><td>Directory entry (32 bytes) </td><td>Directory entry (length varies) + Fnode (512 bytes)</td>
|
||||
</tr><tr>
|
||||
<td>Average wasted space per file </td><td>1/2 cluster (typically 2Kb or more) </td><td>1/2 sector (256 bytes)</td>
|
||||
</tr><tr>
|
||||
<td>Minimum alocation unit </td><td>Cluster (typically 4Kb or more) </td><td>Sector (512 bytes)</td>
|
||||
</tr><tr>
|
||||
<td>Allocation info for files </td><td>Centralized in FAT on home track </td><td>Located nearby each file in its Fnode</td>
|
||||
</tr><tr>
|
||||
<td>Free disk space info </td><td>Centralized in FAT on home track </td><td>Located near free space in bitmaps</td>
|
||||
</tr><tr>
|
||||
<td>Free disk space described per byte </td><td>2Kb ( 1/2 cluster at 8 sectors /clustor)
|
||||
</td><td>4Kb (8 sectors)</td>
|
||||
</tr><tr>
|
||||
<td>Directory structure </td><td>Unsorted linear list, must be searched exhaustivily
|
||||
</td><td>Sorted B-Tree</td>
|
||||
</tr><tr>
|
||||
<td>Directory Location </td><td>Root directory on home track, others scattered
|
||||
</td><td>Localized near seek center of volume</td>
|
||||
</tr><tr>
|
||||
<td>Cache replacement strategy </td><td>Simple LRU
|
||||
</td><td>Modified LRU, sensitive to data type and usage history</td>
|
||||
</tr><tr>
|
||||
<td>Read ahead </td><td>None in MS-DOS 3.3 or earlier, primitive read-ahead optional in MS-DOS 4
|
||||
</td><td>Always present, sensitive to data type and usage history</td>
|
||||
</tr><tr>
|
||||
<td>Write behind </td><td>Not available </td><td>Used by default, but can be defeated on per-handle basis</td>
|
||||
</tr></table>
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="figa.gif" name="figa">
|
||||
<img src="figa.gif" alt="[Fig. A]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE A</b>:
|
||||
To find a piece of data, the binary tree is traversed from the root untill
|
||||
the data is found or an empty subtree is encountered.
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="figb.gif" name="figb">
|
||||
<img src="figb.gif" alt="[Fig. B]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE B</b>:
|
||||
In a balanced B-Tree, data is stored in nodes, more than one data item can
|
||||
be stored in a node, and all branches of the tree are the same length.
|
||||
<p>
|
||||
|
||||
|
||||
<center>
|
||||
<a href="figc.gif" name="figc">
|
||||
<img src="figc.gif" alt="[Fig. C]" border=0></a>
|
||||
</center>
|
||||
<p>
|
||||
<b>FIGURE C</b>:
|
||||
A B+ Tree has internal nodes that point to other nodes and external nodes
|
||||
that contain actual data.
|
||||
|
||||
|
||||
<p>
|
||||
<hr>
|
||||
|
||||
< <a href="app_hpfs.html">[Application Programs and HPFS]</a> |
|
||||
<a href="hpfs.html">[HPFS Home]</a>
|
||||
|
||||
<hr>
|
||||
|
||||
<font size=-1>
|
||||
Html'ed by <a href="http://www.seds.org/~spider/">Hartmut Frommert</a>
|
||||
</font>
|
||||
|
||||
</body></html>
|
||||