Andrew File System

The Andrew File System (AFS ) is a well skalierendes network protocol for distributed network file systems. Tens of thousands of workstations, tens of thousands of users and hundreds of file servers are not uncommon. The integrated in the client caching the protocol suitable for operation over the Internet. Classical network file systems such as NFS has predicted that secondary storage extensions and exchange server from a user perspective is completely transparent possible. This is realized by an extra level of abstraction between the paths by which users work, and the actual data objects of the AFS.

The concept of AFS is holistic. Addition to the exchange of files, it also includes user management, ( Kerberos-based ) authentication, data backup, if required, necessary for the cryptographic components clock synchronization between clients and servers. In practice, today, a separate Kerberos server, and a separate backup system may be used.

The various functions required for AFS (client, file server, database server) are strictly separated from each other and run typically on different physical machines. AFS server processes never run on computers under user control (eg, workstations) as is common, for example, in Windows SMB shares.

A local cache on AFS client relieves the file server and improves performance - especially when operating over WANs. A cache consistency guarantee is built into the protocol. Authentication of users is done on the file servers - not such as NFS ( uncertain) on the clients. Nevertheless not each session a user requires a separate explicit link to a file server, as is the case for example with SMB. Access rights are defined by ACLs, but only per directory. AFS enables users to establish a centrally managed, single name space for all clients of a cell. AFS servers usually operate on Linux, Solaris, or AIX, but other Unix variants are supported as a server platform.

There are several programs that implement the AFS as a protocol. AFS clients are available for a variety of operating systems - typically free of license fees. Powerful AFS server are free of license fees available for Linux and other Unix operating systems. AFS server with special functions are commercially available.

The AFS controlled data replication, but not in real time. The replication needs (which of course is automated ) are triggered by the administrator. It is not economical in the AFS, (eg once per minute) to replicate data often.

• 3.1 weaknesses
• 3.2 Countermeasures

• 5.1 cache

• 7.1 AFS, Transarc AFS, IBM AFS
• 7.2 OpenAFS
• 7.3 Arla
• 7.4 MR - AFS
• 7.5 Hostafs
• 7.6 kAFS
• 7.7 YFS
• 7.8 Looking to the future

• 8.1 Other Restrictions

• 9.1 Furnishings vs. operation
• 9.2 effort for normal users
• 9.3 backup

Structure of the AFS

Independent administrative units in the AFS hot cells. A cell comprising one or more database servers ( the only objects that need to know an AFS client on a cell at the beginning ) and one or more file servers. AFS clients are usually (in Windows this can be otherwise) assigned to a " home cell ", but exists in the existing AFS implementations no central process that results of all clients that book. On file servers are data partitions, which in turn contain instances of volumes. Volume can instances symbolic links to other volume instances include ( in other AFS cells ), which is used to span a tree, which forms the file name space of the AFS. A defined volume (usually root.afs ) is from any AFS client to a defined point ( / afs Unix ) mounted in the file system and forms the root of this tree, but also cycles in the directory structure are possible through the symbolic links.

Cells

Worldwide, there are numerous AFS cells - especially larger institutions such as universities are among them. Cells are managed independently and may also be public. Public cells are characterized by the following properties:

• All AFS database server and AFS file servers have public IP addresses
• The database server of the cell have been made ​​public (either by entry into a special file on OpenAFS or by publication in the DNS).

Cells can also express mutual trust, which enables users of a cell in ACLs of AFS directories can be given rights. This trust is realized by corresponding Kerberos mechanisms.

Volumes

The term volume is within the framework of AFS for two things:

• An entry in the VLDB (Volume Database ), pointing to different instances of a volume on one or more file servers, the same AFS cell.
• Object on a file server that contains directories, files and references to other volumes. This article is used for such an object to better distinguish the notion of a volume instance or instance.

Volumes and volume instances are exclusively managed by the administrator. They have a modifiable maximum size. This is similar to using a quota, but does cover the volume and not individual users. There are four types of volume instances:

For all volume instances each statistic is managed by the file server, divided into the number of hits after Reading / Writing, Local area network / other network and some other criteria are recorded. OpenAFS file server in addition have a mode to spend extensive logging information about accesses to instances - either directly to other programs ( piped ).

File Server

AFS file servers include one or more data partitions, which in turn contain volume instances. The AFS network protocol principle does not care in which format the volumes stored on the data partitions. Allen AFS implementations, however, have in common is that if you have a partition on the file server looks at, one does not recognize the file structure of the AFS name space.

It is therefore not possible to release the data partitions via another file-sharing protocol in addition.

RW instances can be moved between servers during production operation - read and write access to the data of the instance is still possible. Thus, the maintenance of file servers is possible without losing access to the data stored there.

In the now mostly - used AFS implementation ( OpenAFS ) of the file server consists of multiple processes (which partly consist of multiple threads ):

• Volserver - this server process is mainly used by administrators. It provides functions that each whole volume instances relating (eg volume cloning, volume on or off, send volume through the network, ...)
• Salvager - The Salvager tests and repairs the AFS own administrative structures to the host partitions on a file server. That is, for example, after a crash necessary ( and then passes automatically ) to ensure the consistency of the stored data.

Since AFS is only one protocol, but a tape robot behind a file server also, for example, hide the AFS files on tertiary storage media stores (eg MR - AFS).

File server can have multiple IP addresses. AFS clients roam the failure of a file server network interfaces easily to the next. Client test for this reason regular reachability to all file server network interfaces with which they have to do.

Database server

The database servers are networked together and manage two or more databases. Required are:

• PTDB ( Protection DataBase ) - manages user of the cell and user groups. A special feature is that users themselves create groups within certain limits, edit and use ACLs in the AFS. Note: This database is not a directory service for user data such as home directories, e- mail addresses or passwords.
• VLDB (Volume DataBase ) - keeps a record of volumes ( see later in the article ) on file servers. It also stores the list of each file server of the assigned IP addresses.

The following databases are also quite common for:

• BDB (Backup DataBase ) - managed tapes that were described by special AFS server processes to perform backups of data.
• KDB (Kerberos DataBase ) - this database maintains user passwords (actually Kerberos Key ). However, the protocol used between AFS client and server KDB is a precursor of the already outdated protocol Kerberos v4. Newly built cells use today typically have a Kerberos v5 based server that is operated independently of the AFS databases.

All databases per database server managed by one process. Where the protocol is used UBIK. This allows even read and write access to the AFS databases is always possible if still reach more than half of the database server over the network. For read-only access an attainable database server is needed. So there are 5 database server, such as a will be migrated to a new machine and the failure of another would still not cost- write access. If the failed database servers are back online, it will automatically update the database from one another.

The complex synchronization mechanism requires exact synchronization of the internal clocks of the database server. If the times of any two database server differ by more than 10 s, locks the database write access.

Database servers are the only objects that need to know an AFS client when it tries to access a given cell. This may create a local file ( CellServDB ) or via the Domain Name System to happen ( about AFSDB resource record ).

Other server processes

The bosserver comes on all AFS servers to use. Similar to the init process on Unix systems it maintains a list of processes that have to run on a server. The running processes have an AFS server then as a database server, file server, or both ( not recommended). Can be managed over the network This list and a few other things.

In some AFS cells called update server and update clients are used, update the other server software ( for example, file server processes ) as needed.

A so-called butc comes on AFS tape controllers (read: AFS backup servers ) are used to receive data from file servers and save on tape or on disks.

Network protocol

AFS works today exclusively via UDP, but with RX exists an abstraction layer that allows in principle also other protocols such as TCP - there are plans for exactly realizing the OpenAFS.

The Rx protocol is operating in authenticated mode (read: if a user does not work without registration by previously ) always signed - usually encrypted. This refers, for example, also to transfers between AFS Client and AFS file server.

AFS is very sensitive with regard to firewalls. The following ( UDP ) ports must be connected between servers and clients, and between the servers to each other:

• For AFS in general: 7000, 7001, 7002, 7003, 7005, 7007
• If the AFS backup system is used, then in addition: 7021, 7025-7032
• If Kerberos 5 is used, then in addition: 88

Aside from not currently known security vulnerabilities all these ports are considered safe, so they can also be accessing the internet.

AFS works with fixed port numbers, and therefore has no problem with conventional NAT routers.

Security

The safety of AFS is ensured that each AFS server ( database - like file server) receives a cell -wide symmetric key ( shared secret ). This key is also known as the Kerberos server and can therefore be used for this, to authenticate users reliable. The key is 56 bits wide and thus no longer state-of -the-art.

Data transfers are also signed with a 56 -bit-wide session key and encrypted if necessary with an AFS own algorithm called fcrypt.

For anonymous access to the AFS (read: whenever a user has no AFS token) is no possibility of the file server securely authenticate, which neither the integrity nor the confidentiality of data transfers can be ensured for the client.

Weaken

If a file server compromised and the cell key falls into the hands of an attacker, it is this possible to act with superuser privileges on all file servers to read data from all users and also to change it. DFS, the " former successor " of AFS, does away with this problem for AFS is still no solution in sight.

The low key width is also a problem and moves brute- force attacks in the realm of possibility. The use of session keys the risk is still relatively low and not comparable to the weakness of WEP for example.

The lack of integrity check for anonymous access is a critical vulnerability, as in the common AFS client version " OpenAFS " a shared cache will be used. Anonymous fetched from the file server files may also returned logged AFS users when they access those. An attacker can - if you do not take countermeasures - Lever out with a little effort, the integrity check for registered users. This vulnerability is not critical for single-user machines on which users work only authenticated. However, multiuser systems are particularly at risk. It is currently not known practically carried out attack.

Countermeasures

Counter the problem of cell -wide key following organizational measures must be taken:

• Secure AFS server paranoid and only activate the most important services it
• Keep all AFS server in closed spaces and restrict access to server manager
• Keep AFS keys in an encrypted file system. The security of this measure has decreased by more recent evidence about possible physical attacks on DRAM devices

Against the low key width helps only a re-implementation of the security layer of the RPC protocol used ( Rx). There are companies that offer AFS programming services and tackle payment such problems. A regular key change reduces the risk of successful brute- force attacks.

To exclude the described attacks against the integrity of transmitted data, you have to stop on the respective client anonymous AFS access. This is practical only on machines that have the no normal user authenticated access ( shell accounts, FTP, WebDAV, ...). All services must always work on such a system with a token. Also cron jobs should not go forgotten.

File system semantics

To simplify the file name space in AFS is the administrator usually built cycle-free. A guarantee, however, there can not be, as soon as users get the right to create volume mount points or modify rights. This can, for example, for backup software be a problem.

The file system has three types of objects:

• Directories - these contain files, other directories and mount points an ACL that controls access rights.
• Files - files in modern AFS cells (eg from OpenAFS 1.4) - if the client and server support - be larger than 2 GB. You have exactly one data stream, Unix - standard metadata such as user ID and group ID. The Unix Permissions are however not used for Autorisationszwecke. Multiple hard links to files can exist, but only if they are located in the same directory.
• Symbolic links - These act like one is used by Unix. Links whose target has a special form, are interpreted by the AFS client as a volume mount point. In their place, then the contents of the home directory of another volume is mounted.

The administrator of a cell defines the name space by each other depends volumes well structured. Starting with the standard Volume root.cell you then accesses eg on volumes such as home directories, software, projects and temp and depends, for example in the home directories volume more home.ernie with the name home.bert, a .... The path to Bert looks eg like this:

/ afs / meine.zelle / home / bert

Notes:

• The path of a directory / file says nothing about the file server is accessed. The same goes for mount points.
• Also the volumes that you walk through along a path going from the path out not necessarily mean, however, can be determined this example, from the volume mount points.

On operating systems, where the concept of symbolic links is foreign (eg Windows), symbolic links appear as directories in the file name space of the AFS. Newer Windows clients include appropriate extensions to express such links as Junction Points and shell extensions to deal with it.

The AFS protocol supports network-wide file locks, but only so-called advisory locks ( flock () ), no byte range locks ( lockf ()). The OpenAFS Windows client is able to locally Byte Range Locks to emulate since version 1.5.10. Here, local applications can use this kind Locks on the client machine, the AFS client will lock the corresponding files on the file server but simple advisory locks.

The display of free and used memory of the mounted AFS (Unix ) is a fantasy figure. Principle can only be determined per directory in a distributed network file system of the free or used memory. The Windows client is able to report back the free space per directory applications.

AFS client

AFS clients are computers (eg workstations) that can access the AFS file space name. Under Unix operating systems this is a kernel extension necessary. This is done either via a generic file system drivers such as FUSE ( Arla AFS client ) or through a more comprehensive AFS - specific kernel module ( OpenAFS ). In both cases, additional user-space processes are needed to work out the kernel drivers. The OpenAFS Windows clients based on a model developed for AFS Redirector, that works with a Windows service.

Cache

AFS client (also cache manager ) are able to cache large amounts of data from file servers, where not all the files, but little piece adjustable sizes to be stored. The optimal size of such caches depends on usage patterns and can be many gigabytes.

The cache integrity is guaranteed by the AFS. A file fragment that is stored by the cache manager, is valid until the appropriate AFS server is active invalidated. This is done for RW instances such as when the file has been modified by another AFS client, wherein RO instances, for example when the administrator initiates replication.

Be cached active only reads. Write accesses are even buffered, however, when an open for write access file is closed, the close () command blocks until all data has been written to the file server.

The cache is persistent with OpenAFS for Unix. The cache integrity is realized after reboot by comparing the modification time stamp of files with the file server. The persistence of the cache makes might and in local networks, the use of large caches to increase speed sense.

On Windows, the cache consists of a single file that is used by memory mapping. The maximum amount of virtual memory (4 GB on 32 -bit system ) is therefore an insurmountable hurdle for the cache size.

Under OpenAFS for Unix systems, the cache consists of many files in a directory. At the file system where this directory is located, increased demands are made:

• No journal
• High POSIX compatibility
• Under Linux Ext2 best

Openafs also allows the use of the main memory (RAM) instead of a directory on the hard disk for the cache (option afsd - memcache ).

Supported Platforms

AFS is supported on many platforms. This is easier to implement for the AFS server processes, as for AFS client as no kernel extensions are necessary. There are several projects that implement the AFS protocol completely or partially - in this case a throbbing non exhaustive list:

Legend:

For AFS server you should as far as possible recourse to the respective recommended platforms. Although there exist, for example, experimental AFS server versions for Windows or old AFS server versions of Irix, but these are not officially supported or free of errors.

Transarc AFS and its successors have a NFS server on the client, the other platforms for which there is no client via NFS can provide access to the AFS namespace. However, it is known only from Solaris that on current actual OpenAFS client will not yet supported. Principle, however, should any current userspace server process (eg Samba user space NFS server, WebDAV, ...) easily share files from the AFS. Without special adaptations to the server software only anonymous access so but its possible.

AFS implementations, Historical

AFS, Transarc AFS, IBM AFS

AFS was originally a university project at Carnegie Mellon University and included a client and a server implementation. It was marketed by the company Transarc later under the name Transarc AFS. Transarc was acquired by IBM, AFS marketed under the name IBM AFS. IBM AFS was in 2000 under an open source license (IBM Public License ) free - it's called since OpenAFS and is actively developed. Furthermore, however, are most numerous Transarc and IBM AFS server in use.

OpenAFS

OpenAFS is the most actively maintained AFS implementation.

The main focus of the development of OpenAFS is currently for servers on

• Linux
• AIX
• Solaris

General rule is that the AFS server is the host operating system depends only to a small extent. It should therefore also on an older version of Linux to be possible, for example, the server (which typically works exclusively in user space ), to translate and to operate. Exceptions are server versions, the special modifications to the host file system to make (so-called inode server). These require additional kernel modules and are practically no longer used for new AFS installations.

Supported client platforms are

• Linux. Since the quantity needed for the client kernel module is open source and any kernel patches needed, one can translate it for any Linux distributions.
• Windows 2000 and Windows XP
• Mac OS X 10.4 (Tiger)
• AIX
• Solaris. Warning: The OpenAFS client support for Solaris before 2.6 is removed from the development version of OpenAFS - OpenAFS 1.4 still supported from Solaris 2.0.

Clients for older platforms - for example, for older Windows versions can be found on OpenAFS by looking for something in the old OpenAFS releases.

As part of the DCE Distributed File System DFS standards was developed as a successor of AFS. This, inter alia, provides the following benefits:

• A secret key per server, not like with AFS per cell
• ACLs per file and per directory not only

The DFS was despite backward compatibility with AFS but no success, since its use was linked to high license fees.

Arla

The Arla project came at a time when there was no free AFS implementation and Transarc AFS was associated with royalty payments. It was independent of the " AFS -mainstream " (AFS ... OpenAFS ) at KTH developed as open source software. Until now, there is only one client implementation, but this covers some of OpenAFS unsupported platforms.

MR - AFS

MR - AFS (Multi -Resident AFS) was established as a commercial development of Transarc AFS. MR - AFS ' strength is that the file server files from the AFS namespace on tertiary storage ( tape, optical disk, ...) are able to outsource. The file server to write in a HSM file system and leave the actual migration decisions of the HSM software. In this case, normal AFS client with MR - AFS servers exchange data. MR - AFS deals exclusively with the server software. MR - AFS -specific features are incorporated, for example in the command line tools of OpenAFS. The future of MR - AFS is uncertain, as the only developer is already in retirement.

Hostafs

Hostafs is a small AFS server implementation that aims to disguise normal directories as volumes and release by AFS. In this way you can make available, for example, CD-ROMs in the AFS. However Hostafs provides no access protection mechanisms such as ACLs are available - all shares are readable.

KAFS

This AFS implementation consists of a client that is implemented as a Linux kernel module. He is part of the standard Linux kernel. The client, however, is not meant for productive AFS operation, but, for example, to boot from the network, if the administrator wants to really hold everything in AFS. It has no way of authenticating to the AFS access, supports read-only access and speaks only with file servers. The latter means that the to-use file server you have to specify it explicitly - the module can not ask to the vlserver.

YFS

Because of dissatisfaction with the organizational mechanisms of development of the AFS protocol some OpenAFS developers are working on a commercial fork of OpenAFS with Name YFS. This Fork on the one hand to deal with the AFS protocol and with the massively improved YFS protocol. There is currently no official release (as of January 2013).

A look into the future

At the Rechenzentrum Garching is an AFS server (with appropriate flowing into the OpenAFS client client modifications ) with OSD (Object Storage) in development. The metadata ( permissions, timestamps, directory structures) will continue to be managed by the AFS server, the data is, however, to so-called Object Storage Servers, then communicates directly with those of the client. In this way, for example, files can be on multiple servers (striping) and be read and written much more quickly.

Restrictions, limits

• AFS can ( be actively informed about changes on the server machine ) does not reliably work with NAT routers because of the callbacks principle. Rule of thumb: It's not a NAT router in between must apply for each possible pair of computers of an AFS cell - From version 1.4.1 OpenAFS works better with IP NAT together.
• AFS works only with IPv4. The support of IPv6 would require changes to the schemes of the AFS database as well as to RPCs of the database server.
• The AFS client is not designed for extremely large amounts of data. This is due to the organization of the cache manager, which, however, regardless of their size can not many of them manage efficiently while Dateistückchen exorbitant size. This restriction only applies to OpenAFS clients prior to version 1.4.0.
• On Unix operating systems, the widespread OpenAFS client uses the GIDs ( Unix group IDs) 0x7f0 to 0xBF00. In addition to use these groups for any other purpose is a security risk.
• AFS does not support network-wide byte - range locks. The OpenAFS Windows client simulate byte - range locks locally. A similar function, there will be soon also for the OpenAFS Linux client.
• Each computer can be client and server for each an AFS cell. It is not possible, similar to a web server to use several AFS cells via an AFS server. Of course, nothing speaks against server virtualization. Clients, irrespective of their home cell with as many cells exchange data simultaneously.
• In AFS namespace, only the objects directory, file, symbolic link and Volume Mount Points ( a special form of the symbolic link ) are known. Pipes, device files or sockets are not supported.
• Maximum of 254 file servers are allowed per cell.
• 255 data partitions are supported per file server.
• The block size in the AFS is 1 Kbyte and can not be changed.
• Per data partition 4 Tebibyte (32-bit * block size) are easily usable with OpenAFS file servers.
• Some RPCs of the file server cause invalid return values ​​, if you exceed this limit. From the File Server Version 1.6.2 is but it is very simple for regular users.
• Volumes Up to 2 ^ 31-1 blocks are large ( about 2 Tebibyte ). This restriction is marginal, since the goal should always be to volumes can be moved easily - so small - to keep. Since OpenAFS 1.4.0, larger volumes are possible, but the maximum adjustable quota is still 4TiB.
• Volume names can (instance extensions such as. Readonly and. Backup, not added ) a maximum of 22 characters long.
• AFS directories are static data structures with a maximum capacity of 64435 entries ( dentrys ). The number of entries is reduced when one or more entries have names longer than 15 characters.
• Each ACL (ie positive and negative ACLs independently ) of directories can have a maximum of 20 entries.
• AFS does not speak any automatic replication. Data is written to a RW - instance and possibly later copied manually (or scripted ) in RO instances. However, only one instance per - RW Volume exist.

Various programs interfere with it when they run in AFS. A list of known problem children and solutions can be found here.

Other restrictions

• AFS is not suitable for the storage of databases.
• AFS is not suitable as a mail server backend. While there are examples of AFS cells in which messages are placed directly in the home directories of users, but this is technically demanding. Moreover, such solutions scale poorly for many users and the gain is minimal.

Administration expenses

Device vs. operation

Setting up an AFS cell is much more difficult than, for example, the application of an SMB share or an NFS export. The cryptographic security of authentication by Kerberos requires a certain effort, which is independent of the size of the cell. In addition, the design costs of the cell time.

Its advantages plays AFS in the following situations:

• If scalability is important (keyword: Exponential growth of the data )
• When the data is already extremely large. AFS cells with hundreds of terabytes are no problem.
• If the security plays an important role.
• When users require high flexibility in the allocation of rights
• If much is to be automated. AFS can be completely controlled via command -line tools.
• Is mandatory if cross-platform access to data. AFS covers the platforms Unix, Mac OS X and Windows.

If the AFS cell runs first, the work of the AFS administrator on the upgrade and replace if necessary servers limited. Administration costs will be extremely low compared to the number of users and amount of memory. Cells with many terabytes of data and thousands of users come of it may include an administrator from office.

Expenses for normal users

The cost of user should not be underestimated - they are and ACLs are generally a concept that is slowly gaining importance, especially in the Unix world unfamiliar per - directory ACLs.

As a sensible strategy AFS home directories has proven to be provided with certain standard paths that express the level of security (eg ~ / public ~ / secret ) and the user so, aside from exceptional cases, stay away from ACLs.

However, since a user's manual should not be abstract, an AFS administrator is such a write for the own cell and usually even consider local features in it.

Backup

AFS is not supported by many manufacturers of backup solutions. The reasons for this are complex. A separate backup solution is, however, comparatively quickly programmed in the form of some shell scripts.

Institutions, which use AFS

• Max Planck Institute for Human Cognitive and Brain Sciences
• University of Hohenheim - operates the oldest AFS cell in German-speaking
• Max Planck Institute for Plasma Physics
• Chemnitz University of Technology
• Georgius Agricola -Gymnasium Chemnitz
• University of Paderborn
• Max Planck Institute for Human Cognitive and Brain Sciences
• CERN
• Institut national de physique Nucléaire et de physique of particules
• Stanford Linear Accelerator Center
• Deutsches Elektronen -Synchrotron
• Paul Scherrer Institute
• Ruprecht -Karls- University of Heidelberg
• Physics student at the Vienna University of Technology
• Technical University of Braunschweig
• Technical University of Berlin
• Graz University of Technology - SPSC
• Friedrich -Schiller- University Jena
• Leibniz-Rechenzentrum
• Lower Saxony Police
• Vienna University of Economics
• Rheinische Friedrich- Wilhelms-Universität Bonn ( until 1.12.2013 )
• University of Cologne
• Humboldt -Universität zu Berlin
• Georg -August- University Göttingen, at the Institute of computer science
• University of Applied Sciences Munich
• Eberhard Karls University of Tübingen