Chapter 4: Common Definitions and Data Structures

e chap3 Chapter 3: Cache Manager Architecture

Section 3.1: Overview

The AFS Cache Manager is a kernel-resident agent with the following duties and responsibilities:
This differs from the semantics expected from the single-machine, local unix environment, where writes performed on one open file descriptor are immediately visible to all processes reading the file via their own file descriptors. Thus, instead of the standard "last writer wins" behavior, users see "last closer wins" behavior on their AFS files. Incidentally, other DFSs, such as NFS, do not implement full unix semantics in this case either.
To support this transparent access paradigm, the Cache Manager proceeds to:
This chapter takes a tour of the Cache Manager's architecture, and examines how it supports these roles and responsibilities. First, the set of AFS agents with which it must interact are discussed. Next, some of the Cache Manager's implementation and interface choices are examined. Finally, the server's ability to arbitrarily dispose of callback information without affecting the correctness of the cache consistency algorithm is explained.

Section 3.2: Interactions

The main AFS agent interacting with a Cache Manager is the File Server. The most common operation performed by the Cache Manager is to act as its users' agent in fetching and storing files to and from the centralized repositories. Related to this activity, a Cache Manager must be prepared to answer queries from a File Server concerning its health. It must also be able to accept callback revocation notices generated by File Servers. Since the Cache Manager not only engages in data transfer but must also determine where the data is located in the first place, it also directs inquiries to Volume Location Server agents. There must also be an interface allowing direct interactions with both common and administrative users. Certain AFS-specific operations must be made available to these parties. In addition, administrative users may desire to dynamically reconfigure the Cache Manager. For example, information about a newly-created cell may be added without restarting the client's machine.

Section 3.3: Implementation Techniques

The above roles and behaviors for the Cache Manager influenced the implementation choices and methods used to construct it, along with the desire to maximize portability. This section begins by showing how the VFS/vnode interface, pioneered and standardized by Sun Microsystems, provides not only the necessary fine-grain access to user file system operations, but also facilitates Cache Manager ports to new hardware and operating system platforms. Next, the use of unix system calls is examined. Finally, the threading structure employed is described.

Section 3.3.1: VFS Interface

As mentioned above, Sun Microsystems has introduced and propagated an important concept in the file system world, that of the Virtual File System (VFS) interface. This abstraction defines a core collection of file system functions which cover all operations required for users to manipulate their data. System calls are written in terms of these standardized routines. Also, the associated vnode concept generalizes the original unix inode idea and provides hooks for differing underlying environments. Thus, to port a system to a new hardware platform, the system programmers have only to construct implementations of this base array of functions consistent with the new underlying machine.
The VFS abstraction also allows multiple file systems (e.g., vanilla unix, DOS, NFS, and AFS) to coexist on the same machine without interference. Thus, to make a machine AFS-capable, a system designer first extends the base vnode structure in well-defined ways in order to store AFS-specific operations with each file description. Then, the base function array is coded so that calls upon the proper AFS agents are made to accomplish each function's standard objectives. In effect, the Cache Manager consists of code that interprets the standard set of unix operations imported through this interface and executes the AFS protocols to carry them out.

Section 3.3.2: System Calls

As mentioned above, many unix system calls are implemented in terms of the base function array of vnode-oriented operations. In addition, one existing system call has been modified and two new system calls have been added to perform AFS-specific operations apart from the Cache Manager's unix 'emulation' activities. The standard ioctl() system call has been augmented to handle AFS-related operations on objects accessed via open unix file descriptors. One of the brand-new system calls is pioctl(), which is much like ioctl() except it names targeted objects by pathname instead of file descriptor. Another is afs call(), which is used to initialize the Cache Manager threads, as described in the section immediately following.

Section 3.3.3: Threading

In order to execute its many roles, the Cache Manager is organized as a multi-threaded entity. It is implemented with (potentially multiple instantiations of) the following three thread classes:

Section 3.4: Disposal of Cache Manager Records

The Cache Manager is free to throw away any or all of the callbacks it has received from the set of File Servers from which it has cached files. This housecleaning does not in any way compromise the correctness of the AFS cache consistency algorithm. The File Server RPC interface described in this paper provides a call to allow a Cache Manager to advise of such unilateral jettisoning. However, failure to use this routine still leaves the machine's cache consistent. Let us examine the case of a Cache Manager on machine C disposing of its callback on file X from File Server F. The next user access on file X on machine C will cause the Cache Manager to notice that it does not currently hold a callback on it (although the File Server will think it does). The Cache Manager on C attempts to revalidate its entry when it is entirely possible that the file is still in sync with the central store. In response, the File Server will extend the existing callback information it has and deliver the new promise to the Cache Manager on C. Now consider the case where file X is modified by a party on a machine other than C before such an access occurs on C. Under these circumstances, the File Server will break its callback on file X before performing the central update. The Cache Manager on C will receive one of these "break callback" messages. Since it no longer has a callback on file X, the Cache Manager on C will cheerfully acknowledge the File Server's notification and move on to other matters. In either case, the callback information for both parties will eventually resynchronize. The only potential penalty paid is extra inquiries by the Cache Manager and thus providing for reduced performance instead of failure of operation.
This chapter discusses the definitions used in common by the File Server and the Cache Manager. They appear in the common.xg file, used by Rxgen to generate the C code instantiations of these definitions.

Section 4.1: File-Related Definitions

Section 4.1.1: struct AFSFid

This is the type for file system objects within AFS.

Fields

Section 4.2: Callback-related Definitions

Section 4.2.1: Types of Callbacks

There are three types of callbacks defined by AFS-3:

Section 4.2.2: struct AFSCallBack

This is the canonical callback structure passed in many File Server RPC interface calls.
Fields

Section 4.2.3: Callback Arrays

AFS-3 sometimes does callbacks in bulk. Up to AFSCBMAX (50) callbacks can be handled at once. Layouts for the two related structures implementing callback arrays, struct AFSCBFids and struct AFSCBs, follow below. Note that the callback descriptor in slot i of the array in the AFSCBs structure applies to the file identifier contained in slot i in the fid array in the matching AFSCBFids structure.

Section 4.2.3.1: struct AFSCBFids


Fields

Section 4.2.3.2: struct AFSCBs


Fields

Section 4.3: Locking Definitions

Section 4.3.1: struct AFSDBLockDesc

This structure describes the state of an AFS lock.
Fields

Section 4.3.2: struct AFSDBCacheEntry

This structure defines the description of a Cache Manager local cache entry, as made accessible via the RXAFSCB GetCE() callback RPC call. Note that File Servers do not make the above call. Rather, client debugging programs (such as cmdebug) are the agents which call RXAFSCB GetCE().
Fields

Section 4.3.3: struct AFSDBLock

This is a fuller description of an AFS lock, including a string name used to identify it.
Fields

Section 4.4: Miscellaneous Definitions

Section 4.4.1: Opaque structures

A maximum size for opaque structures passed via the File Server interface is defined as AFSOPAQUEMAX. Currently, this is set to 1,024 bytes. The AFSOpaque typedef is defined for use by those parameters that wish their contents to travel completely uninterpreted across the network.

Section 4.4.2: String Lengths

Two common definitions used to specify basic AFS string lengths are AFSNAMEMAX and AFSPATHMAX. AFSNAMEMAX places an upper limit of 256 characters on such things as file and directory names passed as parameters. AFSPATHMAX defines the longest pathname expected by the system, composed of slash-separated instances of the individual directory and file names mentioned above. The longest acceptable pathname is currently set to 1,024 characters.

Generated on Fri Feb 25 15:57:23 2011 by  doxygen 1.5.1