Design and Implementation of a Distributed Content Management System C. D. Cranor, R. Ethington, A. Sehgal† , D. Shur, C. Sreenan‡ and J.E. van der Merwe AT&T Labs - Research † University of Kentucky ‡ University College Cork Florham Park, NJ, USA Lexington, KY, USA Cork, Ireland ABSTRACT The convergence of advances in storage, encoding, and networking technologies has brought us to an environment where huge amounts of continuous media content is routinely stored and exchanged between network enabled devices. Keeping track of (or managing) such content remains challenging due to the sheer volume of data. Storing live continuous media (such as TV or radio content) adds to the complexity in that this content has no well defined start or end and is therefore cumbersome to deal with. Networked storage allows content that is logically viewed as part of the same collection to in fact be distributed across a network, making the task of content management all but impossible to deal with without a content management system. In this paper we present the design and implementation of the Spectrum content management system, which deals with rich media content effectively in this environment. Spectrum has a modular architecture that allows its application to both stand-alone and various networked scenarios. A unique aspect of Spectrum is that it requires one (or more) retention policies to apply to every piece of content that is stored in the system. This means that there are no eviction policies. Content that no longer has a retention policy applied to it is simply removed from the system. Different retention policies can easily be applied to the same content thus naturally facilitating sharing without duplication. This approach also allows Spectrum to easily apply time based policies which are basic building blocks required to deal with the storage of live continuous media, to content. We not only describe the details of the Spectrum architecture but also give typical use cases. Categories and Subject Descriptors C.2.4 [Computer Systems Organization]: Computer-communication Networks-distributed systems; H.3.4 [Information Systems]: Information Storage and Retrieval-systems and software General Terms Design, Management 1. INTRODUCTION Manipulating and managing content is and has always been one of the primary functions of a computer. Initial computing applications include text formatters and program compilers. Content was initially managed by explicit user interaction through the use of files and filesystems. As technology has advanced, both the types of content and the way people wish to use it have greatly changed. New content types such as continuous multimedia streams have become commonplace due to the convergence of advances in storage, encoding, and networking technologies. For example, by combining improvements in storage and encoding, it is now possible to store many hours of TV-quality encoded video on a single disk drive. This has led to the introduction of stand alone digital video recording or personal video recording (PVR) systems such as TiVO [8] and ReplayTV [7]. Another example is the combination of encoding and broadband networking technology. This combination has allowed users to access and share multimedia content in both local and remote area networks with the network itself acting as a huge data repository. The proliferation of high quality content enabled by these advances in storage, encoding, and networking technology creates the need for new ways to manipulate and manage the data. The focus of our work is on the storage of media rich content and in particular the storage of continuous media content in either pre-packaged or live forms. The need for content management in this area is apparent when one consider the following: • Increases in the capacity and decreases in the cost of storage means that even modest desktop systems today have the ability to store massive amounts of content. Managing such content manually (or more correctly manual non-management of such content) lead to great inefficiencies where unwanted and forgotten content waste storage and where wanted content cannot be found. • While true for all types of content the storage of continuous media content is especially problematic. First continuous media content is still very demanding in terms of storage resources which means that a policy-less approach to storing it will not work for all but the smallest systems. Second, the storing of live content such as TV or radio is inherently problematic as these signals are continuous streams with no endpoints. This means that before one can even think about managing such content there is a need to abstract it into something that could be manipulated and managed. 4 • When dealing with stored continuous media there is a need to manage such content at both a fine-grained as well as an aggregate level. For example, an individual PVR user wanting to keep only the highlights of a particular sporting event should not be required to have to store the content pertaining to the complete event. At the same time the user might want to think of content in the aggregate, e.g. remove all of the content that I have not watched for the last month except that content which was explicitly marked for archival. • As indicated above, trying to keep track of content on a standalone system without a content management system is very difficult. However, when the actual storage devices are distributed across a network the task of keeping track of content is almost impossible. This scenario is increasingly common in network based content distribution systems and is likely to also become important in home-networking scenarios. It would seem clear then that a content management system that can efficiently handle media rich content while also exploiting the networked capability of storage devices is needed. This system should allow efficient storage of and access to content across heterogeneous network storage devices according to user preferences. The content management system should translate user preferences into appropriate low-level storage policies and should allow those preferences to be expressed at a fine level of granularity (while not requiring it in general). The content management system should allow the user to manipulate and reason about (i.e. change the storage policy associated with) the storage of (parts of) continuous media content. Addressing this distributed content management problem is difficult due to the number of requirements placed on the system. For example: • The content management system must operate on a large number of heterogeneous systems. In some cases the system may be managing content stored on a local filesystem, while in others the content may be stored on a separate network storage appliance. The content manager may be responsible for implementing the policies it uses to reference content or that role may be delegated to a separate computer. A application program interface (API) and associated network protocols are needed in order for the content management system to provide a uniform interface. • The content management system should be flexible and be able to handle differing requirements for content management policies. These policies reflect what content should be obtained, when it should be fetched, how long it should be retained, and under what circumstances it should be discarded. This means that the content management system should allow multiple applications to reference content with a rich set of policies and that it should all work together seamlessly. • The content management system needs to be able to monitor references for content and use that information to place content in the right location in the network for efficient application access. • The content management system must handle the interaction between implicit and explicit population of content at the network edge. • The content system must be able to efficiently manage large sets of content, including continuous streams. It needs to be able to package this content in such a way that it is convenient for users to access. To address these issues we have designed and implemented the Spectrum content management system architecture. Our layered architecture is flexible - its API allows the layers to reside either on a single computer or on multiple networked heterogeneous computers. It allows multiple applications to reference content using differing policies. Note that the Spectrum architecture assumes the existence of a content distribution network (CDN) that can facilitate the efficient distribution of content (for example, the PRISM CDN architecture [2]). The rest of this paper is organized as follows. Section 2 describes the architecture of our content management system. In Section 3 we describe both our implementation of the Spectrum architecture and examples of its use. Related work is described in Section 4, and Section 5 contains our conclusion and suggestions for future work. 2. THE SPECTRUM DISTRIBUTED CONTENT MANAGEMENT SYSTEM ARCHITECTURE The Spectrum architecture consists of three distinct management layers that may or may not be distributed across multiple machines, as shown in Figure 1. The three layers are: content manager: contains application specific information that is used to manage all of an application"s content according to user preferences. For example, in a personal video recorder (PVR) application the content manager receives requests for content from a user interface and interacts with the lower layers of the Spectrum architecture to store and manage content on the device. policy manager: implements and enforces various storage polices that the content manager uses to refer to content. The policy manager exports an interface to the content manager that allows the content manager to request that a piece content be treated according to a specific policy. Spectrum allows for arbitrary policies to be realized by providing a fixed set of base-policy templates that can easily be parameterized. It is our belief that for most implementations this will be adequate (if not, Spectrum can easily be extended to dynamically load new base-policy template code at run time). A key aspect of the policy manager is that it allows different policies to be simultaneously applied to the same content (or parts of the same content). Furthermore content can only exist in the system so long as it is referenced by at least one existing policy. Policy conflicts are eliminated by having the policy manager deal exclusively with retention policies rather than with a mix of retention and eviction policies. This means that content with no policy associated with it is immediately and automatically removed from the system. This approach allows us to naturally support sharing of content across different policies which is critical to the efficient storage of large objects. Note that a key difference between the content manager and the policy manager is that the content manager manages references to multiple pieces of content, i.e. it has an applicationview of content. On the other hand, the policy manager is only concerned with the policy used to manage standalone pieces of content. For example, in a PVR application, the content manager layer would know about the different groups of managed content such as keep-indefinitely, keep for one day, and keep if available diskspace. However, at the policy manager level, each piece of content has 5 Content Manager Policy Manager Storage Manager Content Manager Content Manager Content Manager Policy Manager Policy Manager Policy Manager Storage Manager Storage Manager Storage Manager Remote Invocation Figure 1: The components of the Spectrum architecture and the four ways they can be configured its own policy (or policies) applied to it and is independent from other content. storage manager: stores content in an efficient manner while facilitating the objectives of the higher layers. Specifically the storage manager stores content in sub-object chunks. This approach has advantages for the efficient retrieval of content but more importantly allows policies to be applied at a subobject level which is critically important when dealing with very large objects such as parts of continuous media, e.g. selected pieces of TV content being stored on a PVR. Note that the storage manager has no knowledge of the policies being used by the content and policy managers. Another unique part of our approach is that the interfaces between the layers can either be local or distributed. Figure 1 shows the four possible cases. The case on the far left of the Figure shows the simplest (non-distributed) case where all the layers are implemented on a single box. This configuration would be used in selfcontained applications such as PVRs. The next case over corresponds to the case where there is a centralized content manager that controls distributed storage devices each of which is responsible for implementing policy based storage. In this case although the remote devices are controlled by the central manager they operate much more independently. For example, once they receive instructions from the central manager they typically operate in autonomous fashion. An example of this type of configuration is a content distribution network (CDN) that distributes and stores content based on a schedule determined by some centralized controller. For example, the CDN could pre-populate edge devices with content that is expected to be very popular or distribute large files to branch offices during off-peak hours in a bandwidth constrained enterprise environment. Allowing a single policy manager to control several storage managers leads to the next combination of functions and the most distributed case. The need for this sort of separation might occur for scalability reasons or when different specialized storage devices or appliances are required to be controlled by a single policy manager. The final case shows a content manager combined with a policy manager controlling a remote storage manager. This separation would be possible if the storage manager is somewhat autonomous and does not require continuous fine grained control by the policy manager. We now examine the function of the three layers in detail. 2.1 Content Manager The content manager layer is the primary interface through which specific applications use the Spectrum architecture. As such the content manager layer provides an API for the application to manipulate all aspects of the Spectrum architecture at different levels of granularity. The content manager API has functions that handle: Physical devices: This set of functions allows physical storage devices to be added to Spectrum thereby putting them under control of the content manager and making the storage available to the system. Physical devices can be local or remote - this is the only place in the architecture where the application is required to be aware of this distinction. Once a device is mapped into the application through this interface, the system tracks its type and location. Users simply refer to the content through an application-provided label. Stores: Stores are subsets of physical storage devices. Through these functions an application can create a store on a physical device and assign resources (e.g. disk space) to it. Stores can only be created in physical devices that are mapped into the system. Policy Groups: Policy groups are the means whereby an application specifies, instantiates, and modifies the policies that are applied to Spectrum content. Typical usage of this set of functions is to select one of a small set of base policies and to parameterize this specific instance of the policy. Policy groups are created within existing stores in the system. The Spectrum architecture has policies that are normally associated with storage that aim to optimize disk usage. In addition a set of policies that take a sophisticated time specification enable storage that is cognizant of time. For example, a simple time-based policy could evict content from the system at a certain absolute or relative time. A slightly more involved time-based policy enabled by the Spectrum architecture could allow content to be stored in rolling window of a number of hours (for example, the most recent N-number of hours is kept in the system). Time-based polices are of particular use when dealing with continuous content like a live broadcast. 6 Content: At the finest level of granularity content can be added to or removed from the system. Content is specified to the system by means of a uniform resource locator (URL) which concisely indicates the location of the content as well as the protocol to be used to retrieve it. Optionally a time specification can be associated with content. This allows content to be fetched into the system at some future time, or at future time intervals. Again, this is particularly useful for dealing with the storage and management of live content. 2.2 Policy Manager The policy manager layer of the Spectrum architecture has two main types of API functions. First, there are functions that operate on managed storage areas and policy-based references (prefs) to content stored there. Second, there are sets of functions used to implement each management policy. The first class of functions is used by the content manager layer to access storage. Operations include: create, open, and close: These operations are used by the content manager to control its access to storage. The policy manager"s create operation is used to establish contact with a store for the first time. Once this is done, the store can be open and closed using the appropriate routines. Note that the parameters used to create a store contain information on how to reach it. For example, local stores have a path associated with them, while remote stores have a remote host and remote path associated with them. The information only needs to be passed to the policy manager once at create time. For open operations, the policy manager will use cached information to contact the store. lookup: The lookup operation provides a way for the content manager to query the policy manager about what content is currently present for a given URL. For continuous media time ranges of present media will be returned. resource: The resource routines are used to query the policy manager about its current resource usage. There are two resource routines: one that applies to the store as a whole and another that applies to a particular policy reference. The resource API is extensible, we currently support queries on disk usage and I/O load. pref establish/update: The pref establish operation is used by the content manager to reference content on the store. If the content is not present, this call will result in the content being fetched (or being scheduled to be fetched if the content is not currently available). Parameters of this function include the URL to store it under, the URL to fetch data from if it is not present, the policy to store the content under, and the arguments used to parameterize the policy. The result of a successful pref establish operation is a policy reference ID string. This ID can be used with the update operation to either change the storage policy parameters or delete the reference entirely. The second group of policy manager functions are used to implement all the polices supported by Spectrum. We envision a small set of base-level policy functions that can be parameterized to produce a wide range of storage polices. For example, a policy that implements recording a repeating time window can be parameterized to function daily, weekly, or monthly. Note that the policy manager is only concerned with executing a specific policy. The higher-level reasons for choosing a given policy are handled by the content and application manager. A base policy is implemented using six functions: establish: called when a pref is established with the required URLs and base policy"s parameters. The establish routine references any content already present in the store and then determines the next time it needs to take action (e.g. start a download) and schedules a callback for that time. It can also register to receive callbacks if new content is received for a given URL. update: called to change the parameters of a pref, or to discard the policy reference. newclip: called when a chunk of new content is received for a URL of interest. The base policy typically arranges for newclip to be called for a given URL when the pref is established. When newclip is called, the base policy checks its parameters to determine if it wishes to add a reference to the clip just received. callback: called when the pref schedules a timer-based callback. This is a useful wakeup mechanism for prefs that need to be idle for a long period of time (e.g. between programs). boot/shutdown: called when the content management system is booting or shutting down. The boot operation is typically used to schedule initial callbacks or start I/O operations. The shutdown operation is used to gracefully shutdown I/O streams and save state. 2.3 Storage Manager The role of Spectrum"s storage manager is to control all I/O operations associated with a given store. Spectrum"s storage manager supports storing content both on a local filesystem and on a remote fileserver (e.g. a storage appliance). For continuous media, at the storage manager level content is stored as a collection of time-based chunks. Depending on the underlying filesystem, a chunk could correspond to a single file or a data node in a storage database. The two main storage manager operations are input and output. The input routine is used to store content in a store under a given name. The output routine is used to send data from the store to a client. For streaming media both the input and output routines take time ranges that schedule when the I/O operation should happen, and both routines return an I/O handle that can be used to modify or cancel the I/O request in the future. Much like the policy manager, the storage manager also provides API functions to create, open, and close stores. It also supports operations to query the resource usages and options supported by the store. Finally, the storage manager also has a discard routine that may be used by the policy manager to inform the store to remove content from the store. 3. IMPLEMENTATION AND USE CASES In this section we describe our implementation of Spectrum and describe how it can be used. 3.1 Implementation We have implemented Spectrum"s three layers in C as part of a library that can be linked with Spectrum-based applications. Each layer keeps track of its state through a set of local data files that persist across reboots, thus allowing Spectrum to smoothly handle power cycles. For layers that reside on remote systems (e.g. a remote store) only the meta-information needed to contact the remote 7 Content Manager Policy Manager Storage Manager Storage Fetcher Program Listings Graphical User Interface Network Enabled DVR Program Information Content DVR Application Figure 2: Spectrum in a Network Enabled DVR node is stored locally. Our test application uses a local policy and storage manager to fetch content and store it in a normal Unixbased filesystem. To efficiently handle communications with layers running on remote systems, all Spectrum"s API calls support both synchronous and asynchronous modes through a uniform interface defined by the reqinfo structure. Each API call takes a pointer to a reqinfo structure as one of its arguments. This structure is used to hold the call state and return status. For async calls, the reqinfo also contains a pointer to a callback function. To use a Spectrum API function, the caller first chooses either the sync or async mode and allocates a reqinfo structure. For sync calls, the reqinfo can be allocated on the stack, otherwise it is allocated with malloc. For async calls, a callback function must be provided when the reqinfo is allocated. Next the caller invokes the desired Spectrum API function passing the reqinfo structure as an argument. For sync calls, the result of the calls is returned immediately in the reqinfo structure. For successful async calls, a call in progress value is returned. Later, when the async call completes or a timeout occurs, the async callback function is called with the appropriate information needed to complete processing. The modular/layered design of the Spectrum architecture simplifies the objective of distribution of functionality. Furthermore, communication between functions is typically of a master-slave(s) nature. This means that several approaches to distributed operation are possible that would satisfy the architectural requirements. In our implementation we have opted to realize this functionality with a simple modular design. We provide a set of asynchronous remote access stub routines that allow users to select the transport protocol to use and to select the encoding method that should be used with the data to be transferred. Transport protocols can range simple protocols such as UDP up to more complex protocols such as HTTP. We currently are using plain TCP for most of our transport. Function calls across the different Spectrum APIs can be encoded using a variety of formats include plain text, XDR, and XML. We are currently using the eXpat XML library [4] to encode our calls. While we are current transferring our XML encoded messages using a simple TCP connection, in a real world setting this can easily be replaced with an implementation based on secure sockets layer (SSL) to improve security by adding SSL as a transport protocol. An important aspect of Spectrum is that it can manage content based on a given policy across heterogenous platforms. As we explained previously in Section 2.2, envision a small set of base-level policy functions that can be parameterized to produce a wide range of storage polices. In order for this to work properly, all Spectrumbased applications must understand the base-level policies and how they can be parameterized. To address this issue, we treat each base-level policy as if it was a separate program. Each base-level policy should have a well known name and command line options for parameterization. In fact, in our implementation we pass parameters to base-level policies as a string that can be parsed using a getopt-like function. This format is easily understood and provides portability since byte order is not an issue in a string. Since this part of Spectrum is not on the critical data path, this type of formatting is not a performance issue. 3.2 Using the Spectrum Content Management System In this section we show two examples of the use of the Spectrum Content Management System in our environment. The focus of our previous work has been content distribution for streaming media content [2] and network enabled digital video recording [3]. The Spectrum system is applicable to both scenarios as follows. Figure 2 shows the Network Enabled DVR (NED) architecture. In this case all layers of the Spectrum architecture reside on the same physical device in a local configuration. The DVR application obtains program listings from some network source, deals with user presentation through a graphical user interface (GUI), and interface with the Spectrum system through the content management layer APIs. This combination of higher level functions allows the user to select both content to be stored and what storage policies to 8 Content Manager Centralized Content Management station Content InformationUser Interface Policy Manager Storage Manager Storage Fetcher Edge Portal Server Policy Manager Storage Manager Storage Fetcher Edge Portal Server Distributed Content To Media Endpoints To Media Endpoints Figure 3: Spectrum in a Content Distribution Architecture apply to such content. Obtaining the content (through the network or locally) and the subsequent storage on the local system is then handled by the policy and storage managers. The use of Spectrum in a streaming content distribution architecture (e.g. PRISM [2]) is depicted in Figure 3. In this environment streaming media content (both live, canned-live and on-demand) is being distributed to edge portals from where streaming endpoints are being served. In our environment content distribution and storage is done from a centralized content management station which controls several of the edge portals. The centralized station allows administrators to manage the distribution and storage of content without requiring continuous communication between the content manager and the edge devices, i.e. once instructions have been given to edge devices they can operate independently until changes are to be made. 3.3 Spectrum Operational Example To illustrate how Spectrum handles references to content, consider a Spectrum-based PVR application programmed to store one days worth of streaming content in a rolling window. To set up the rolling window, the application would use the content manager API to create a policy group and policy reference to the desired content. The establishment of the one-day rolling window policy reference would cause the policy manger to ask the storage manager to start receiving the stream. As each chunk of streaming data arrives, the policy manager executes the policy reference"s newclip function. The newclip function adds a reference to each arriving chunk, and schedules a callback a day later. At that time, the policy will drop its now day-old reference to the content and the content will be discarded unless it is referenced by some other policy. Now, consider the case where the user decides to save part of the content (e.g. a specific program) in the rolling window for an extra week. To do this, the application requests that the content manager add an additional new policy reference to the part of the content to preserved. Thus, the preserved content has two references to it: one from the rolling window and one from the request to preserve the content for an additional week. After one day the reference from the rolling window will be discarded, but the content will be 9 ref2, etc. base data url1 url2 (media files...) (media files...) meta store (general info...) url1 chunks prefs ranges media chunks, etc.url2 poly host ref1 ref1.files ref1.state Figure 4: Data layout of Spectrum policy store preserved by the second reference. After the additional week has past, the callback function for the second reference will be called. This function will discard the remaining reference to the content and as there are no remaining references the content will be freed. In order to function in scenarios like the ones described above, Spectrum"s policy manager must manage and maintain all the references to various chunks of media. These references are persistent and thus must be able to survive even if the machine maintaining them is rebooted. Our Spectrum policy manager implementation accomplishes this using the file and directory structure shown in Figure 4. There are three classes of data stored, and each class has its own top level directory. The directories are: data: this directory is used by the storage manager to store each active URL"s chunks of media. The media files can be encoded in any format, for example MPEG, Windows Media, or QuickTime. Note that this directory is used only if the storage manager is local. If the policy manager is using an external storage manager (e.g. a storage appliance), then the media files are stored remotely and are only remotely referenced by the policy manager. meta: this directory contains general meta information about the storage manager being used and the data it is storing. General information is stored in the store subdirectory and includes the location of the store (local or remote) and information about the types of chunks of data the store can handle. The meta directory also contains a subdirectory per-URL that contains information about the chunks of data stored. The chunks file contains a list of chunks currently stored and their reference counts. The prefs file contains a list of active policy references that point to this URL. The ranges file contains a list of time ranges of data currently stored. Finally, the media file describes the format of the media being stored under the current URL. poly: this directory contains a set of host subdirectories. Each host subdirectory contains the set of policy references created by that host. Information on each policy reference is broken up into three files. For example, a policy reference named ref1 would be stored in ref1, ref1.files, and ref1.state. The ref1 file contains information about the policy reference that does not change frequently. This information includes the base-policy and the parameters used to create the reference. The ref1.files file contains the list of references to chunks that pref ref1 owns. Finally, the ref1.state file contains optional policy-specific state information that can change over time. Together, these files and directories are used to track references in our implementation of Spectrum. Note that other implementations are possible. For example, a carrier-grade Spectrum manager might store all its policy and reference information in a high-performance database system. 10 4. RELATED WORK Several authors have addressed the problem of the management of content in distributed networks. Much of the work focuses on the policy management aspect. For example in [5], the problem of serving multimedia content via distributed servers is considered. Content is distributed among server resources in proportion to user demand using a Demand Dissemination Protocol. The performance of the scheme is benchmarked via simulation. In [1] content is distributed among sub-caches. The authors construct a system employing various components, such as a Central Router, Cache Knowledge base, Subcaches, and a Subcache eviction judge. The Cache Knowledge base allows sophisticated policies to be employed. Simulation is used to compare the proposed scheme with well-known replacement algorithms. Our work differs in that we are considering more than the policy management aspects of the problem. After carefully considering the required functionality to implement content management in the networked environment, we have partitioned the system into three simple functions, namely Content manager, Policy manager and Storage manager. This has allowed us to easily implement and experiment with a prototype system. Other related work involves so called TV recommendation systems which are used in PVRs to automatically select content for users, e.g. [6]. In the case where Spectrum is used in a PVR configuration this type of system would perform a higher level function and could clearly benefit from the functionalities of the Spectrum architecture. Finally, in the commercial CDN environment vendors (e.g. Cisco and Netapp) have developed and implemented content management products and tools. Unlike the Spectrum architecture which allows edge devices to operate in a largely autonomous fashion, the vendor solutions typically are more tightly coupled to a centralized controller and do not have the sophisticated time-based operations offered by Spectrum. 5. CONCLUSION AND FUTURE WORK In this paper we presented the design and implementation of the Spectrum content management architecture. Spectrum allows storage policies to be applied to large volumes of content to facilitate efficient storage. Specifically, the system allows different policies to be applied to the same content without replication. Spectrum can also apply policies that are time-aware which effectively deals with the storage of continuous media content. Finally, the modular design of the Spectrum architecture allows both stand-alone and distributed realizations so that the system can be deployed in a variety of applications. There are a number of open issues that will require future work. Some of these issues include: • We envision Spectrum being able to manage content on systems ranging from large CDNs down to smaller appliances such as TiVO [8]. In order for these smaller systems to support Spectrum they will require networking and an external API. When that API becomes available, we will have to work out how it can be fit into the Spectrum architecture. • Spectrum names content by URL, but we have intentionally not defined the format of Spectrum URLs, how they map back to the content"s actual name, or how the names and URLs should be presented to the user. While we previously touched on these issues elsewhere [2], we believe there is more work to be done and that consensus-based standards on naming need to be written. • In this paper we"ve focused on content management for continuous media objects. We also believe the Spectrum architecture can be applied to any type of document including plain files, but we have yet to work out the details necessary to support this in our prototype environment. • Any project that helps allow multimedia content to be easily shared over the Internet will have legal hurdles to overcome before it can achieve widespread acceptance. Adapting Spectrum to meet legal requirements will likely require more technical work. 6. REFERENCES [1] K. . Cheng and Y. Kambayashi. Multicache-based Content Management for Web Caching. Proceedings of the First International Conference on Web Information Systems Engineering, Jume 2000. [2] C. Cranor, M. Green, C. Kalmanek, D. Shur, S. Sibal, C. Sreenan, and J. van der Merwe. PRISM Architecture: Supporting Enhanced Streaming Services in a Content Distribution Network. IEEE Internet Computing, July/August 2001. [3] C. Cranor, C. Kalmanek, D. Shur, S. Sibal, C. Sreenan, and J. van der Merwe. NED: a Network-Enabled Digital Video Recorder. 11th IEEE Workshop on Local and Metropolitan Area Networks, March 2001. [4] eXpat. expat.sourceforge.net. [5] Z. Ge, P. Ji, and P. Shenoy. A Demand Adaptive and Locality Aware (DALA) Streaming Media Server Cluster Architecture. NOSSDAV, May 2002. [6] K. Kurapati and S. Gutta and D. Schaffer and J. Martino and J. Zimmerman. A multi-agent TV recommender. Proceedings of the UM 2001 workshop, July 2001. [7] ReplayTV. www.sonicblue.com. [8] TiVo. www.tivo.com. 11

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O B I B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I I O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

continuous media storage
home-networking scenario
high-performance database system
carrier-grade spectrum manager
distribute content management

spectrum content management system
application program interface
content distribution network
uniform resource locator
policy manager
network enabled dvr


A Cross-Layer Approach to Resource Discovery and Distribution in Mobile ad-hoc Networks Chaiporn Jaikaeo Computer Engineering Kasetsart University, Thailand (+662) 942-8555 Ext 1424 cpj@cpe.ku.ac.th Xiang Cao Computer and Information Sciences University of Delaware, USA (+1) 302-831-1131 cao@cis.udel.edu Chien-Chung Shen Computer and Information Sciences University of Delaware, USA (+1) 302-831-1951 cshen@cis.udel.edu ABSTRACT This paper describes a cross-layer approach to designing robust P2P system over mobile ad-hoc networks. The design is based on simple functional primitives that allow routing at both P2P and network layers to be integrated to reduce overhead. With these primitives, the paper addresses various load balancing techniques. Preliminary simulation results are also presented. Categories and Subject Descriptors C.2.4 [Distributed Systems]: Distributed Applications General Terms Algorithms and design 1. INTRODUCTION Mobile ad-hoc networks (MANETs) consist of mobile nodes that autonomously establish connectivity via multi-hop wireless communications. Without relying on any existing, pre-configured network infrastructure or centralized control, MANETs are useful in situations where impromptu communication facilities are required, such as battlefield communications and disaster relief missions. As MANET applications demand collaborative processing and information sharing among mobile nodes, resource (service) discovery and distribution have become indispensable capabilities. One approach to designing resource discovery and distribution schemes over MANETs is to construct a peer-to-peer (P2P) system (or an overlay) which organizes peers of the system into a logical structure, on top of the actual network topology. However, deploying such P2P systems over MANETs may result in either a large number of flooding operations triggered by the reactive routing process, or inefficiency in terms of bandwidth utilization in proactive routing schemes. Either way, constructing an overlay will potentially create a scalability problem for large-scale MANETs. Due to the dynamic nature of MANETs, P2P systems should be robust by being scalable and adaptive to topology changes. These systems should also provide efficient and effective ways for peers to interact, as well as other desirable application specific features. This paper describes a design paradigm that uses the following two functional primitives to design robust resource discovery and distribution schemes over MANETs. 1. Positive/negative feedback. Query packets are used to explore a route to other peers holding resources of interest. Optionally, advertisement packets are sent out to advertise routes from other peers about available resources. When traversing a route, these control packets measure goodness of the route and leave feedback information on each node along the way to guide subsequent control packets to appropriate directions. 2. Sporadic random walk. As the network topology and/or the availability of resources change, existing routes may become stale while better routes become available. Sporadic random walk allows a control packet to explore different paths and opportunistically discover new and/or better routes. Adopting this paradigm, the whole MANET P2P system operates as a collection of autonomous entities which consist of different types of control packets such as query and advertisement packets. These packets work collaboratively, but indirectly, to achieve common tasks, such as resource discovery, routing, and load balancing. With collaboration among these entities, a MANET P2P system is able to ‘learn" the network dynamics by itself and adjust its behavior accordingly, without the overhead of organizing peers into an overlay. The remainder of this paper is organized as follows. Related work is described in the next section. Section 3 describes the resource discovery scheme. Section 4 describes the resource distribution scheme. The replica invalidation scheme is described in Section 5, followed by it performance evaluation in Section 6. Section 7 concludes the paper. 2. RELATED WORK For MANETs, P2P systems can be classified based on the design principle, into layered and cross-layer approaches. A layered approach adopts a P2P-like [1] solution, where resource discovery is facilitated as an application layer protocol and query/reply messages are delivered by the underlying MANET routing protocols. For instance, Konark [2] makes use of a underlying multicast protocol such that service providers and queriers advertise and search services via a predefined multicast group, respectively. Proem [3] is a high-level mobile computing platform for P2P systems over MANETs. It defines a transport protocol that sits on top of the existing TCP/IP stack, hence relying on an existing routing protocol to operate. With limited control over how control and data packets are routed in the network, it is difficult to avoid the inefficiency of the general-purpose routing protocols which are often reactive and flooding-based. In contrast, cross-layer approaches either relies on its own routing mechanism or augments existing MANET routing algorithms to support resource discovery. 7DS [4], which is the pioneering work deploying P2P system on mobile devices, exploits data locality and node mobility to dissemination data in a single-hop fashion. Hence, long search latency may be resulted as a 7DS node can get data of interest only if the node that holds the data is in its radio coverage. Mohan et al. [5] propose an adaptive service discovery algorithm that combines both push and pull models. Specifically, a service provider/querier broadcasts advertisement/query only when the number of nodes advertising or querying, which is estimated by received control packets, is below a threshold during a period of time. In this way, the number of control packets on the network is constrained, thus providing good scalability. Despite the mechanism to reduce control packets, high overhead may still be unavoidable, especially when there are many clients trying to locate different services, due to the fact that the algorithm relies on flooding, For resource replication, Yin and Cao [6] design and evaluate cooperative caching techniques for MANETs. Caching, however, is performed reactively by intermediate nodes when a querier requests data from a server. Data items or resources are never pushed into other nodes proactively. Thanedar et al. [7] propose a lightweight content replication scheme using an expanding ring technique. If a server detects the number of requests exceed a threshold within a time period, it begins to replicate its data onto nodes capable of storing replicas, whose hop counts from the server are of certain values. Since data replication is triggered by the request frequency alone, it is possible that there are replicas unnecessarily created in a large scope even though only nodes within a small range request this data. Our proposed resource replication mechanism, in contrast, attempts to replicate a data item in appropriate areas, instead of a large area around the server, where the item is requested frequently. 3. RESOURCE DISCOVERY We propose a cross-layer, hybrid resource discovery scheme that relies on the multiple interactions of query, reply and advertisement packets. We assume that each resource is associated with a unique ID1 . Initially, when a node wants to discover a resource, it deploys query packets, which carry the corresponding resource ID, and randomly explore the network to search for the requested resource. Upon receiving such a query packet, a reply packet is generated by the node providing the requested resource. Advertisement packets can also be used to proactively inform other nodes about what resources are available at each node. In addition to discovering the ‘identity" of the node providing the requested resource, it may be also necessary to discover a ‘route" leading to this node for further interaction. To allow intermediate nodes to make a decision on where to forward query packets, each node maintains two tables: neighbor 1 The assumption of unique ID is made for brevity in exposition, and resources could be specified via attribute-value assertions. table and pheromone table. The neighbor table maintains a list of all current neighbors obtained via a neighbor discovery protocol. The pheromone table maintains the mapping of a resource ID and a neighbor ID to a pheromone value. This table is initially empty, and is updated by a reply packet generated by a successful query. Figure 1 illustrates an example of a neighbor table and a pheromone table maintained by node A having four neighbors. When node A receives a query packet searching for a resource, it makes a decision to which neighbor it should forward the query packet by computing the desirability of each of the neighbors that have not been visited before by the same query packet. For a resource ID r, the desirability of choosing a neighbor n, δ(r,n), is obtained from the pheromone value of the entry whose neighbor and resource ID fields are n and r, respectively. If no such entry exists in the pheromone table, δ(r,n) is set to zero. Once the desirabilities of all valid next hops have been calculated, they are normalized to obtain the probability of choosing each neighbor. In addition, a small probability is also assigned to those neighbors with zero desirability to exercise the sporadic random walk primitive. Based on these probabilities, a next hop is selected to forward the query packet to. When a query packet encounters a node with a satisfying resource, a reply packet is returned to the querying node. The returning reply packet also updates the pheromone table at each node on its return trip by increasing the pheromone value in the entry whose resource ID and neighbor ID fields match the ID of the discovered resource and the previous hop, respectively. If such an entry does not exist, a new entry is added into the table. Therefore, subsequent query packets looking for the same resource, when encountering this pheromone information, are then guided toward the same destination with a small probability of taking an alternate path. Since the hybrid discovery scheme neither relies on a MANET routing protocol nor arranges nodes into a logical overlay, query packets are to traverse the actual network topology. In dense networks, relatively large nodal degrees can have potential impacts on this random exploring mechanism. To address this issue, the hybrid scheme also incorporates proactive advertisement in addition to the reactive query. To perform proactive advertisement, each node periodically deploys an advertising packet containing a list of its available resources" IDs. These packets will traverse away from the advertising node in a random walk manner up to a limited number of hops and advertise resource information to surrounding nodes in the same way as reply packets. In the hybrid scheme, an increase of pheromone serves as a positive feedback which indirectly guides query packets looking for similar resources. Intuitively, the amount of pheromone increased is inversely proportional to the distance the reply packet has traveled back, and other metrics, such as quality of the resource, could contribute to this amount as well. Each node also performs an implicit negative feedback for resources that have not been given a positive feedback for some time by regularly decreasing the pheromone in all of its pheromone table entries over time. In addition, pheromone can be reduced by an explicit negative response, for instance, a reply packet returning from a node that is not willing to provide a resource due to excessive workload. As a result, load balancing can be achieved via positive and negative feedback. A node serving too many nodes can either return fewer responses to query packets or generate negative responses. 2 The 3rd International Conference on Mobile Technology, Applications and Systems - Mobility 2006 Figure 1: Example illustrating neighbor and pheromone tables maintained by node A: (a) wireless connectivity around A showing that it currently has four neighbors, (b) A"s neighbor table, and (c) a possible pheromone table of A Figure 2: Sample scenarios illustrating the three mechanisms supporting load-balancing: (a) resource replication, (b) resource relocation, and (c) resource division 4. RESOURCE DISTRIBUTION In addition to resource discovery, a querying node usually attempts to access and retrieve the contents of a resource after a successful discovery. In certain situations, it is also beneficial to make a resource readily available at multiple nodes when the resource can be relocated and/or replicated, such as data files. Furthermore, in MANETs, we should consider not only the amount of load handled by a resource provider, but also the load on those intermediate nodes that are located on the communication paths between the provider and other nodes as well. Hence, we describe a cross-layer, hybrid resource distribution scheme to achieve load balancing by incorporating the functionalities of resource relocation, resource replication, and resource division. 4.1 Resource Replication Multiple replicas of a resource in the network help prevent a single node, as well as nodes surrounding it, from being overloaded by a large number of requests and data transfers. An example is when a node has obtained a data file from another node, the requesting node and the intermediate nodes can cache the file and start sharing that file with other surrounding nodes right away. In addition, replicable resources can also be proactively replicated at other nodes which are located in certain strategic areas. For instance, to help nodes find a resource quickly, we could replicate the resource so that it becomes reachable by random walk for a specific number of hops from any node with some probability, as depicted in Figure 2(a). To realize this feature, the hybrid resource distribution scheme employs a different type of control packet, called resource replication packet, which is responsible for finding an appropriate place to create a replica of a resource. A resource replication packet of type R is deployed by a node that is providing the resource R itself. Unlike a query packet which follows higher pheromone upstream toward a resource it is looking for, a resource replication packet tends to be propelled away from similar resources by moving itself downstream toward weaker pheromone. When a resource replication packet finds itself in an area with sufficiently low pheromone, it makes a decision whether it should continue exploring or turn back. The decision depends on conditions such as current workload and/or remaining energy of the node being visited, as well as popularity of the resource itself. 4.2 Resource Relocation In certain situations, a resource may be required to transfer from one node to another. For example, a node may no longer want to possess a file due to the shortage of storage space, but it cannot simply delete the file since other nodes may still need it in the future. In this case, the node can choose to create replicas of the file by the aforementioned resource replication mechanism and then delete its own copy. Let us consider a situation where a majority of nodes requesting for a resource are located far away from a resource provider, as shown on the top of Figure 2(b). If the resource R is relocatable, it is preferred to be relocated to another area that is closer to those nodes, similar to the bottom of the same figure. Hence network bandwidth is more efficiently utilized. The 3rd Conference on Mobile Technology, Applications and Systems - Mobility 2006 3 The hybrid resource distribution scheme incorporates resource relocation algorithms that are adaptive to user requests and aim to reduce communication overhead. Specifically, by following the same pheromone maintenance concept, the hybrid resource distribution scheme introduces another type of pheromone which corresponds to user requests, instead of resources. This type of pheromone, called request pheromone, is setup by query packets that are in their exploring phases (not returning ones) to guide a resource to a new location. 4.3 Resource Division Certain types of resources can be divided into smaller subresources (e.g., a large file being broken into smaller files) and distributed to multiple locations to avoid overloading a single node, as depicted in Figure 2(c). The hybrid resource distribution scheme incorporates a resource division mechanism that operates at a thin layer right above all the other mechanisms described earlier. The resource division mechanism is responsible for decomposing divisible resources into sub-resources, and then adds an extra keyword to distinguish each sub-resource from one another. Therefore, each of these sub-resources will be seen by the other mechanisms as one single resource which can be independently discovered, replicated, and relocated. The resource division mechanism is also responsible for combining data from these subresources together (e.g., merging pieces of a file) and delivering the final result to the application. 5. REPLICA INVALIDATION Although replicas improve accessibility and balance load, replica invalidation becomes a critical issue when nodes caching updatable resources may concurrently update their own replicas, which renders replicas held by other nodes obsolete. Most existing solutions to the replica invalidation problem either impose constrains that only the data source could perform update and invalidate other replicas, or resort to network-wide flooding which results in heavy network traffic and leads to scalability problem, or both. The lack of infrastructure supports and frequent topology changes in MANETs further challenge the issue. We apply the same cross-layer paradigm to invalidating replicas in MANETs which allows concurrent updates performed by multiple replicas. To coordinate concurrent updates and disseminate replica invalidations, a special infrastructure, called validation mesh or mesh for short, is adaptively maintained among nodes possessing ‘valid" replicas of a resource. Once a node has updated its replica, an invalidation packet will only be disseminated over the validation mesh to inform other replica-possessing nodes that their replicas become invalid and should be deleted. The structure (topology) of the validation mesh keeps evolving (1) when nodes request and cache a resource, (2) when nodes update their respective replicas and invalidate other replicas, and (3) when nodes move. To accommodate the dynamics, our scheme integrates the components of swarm intelligence to adaptively maintain the validation mesh without relying on any underlying MANET routing protocol. In particular, the scheme takes into account concurrent updates initiated by multiple nodes to ensure the consistency among replicas. In addition, version number is used to distinguish new from old replicas when invalidating any stale replica. Simulation results show that the proposed scheme effectively facilitates concurrent replica updates and efficiently perform replica invalidation without incurring network-wide flooding. Figure 3 depicts the idea of ‘validation mesh" which maintains connectivity among nodes holding valid replicas of a resource to avoid network-wide flooding when invalidating replicas. Figure 3: Examples showing maintenance of validation mesh There are eight nodes in the sample network, and we start with only node A holding the valid file, as shown in Figure 3(a). Later on, node G issues a query packet for the file and eventually obtains the file from A via nodes B and D. Since intermediate nodes are allowed to cache forwarded data, nodes B, D, and G will now hold valid replicas of the file. As a result, a validation mesh is established among nodes A, B, D, and G, as depicted in Figure 3(b). In Figure 3(c), another node, H, has issued a query packet for the same file and obtained it from node B"s cache via node E. At this point, six nodes hold valid replicas and are connected through the validation mesh. Now we assume node G updates its replica of the file and informs the other nodes by sending an invalidation packet over the validation mesh. Consequently, all other nodes except G remove their replicas of the file from their storage and the validation mesh is torn down. However, query forwarding pheromone, as denoted by the dotted arrows in Figure 3(d), is setup at these nodes via the ‘reverse paths" in which the invalidation packets have traversed, so that future requests for this file will be forwarded to node G. In Figure 3(e), node H makes a new request for the file again. This time, its query packet follows the pheromone toward node G, where the updated file can be obtained. Eventually, a new validation mesh is established over nodes G, B, D, E, and H. To maintain a validation mesh among the nodes holding valid replicas, one of them is designated to be the focal node. Initially, the node that originally holds the data is the focal node. As nodes update replicas, the node that last (or most recently) updates a 4 The 3rd International Conference on Mobile Technology, Applications and Systems - Mobility 2006 corresponding replica assumes the role of focal node. We also name nodes, such as G and H, who originate requests to replicate data as clients, and nodes B, D, and E who locally cache passing data as data nodes. For instance, in Figures 3(a), 3(b), and 3(c), node A is the focal node; in Figures 3(d), 3(e), and 3(f), node G becomes the focal node. In addition, to accommodate newly participating nodes and mobility of nodes, the focal node periodically floods the validation mesh with a keep-alive packet, so that nodes who can hear this packet are considered themselves to be part of the validation mesh. If a node holding a valid/updated replica doesn"t hear a keep-alive packet for a certain time interval, it will deploy a search packet using the resource discovery mechanism described in Section 3 to find another node (termed attachment point) currently on the validation mesh so that it can attach itself to. Once an attachment point is found, a search_reply packet is returned to the disconnected node who originated the search. Intermediate nodes who forward the search_reply packet will become part of the validation mesh as well. To illustrate the effect of node mobility, in Figure 3(f), node H has moved to a location where it is not directly connected to the mesh. Via the resource discovery mechanism, node H relies on an intermediate node F to connect itself to the mesh. Here, node F, although part of the validation mesh, doesn"t hold data replica, and hence is termed nondata node. Client and data node who keep hearing the keep-alive packets from the focal node act as if they are holding a valid replica, so that they can reply to query packets, like node B in Figure 3(c) replying a request from node H. While a disconnected node attempting to discover an attachment point to reattach itself to the mesh, the disconnected node can"t reply to a query packet. For instance, in Figure 3(f), node H does not reply to any query packet before it reattaches itself to the mesh. Although validation mesh provides a conceptual topology that (1) connects all replicas together, (2) coordinates concurrent updates, and (3) disseminates invalidation packets, the technical issue is how such a mesh topology could be effectively and efficiently maintained and evolved when (a) nodes request and cache a resource, (b) when nodes update their respective replicas and invalidate other replicas, and (c) when nodes move. Without relying on any MANET routing protocols, the two primitives work together to facilitate efficient search and adaptive maintenance. 6. PERFORMANCE EVALUATION We have conducted simulation experiments using the QualNet simulator to evaluate the performance of the described resource discovery, resource distribution, and replica invalidation schemes. However, due to space limitation only the performance of the replica invalidation is reported. In our experiments, eighty nodes are uniformly distributed over a terrain of size 1000×1000 m2 . Each node has a communication range of approximately 250 m over a 2 Mbps wireless channel, using IEEE802.11 as the MAC layer. We use the random-waypoint mobility model with a pause time of 1 second. Nodes may move at the minimum and maximum speeds of 1 m/s and 5 m/s, respectively. Table 1 lists other parameter settings used in the simulation. Initially, there is one resource server node in network. Two nodes are randomly picked up every 10 seconds as clients. Every β seconds, we check the number of nodes, N, which have gotten data. Then we randomly pickup Min(γ,N) nodes from them to initiate data update. Each experiment is run for 10 minutes. Table 1: Simulation Settings HOP_LIMIT 10 ADVERTISE_HOP_LIMIT 1 KEEPALIVE_INTERVAL 3 second NUM_SEARCH 1 ADVERTISE_INTERVAL 5 second EXPIRATION_INTERVAL 10 second Average Query Generation Rate 2 query/ 10 sec Max # of Concurrent Update (γ) 2 Frequency of Update (β) 3s We evaluate the performance under different mobility speed, the density, the maximum number of concurrent update nodes, and update frequency using two metrics: • Average overhead per update measures the average number of packets transmitted per update in the network. • Average delay per update measures how long our approach takes to finish an update on average. All figures shown present the results with a 70% confidence interval. Figure 4: Overhead vs. speed for 80 nodes Figure 5: Overhead vs. density Figure 6: Overhead vs. max #concurrent updates Figure 7: Overhead vs. freq. Figure 8: Delay vs. speed Figure 9: Delay vs. density The 3rd Conference on Mobile Technology, Applications and Systems - Mobility 2006 5 Figure 10: Delay vs. max #concurrent updates Figure 11: Delay vs. freq. Figures 4, 5, 6, and 7 show the overhead versus various parameter values. In Figure 4, the overhead increases as the speed increase, which is due to the fact that as the speed increase, nodes move out of mesh more frequently, and will send out more search packets. However, the overhead is not high, and even in speed 10m/sec, the overhead is below 100 packets. In contrast, the packets will be expected to be more than 200 packets at various speeds when flooding is used. Figure 5 shows that the overhead almost remains the same under various densities. That is attributed to only flooding over the mesh instead of the whole network. The size of mesh doesn"t vary much on various densities, so that the overhead doesn"t vary much. Figure 6 shows that overhead also almost remains the same under various maximum number of concurrent updates. That"s because one more node just means one more flood over the mesh during update process so that the impact is limited. Figure 7 shows that if updates happen more frequently, the overhead is higher. This is because the more quickly updates happen, (1) there will be more keep_alive message over the mesh between two updates, and (2) nodes move out of mesh more frequently and send out more search packets. Figures 8, 9, 10, and 11 show the delay versus various parameter values. From Figure 8, we know the delay increases as the speed increases, which is due to the fact that with increasing speed, clients will move out of mesh with higher probability. When these clients want to update data, they will spend time to first search the mesh. The faster the speed, the more time clients need to spend to search the mesh. Figure 9 shows that delay is negligibly affected by the density. Delay decreases slightly as the number of nodes increases, due to the fact that the more nodes in the network, the more nodes receives the advertisement packets which helps the search packet find the target so that the delay of update decreases. Figure 10 shows that delay decreases slightly as the maximum number of concurrent updates increases. The larger the maximum number of concurrent updates is, the more nodes are picked up to do update. Then with higher probability, one of these nodes is still in mesh and finishes the update immediately (don"t need to search mesh first), which decreases the delay. Figure 11 shows how the delay varies with the update frequency. When updates happen more frequently, the delay will higher. Because the less frequently, the more time nodes in mesh have to move out of mesh then they need to take time to search the mesh when they do update, which increases the delay. The simulation results show that the replica invalidation scheme can significantly reduce the overhead with an acceptable delay. 7. CONCLUSION To facilitate resource discovery and distribution over MANETs, one approach is to designing peer-to-peer (P2P) systems over MANETs which constructs an overlay by organizing peers of the system into a logical structure on the top of MANETs" physical topology. However, deploying overlay over MANETs may result in either a large number of flooding operations triggered by the routing process, or inefficiency in terms of bandwidth usage. Specifically, overlay routing relies on the network-layer routing protocols. In the case of a reactive routing protocol, routing on the overlay may cause a large number of flooded route discovery message since the routing path in each routing step must be discovered on demand. On the other hand, if a proactive routing protocol is adopted, each peer has to periodically broadcast control messages, which leads to poor efficiency in terms of bandwidth usage. Either way, constructing an overlay will potentially suffer from the scalability problem. The paper describes a design paradigm that uses the functional primitives of positive/negative feedback and sporadic random walk to design robust resource discovery and distribution schemes over MANETs. In particular, the scheme offers the features of (1) cross-layer design of P2P systems, which allows the routing process at both the P2P and the network layers to be integrated to reduce overhead, (2) scalability and mobility support, which minimizes the use of global flooding operations and adaptively combines proactive resource advertisement and reactive resource discovery, and (3) load balancing, which facilitates resource replication, relocation, and division to achieve load balancing. 8. REFERENCES [1] A. Oram, Peer-to-Peer: Harnessing the Power of Disruptive Technologies. O"Reilly, March 2000. [2] S. Helal, N. Desai, V. Verma, and C. Lee, Konark - A Service Discovery and Delivery Protocol for ad-hoc Networks, in the Third IEEE Conference on Wireless Communication Networks (WCNC), New Orleans, Louisiana, 2003. [3] G. Krotuem, Proem: A Peer-to-Peer Computing Platform for Mobile ad-hoc Networks, in Advanced Topic Workshop Middleware for Mobile Computing, Germany, 2001. [4] M. Papadopouli and H. Schulzrinne, A Performance Analysis of 7DS a Peer-to-Peer Data Dissemination and Prefetching Tool for Mobile Users, in Advances in wired and wireless communications, IEEE Sarnoff Symposium Digest, Ewing, NJ, 2001, (Best student paper & poster award). [5] U. Mohan, K. Almeroth, and E. Belding-Royer, Scalable Service Discovery in Mobile ad-hoc Networks, in IFIP Networking Conference, Athens, Greece, May 2004. [6] L. Yin and G. Cao, Supporting Cooperative Caching in Ad Hoc Networks, in IEEE INFOCOM, 2004. [7] V. Thanedar, K. Almeroth, and E. Belding-Royer, A Lightweight Content Replication Scheme for Mobile ad-hoc Environments, in IFIP Networking Conference, Athens, Greece, May 2004. 6 The 3rd International Conference on Mobile Technology, Applications and Systems - Mobility 2006

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

manet p2p system
manet routing protocol
mobile ad-hoc network

manet
hybrid discovery scheme
neighbor discovery protocol
negative feedback
query packet
replica invalidation
validation mesh
invalidation packet
route discovery message
concurrent update
resource discovery