Ebook: Self-Organization and Autonomic Informatics (I)

Today's IT systems with its ever-growing communication infrastructures and computing applications are becoming more and more large in scale, which results in exponential complexity in their engineering, operation, and maintenance. Conventional paradigms for run-time deployment, management, maintenance, and evolution are particularly challenged in tackling these immense complexities. Recently, it has widely been recognized that self-organization and self-management/regulation offer the most promising approach to addressing such challenges. Consequently, a number of autonomic/adaptive computing initiatives have been launched by major IT companies, like IBM, HP, and others.

Self-organization and adaptation are concepts stemming from the nature and have been adopted in systems theory. They are considered to be the essential ingredients of any living organism and, as such, are studied intensively in biology, sociology, and organizational theory. They have also penetrated into control theory, cybernetics and the study of adaptive complex systems. The original idea was to understand complex systems behaviour by understanding the systems' self-organization and adaptation mechanisms, i.e., to understand a system by observing the behaviour of its components and their interactions. However, as stated, the study of self-organization and adaptation has mainly been related to living systems so far.

Computing and communication systems are basically artificial systems. This prevents conventional self-organization and adaptation principles and approaches from being directly applicable to computing and communication systems. The complexity attributes in terms of openness, scalability, uncertainty, discrete-event dynamics, etc. have varied contexts in large-scale complex IT systems, and are too prominent to be solved by procedures pre-defined at off-time. Rather, they have to be tackled by means of run-time perception of the complexity patterns and the run-time enforcement of self-organization and adaptation policies. The current knowledge about large-scale complex IT systems is still very limited, and a framework has yet to be established for their self-organization and adaptation.

The methodology of multi-agent systems and the technology of Grid computing have shed lights for the exploration into the self-organization and adaptation of large-scale complex IT systems. A multi-agent system is one that consists of a collection of autonomous and intelligent agents that have the ability to interact with each other and, thus, may by themselves constitute organizations at run-time. The global behaviour of a multi-agent system stems from the emergent interactions among the agents. Multi-agent systems have been studied widely, not only in computer science, software engineering and artificial intelligence, but even more widely under the heading “systems theory” in economics, management science and sociology. In fact, multi-agent systems permeate social, economic, and technical domains. Essentially, multi-agent systems provide a generic model for large-scale complex IT systems. Exploring and understanding the self-organization and adaptation of multi-agent systems is of profound significance for engineering the self-organization and self-management/regulation of large-scale complex IT systems, comprised of communication infrastructures and computing applications.

Grid computing is the new generation of distributed and networked information and computing systems which have the capacity to enable users and applications, in an emergent manner, to transcend the organizational boundaries and to gain access to the distributed heterogeneous computing resources owned and administrated locally by different organizations. A Grid computing system is by nature a large-scale, complex, and open multi-agent system. Grid computing compounds various areas such as distributed computing resource management, semantic web technology, service-oriented architecture and service management, distributed workflow management, distributed problem solving environment, etc. A Grid commutating system exposes all the complexity attributes typical of large-scale complex IT systems. Investigating the self-organization and autonomic systems for Grid computing has remained a huge challenge.

This book provides in-depth thoughts about the above discussed challenges as well as a range of state-of-the-art methodologies and technologies for the entirely new area, that is, self-organization and autonomic systems in computing and communications. We refer to this newly emerging area as Self-Organization and Autonomic Informatics, which has represented the future generation of IT systems, comprised of communication infrastructures and computing applications, which are inherently large-scale, complex, and open.

The 16 full-length and 17 short papers included in this book are carefully selected from the 58 initial manuscripts based on a rigorous peer review process that was conducted by the 86 technical reviewers worldwide who are experts or active researchers in the related areas. The contents of the book are structured as five parts, i.e.,

Part I: Self-Organization and Adaptation in General;

Part II: Self-Organization/Adaptation of Multi-Agent Systems;

Part III: Self-Organization/Adaptation for Grid Computing;

Part IV: Autonomic Computing in General; and

Part V: Autonomic Communications.

We are sure that you will find the book interesting.

Editors: Professor Hans Czap, University of Trier, Germany; Professor Rainer Unland, University of Duisburg-Essen, Germany; Professor Cherif Branki, University of Paisley, UK; Professor Huaglory Tianfield, Glasgow Caledonian University, UK

In a complex dynamic system the centralised control and local monitoring of system behaviour is not achievable by scaling up simple feedback adaptation and control models. This paper proposes using a variety of concepts from distributed artificial intelligence (DAI) to logically model an abstract system control using adaptable agent federations to induce self-organisation in a swarm type system. The knowledge acquisition and updates are handled through a modal logic of belief for team dynamics and the system as a whole evolves to learn from local failures that have minimal impact on the global system. Self-governance emerges from innate (given) action thresholds that are adapted dynamically to system demands. In this way it is shown that such a system conforms to the prerequisites that have been specified as necessary for a system to exhibit self-organisation and the intrinsic benefits of agent teamwork are established for a robust, reliable and agile system. The approach is illustrated by looking at team formation in a swarm scenario from a proposed NASA project. The Situation Calculus is used to formalise the dynamic nature of such systems with a dynamic logic implementation to reason about the ensuing programs. Subsequently the model is encoded using the Neptune scripting language and compiled to an object-oriented system for its deployment on distributed systems architecture.

When designing self-organising emergent multi-agent systems (MASs), a fundamental engineering issue is to achieve a macroscopic behaviour that meets the requirements and emerges only from the behaviour of locally interacting agents. Agent-oriented methodologies today are mainly focussed on engineering the microscopic issues, i.e. the agents, their rules, how they interact, etc, without explicit support for engineering the required macroscopic behaviour. As a consequence, the macroscopic behaviour is achieved in an ad-hoc manner. This paper proposes a way to define a full life-cycle methodology based on an industry-ready software engineering process, i.e. the Unified Process, which is customised to explicitly focus on engineering macroscopic behaviour of self-organising emergent MASs. As such, the MAS paradigm is integrated into an existing and widely accepted methodology and we can systematically develop a solution that exhibits the required macroscopic behaviour.

This paper motivates research into implementing nature-inspired algorithms in decentralised, asynchronous and parallel environments. These characteristics typify environments such as Peer-To-Peer systems, the Grid and autonomic computing which demand robustness, decentralisation, parallelism, asynchronicity and self-organisation. Nature-inspired systems promise these properties. However, current implementations of nature-inspired systems are only loosely based on their natural counterparts. They are generally implemented as synchronous, sequential, centralised algorithms that loop through passive data structures. For their successes to be relevant to the aforementioned new computing environments, variants of these algorithms must work in truely decentralised, parallel and asynchronous Multi-Agent System (MAS) environments. A general methodology is presented for engineering the transfer of nature-inspired algorithms to such a MAS framework. The concept of pheromone infrastructures is reviewed in light of emerging standards for agent platform architecture and interoperability. These ideas are illustrated using a particularly successful nature-inspired algorithm, Ant Colony System for the Travelling Salesman Problem.

In Multiagent systems there are several agents with cooperative or competitive goals. Here, we are especially interested in zero-sum games which contain exactly two players with fully opposite goals. We describe a method based on Maximum-Expected-Utility [7] principle that learns the ingenuity of the opponent based on the moves of the opponent through a game and exploits this knowledge to play better against that opponent. Then we demonstrate an application of proposed method in the popular board game of Connect-4. The results show that the proposed method is superior compared to previous methods for adversarial environments especially when there is not adequate training for appropriate adaptation against an opponent.

Bayesian approach to decision making is successfully applied in control theory for design of control strategy. However, it is based on on the assumption that a decision-maker is the only active part of the system. Relaxation of this assumption would allow us to build a framework for design of control strategy in multi-agent systems. In Bayesian framework, all information is represented by probability density functions. Therefore, communication and negotiation of Bayesian agents also needs to be facilitated by probabilities. Recent advances in Bayesian theory make formalization these tasks possible. In this paper, we bring the existing theoretic results together and show their relevance for multi-agent systems. The proposed approach is illustrated on the problem of feedback control of an urban traffic network.

The motivation of this paper is to realize an energy-driven self-organising architecture of Social Behaviour Networks(SoBeNet) for the Web application. Internet agents can sense changes in the web environment via virtual web sensors and behavior selection is based on the energy spreading mechanism from the bottom-up paradigm of AI. There is no global coordinator module to control behavior selection and the cooperation between the two agents is implicitly. A social behavior network is formed spontaneously.

In today's hyper-competitive business environments virtual organisations are becoming highly dynamic and unpredictable. Individuals may want to work together across organisation boundaries but do not have much prior knowledge of others. The semantic web and its associated new standards appear very promising as candidates to support a new generation of virtual organisations. In this paper a behaviour based organisation, Social Behaviour Networks, is proposed. In order to sense the changes on the web this paper focuses on a virtual sensor for allocating tasks amongst agents based on the announcements of tasks and capabilities of agents in DAML (DARPA Agent Markup Language). Due to the autonomy of agents the announcements are often vague and in a very high dimensional space. The ontology can provide useful information for achieving variable-resolution sensing from an individual agent's perspective and reducing the dimensions of the virtual space. The variable-resolution virtual sensors are based on hierarchical clustering analysis to reveal the level of similarity of announcements in the web.

The development of multi-agent based solution for outdoor mobile robot navigation is a complex multi-level process. Model Driven Generative Domain Engineering is one domain engineering method aim to developing optimized, reusable architectures, components and aspects for application engineering. According to MDGDE, we designed a set of event-driven agents, which enable the robot to initiate action adaptive to the dynamical changes in the environment. This paper describes our approach as well as its motivations and our practice.

The main goal of any testbed is to facilitate the trial and evaluation of ideas that have promise in the real world. In fact, in a real platform several physical or hardware restrictions exist and maybe arise for an agent or robot. These restrictions affect the performance of implemented algorithms. To the best of our knowledge, at present, only RoboCup Soccer Server is a common testbed for simulating a soccer game, but it can not support the above mentioned physical limits on real robot. In order to overcome these problems, we designed and implemented a realistic simulation testbed called SharifCE. In addition, this testbed allows the user to define a probability for a fault occurred on any internal part of the robot. That is done by a fault injection procedure. Our experimental results convinced us that SharifCE Testbed is quite appropriate for Multiagent Systems (MAS); because not only it is similar to a real platform, but also it supports the necessities such as Movement, Communication, Supervision, Cooperation and Learning. As an experimental result, a practical implementation of this testbed is presented

We present a novel approach to enable decision-making in a highly distributed multiagent environment where individual agents need to act in an autonomous fashion. Our architecture framework integrates risk management, knowledge management, and agent deliberation to enable sophisticated, autonomous decision-making. Instead of a centralized knowledge repository, our approach supports a highly distributed knowledge base in which each agent manages a fraction of the knowledge needed by the entire system. Our approach also addresses the fact that the desired knowledge is often highly dynamic, context-sensitive, incomplete, or uncertain. Thus risk management becomes an integral component which enables context-based, situation-aware decision making, which in turn supports autonomous, self-managing behavior of the agents. A prototype system demonstrating the feasibility of our approach is being developed as part of an ongoing funded research project.

The medical milieu is an open environment characterized by a variety of distributed, heterogeneous and autonomous information resources. Coordination, cooperation and exchange of information is important to the medical community. Efficient storage and acquisition of medical knowledge requires structured and standardized organization of data. We design a new ontology, called Generic Human Disease Ontology (GHDO), for the representation of knowledge regarding human diseases. The concepts of the GHDO ontology are organized into the following four 'dimensions': Disease Types, Symptoms, Causes and Treatments. We align and merge existing ontologies against the four dimensions of GHDO. The designed ontology makes our query system suitable for different user categories. The process of problem decomposition into smaller sub-problems within a multi-agent system becomes much easier as well. We also design a multi-agent system framework over different information resources. The multi-agent system uses the common GHDO ontology for query formulation, information retrieval and information integration. This intelligent dynamic system provides opportunities to collect information from multiple information resources, to share data efficiently and to integrate and manage scientific results in a timely manner.

Customer relationship management encompasses all the aspects of the interaction with customer and it joins all the customer related elements within an organization together in an intelligent manner. Despite its numerous benefits for organizations, there are some serious concerns regarding the implementation of CRM. Customer relationship management projects are usually complicated, long-term and resource-consuming with outstanding results for far future. The new conceptual model of CRM introduced here is a Multi-Agent System (MAS) called 'agent-based CRD'. It is the process of developing electronic context and content within customer relationships on a collaborative basis using intelligent agents. Agent-based CRD comprises three essential building blocks: single view, intelligent electronic dialogue, and opportunity spotting. A general structure for agent-based CRD framework, which is easy to understand and implement is presented graphically, theoretically, and technically in full details.

This document presents a study for the securing of Java-based systems based in the Mobile Agent paradigm. Security is an important issue for the widespread development of applications based on software agent technology. It is generally agreed that without the proper countermeasures in place, use of agent-based applications will be severe impeded. This document gives an overview of the threats associated with software agent systems focused on the elements of our simplified model: Agents and Agent Platforms and also describe a general method for controlling the behavior of mobile agent-system entities through the allocation of privileges. This approach overcomes a number of problems in existing agent systems and provides a means for attaining improved interpretability of agent systems designed and implemented independently by different manufacturers.

This paper employs the methodology of Agent-Based Computational Economics (ACE) to investigate under what conditions trust can be viable in markets. The emergence and breakdown of trust is modeled in a context of multiple buyers and suppliers. Agents adapt their trust in a partner, the weight they attach to trust relative to profitability, and their own trustworthiness, modeled as a threshold of defection. Adaptation occurs on the basis of realized profit. Trust turns out to be viable under fairly general conditions.

Grid computing environments are being extended in order to present some features that are typically found in pervasive computing environments. In particular, Grid environments have to allow mobile users to access to their services and resources, and self-adapt based on mobile user location and context. Moreover, mobile users have to have the possibility of changing their location without wondering about their pending computations. This requires that the environment be able to self-manage mobile users implicit disconnections, who may also reappear later and be willing of resuming their computations. In this paper, we present a session manager service for distributed federations of pervasive grids. The service makes mobile users able of leaving a pervasive grid and resuming later, and even in another grid of the federation, their computations. The service relies on the mobile agents technology. In particular, a personal agent is associated to every mobile user active in the environment. The personal agent offers the list of application services available at the location of the user and handles the list of activated services. If the user moves in another grid, the personal agent migrates in the new grid, handles the list of pending services, and updates the list of available services for the new location.

Complex systems such as those in evolution, growth and depinning models do not evolve slowly and gradually, but exhibit avalanche dynamics or punctuated equilibria. Self-Organized Criticality (SOC) and Highly Optimized Tolerance (HOT) are two theoretical models that explain such avalanche dynamics. We have studied avalanche dynamics in two vastly different grid computing systems: Optimal Grid and Vishva. Failures in optimal grid cause an avalanche effect with respect to the overall computation. Vishva does not exhibit failure avalanches. Interestingly, Vishva exhibits load avalanche effects at critical load density, wherein a small load disturbance in one node can cause load disturbances in several other nodes. The avalanche dynamics of grid computing systems implies that grids can be viewed as SOC systems or as HOT systems. An SOC perspective suggests that grids may be sub-optimal in performance, but may be robust to unanticipated uncertainties. A HOT perspective suggests that grids can be made optimal in performance, but would then be sensitive to unanticipated perturbations. An ideal approach for grid systems research is to explore a combination of SOC and HOT as a basis for design, resulting in robust yet optimal systems.

We investigate approaches for the automated configuration of distributed Grid services. In particular, we implement several approaches for combining configuration information specified by a group of collaborating institutions (a Virtual Organization or VO) with local configuration parameters. We describe our implementation of merging strategies for configuring the Globus Replica Location Service. Based on our initial work, we describe outstanding issues for merging local and VO configuration policies and for resolving conflicting policies.

We consider that the concept derived of the integration of pervasive, context-aware and grid computing is suitable for building the next generation of the grid applications which assist to nomadic and mobile users. To demonstrate our ideas we have implemented a pilot application, called GeneAl, using the API provided by EXEHDA middleware. EXEHDA is adaptive, service oriented and was conceived to support the execution of pervasive grid applications. The main concept embedded in middleware and application design is the context-awareness expressed by their adaptive behavior. This is also a key to provide functionality adapted to the constraints and unpredictability of the large-scale mobile environment. To achieve this objective, EXEHDA employs a lot of strategies in its services to allow the adaptation to the current state of the execution context, such as on-demand adaptive service loading and dynamic discovery and configuration. The middleware manages and implements the follow-me semantics for pervasive grid applications. In that sense, it provides services for distributed adaptive execution, context recognition, pervasive storage and access, anonymous and asynchronous communications.

A Grid information system should rely upon two basic features: the replication and dissemination of information about Grid services and resources, and an intelligent distribution of such information among Grid hosts. This paper examines an approach based on ant-based systems to replicate and map Grid services information on Grid hosts according to a given semantic classification of such services. Information is disseminated by agents (ants), which traverse the Grid by exploiting the P2P interconnections among Grid hosts. An entropy index is used to evaluate the performance of the proposed Ant-based Replication and MApping protocol (ARMAP), and control the dissemination of resource information. This approach enables the use of a semi-informed search algorithm which can drive query messages towards a cluster of peers having information about resources belonging to the requested class. A simulation analysis has been performed to evaluate the performance of the ARMAP protocol.

The emerging Grid technologies hold out the promise of a global information channel that is far more powerful and uniquely distinct from the existing internet framework. This paper aims to provide a state of the art of work addressing the need of resource discovery in computational Grids. The various resource discovery models, techniques and approaches are discussed along with their pros & cons, and recommendations are made with respect to practical implementation and directions of future research in Grid resource discovery.

This paper presents a high level model for the reliable use of mobile distributed services within Grid applications arising from research within the European research project Akogrimo. In the paper we discuss reliability and mobility in respect to static and mobile Grid services. We then introduce current research from the project into a high level blueprint/model for the architecture surrounding the provision of reliable distributed Grid services in a mobile environment.