Cognition Based Adaptive Control Mechanism

This work was supported by the National High Technology Research and Development Program ("863" Program) of China under Grant No. 2007AA01Z209 and the National Basic Research Program of China under Grant No. 2009CB320405.

   For the inter-discipline established by introducing basic cognition concepts into wireless communication, Simon Haykin contributes to a book titled Cognitive Wireless Communication Networks which covers the relation between human intelligence and wireless communication system. From perspectives of the communication system, Mr. Haykin describes the basic functions of Cognitive Radio (CR), including observation, learning, memorizing, and decision-making. That is, the CR system is able to respond to the information in its environment in light of prior information obtained from historical information and current observation result. These basic functions can be summarized as follows:

• Cognize the environment, including radio environment, intra- and inter-system information;
• Learn from environment;
• Use policy to control system parameters, including power, encoding modulation scheme, access scheme, and route selection;
• Fulfill inter-node communications through the self-organization and cooperation of the nodes;
• Control the communication process between nodes through reasonable resource allocation;
• Generate intentions and self-consciousness.

   The basis objectives of a CR system are to realize user-required flexible and reliable communications in complicated communication systems, and to make efficient use of the spectrum resource. In a word, the cognition in a radio system is a process of information acquisition, analysis, storage and response performed to obtain the knowledge contained in the external environment.

1 Cognition Loop Based Adaptive Control Mechanism
According to the basic functions and objectives of the CR system, it is obvious that the system is, in principle, a global closed-loop feedback control system. From perspectives of cybernetics and artificial intelligence, this system is realized through the cognition-based adaptive feedback control mechanism[1]. Therefore, a loop-shaped block diagram structure can be used to depict the dependencies between the system’s basic functions, objectives and inner entities. When Joseph Mitola III proposed the basic concept of CR in 1999, he used the cognitive computer loop to describe the process of CR fulfillment from the perspective of software computer[2]. Simon Haykin used the cognitive signal-processing loop to describe the signal processing, learning and adaptation processes of the CR system from the perspective of signal processing and cybernetics[3]. The cognitive computer loop is a highly abstract conceptive model without any implementation details. Therefore, actual application examples are necessary to help it take shape. The cognitive signal processing loop is what makes the cognitive computer loop take shape. However, it takes into consideration only signal processing at the physical layer of single transmission-reception pairs, not including vertical inter-layer design parameters or horizontal multi-system coordination and competition scenarios. For this reason and according to the major research contents and methods for the CR system in recent years, this paper will expand the cognition loop model in two aspects: Firstly, the horizontal multi-system coordination and competition concepts will be included, and the scenarios of multi-system sharing of radio resources will be taken into consideration; secondly, the vertical layer-by-layer and cross-layer design concepts will be included for
physical-layer parameter configuration and optimisation objectives, as well as for fulfilling joint optimisation and parameter configuration when the link layer and network layer issues are together taken into account.

   As the cognition science is concerned, a cognition system comprises two major components: Environment; and the intelligent system located in the environment. The intelligent system observes to learn how the environment affects it. At the same time, the intelligent system reacts to the environment. As shown in Figure 1, in the cognition loop model put forward in this paper, the intelligent system is defined as network nodes that are dependent of each other to reach a certain communication objective. Specifically, the network nodes include communication transmission-reception pairs at the physical layer, and the source node, destination node and trunk node at the network layer. The environment can be either external or internal environment. The external environment refers to radio resources and external system information; while the internal environment refers to the events triggered inside the intelligent system and the inner status. The intelligent system senses and obtains the status information of the radio resource. The external system transfers its own information as messages to the system. But the inner event and status of the intelligent system can be obtained directly. The actions of the intelligent system are determined with learned policies, but the learning process is determined by the following two factors together: The utilization decided by the layer-by-layer or inter-layer optimisation objectives; and the information obtained by the intelligent system from environment. The actions act on the external environment and the internal environment as well.

2 Environment Modeling

2.1 Modeling of Radio Resource
The radio resource model refers to the availability model and quality model. The availability model includes temporally and spatially available frequencies; while the quality model is described with the fading status of radio channel.

   (1) Availability Model
   This model is determined by the spectrum opportunity (also called spectrum hole or white space spectrum) model[4] and the interference temperature model[5].

   The statistical features of the licensed system’s traffic load determine the spectrum opportunity model, which is generally modeled as an on/off process suitable for a certain distribution. When according to the traffic characteristics of the licensed system, the spectrum opportunities can be either of the timeslot or continuous type.

   The interference temperature model means that even if a frequency is occupied by a licensed system, the CR system is able to use the frequency as long as the interference introduced into the CR system does not exceed the interference temperature threshold.

   (2) Quality Model
   The quality model of radio resources (channel and frequency) is determined by Signal-to-Noise Ratio (SNR) of the receiver end, while the SNR changes mainly depend on the fading of radio channels. In most cases, the channel fading is either large-scale fading determined by the distance between reception and transmission pairs or the small-scale fading as a result of multi-path effects. With the large-scale fading, the location of reception and transmission pairs can be indicated with the two-dimensional coordinator, yet the location changes can be modelled as a time-homogeneous discrete-time Markov process. The SNR determined by small-scale fading can be described by a quite common discrete-time Markov process[6]. With this model, the continuous SNR falling in different segments is quantified as discrete status, and the transfer probability of status is determined by the duration of the status, Doppler frequency shifts, upper threshold and bottom threshold values of the quantification segment, and also the average SNR.

2.2 Modeling of External System
The problems of resource competition or resource co-use are unavoidable when several systems share the radio resources for communications. If a central control unit is available to help with the resource configuration among several systems, it will be unnecessary to establish the external system model for every system to address the issue of inter-system interactions. Alternatively, if every system shares the resources with another in the self-organized way, then each of them will have to face the external system environments of others. Information of the external system includes their policies, utilization, and environment. Such information is not directly observable, but encapsulated in the protocols and obtained as messages by its own system.

   Since the external system and the system itself are rivals for resource utilization, but they simultaneously expect cooperation for win-win results, the information contained in the message fed back by the external system can be either true or not true, which generally depends on the self-organization modes of the multiple systems. There are three self-organization modes as follows:

   (1) Cooperation
   In this self-organization mode, there is no competition for resource utilization among systems, or certain a confining agreement is reached to prevent and lessen the competition for better resource utilization. When the systems cooperate, they are prone to providing their own true information to the external system. For example, in some research of coordinated spectrum cognition without energy confinement, there is no inter-system conflict for resource utilization and in this case, the cooperative network nodes will notify each other of true cognition information so as to obtain higher spectrum inspection probability and lower virtual alarm probability[7].

   (2) Competition
   In this mode, systems are competing for resource utilization and there is neither confining agreement to lessen such conflicts nor a third-party mediation mechanism. Therefore, the systems are prone to transferring untrue information to each other[8].

   (3) Cooperation plus Competition
   In this mode, some systems form an alliance or small group. Systems in the same alliance do not compete for resource utilization because they have reached a confining cooperation agreement. Therefore, they are prone to providing true information to each other. However, a system in an alliance will compete with a system outside the alliance, and the information transferred between the two is likely untrue. The existence of alliance will improve the resource utilization inside the alliance but degrade the total utilization (social utilization) of all the systems as a whole and impair the fairness among systems. To encourage or restrict the existence of alliances depends on the design objectives. For example, if there is a need to guarantee the resource utilization of some higher level systems, these systems should better set up an alliance; and if fairness and total resource utilization are more important, some rules should be designed to restrict the existence of alliance[9].
It is noteworthy whether the systems sharing resources in a self-organized way are willing to transfer true information depends on whether their own interests are guaranteed or not, and, furthermore, whether transferring true information will bring about better, or at least not worse, utilization than transferring untrue information. That is to meet the so-called incentive consistent restraint. On the one hand, if there is no utilization conflict among systems or there is an agreement for confinement among systems, transferring true information will be a better choice for improved utilization. On the other hand, if there is utilization conflict among systems, then some third party mechanism for mediation or rules can be introduced to punish systems transferring false information so that the systems are more likely to transfer their true information and the total system performance will achieve Pareto improvement. To sum up, the environment formed by external systems can be deemed as the social environment that is affected by the subjective elements of the external intelligent systems.

2.3 Modeling of Events and States Inside a System
As the CR system is concerned, it has internal events of handover request, call request, packet arrival and route request, by which various layers trigger the communication process. Such events are generally modeled as the random processes that conform to the Poisson distribution and Pareto distribution. On the other hand, by introducing the devices of queue and register, the continuous events are converted as discrete states: the number of requests in waiting state in the queue, and number of packets inside the register.

3 Observation of Intelligent System
The CR system can acquire its internal events and states directly and thus there is no need for a special observation method. However, some specific observation methods are necessary so that the CR system can cognize and estimate the radio environment and obtain external system’s information.

3.1 Cognition and Estimation of Radio Environment
   (1) Cognition of Availability
   There are two types of method for spectrum cognition. As for the research of single-node spectrum cognition, there are detections based on matched filter, and such blind detections as energy detection, detection of period features and Covariance matrix-based detection. On the other hand, multi-node coordinated spectrum cognition is introduced to improve cognition sensitivity and overcome hidden terminal problems. The multi-node coordinated spectrum cognition can also make system design simpler. Because multi-node detection enhances the detection accuracy of the whole system, the requirement on performance of the single-node detection can be lowered on condition that the system requirements are satisfied[10].

   (2) Estimation of Quality
   The large-scale fading information can be estimated in light of the geographical location fed back by the receiver to the transmitter. To obtain small-scale fading information, the transmitter should first send pilot to the receiver, and then the receiver uses the pilot to estimate the channel status and feeds back the channel status information to the transmitter. To ensure that the reported information is correct and prompt, dedicated control channel can be allocated to transmit such information.

3.2 Acquisition of External System Information
Information of the external systems cannot be observed directly, but encapsulated in protocols and obtained as messages by the system instead. Therefore, protocol design is the foundation of inter-system information interaction. The protocol design based on message transfer is largely dependent on the self-organization structure of multiple systems:

   (1) Distribution Structure
   With this structure, there is no dedicated central controller responsible for receiving and transferring messages. For this reason, the design of protocols can refer to the Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) of IEEE802.11 standard. Since the CSMA/CA has been designed with reserved fields for protocol expansion, the fields can be used to pack the system messages[11].

   (2) Scattered Structure
   This structure is the type between centralized and distributed modes, that is to say, the decision and policy are distributed rather than dependent on the central controller, but the protocol, signaling and control information depend on a weakened central controller for collection and transferring intentions[12].

4 Structure of Intelligent System
The intelligent system is made up of three functional modules: reasoning, utilization and learning; and two storage modules: information base and policy base.

4.1 Functional Modules
   (1) Reasoning
   If the environment is partially observable or unobservable, it is necessary to obtain environment information through reasoning. Reasoning is a thinking process which draws an unknown conclusion through one or more known judgments. The intention of reasoning is to obtain unknown information from known information, especially those unknown information that cannot possibly be obtained through observation. Common reasoning is the Bayesian reasoning that is based on probability distribution.

   (2) Utilization
   Preference is the sense of satisfaction related with the optimization objective and obtained by the intelligent system from the environment, while utilization is a measure of the preference, that is, the standard and reference for measuring the positive or passive effects obtained by the system. Utilization is essentially a kind of function, that is, the satisfaction degree of the system in this environment deduced from the environment information[13].

   (3) Learning
   During the learning process, the intelligent system obtains current information of environment, makes exploratory actions on the environment, and obtains appraisal of the actions fed back by the environment and environment transfer status. If an action of the intelligent system wins award of the environment (positive utilization), then the intelligent system will be more likely to produce this action again in the future; otherwise it will be less likely to do it again. With the interactions of system’s actions, environment feedback and utilizations, the learning function module gradually modifies the mapping from system information to actions so as to optimize the system performance[14].

4.2 Storage Modules
   (1) Information Base
   If the observation result of environment is true and complete, the result will enter the information base directly, for example, the internal event and status of the CR system. Meanwhile, some of the observable or unobservable environment information, after the reasoning process, is stored as the confidence level in the information base. Confidence level refers to the probability measure of all possible values of information, for example, the result obtained from Bayesian reasoning takes the form of confidence level[11].

   (2) Policy Base
   Policies are the action rules of the intelligent system in a given environment. In other words, it defines when the system takes what actions[15]. A policy can be either fixed or mixed (random). With a fixed policy, the action corresponding to a given environment is unique. With a mixed policy, what action to be taken for the given environment depends on the probability distribution, and such distribution is the probability measure of all possible actions determined by the current environment information. This shows that the policy reflects the relation between the environment information and action.

   To realize the CR system, the problem of storage price is unavoidable and should be taken into consideration. The presentation form of information or policies in two storage modules and the method to compress the storage size are key factors for the system fulfillment. Two available presentation forms are radio-based extensible markup language and policy descriptive language based on policy core information model. Two methods for compressing the storage size are feature-based extraction and nerve network compressed storage.

5 Actions of Intelligent System
Actions of the intelligent system are the system’s decision variables in the case of a given environment information. There are two types of action as follows:

   (1) The actions for processing communication transactions, for example, transmit power, encoding, modulation, access, scheduling, and route selection. Such actions correspond to the layer-by-layer and inter-layer optimization objectives, for example, the energy-efficient transmission corresponds to adaptive rate control (encoding and modulation).

   (2) The actions for determining in what form to feed back the system’s information to the external systems, that is, to determine the truthfulness of the system’s information in the feedback information. As stated before, such actions correspond to the system’s different self-organization modes. Actions can, on the one hand, change the environment’s transfer rule, and also provide information to external system on the other.

6 Conclusion
The CR system is capable of cognition, reasoning, learning and decision making. Its ultimate intention is to make highly efficient use of the spectrum resource for a complicated system and to make the complicated system less dependant on human for its operation and maintenance. These objectives set higher requirements on the system’s adaptability, intelligence and also the coordination between systems. Besides, the research on system’s adaptability and self control capability in light of environment’s status transfer has to be considered in the architecture design of future wireless communication system. This should imply technical renovations and new challenges in the theoretical study of wireless communication. The CR system-related theoretical study and standard formulation are now undertaken in academic and industrial arenas and some achievements have been made. It is expectable that as a response to scientific and technological developments and market requirements, the research of CR system will see more progress and applications.

References
[1] HAYKIN S. Cognitive radio: brain-empowered wireless communications [J]. IEEE Journal on Selected Areas in Communications, 2005, 23(2):
201-220.
[2] MITOLA J III. Cognitive radio architecture: the engineering foundations of radio XM [M]. New York, NY, USA: John Wiley & Sons, 2006: 134-138.
[3] HOSSAIN E, BHARGAVA V. Cognitive wireless communication networks [M]. New York, NY, USA: Springer, 2007: 1-43.
[4] HYOI KL, SHIN K G.  Efficient discovery of spectrum opportunities with MAC-layer sensing in cognitive radio networks [J]. IEEE Transactions on Mobile Computing, 2008, 7(5): 533-545.
[5] CLANCY T. Formalizing the interference temperature model [J]. Journal on Wireless Communications and Mobile Computing, 2007, 7(9): 1077-1086.
[6] WANG H S, MOSYERI N. Finite-state Markov channel: a useful model for radio communication channels [J]. IEEE Transactions on Vehicular Technology, 1995, 44 (1): 163-171.
[7] GHASEMI A, SOUSA E S. Collaborative spectrum sensing for opportunistic access in fading environment [C]//Proceedings of 1st IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks (DySPAN’05), Nov 8-11, 2005, Baltimore, MD, USA. Piscataway, NJ, USA: IEEE, 2005: 131-136.
[8] NEEL J, REED J H, GILLES R P. The role of game theory in the analysis of software radio networks [C]//Proceedings of 2002 SDR Forum Technical Conference: Vol.2, Nov 12-14, 2002, San Diego, CA, USA. NP-3-02.
[9] MATHUR S, SANKAR L, MANDAYAM N B. Coalitions in cooperative wireless networks [J]. IEEE Journal on Selected Areas in Communications, 2008, 26(7): 1104-1115.
[10] AKYILDIZ F, LEE W, VURAN M C, et al. Next generation/dynamic spectrum access/cognitive radio wireless networks: a survey [J]. Elsevier Computer Networks, 2006, 50(13): 2127-2159.
[11] ZHAO Q, TONG L, SWAMI A, et al. Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: a POMDP framework [J]. IEEE Journal on Selected Areas in Communications, 2007, 25(3): 589-600.
[12] GANDHI S, BURAGOHAIN C, CAO L, et al. Towards real-time dynamic spectrum auctions [J]. Elsevier Computer Networks, 2008, 52 (5): 879-897.
[13] JIANG ZHIMEI, GE YE, LI YE. Max-utilization wireless resource management for best-effort
traffic [J]. IEEE Transactions on Wireless Communications. 2005, 4(1): 100-111.
[14] CLANCY C, HECKER J, STUNTEBECK E, et al. Applications of machine learning to cognitive radio networks [J]. IEEE Wireless Communications Magazine, 2007, 14 (4): 47-52.
[15] ZHU JIANG, WEI NING, XU BINGYANG, et al. Optimal and suboptimal access and transmission polices for dynamic spectrum access over fading channels in cognitive radio networks [J]. Chinese Journal of Electronics. 2008, 17(4): 726-732.

[Abstract] According to the basic functions and objectives of Cognitive Radio (CR) systems, the cognition-based adaptive control mechanism is the generalization of the research contents and approaches of cognitive radio systems. Therefore, the mechanism is described by a cognition loop, which contains the following parts: environment, inner structure of intelligent systems, observation and action.