Methodology usually proceeds from one or more of insight, intuition, or experience. Accordingly, elaboration of a methodology of interaction could be based upon any of: a deep understanding of the why of interaction; a deep understanding of the how of interaction; a feel" for the way an interaction system ought to be; or experiences with building or using effective and satisfying interaction in several contexts.
This paper strives to express a philosophical view of interaction between man and machine. The goal of this statement is a more precise, deeper understanding of both the why and the how of interaction. Upon such a foundation, subsequent work may more readily erect interactive structures and mechanisms and define trade-off criteria that allow more expeditious tailoring of systems to the anticipated using community.
Several underlying concepts require preliminary remarks.
Interaction is viewed as a bi-directional process. It is a process among two or more parties. Each party can exert control only on those parts of the process that are their own: knowing what others are expressing and producing one's own expressions. The circumstances of interaction considered in this paper do not include ones in which there is any interpreter, controller, moderator or referee to the interaction among the parties.
Each party to the interactive process is considered knowledgeable in the following sense. Each party is separately aware of information accepted as being confirmed in the surrounding world {i.e. facts). These facts may take several forms (e.g. empirical data or validated rules, inferences, and relationships, etc.). Each party is further assumed to possess a set of beliefs accepted on faith as true without verification (e.g. value systems, projected data, or hypothesized rules, inferences, relationships, etc.).
Every human possesses a rich belief system that underlies their use of explicit knowledge or facts. There are no belief systems embedded in current intelligent machines. This leads to restricting human expression input to a machine to only that capable of unambiguous decoding within a machine-based fact system. Fortunately, frame theory and other related AI developments will allow the design of hypothesis formation (abduction) into machines on a more routine basis in the next few years.
A five layer descriptive formalism for language is used rather than the familiar three layer formalism. The familiar lexical, syntactic and semantic layers are used with the accepted interpretations. Two additional layers are used based upon the logical foundations of C.S. Pierce in the last century and the formal linguistic work of E.D. Pendergraft over the past fifteen years.
The layer of abstraction beyond semantics is labelled the Pragmatic level. In natural language, semantic alternatives are organized at this level around cultural, sociological or other environments that serve to convey implication, confine meaning and limit misunderstanding. (Foley's notion of concept, in his paper for this workshop, is an example of a pragmatic environment.) In general, pragmatic environments form the contexts or connotations within which semantic alternatives are resolved. Pragmatic analysis is the formal mechanism by which this occurs.
The fifth and highest layer of abstraction is labelled the Semiotic level. In natural language, pragmatic environments are organized at this level around common tokens that signify the same aim, purpose or plan among the pragmatic environments. In general, a semiotic is a set of contexts or connotations that share the same abstract intention. (Again, in his paper for this workshop, Foley intuitively recognizes this when he sets out interactive text editor as an intent within which to seek concepts. Interactive text editor is one of many computer-based capabilities that belong to the semiotic of Interactive.)
Entrainment is a cooperative process between two or more interacting entities. The outcome of a successful entraining process is the inducement of stability (homeostasis) in the interaction. In the presence of this stability, other more complex processes can occur ranging from communication to learning.
Communication is an interactive act of conveying data, information or messages between a sender and a receiver. In the communicating process between people or between a person and an intelligent machine, the stability achieved via entrainment is known as congruence.
Congruence is identified as an agreement, tacit or explicit, between the sender and receiver of a message. Congruence agreements are arrived at about shared understanding of denotation {literal content), connotation (implied content) and intention (purpose) of the message.
Entrainment occurs within the receiver and is induced by the symbol or token content of the transmitted message. The sender embeds symbols or tokens in the message that are designed to require interpretation or some other response by the receiver. This response, in turn, is itself a new message in the opposite direction between the parties in the interactive dialogue. The continuing entraining process allows the parties to the dialogue to achieve greater congruence in their interaction.
Man-machine interaction occurs because of a purpose, or intention, on the part of the human. Intention is taken to mean a plan of action, a design, or an aim that guides action. Interaction occurs within a scope of suggested or associated meanings or implications - within a scope of connotations. Interaction between human and machine is a process; an interactive dialogue controlled via the medium of language. The purpose of the dialogue is to accomplish the intention of the human within mutually accepted connotations. Interactive dialogue is taken to be an interchange of entraining linguistic expressions among two or more parties such that knowledge, belief, understanding, or awareness is focused and expanded as a consequence, for at least one of these parties.
In the context of language formalisms, it is usual to consider lexical, syntactic and semantic elements with regard to understanding linguistic expressions. The controlling nature of language in interaction requires additional considerations. Connotations are the pragmatic issues which bound semantic alternatives. Intention represents a semiotic, a specialization of the bounded semantic alternatives. Intention and connotation combine to eliminate ambiguity of meaning.
In the specific context of man-machine interaction, the term intelligent interaction is used to denote the processes and behaviors initiated and carried out by the human.
The human capacity to function intelligently increases by using processes which: add accumulated facts to the knowledge base; transform beliefs into knowledge based on validations in the real world; eliminate beliefs due to failed evaluations in the real world; or reclassify the relational structure within and between the belief and knowledge systems based on apparent misfits or mismatches that are perceived within or between them.
Human activities in man-machine interaction are perceived as being variations of goal-directed, problem solving behaviors. This problem solving behavior is constrained by several facets of suggestive or associative implications that exist within the human's realm of knowledge, belief, understanding or awareness: (1) facts; (2) relations amongst facts; (3) beliefs; (4) relations amongst beliefs; (5) correlations (or lack thereof) between facts and beliefs; and (6) correlations (or lack thereof) between the relations amongst facts and the relations amongst beliefs.
Toward intelligent interaction, the human task is four-fold. The first task is to get oriented in the problem space and get a sense of the prevailing conditions. More formally, the task is to search for patterns, associations or relations in the presenting set of circumstances, data or facts in accordance with the constraining conditions.
The second task is to understand the character of the sought after solution and establish solution characteristics as a goal. Formally, this task is to search for patterns of outcome that satisfy intent, also constrained by connotations.
The third task is to achieve some sense or concept of the route or path by which to proceed from the encountered condition of the problem to the desired solution. Formally, this task is to either search for patterns of induction that proceed from problem to solution or to search for patterns of abduction used to form hypotheses about solution which may be tested for validity.
The fourth task is to design the solution and plan its implementation. Formally, this task is to achieve a series of decisions on the results of the searches for presenting patterns, outcome patterns and intermediate patterns of either induction and/or abduction.
A simple scenario in interactive text editing may better serve as an example of the four-fold human task. The scenario is about checking the wording in a single paragraph of text for compliance with a technical concept and to make changes where appropriate. First, assuming the subject paragraph is visible on the display, the paragraph is read by the human. The act of reading the paragraph and acquiring a sense of its wording as to the desired direction is the first task. Still looking at the paragraph, and assuming a change is to be made, the person conceives of a preferred sense of wording in one sentence or part. This constitutes the second task. Continuing to examine the target sentence or phrase, the human conceives of a way to change current wording to the desired sense and some corresponding new wording. This is the third task. Lastly, the human decides what editing actions to take in order to accomplish the desired change. This is the fourth task. At this point in the scenario, the entire four-fold task is repeated - now focused at the more detailed level of use of the interactive text editor to accomplish the change. Namely, decide where to position the cursor for the first change, decide what change is to be made in that position, select the editing command necessary to accomplish the first change, and key stroke the command and attributes necessary to accomplish the desired wording in this sentence or part thereof.
The human uses internal beliefs and external knowledge to arrive at choices that span the area of discourse and partition the domain of alternatives into non-intersecting subsets so that a decision can be reached. The perceived patterns may be formally fuzzy and this may give rise to fuzzy subsets of alternatives across which decisions must be reached.
The term interaction intelligence is used to denote the processes and behaviors initiated and carried out by the machine.
The capacity for interaction intelligence is increased either by expanding the scope of any rules' applicability or by strengthening the entire rule system through the elimination or simplification of rules. This is more clearly seen by considering the formal notion of a theorem. The strength and generality of a theorem is increased by eliminating or simplifying conditions for its applicability. At one level of a computer system, this is equivalent to expanding the scope of any rule's applicability. At another level, the set of rules adhered to within a computer system is itself a theorem about logical behavior. Strengthening the rule system occurs by simplification of any rule or elimination of any rule such that the set of possible logical behaviors of the system is at least as rich a set after simplification and/or elimination as was the case formerly. In the machine's realm of interaction intelligence, the several-fold tasks are: (1) record and respond to communiques of the interaction; (2) signal when the mutually agreed connotative bounds may have been exceeded; (3) reveal and portray elements of machine-based knowledge; (4) maintain and apply rules used to access machine-based knowledge; (5) maintain and apply rules used to access procedures for revision of machine-based knowledge; (6) maintain and apply rules used to access and invoke machine-based evaluation of alternatives; and (7) maintain and apply rules to access and invoke machine-based induction or abduction processes.
Returning to the interactive text editor example, we see the following. Each time the human exercises an input mechanism, the machine act of acquiring the input content and moving to an appropriate state as a result is an example of the first task. Although any given input activity may be appropriate for some machine state, a given input activity or entity may be inappropriate in a given state. For example, issuing a second editing command call before filling in the required attributes of the pending editing command would invoke the second task of the machine. Exhibiting the menu of next possible editing commands and files that can be accessed for further editing illustrates the third task. Carrying out the access to a file or record for editing illustrates the fourth task. Carrying out a word substitution command illustrates the fifth task. Automatically changing the edited result into clean copy (hyphenated, left-justified, etc.) would illustrate the sixth task. If the interactive text editor were part of a text processing system that accommodates space planning for graphic art, physically identifying the shape of a piece of art and requesting a proposed space allocation in the vicinity of a particular text string would be an example of the seventh task.
The machine uses the rule system to carry out the tasks. The availability of abductive hypothesis formation within the machine would enlarge the range of allowable human expressiveness acceptable as input. The result would be more effective entrainment within machine systems for human intention.
Interaction intelligence and intelligent interaction each contain appropriate elements of an equilibrium principle. This equilibrium principle is vested in procedures that detect incorrectness and furnish instructions on what not to do. The net result of these procedures, in accordance with the prevailing principle, are several fold: (1) the interactive dialogue is kept on track with regard to intent; (2) the dialogue as a process has a self-organizing property; and (3) the necessary revisions within or between knowledge and belief systems are triggered, both in the human and in the machine.
The procedures for interactive equilibrium are generally oriented to both form and content. Lexical equilibrium insures that only mutually acceptable discourse tokens are used. Syntactic equilibrium insures that the structure of expressions is acceptable to both sender and receiver. Semantic equilibrium is maintained through establishing agreements as to both connotation and intention among meaning alternatives.
Another dimension to the interaction equilibrium procedures involves the condition of tentative equilibrium. An equilibrium condition is viewed as tentative due to involvement of either fuzzy or uncertain or unconfirmed elements of the knowledge base. These elements may be lexical discourse tokens, syntactic rules, semantic alternatives, connotative implications, or intention equivalencies. Skepticism or doubt occur in the presence of tentative equilibrium, are generally undesirable, and often unproductive for effective interaction. Refutation of the unconfirmed elements leads to separate internal revisions of each party's knowledge base. Any congruence that may have been established is disrupted and would have to be reestablished (re-entrained) on a new basis. Confirmation can often occur through definitional extension and/or extrapolation from the previously confirmed to the currently tentative. Interaction equilibrium procedures that are effective provide a powerful set of receiver replies equivalent to: say another way, or I didn't understand that, or explain, or elaborate and others. Similarly powerful sender interjections are furnished equivalent to: it's the same as ..., or it's like this, or it's analogous to ..., etc.
Sender messages that violate confirmed elements of the receivers knowledge base evoke a different category of receiver replies. One reply is equivalent to I cannot accept .... Another reply is equivalent to something of that form is not acceptable. And so on. It is the receiver's burden to: detect the presence of the misfit; decide as to the extent of unacceptability or tentative acceptance; and, notify the sender of the disruption of congruence (failure or limited entraining success) by the last message.
Caution should prevail in using such a philosophical view of interaction. Such a philosophy is not meant as a blueprint for design. Rather, it could better serve as a powerful guideline on methodological strategy and tactics. It may even serve as a guide to trade-off criteria.
Methodology of interaction and the design of interactive man-machine systems need to accommodate mundane weaknesses of both human users and human designers of machine systems. For example, human users can fail as dialogue participants and human designers can fail as anticipators of desirable ranges of users expressiveness. These two diverse illustrations follow:
In the first case, time sense leads to a human decision to disregard capabilities for effective and satisfying interaction. A human is troubled by the pressures of time. Behavior or constraints that are perceived to be subject to time pressure translate to self-invoked constraints on the depth or scope of search for relevant patterns. In effect, time pressure, in terms of immediacy, shrinks a users concern for both search scope and search depth. This leads to constrained consideration by the human of interpretations within connotation, intention and categories of representation. It, furthermore, leads to the human using a rigid system of categorization within the underlying systems of belief and knowledge. (There is no effort at clarification; acceptance or rejection become the only choices based upon sparse information.) Dependent upon the degree of time pressure, self-constraints are imposed in a stratified manner corresponding to the degree of stress or threat perceived versus that which may be tolerated.
In the second case, concern for potential ambiguity leads to a designer's decision to limit a machine's capability for supplying effective and satisfying interaction to the human. Designers of machines are troubled by ambiguity with regard to decidability as to the current rule that is to be invoked as a consequence of the last input by the human. This leads to designer constraints on what is acceptable from the human in the interactive dialogue as to valid contextual usages, classes of input types, and sets of symbols in the language mechanism. Furthermore, it leads to designs which are oriented towards a fixed set of expected outcomes for the machine in response to the human. The manner by which designer constraints are imposed in the machine domain is by over-specification of the machine mechanisms. The extent of over-specification is a function of the degree of expectation of the presence or potential for ambiguity for each type of option possible on the part of the human.
In both cases, design methods for interaction founded on an understanding of why and how interaction leads to possible solutions. In the case of the time harassed human, time pressure (an admitted non-issue technically) can philosophically be seen as an impediment to resolution of a tentative equilibrium condition. Continued application of this pressure leads to the likelihood that the human will erroneously not accept an otherwise valid expression from the machine and will react incorrectly. The solution is to allow to the human a receiver reply equivalent to: slow down for a while so that I can comprehend more fully .... The implications on total system design in this instance, although not very profound, are highly relevant to a system design well matched to user needs.
In the case of a worried designer, ambiguity can philosophically be seen as the concern for lack of user specified connotation and intent in a dialogue expression. The implication here for a design solution is far more profound. The system would need to detect the requirement for and request input of clarifying expressions for connotation and intent. The dialogue expressiveness allowed to the human must support such a response. And so on.
This paper has presented the position that methodology of interaction needs to be based on more than an understanding of machines, their structures and their mechanisms. An effective methodology of interaction needs to be founded on a clear understanding of the purposes of interaction and its logical nature. A sketch of a philosophical approach to such an understanding has been provided.
