Towards an Architecture Framework for AI Systems

AI systems may radically differ from other software intensive systems – How to address that in system architecture and engineering?

From the viewpoint of software and system engineering, AI is today mainly seen and developed as evolution of smart/intelligent features of existing systems (e.g. enterprise systems/applications,  robotics and bots) enabled by machine learning, increasing computing power and availability of data. On the other hand, R&D on AI and cognitive architectures has produced framework implementations for general intelligent agents (e.g. SOAR and ACT-R), which have been used in building various AI systems – still largely relying on handcrafting the knowledge, behavior and learning of the system.

The way towards wider/general purpose AI systems envisions very different kind of systems; Systems autonomously maintaining themselves, operating, learning and interacting over extended periods as part of society and culture, as presented in the vision of Software Social Organisms. Acknowledging that advanced AI systems may highly differ from the existing software intensive systems (in terms of applicable methodologies, underlying technologies, organization, development, training, operation, maintenance, governance and way of interaction) suggests that an architecture framework for AI systems would be beneficial for analysis and development these systems – both for industry and academia. In this post, we take a step towards such an architecture framework for AI systems and discuss the potential benefits it could provide for the R&D community.

In previous post the AI Cube Framework was introduced as a big picture and high abstraction level analysis tool for AI systems. In that somewhat multidimensional and complex framework the central concept of AI genome was also introduced. In this post, we describe the concept of AI genome in more detail deriving an initial outline of an architecture framework for AI systems (targeting towards conformance with ISO/IEC/IEEE 42010:2011). Specifically, the stakeholders and system life-cycle stages need to be shown and explained. Therefore, an alternative view on AI genome is presented in the Figure 1. below.


Figure 1. AI Genome; The main aspects of interests and stakeholders around evolution of AI systems.

The Figure 1 illustrates the AI genome in more detail, presenting the four main aspects of interest in evolution of AI systems and identifies related stakeholders of each aspect;

  1. AI Research and Development  – Continuously evolving research community of AI related sciences including researchers and developers making their results and assets available as baseline for building AI systems. These include e.g. knowledge in form of scientific publications, software and computing platforms, algorithms, data sets as well as processes for building and operating AI systems.
  2. AI Craft/Production – AI System stakeholders during its life-cycle; Professionals involved in building an AI System in various life-cycle phases (Concept, Design, Development, Deployment, Training, Operation & Learning, Maintenance & Versioning, Undeployment, Re-deployment, Deletion) and methodologies/processes that those professionals apply. We know much about agile and lean software development and reinforcement learning, but are these processes and methodologies applicable for building AI systems that interact in natural environment, where an individual bug or learning iteration might cause e.g. loss of lives or substantial economic losses?
    • There is room and need for more research in applicable processes and methodologies for AI system crafting/production, with safety and quality as guiding principles instead of speed and development cost.
    • Another major area, which we know very little of today, is governance or institutional control of AI systems – presenting an opportunity for more research.
  3. AI Systems – AI System collaborators (comparable to users of existing information, communication and automation technology systems), which are interacting with the AI system during its operational life-cycle phase to achieve a common goal or gain some value. Perhaps the closest example of this kind of stakeholder is driver of an “autonomous” car, who is collaborating with his car to get from place A to place B safely (truly autonomous car would not need a driver – only a collaborator who gives the destination where to go safely). The AI System – collaborator interface presents many topics to consider in development of AI systems:
    • Natural multi-modal interaction between the AI system and it’s collaborators (e.g. UX of conversational interface)
    • Expressing the capabilities of an AI system in a way that is understandable for collaborators (who are not technology experts but maybe e.g. healthcare professionals in their daily work)
    • Sharing of responsibility and control between provider/manufacturer and collaborators in situations where capabilities of AI systems are exceeded or the system fails. (For example, you can imagine great examples of this kind of situations when collaborating with “autonomic” cars – can I sleep and trust the car to handle every situation that we might encounter? At what point the car hands out the control to the driver – is it already too late at that point to avoid an accident?)
  4. Environment – The natural environment as a stakeholder, often considered under umbrella of environmental and social responsibility in different organizations. Regarding AI systems, the related requirements are related to energy, material, ethical and cultural impacts of the system. In addition, an AI System might have positive or negative value for the environment. At first this might seem irrelevant/distant, but think again:
    • Energy usage of a computer cluster matching human capability in some task vs. human energy usage. Is the energy produced by burning coal or by harnessing wind/solar/geothermal, and what is the related environmental footprint?
      • Perhaps it would be wise to measure also energy and environmental performance of AI Systems with human equivalent in addition to just measuring performance in a task. Metrics are not there yet but are not hard to come up with (e.g. grams of CO2/flop and grams of CO2/task). Perhaps the challenge for AI systems development is to match human performance in these metrics as well.
    • Materials used in building AI systems; availability, recycling, waste, alternative uses, toxicity, safety. Related especially to embodiment and hardware used for building AI systems.
    • Ethics when using human or other biological intelligence as a source for data and training of AI systems? Ethics of AI, especially in autonomous agents interacting in the natural environment (instead of closed game worlds or simulations)
    • Culture of humans may be impacted in many ways via introduction of new AI systems (e.g. convincing image/video/story generation by AI systems to match human performance vs. fake news?) It might be worthwhile to consider possible cultural impacts as well when designing new AI systems?

The Figure 1 also presents the main subsystems of an AI system;

  • Embodiment – The physical material structure of the AI system enabling implementation of other subsystems and bounding interfaces with the natural environment. Even though often embodied AI refers to robotic applications and instances of AI systems, all AI systems are embodied in one way or another (or they are fairy tale). AI System might be implemented as system of interconnected computers running software and algorithms across a network (Cloud based AI) or as a single autonomous robot with embedded sensors and actuators (Robotic AI). Also hybrid distributed embodiments are possible, especially in IoT and real-time sensitive AI applications, where the computing is preferred near the sensing and actuation point of natural environment, while part of the computing might also take place in separate computing cluster.  Also many new kinds of embodiments are likely to be seen in future AI systems as a result of energy efficiency challenges of current computing architectures and progress in R&D of materials and computing (e.g. quantum computing, neuromorphic computing and optical computing).
  • Perception and Actuation – “Sensorimotor” subsystem of the AI system including all mechanisms related to sensing and actuation of the system with the environment (via the embodiment subsystem). This may include functionalities for one or more sensing modalities (e.g. Vision (graphics), Audition (sound), Tactition vibration/motion) and Equilibrioception (balance)), as well as for actuation modalities (e.g. text generation, sound/speech generation and locomotion).
  • Memory – Subsystem including all different memory mechanisms of an AI System. This may include both short-term/working memory and long-term memory (possibly as further subsystems).  For example, the short-term/working memory may further be divided to sensory and context memory subsystems, and the long-term memory may further be divided to declarative, procedural, episodic and associative associative memory systems.
  • Behavior – Subsystem including all behavior related mechanisms of the AI System. These may include e.g. attention, goals, deliberation, decision making/action selection, reactions and reasoning mechanisms.
  • Cognition – Subsystem including all higher cognition related mechanisms accumulating the internal world model of the system, and possibly adapting the behavior of the AI System over time. These may include self- and context awareness, learning, meta-cognition, motivation, emotion and reward mechanisms.

The subsystem organization presented here does not propose any specific embodiment, perception, actuation, memory, behavior or cognition system organization, but has the purpose of being able to serve analysis and design of AI systems with various organizations in these subsystems. However, we refer to read more about recent development and proposals on standard model of the mind and review on cognitive architectures closely related to the three subsystems of memory, behavior and cognition.

Furthermore, Figure 1 illustrates the role of internal world model and internal data processing subsystems of an AI system;

  • Internal world model – The natural environment including human culture as the AI system comprehends it internally. All knowledge contained by the AI System as a result of training and learning while in operation. This includes for example all objects, concepts, other intelligent entities and rules/policies, which the AI system may use in operation throughout all the subsystems. The internal world model may be symbolic (e.g. ontology based) and explicitly accessible for human monitoring and governance, or it may be fully internal non symbolic and not accessible for humans (e.g. emergent architectures)
  • Internal data processing – The internal continuous data processing and analysis pipeline of the AI System defining the mechanism converting data flow from perception subsystem into information, knowledge, wisdom as modifications of the internal world model and behavior. For example, the data processing involved in using the sensors (data), to identify an approaching object classifying it as a car (information), identifying the potential danger of an approaching car (knowledge), comprehending the danger of the situation resulting to action of moving away from the estimated trajectory of the moving car (wisdom).

As illustrated in Figure 1, the external source of data for AI Systems is natural environment including the subculture/domain of collaborators of the system, which can be perceived via interactions with the environment. These interactions may include for example conversation with collaborators and perceiving recorded human culture in various formats (e.g. videos, images, text and audio).

The approach taken here for outlining an architecture framework for AI systems is a hybrid model driven approach combining agent-orientation and holistic two-dimensional system orientation, where system thinking is applied both to the external environment and to the internal organization of the AI system. The agent-orientation is applied by defining AI system as an autonomous physically bounded entity/organism interacting with its environment. The model driven approach is adopted to enable defining the AI System and the subsystems independently of the technological platform used for it’s implementation, based purely on stakeholder requirements towards the system (~Platform Independent Model – PIM). The approach enables also designing multiple different technology specific system architectures and implementations (~Platform Specific Model – PSM) from one PIM.

The main benefit of the architecture framework would be bringing forward, transparent and into consideration the internal world model, internal data processing, embodiment, perception, actuation, cognitive, behavior and memory mechanisms of an AI system already in the concept and design phase of the system life-cycle. This would also enable gathering and consideration of all stakeholder requirements towards these mechanisms and the overall system early in the development process. In current AI system implementations these mechanisms are often only partly considered, defined and transparent, but those would serve as exemplars for analysis within the framework outlined.

From the viewpoint of analyzing progress in AI research, technology and systems, the outline of architecture framework presented divides AI systems into 7 subsystems and identifies the basic life-cycle phases related to these. This enables finding, defining and mapping AI system performance metrics, challenges and leaderboards in modular fashion, corresponding to the subsystems and life-cycle phases identified. In other words, the architecture framework can be used to create corresponding metrics, challenges and leaderboard framework for evaluating and monitoring performance and progress in various AI subsystems – indicating the R&D progress on the field widely from the viewpoint of AI systems engineering. The architecture framework outlined also raises questions regarding need for new metrics, challenges and leaderboards on aspects such as:

  • How to measure the breadth/depth of knowledge acquired by an AI system (breadth/depth of internal world model)?
  • How to measure degree of autonomy of an AI system? (related: How to measure/estimate life-time costs of an AI system?)
  • How to measure adaptation/evolution of an AI system? (related: How to measure/estimate life-time costs of an AI system?)
  • How to measure modal capabilities of an AI system?
  • How to measure cross-modal understanding of an AI system? (to avoid parallel systems per modality)
  • How to measure energy efficiency, ethical compliance, cultural impact and material footprint of AI systems? (related: Environmental and social responsibility of organizations involved in using, building, providing and operating AI systems)

We believe that the approach and outlined architecture framework for AI Systems presented here is novel in a way that it combines multiple architectural approaches, used widely independently of each other today, towards an architectural approach and framework for improving analysis and design of highly complex AI systems. If you think differently, please let us know (with pointers to similar works or alternative views on the topic) – those are welcome and greatly appreciated. If you are interested to collaborate on our way forward on AI system architectures and Opentech AI in general – please do not hesitate to contact us!


Leave a Reply

%d bloggers like this: