##,. ####,. #######,. ######,. ##, #####,. #,.#,. ##,. ##, ##,. ##,. ##,. ##, ##,. #,. ##,.##,. ##,. ##,. ##,. ##,. ##, ##,. ##,. ##,. ##,. #######,. ##,. ##,. ##, ##, ########,. ##,. #, ##,. ##,. ##,. ##, .###, ##,. ##,. ##,.. #, ##,. ##,. ##,. ##, ##,.. ##, ##,. ##,. ######, #######,. #######, ##, ######, ## http://www.agedis.de #############.. ##.. +===================================================+ +======= AGEDIS Newsletter =======+ +======= June 2002 =======+ +===================================================+ The AGEDIS NEWSLETTER is E-mailed quarterly to subscribers worldwide to provide information on the AGEDIS EC project in particular and on software testing and software test generation in general. Permission to copy and/or re-distribute is granted, and secondary circulation is encouraged by recipients of the AGEDIS newsletter provided that the entire document/file is kept intact and this complete copyright notice appears with it in all copies. Information on how to subscribe or unsubscribe can be found at the end of this issue. (c) Copyright 2002 by imbus AG, Germany. For best viewing quality we recommend the use of non-proportional fonts, like Courier in your email displayer. ======================================================================== o AGEDIS — The Halfway Mark by A. Hartman o Design of the AGEDIS Compiler by Sergey Olvovsky o The AGEDIS Test Execution Environment Designed to Run Abstract Tests for Distributed Software by A. Hartman, A. Kirshin, and K. Nagin o About us: France Telecom R&D by Yves-Marie Quemener o AGEDIS newsletter Article Submission, Subscription Information ======================================================================== AGEDIS — The Halfway Mark by A. Hartman (IBM) The AGEDIS project has just reached an important milestone in its target of creating tools for automated test generation and execution. Our group has been working together for 18 months, and we are beginning to see the fruits of our labor. On Friday June 14, we demonstrated the prototype test model compiler, test generator, test execution engine, and test suite browser to the project review board. The review board consists of Prof. Antonia Bertolino of the University of Pisa, Alan Wills, Technical Director of Trireme International, and Vassilis Kopanas of the European Commission. Our demonstration was a convincing proof of concept. We began by creating a UML model of a distributed client server application annotated with test purposes. This model was then run through the compiler and then the test generator prototypes to provide a suite of abstract test cases. The test cases are abstract in the sense that they are written in the language of the model rather than the application under test. At this point, we created a set of test execution directives that bridged the gap between the model and the application under test. We then fed these directives and the abstract test suite to the AGEDIS test execution engine, which proceeded to stimulate and observe a distributed implementation of the application we had modelled. One client was situated in the review room while a second client and the server were located in a building across the street. The reviewers were impressed by the ease of use of the execution engine. In addition, they had favorable comments on other aspects of the project. They gave us valuable feedback for making the tools more useful for our customers, and we plan to focus on usability issues as we move from the prototype to the completed tool set in the next twelve months. The reviewers were also very pleased with our overall testing philosophy, and our commitment to integrating our tools with existing tools and practices in the industry. In addition, we were able to demonstrate convincingly that the AGEDIS tools are being developed to the highest standards of reliability, and that we are serious in our commitment to quality. We did this by using the AGEDIS test execution engine to do automated testing of itself. [see also on the AGEDIS website: http://www.agedis.de/documents/TestGUIJavaApplicationswithAGEDIS.pdf , the Ed.] ======================================================================== Design of the AGEDIS Compiler by Sergey Olvovsky (IBM) The AGEDIS compiler uses the XML representation of a model as its input. This XML representation is created by an AGEDIS user and is written in AML (Agedis Modelling Language). The model contains a set of class, object, and state diagrams. The compiler's output is a semantically equivalent model, written in IF (Intermediate Format) language. The AML and IF language specifications can be found on the AGEDIS website: www.agedis.de This short article presents the main requirements for the AGEDIS compiler design and the decisions that were made to satisfy these requirements. There were a number of challenges that had to be met by the compiler design. 1. The AML language specification was known to be incomplete and subject to future changes. 2. The XML format of the AML representation had not been finalized before work on the compiler began. 3. Ease of XML format change was required, since there is a need to support the many possible XML output forms produced by different AML modelling tools. 4. It was foreseen that we would need to perform many different operations on the internal representation of the AML models. These operations include different types of syntactical and semantical checks of models, such as: - Access to model data. - Output of different information (not only model translation, but also AML to IF mapping, model architecture, etc.). - Implementation of APIs required by future productivity tools. - And more. The following technical requirements for the design were derived from the nonfunctional requirements described above: 1. The XML reader should be a separate block that can be easily substituted by some other front-end parser. 2. The internal representation of AML models should be as separate as possible from the rest of the compiler code. This would enable us to localize changes to AML. 3. Operations code should be separated from the internal representation of the AML and IF models. This would enable us to add new operations with minimal changes to the existing code base. DESIGN OVERVIEW The compiler contains two main structures: the AML and IF model representations. The AML model representation is created by the XML front-end parser. The IF representation is created from the AML structure, by performing mapping operations. The AGEDIS compiler also performs additional operations, completes the building of the AML and IF internal representations, outputs the IF model, and so forth. Model representation elements are used to mirror the actual elements of the AML and IF models. These elements include classes, objects, associations, and states, in the AML model representation, and include processes, signal routes, and states in the IF model representation. DESIGN SOLUTIONS In response to the design challenges presented above, we implemented the following design solutions: 1. The IF and AML model representations have a well-defined initialization facade. The facade contains model building routines that can add model elements together with their context. For example, they can add states to state machines, or objects together with information on the classes that they instantiate. The building routines for the AML and IF model representations have no knowledge of how the data is represented internally. The building schema roughly follows the Builder design pattern - a separate block that creates the internal model by using the facade routines. The advantages to this solution are as follows: - The XML format can be easily changed. Such a change only requires changing the front end parser, which reads the XML file. There is no need to understand the internal representation of AML models - they are built by the facade methods. The facade methods hide the actual creation of new objects and the internal model classes (such as AmlClass/AmlObject/AmlState/etc.) and the ways in which these objects are attached to the model skeleton. - Alternative input parsers can easily be attached to support multiple input formats such as the XMI output of different tools. The building block containing the parser is separated from the AML model representation by a facade. Thus attaching a new parser is just a matter of adding a new compiler configuration. In this new configuration, the AML model will be stimulated through its initialization facade by the new builder. Writing a new builder is easy for the same reason - there is no need to understand the internal model representation. - Changes to the internal model representation are isolated from the input parser. The facade routines are updated to build the internal model in a new way, possibly creating new internal representation classes. At the same time, the parser need not change at all - it will continue using the same facade methods. This is especially important in the case of multiple input formats, since without such insulation, any change to the AML model representation would have caused a change in all of the parsers. - Changes to the mapping of AML to IF are localized and do not alter the IF model representation. 2. All the functionality of the compiler, such as output, mapping AML to IF, parsing of IF code inside AML models, etc. with the exception of the XML parsing, is handled by operations on AML and IF internal model representations. The logic of these operations is separated from the model representation. The internal model representation elements provide only basic handling routines (mainly getters and setters). A visitor pattern was applied to the models in order to enable the addition of new operations, without adding any new code to the elements handled by the operation. Each visitor is an operation, performed on the model representation. There are output visitors, semantic check visitors, etc. These visitors access and alter the model elements data through getters and setters. The logic of each operation is implemented inside the visitor code. Each structural element of the AML or IF model implements a method known as acceptVisitor(visitor). This method calls the visitor's handle routine, specific to the structural element (handleAssociation, handleSimpleState, etc.) with the element itself as a parameter. Thus, to add a new operation, one needs only to write the operation code inside a concrete new visitor, derived from the abstract visitor class and invoke the acceptVisitor method of either AML or IF model's internal representation. The visitor code should handle all the model elements, relevant to this operation, by providing handler methods for these elements (example - handleAmlClass(AmlClass) or handleAmlSimpleState(AmlSimpleState) - for AML classes and simple states respectively). The advantages to this solution are as follows: - New operations can be easily added. - The functionality of the operations is encapsulated within specific operations, thus simplifying maintenance. - The model representation structure is not changed when operations are added or modified. CONCLUSION The compiler design proved itself robust enough to accommodate all the changes and requirement updates to this point. The XML format has been changed many times during the prototype stage of the project. Internal AML model representation has been altered in a major way after the introduction of test directives. New and unforeseen operations on the AML model were implemented (such as tracing the triggering operation call for any return statement). All these changes were easily accommodated by the current design. ============================================================================== The AGEDIS Test Execution Environment Designed to Run Abstract Tests for Distributed Software by A. Hartman, A. Kirshin, and K. Nagin (IBM) < Email: {hartman,kirshin,nagin} @il.ibm.com > ABSTRACT This article addresses the problem of executing abstract tests on distributed software. Although our focus is on specification based behavior testing that exploits model based test generation, we provide an environment that supports both automatic and manual test generation. Tests derived from a specification are by nature abstract. The tester must manually convert the abstract tests into their concrete executable form. The AGEDIS test execution environment is designed to transform abstract tests and execute these tests efficiently to verify distributed software. 1. INTRODUCTION In recent years, software modeling has enjoyed tremendous popularity due to the widespread adoption of object-oriented models as an aid to software design. The use of software models for the generation of abstract tests has also been reported in both academic settings and in practical experiments. However, the adoption of a specification based modeling strategy for generating test suites has yet to gain industrial momentum. One of the impediments to its adoption is that most current test execution environments are not designed to run the generated tests. AGEDIS has created a testing architecture to validate the behavior of distributed programs. The distributed application under test is not restricted to any particular programming language or operating platform. An abstract test case describes test logic without concerning itself with platform or language issues. In order to run the test, the execution environment must deal with the messy details of concrete execution. Furthermore, the abstract tests designed to validate basic functional behavior may be reused and extended to verify additional software requirements such as concurrency. 2. SYSTEM UNDER TEST An SUT (System Under Test) may be perceived of as a set of objects. The abstract test describes the expected behavior, state, and output of an object in response to a sequence of stimuli from the tester. The concrete implementation of the SUT runs within different process address spaces; the processes reside within hosts interconnected by a network. 3. TEST EXECUTION ENVIRONMENT The AGEDIS test execution environment is an infrastructure that supports the testing of distributed software. Software testing environments are applications that assist testers in executing tests and collecting results against the application under test. Testing environments usually parallel the applications they test. They run under the same operating system and use the same programming languages to interact with the SUT. They also provide services to support interfaces and libraries for testing scripts. The cost of creating and supporting these testing environments is a significant part of the overall development cost. The cost and complexity of the testing environment increases when the software under test is distributed. Distributed software may run in different host environments, separated by a network. Different programming languages may be used to code the different components in the software under test. For example, a server may run under Linux and its clients run under Windows. The server and clients communicate over a TCPIP network. The server may be coded in Java and the clients in C++. The testing environment must provide testing solutions for all of these heterogeneous application environments and languages. The creation and maintenance of complex testing environments is a distraction for a test team. Their main focus is on validating the software under test. The primary goal of the AGEDIS test execution environment is to make software testing more efficient by providing an infrastructure to support these heterogeneous application environments and languages. Naturally, the infrastructure must scale down to support simpler single language non-distributed software testing. 4. ABSTRACT TEST EXECUTION An abstract test for behavior testing is a scenario that involves communication between the tester and the system under test. It specifies a sequence of tester stimuli to the SUT, observation of SUT responses, and the assignment of a verdict based on the observations. An abstract test is designed to exercise a particular execution sequence or verify compliance with a specific requirement. The word "abstract" refers to the fact that the test case is platform and programming language independent. AGEDIS has defined its own non-proprietary abstract test suite format. The test execution environment is responsible for the concretization of the abstract test. It must implement the test logic on a set of operating platforms using a set of different programming languages. If the test scenario occurs on multiple hosts, it is required to distribute the test stimuli and observations over a network. Abstract tests have the advantage that they express the testing logic without becoming bogged down in the details of how to execute the test. Executable tests are written in the programming language of the system under test or in a test scripting language. Thus, the abstract test must be translated in order to execute it on the SUT. The test execution engine bridges the gap between the abstract test and the SUT by mapping the abstract stimuli to method invocations, mouse clicks, or system commands, and the abstract observations to actual value checking. The test execution environment executes the translated test directly and checks the observed results against the expected results as predicted by the model or as described in the abstract test. The abstract test is a required input to the test execution environment, telling it what to execute against the SUT. Test execution directives are another input to the environment, and provide information on how to execute the abstract test. These directives include information about target platforms, programming languages, actual input data, data types, and other mappings needed to make the abstract test concrete. 5. TEST EXECUTION ENVIRONMENT DESIGN Our test execution environment design supports the basic functionality for testing distributed systems: starting processes, creating SUT objects on remote hosts, controlling and observing them, and logging a trace of the test execution. The test driver manages the test execution and communicates with the SUT via the host managers and process controllers. The host managers are replicated on each target host. The processor controllers interact directly with the SUT objects. We describe the components in greater detail below. The test driver and SUT may run on the same host or on different hosts, and they do not need to be coded in the same programming language. The driver communicates with host managers and process controllers in order to accomplish the following tasks: · Create, destroy, and locate process controllers (via host manager). · Create and destroy SUT objects (via process controllers). · Control input to the SUT objects (via process controllers). Control means invoking procedures and functions, and sending input messages. · Observe output and state of the SUT objects (via process controllers). Observation means querying an object's state, recording function responses, publishing asynchronous messages, and getting an output. Only the test driver component understands the syntax and semantics of the abstract test cases. The other components handle real world interactions. This separation of concerns makes the test execution environment design well-suited for the testing of distributed software. The test driver is, by far, the most complicated component in the test execution environment. It is only implemented once since it does not have SUT platform or language dependencies. On the other hand, the host manager and process controller designs are simpler, but their implementation is dependent on the target SUT. Host managers must be implemented in all target SUT platforms. A process controller runs in the same process as the SUT object, and it simplifies interfacing with the SUT when the process controller and SUT are in the same programming language. Thus, the process controller must be implemented in all target SUT platforms and programming languages. (Next month we will provide more details about the AGEDIS Test Execution environment.) ============================================================================== About us: France Telecom R&D by Yves-Marie Quemener (France Telecom) < Email: yvesmarie.quemener@rd.francetelecom.fr > France Telecom R&D is the Research and Development center for the France Telecom group. Our motto is to be "Far ahead and close to you" in relation to France Telecom and its subsidiaries including Orange, Wanadoo, and Equant. As such, we focus on supporting the development of France Telecom towards mobile telephony, internet, data transfer, and international activities. France Telecom R&D is one of the biggest telecommunications Research & Development centers in the world with 3600 people (including 3000 researchers and engineers), 4067 French or foreign patents, and 15 start-ups created in 30 months on the initiative of France Telecom R&D engineers (as of 31/10/2000). We are organised into seven R&D departments: - multimedia services on brodcast networks, internet and intranet. - mobile services and radio systems. - advanced data and voice services. - architecture, integration and network command. - transport and acces networks. - human interaction. - software techniques. The first five deal with technical fields immediately useful to our clients inside the France Telecom group. The last two deal with inter-disciplinary techniques which should be useful for all the other R&D departments. The people leading the France Telecom participation in AGEDIS, are part of the software techniques department, but they will be joined during last year of the project by people from the architecture, integration and network command department and/or by people from the entity responsible for developing the France Telecom IS, who will use the AGEDIS tools for their own testing needs. Members of the software techniques department have experience in the automatic generation of conformance test suites for telecommunication protocols, following the ISO 9646 standard. Some of this experience has been incorporated inside the test generation tool "Test Composer" developed by Verilog (now Telelogic), in collaboration with people from Irisa and Verimag. We are pursuing this line of work in the AGEDIS project, trying to enlarge the number of users of automatic testing techniques (whether generation or execution) inside the France Telecom group. ======================================================================== ----->>> AGEDIS newsletter ARTICLE SUBMISSION POLICY <<<------ ======================================================================== AGEDIS newsletter is E-mailed quarterly to subscribers worldwide. Submission policy of AGEDIS newsletter is: o Length of submitted articles should not exceed 350 lines (about four pages). Longer articles are OK but may be serialized. o Length of submitted event announcements should not exceed 60 lines. o Publication of submitted items is determined by the AGEDIS consortium, and may be edited for style and content as is seen necessary. DISCLAIMER: Articles and items appearing in AGEDIS newsletter represent the opinions of their authors or submitters; AGEDIS newsletter disclaims any responsibility for their content. ======================================================================== --->>> AGEDIS newsletter SUBSCRIPTION INFORMATION <<<--- ======================================================================== To SUBSCRIBE to AGEDIS newsletter visit the AGEDIS web site to UNSUBSCRIBE and CHANGE your address send email to Johannes Trost . Please, when using either method to subscribe or unsubscribe, type the correctly and completely. Requests to unsubscribe that do not match an email address on the subscriber list are ignored. imbus AG Kleinseebacher Strasse 9 D-91069 Moehrendorf Germany ## End ##