The 4 layers of automation project
When we look at automation project we module it to 4 layers: SystemObject, Tests, Scenario manager and management.
System Under Test (or SUT)
The SUT is your testing environment. When planning an automation project one of the first stages is to define the SUT.
The SUT is not only the device/software you are testing, it's all the managed object (SystemObject) in your testing environment. It can include:
- The device/software you are testing.
- Stations or PCs that are part of your setup and should be managed.
- Generator of any kind.
- Any tools you are using to interact with the device/software you are testing (like database utils).
The first layer is the layer of what we call SystemObject. For every managed object there is a SystemObject. The SystemObjects are management layer that will manage your SUT.
What is expected from the SystemObject layer:
- Hide the complexity - this layer is a mediator, it expose to the layer of the tests a simple as possible interface that 'speaks' in the language of the QA engineer. All the issues of the connectivity (how we connect to the SUT and managed it) should be hidden from the person who write the tests.
- Reporting - Every system object operation (method) should leave a remark, by reporting it's action. In some cases it should collect information for analyzing.
- Error handling - When the step is failing an exception of some kind should be thrown. This exception should be catch by the framework of the tests (end not by the tests themselves).
- Analyzing results - The SystemObject layer should enable a unite mechanism to analyze results.
Tests (Layer 2)
When you have finished your SystemObjects the writing of tests should be simple. It usually can be done by a QA engineer with programming orientation.
The guidelines for writing tests are the following:
- Keep it simple as possible - Don't enter complicate logic into the test. Define a fixed SUT, don't try to write tests that will have to run on variety of SUTs.
- Keep it as short as possible - The preferable situation is that you have big number of small tests rather then small number of big tests. Use fixtures to configure your SUT. Share the fixtures between tests.
- Use reports and documentation - If the tests will well documented analyzing scenario execution results will be faster. It's critical if you plan that your project will scale to hundreds and thousands of tests.
- Look under the lamp - Don't start to automate the most complex part of your system. Start with the simple and stable parts.
Scenario manager (Layer 3)
The Scenario manager is an application that can be run from every desktop in your organization. It's used to:
- group tests to scenario,
- manage scenarios,
- set parameters to tests,
- execute scenarios,
- analyze scenario execution results,
- and publish the results to central reporting server.
Management (Layer 4)
This layer is a LMS (Lab Management System) application. It supports the management of multi setups automation lab.
Using this layer you will be able to:
- View an online status of your automation setups. (Not implemented yet).
- Schedule automation task to automation setup. (Not implemented yet).
- View historical reports. (Implemented).
The full AQUA solution include a centralize reports server. The report server is J2EE base application that include database. Using the report application you will be able to:
- access the scenario execution results from any place in the organization,
- view historical execution results,
- compare between scenarios executions,
- run variety of reports (things that managers like...).
For more information about the reports server contact info@aquasw.com.