|
Record / Replay tools
can be made to work ONCE but OFTEN FAIL in regression
testing operations!
Many industry
experts have written articles detailing the ROI measurements
for record replay tools. The basic analysis starts with the
investment in the tool and skills required to use the tools
and conclude that the initial investment is returned when a
test can be rerun WITHOUT modification some number of times.
Unfortunately we need to extensively test applications that are evolving based upon business needs.
We agree that
record replay tools are REALLY worth the investment because
our test authoring and update service allows your QA people to
overcome many of the problems that increase the labor costs of
updating automated tests!
The ScriptGen architecture ASSUMES that the application will be changing. The solution is to use a database to store all the test objects and use DB functionality to maintain the tests.
Record Replay tests
are very difficult to maintain because they are a monolithic
combination of text script and supporting files that cannot be
externally evaluated.
Sample project: Let's
say a project has 2,000 tests for a site containing 500 UI
objects. How would you address these the real world situations?
- One of the 500 objects is deleted in version 2 of the
site. How many tests are affected by the UI change?
- There is NO way to know if the tests are either documents like Excel or record replay
monoliths.
- In ScriptGen when an object is deleted from the
Object Tree the ScriptGen database is searched resulting in:
- Any step using the object is highlighted in EVERY test
script.
- The test status is immediately changed to "REWORK" so
it is easily identified.
- Every step in the project using the deleted object is commented out and the manual and automated test is regenerated.
- How do you fix a record replay test that is broken when a
UI object is deleted?
- Most importantly the person repairing the test MUST have
detailed technical skills using the automated test tool.
- First of all you run all the tests and then rerun the tests
that fail. If this version of the application deleted more
than 1 object you must identify which object caused the
failure and then decide how to use the test tool to fix the
failing part of the test.
- Using ScriptGen, the Subject Matter Expert with NO
knowledge of test tool operations, uses a GUI in a browser
to repair the test logic before execution.
- The POSITIVE results in QA ROI are immediately apparent!
Record Replay basics:
All Windows operating
systems communicate with the application via a message
format connecting any user action with either the keyboard or
the mouse to the associated application.
The operating system
messages identify the UI object, the user action taken, and any
data associated with this specific object class and action. The
format of the test tool instruction varies with the test tool
but the basic architecture is the same.
All Record / Replay
tools store information about the UI object in some form of an
Object Repository. There are 2 basic problems that must be
handled by the Object Repository.
- During replay the test tool must be able to identify the
exact UI object acted upon during the record operation.
- To do this the test tool stores enough physical attributes of the
UI object to assure unique identification during replay.
- The object name used in the script must make sense to the human readers of the test tool script.
- All object repositories associate a "logical name" and
"internal name" for each object.
- The logical name can be changed in the object repository to something that makes
sense for the test tool user. This modification means that any other recorded operations using this object will have the new logical name in the script.
- The internal name is not changed so that the object can
continue to be identified.
ScriptGen maintains
the logical name of objects and generates the correct names in
the manual test narratives and automated scripts. ScriptGen also provides
functionality to assist the maintenance activities for the
object repository in the automated test tools.
Unattended test
execution:
If the automated
test tool requires a person to run the test by starting the
test and observing the test result, the test may run correctly
but still waste QA resources and decrease ROI.
Test management
tools like HP Quality Center can very efficiently run
automated tests in unattended mode.
SpiralTest provides
a secure windows application, SGAgent, that consumes a
webservice linked to your QA project via a transaction table.
SGAgent polls the webservice and downloads any changes to your
test suite like additions, updates, or deletes. The changes
are made to the XML test files in a directory on your LAN or
pushed directly into a SpiralTest configured project on your
QC instance. Once a test update is pushed into QC it is
immediately available for unattended execution using standard
QC functionality.
ScriptGen
architecture is designed to handle many different test tools
like WinRunner, QTP, and others.
© Copyright
SpiralTest, Inc. 1999-2010, All Rights Reserved.
Free
website templates
|
| |
