EasyAccept – A Tool to Create Acceptance Tests in Java

User Manual

Version 0.7

March 12, 2005

Requirements

EasyAccept is a tool developed to help development teams create acceptance tests. Such tests are black box and aim to capture the functional requirements for a software system as expressed by a client. A client is typically a non-IT person with a need to be solved through a software system. EasyAccept is being developed with the following requirements in mind:

It should be easy for a client to write acceptance tests him or herself. Since knowing what to test needs some training (e.g. to test limit conditions, etc.), this task will usually be performed together with an experienced software person.
Even if a software person helps a client to write tests, the client must be able to thoroughly understand the written tests without any help (after learning a bit about test language syntax).
EasyAccept should be reasonably easy to retrofit into existing software, as long as business logic has been separated from user interface considerations in the software to be tested.
It should be easy to automate the execution of acceptance tests.
It should be easy to test error conditions and non-error conditions.

Basic Decisions

In order to satisfy the above requirements, the following decisions were taken:

Tests are written using a simple script language. A script can call business logic and check that it produces the proper output.
The script language is not object-oriented. It seems easier not to burden the client with the notion of objects, encapsulation, state, etc.

Internal Commands

COMMAD	DESCRIPTION
stringDelimiter	Built-in command which changes the string delimiter to the given delimiter. By default, the string delimiter is ".
expect	Built-in command which is used to check that a (business logic) command worked correctly.
expectError	Built-in command which is used to check error conditions of a (business logic) command.
equalFiles	Built-in command which compares two files.
stackTrace	Built-in command which is used to obtain a stack trace when debugging. This is useful when unexpected exceptions occur and one wishes a stack trace to see what is happening.
quit	Built-in command to quit EasyAccept

Examples

Executing business logic

A script is written in a text file. A command is written on a single line. A command is simply executed. For example:

createUser id=id1 name="John Doe" birthdate=1957/02/04

will simply call the createUser business method passing three parameters to it.

Checking business logic execution

Special built-in commands can be used to check that a (business logic) command worked correctly. For example:

createUser name="John Doe" birthdate=1957/02/04

expect "John Doe" getUserName id=id1

In the above two lines, the first line calls a business logic command. EasyAccept will accept that it has functioned correctly if it does not produce an error (an Exception, in programmer parlance). The next line also calls business logic (getUserName with parameter id1) but checks that it returned the string “John Doe”.

Checking error conditions

A special built-in command can be used to check that a (business logic) command produces an error (using an exception). For example:

expectError "Unacceptable date." createUser name="John Doe" birthdate=1957/02/20

In the above line, a business logic command is called (createUser). EasyAccept will accept that it has functioned correctly if it produces an error (an Exception, in programmer parlance) and if the Exception's error message is "Unacceptable date."

Checking voluminous output

When you want to use the expect built-in command but the string to be checked is large, it may be better to leave the string in a text file and have the business logic command produce output in another file. Then, the built-in command equalFiles can be used to check the command's output.

# this shows that John Doe exists
expect "John Doe" getUserName id=id1
produceReport id=id1 outputFile=rep.txt
equalFiles file1=expected-report.txt file2=rep.txt

In the above example, the command produceReport will produce a report concerning John Doe and the report will be left in file rep.txt. The next line checks that the rep.txt file is equal to the expected-report.txt file. This last file (the expected report) should be produced beforehand (by hand, for example) and should contain exactly the output desired for the produceReport command.

Debugging

When any command produces an unexpected exception and you would like to examine a stack trace of the situation that led to the exception, use the stacktrace command as shown below.

# the following command produces an exception
someCommand param=someValue

In this case, in order to see details of the exception that was produced, temporarily use the following command during debugging:

# the following command produces an exception
stackTrace someCommand param=someValue

The language

A script resides in a file. Each line consists of name=value pairs. The following are examples of acceptable name=value pairs:

name=value

name

name=

name=""

=value

In the second and third cases, the value is null; in the fourth case, the value is empty; in the fifth case, there is no name.

In a line, the first name-value pair represents a command to be executed (the name) and typically does not provide a value. The name of the command must match a method available in the business logic. Parameters are passed as given in the other name=value pairs. In reality, since Java does not provide parameter names through reflection, the order of the parameters in the test must match the order in the business logic. The names themselves are not used and serve only as documentation in the tests. Remember that the parameter order is important.

A line starting with # is a comment. The line continuation character is \. A \ character itself must be given as \\.

The default string delimiter is ". This may be changed (see below).

Built-in commands

EasyAccept has several built-in commands used to perform special testing actions. They are described below.

equalFiles

equalfiles OKfile fileToTest

This command is a test of file contents. It receives two files, and the test passes if the two files have identical contents. By convention, the first file contains the correct (expected) output and the second file contains the file to be tested. This command is typically used after a business logic command that has produced its output in a file. There are two ways of using equalfiles command:

1) With Relative path (relative to directory where EasyAccept is executed)

equalfiles ./src/easyaccept/script/test/script1.txt ./src/easyaccept/script/test/script1.txt

2) With AbsolutePath:

equalfiles c:/projetos/script1.txt c:/script2.txt

expect

expect expectedString businessLogicCommand paramName=paramValue ...

This command executes the businessLogicCommand passing the specified parameters. The businessLogicCommand must return a string which is compared with expectedString. The test passes if the strings are equal. No errors (exceptions) may occur.

expectError

expectError expectedErrorString businessLogicCommand paramName=paramValue ...

This command executes the businessLogicCommand passing the specified parameters. The businessLogicCommand must return an error (produce an exception, in Java) using an error string. This string is compared with expectedErrorString. The test passes if the strings are equal. If no exceptions are thrown, the test does not pass.

stackTrace

stackTrace <any other command, including built-in commands>

This command executes the command indicated and, if an exception is thrown, a full stack trace is printed. This is useful for debugging and will not normally be used permamently in scripts.

stringDelimiter delimiter

stringDelimiter=delimiter_character

This command changes the string delimiter to the given character. By default, the string delimiter is ".

quit

quit

This command ends EasyAccept execution.

Instructions for the Programmer

In order to expose the business logic of your program, you must write a façade. EasyAccept will instantiate the façade once and all public methods contained in the façade will be callable from a test script. Remember to separate the business logic from the user interface. Your façade should not print anything anywhere. It should communicate itself with the outside by accepting parameters, returning results or throwing exceptions. Parameters and return values cannot be objects.

In order to call EasyAccept to test a program, use the following syntax:

     java -classpath ... easyaccept.EasyAccept <FacadeClass> <scriptFile> [<scriptFile>] ...

where:

<FacadeClass> - corresponds to the full class name (ex: easyaccept.script.TestFacade) of facade of he system to be tested.
<scriptFile> - corresponds to the full name of a file or a directory. If the argument is a file, it can have any extension and should be composed by test commands written in EasyAccept script language. However, if it is not a EasyAccept script, EasyAccept will still try to execute each line of the file – resulting in many errors. If the argument is a directory, EasyAccept will look for all files in this directory and its subdirectories, include them in a queue and execute all test commands of each file found.

A EasyAccept exit code of 0 implies that all tests have passed.