Test Guidelines¶

Tests are an important part of the development work to ensure the functionality of the code. This wiki page covers the guideline regarding unit tests and states difficulties to define integration tests for new devices.

Unit Tests¶

Unit tests are used to test the correct behavior of functions. The Open eCard project uses for test creation the TestNG Framework which is inspired by JUnit and NUnit but provides new functionalities and is easier to use.

Test files have to be placed into the separate source tree src/test relative to the root directory of the project or module. The Java files should be placed into src/test/java while supporting files like images, XML files or whatever the test requires has to be located in src/test/resources. Most Java IDEs have integrated support for this structure and create it automatically but if you just use a editor without project support you should know this. In the java directory the same package structure as in the source tree should be applied.

There is no strict rule which methods have to be tested. Here we give a brief overview of thing which should be tested and which not.

• Do not test private methods.
• Do not test getters or setters except they contain a lot of logic for input validation or something similar.
• At least one unit test per non trivial public method.
• Methods with parameters should be tested with correct and incorrect input data to ensure a well defined error handling.
• Do not specify tests which require a specific infrastructure e.g. a working network connection or a special kind of smart cards attached to the test executing machine.
• The unit test should be as small as possible.
• The name of the test file should be as descriptive as possible, e.g. if you test a specific method than chose something like WrongInputInFooBarTest.java or if you test all method of an object use something like ObjectNameTest.java
• Every unit test should be documented so that everyone is able to understand whats the expected result.
• Every unit test should contain an Assert.assert... call but also the annotation with @Test(expectedExceptions=...) is legit in case of a test with wrong data.

Usage of TestNG¶

Here we provide a short introduction into the creation of unit tests with TestNG. For detailed information visit the TestNG Documntation.

Specifying a test works the following way:

package de.foo.bar

import org.testng.annotations.Test;

public class foo {

    @Test
    public void foobarTest() {

    }
}

So all you have to do is to annotate a public method with the @Test annotation which instructs the TestNG Framework to interpret the function as test. But currently the test is always evaluated to passed because no real rest method is called. So lets add some more detail to the test.

package de.foo.bar

import org.testng.annotations.Test;
import org.testng.Assert;

public class foo {

    @Test
    public void foobarTest() {
        BarObject bar = new BarObject();
        bar.open(true);
        Assert.assertNotNull(bar);
        Assert.assertTrue(bar.isOpen());
    }
}

The test creates now a BarObject and sets the bar to open. Now the assertNotNull() method checks that the bar is not NULL that's obviously true. Furthermore a check is performed which test whether the bar is open and the bar is so the conditions is also true and the test is passed. If a condition in an Assert statement is not fulfilled the test is evaluated to failed.

Another feature of TestNG is to evaluate methods which throw exception in certain cases.

package de.foo.bar

import org.testng.annotations.Test;

public class foo {

    public static class Beer {

        public boolean open(int power) throws BottleBrokenExcepion {

            if (power > 75) {
                throw new BottleBrokenException();
            } else if (power < 25) {
                return false;
            } else {
                return true;
            }
        } 

    }

    @Test(expectedExceptions = BottleBrokenException.class)
    public void beerTest() {
        Beer beer = new Beer();
        bee.open(80);
    }
}

In some case it might be necessary to create an object with a lot of methods if we would do that in the test method the method body would grow and grow. TestNG provides an annotation for methods to do such things before the test is launched.

package de.foo.bar

import org.testng.annotations.Test;
import org.testng.Assert;

public class foo {

    private Cocktail cocktail;

    @BeforeClass
    public void init() {
        cocktail.add(new Limejuice());
        cocktail.add(new Vodka());
        cocktail.add(new Beer());
        cocktail.add(new Wine());
        cocktail.shake();
    }

    @Test
    public void cocktailTest() {
        Assert.assertFalse(cocktail.tastsGood());
    }
}

Note: There are several other @Before* annotations available see the TestNG documentation for the exact meaning.

Integration Test¶

There are some aspects of the test development which can not be covered by a simple unit test because they require for instance a specific card attached to the test system or a special network configuration or the ability to interact with a certain communication endpoint. It is not possible to automate all of these things.

The developer team endeavors to keep the tests running on all systems. This means tests requiring e.g. a direct http connection are rejected because there is the possibility that a developer works behind a proxy so the test is probably not recognizing this and fails because a connection could not be established.

Test with cards or specific types of readers are also not recommend or wanted because you can not expect anyone to own a specific card or reader. But you probably may want to test your implemented behavior locally with your card or reader so what to do in such cases?
At first the details should be discussed with the responsible release manager and the testers. Besides of this you are free to write a unit test and disable the test per default with the annotation @Test(enabled=false) which means the test is not executed if someone invokes the tests.

Other tests have to be done totally by hand in case there are some specifications for the UI or the correct ResultMinors in case of a failed authentication with the German eID card. Such tests are described in the Quality Management.