Posts Tagged quality

Selenium Best Practices

Posted by mestachs in java on August 13, 2012

It’s a summary (and few extras) of test_design_considerations

Use PageObjects pattern
Be fluent with 
  - return this, varargs, generics, 
  - reuse your model and jodatime
Be robust and portable 
  - Prefered selector order : id > name > css > xpath 
  - Avoid Thread.sleep prefer Wait or FluentWait
  - Use relative URLs
  - Don’t rely on specific Driver implementation
  - Create your dataset
Know your new tool
  - Keep up to date (versions and usage pattern)
  - Troubleshooting 
      - jre 1.6
      - IE (zoom, Protected mode setting )
      - Firefox/firebug startpage
  - How to deal with UI components like... fileupload, datepicker, ajaxtables,...
  - Detect when selenium isn't the good tool for the job
  - Don't be afraid to hack around selenium

Use PageObjects pattern

Page Object is a Design Pattern which has become popular in test automation for enhancing test maintenance and reducing code duplication. A page object is an object-oriented class that serves as an interface to a page of your Application Under Test. The tests then use the methods of this page object class whenever they need to interact with that page of the UI. The benefit is that if the UI changes for the page, the tests themselves don’t need to change, only the code within the page object needs to change. Subsequently all changes to support that new UI are located in one place.

The Page Object Design Pattern provides the following advantages :

  • There is a clean separation between test code and page specific code such as locators (or their use if you’re using a UI map) and layout.
  • There is  a single repository for the services or operations offered by the page rather than having these services scattered through out the tests.

For example you have a LoginPage that you can reuse in all your integration test.
if the logon has now a new option or the layout changed a bit… adapt the LoginPage… that’s it !

More on page object : page-objects-in-selenium-2-, PageObjectFactory, simple example based on google search page

Be fluent !

With return this

It makes often your tests more readable if your “void” methods return “this”.


   public class CampaignCreatePage extends AbstractPage {
       ...
    public CampaignCreatePage withCampaignName(String campaignName) {
        name.sendKeys(campaignName);
        return this;
    }

So you can chain the calls like :

    @Test
    public void shouldntSaveWithoutAnyErrors() throws Exception {
        CampaignCreatePage createPage = openCampaignCreatePage();
        createPage.hasNumberOfErrors(4);
        createPage.withCampaignName("SELENIUM").save();
        createPage.hasNumberOfErrors(3);
        createPage.withAgent("firefox").save();
        createPage.hasNumberOfErrors(2);
        createPage.withAssignee("tom").save();
        createPage.hasNumberOfErrors(1);
    }

With varargs

You can also use varargs to make the api cleaner

    public CampaignCreatePage withCampaignTargetsIds(long... targets) {
        targetsIds.sendKeys(StringUtils.join(target,","));
        return this;
        }

In the test this give us something like

    @Test
    public void shouldntSaveTargetUnknownIds() throws Exception {
        CampaignCreatePage createPage = openCampaignCreatePage();
        createPage.withCampaignName("SELENIUM").withAgent("firefox").withAssignee("tom").withProduct(0).withCampaignTargetsIds(1, 2, 3, 4);
        createPage.save().hasNumberOfErrors(1);
    }

With generics

If you have a component that don’t have access to your pageObject. For example, a generic menu where plugins can contribute links to their own pages.
you can use this trick :

    public  T clickOnMenu(String path, Class pageClassToProxy) {
        clickOnMenu(path);
        return PageFactory.initElements(getDriver(), pageClassToProxy);
    }

 

    @Test
    public void testBrodocWithoutMemberOrProspectIdentified() {
        processLogon();
        MemberIdentificationPage page = openMemberIdentificationPage();
        page = page.getMenuElement().clickOnMenu(LuceneSearchPage.URL, LuceneSearchPage.class);
        page.hasInfoMessage();
        page.withNameFilter("mestachs").search().getResults().hasResults()
 }

By reusing your model and joda

For example the CampaignSearchPage can reuse the SearchCampaignParameter

...
     public CampaignSearchPage searchFor(SearchCampaignParameter parameter) {
        clearForm();
        selectRadio("campaignTargetType", parameter.getPersonType());
        sendString(campaignName, parameter.getCampaignName());
        sendDate(creationDateFrom, parameter.getCreationDateFrom());
        sendDate(creationDateTo, parameter.getCreationDateTo());
        sendDate(campaignDateFrom, parameter.getCampaignDateFrom());
        sendDate(campaignDateTo, parameter.getCampaignDateTo());
        findCampaigns.click();
    }
...

Be robust and portable

Preferred selector order : id > name > css > xpath

To locate an element we can use

  • the element’s ID
  • the element’s name attribute
  • an XPath statement
  • by a links text
  • document object model (DOM)

In practice, you will discover

  • id and name are often the easiest and sure way.
  • xpath are often brittle. for example you have 3 tables displayed but sometimes there are no data and the table isn’t rendered, your xpath locating the second table will not always return the intented one.
  • css are the way to go in conjunction of id and name !
  • locating links in an internationalized application is sometimes hard… try to use partial href.

various selectors articles : why-jquery-in-selenium-css-locators-is-the-way-to-go, css selector demystified, jquery selectors

Avoid Thread.sleep prefer Wait or FluentWait

Instead of sleep

    public void clickOnContactType() {
        getDriver().findElement(By.id("contacttypelink")).click();
        sleep(500);
    }

Prefer this fluentWait

    public void clickOnContactType() {
        getDriver().findElement(By.id("contacttypelink")).click();
        SeleniumUtil.fluentWait(By.name("handle"), getDriver());
    }

It’s more robust, deterministic, in case of element not found… the exception will be clearer.

Another alternative is

(new WebDriverWait(driver, 30)).until(new ExpectedCondition() {
                public Boolean apply(WebDriver d) {
                    return d.getTitle().toLowerCase().startsWith("java developer");
                }

More on this http://www.thoughtworks-studios.com/twist-agile-test-automation/2.3/help/how_do_i_handle_ajax_in_selenium2.html

Use relative URLs

Avoid hardcoding hosts

getDriver().get("http://nl.wikipedia.org/wiki/Wiki");
WikiMainPage page = PageFactory.initElements(getDriver(), WikiMainPage.class);
page.search("sdfsdf");

Prefer configuration and pass relative urls like

openPartialUrl("/");

That use an abstract method


    protected void openPartialUrl(String partialurl) {
        getDriver().get(getUrlPrefix() + partialurl + "?siteLanguage=" + this.settings.getLanguage());
    }
    private static String getPrefix() {
        return StringUtils.replace(System.getProperty("selenium.website.url", HTTP_PREFIX), "@localhost@", getCanonicalHostName());
    }
    private static String getCanonicalHostName() {
        try {
            return java.net.InetAddress.getLocalHost().getCanonicalHostName();
        } catch (Exception e) {
            return "127.0.0.1";
        }
    }

It is easy then to launch these tests against another server (integration,acceptance,… or jetty in integration tests)

Don’t rely on specific Driver implementation

Don’t assume that driver will be an instance of FireFoxDriver or InternetExplorerDriver. For example when running the integration tests on your continuous build (linux) you will receive a RemoteDriver.

It’s quite easy to create a small framework around selenium using :

  • LabelledParameterized : to be able to run with different browsers IE,FF,… or with different user language fr/nl/en
  • ScreenShotWatchMan : that takes screenshot on exception and logs the html sources. (see ./target/screenshot/*.png)

Create your dataset

Avoid something like assuming that the data is always there.

CampaignConsultPage consult = openCampaignConsultPage(2132103215L);
CampaignEditPage edit = consult.goToEditPage();
consult = edit.save();
consult.hasInfoMessage();

Create a new campaign and check consult

CampaignCreatePage createPage = openCampaignCreatePage();
createPage.withCampaignName("SELENIUM" + new DateTime()).withAgent("tom").withAssignee("tom").withProduct(0);
createPage.save().hasNumberOfErrors(0);
createPage.hasInfoMessage();
Long campaignId = createPage.getCampaignId();
CampaignConsultPage consult = openCampaignConsultPage(campaignId);
CampaignEditPage edit = consult.goToEditPage();
consult = edit.save();
consult.hasInfoMessage();

Know your new tool

Keep up to date

Keep your binaries and knowledge up to date !
This official blog contains announcements and also some a lot of links to possible application of selenium
StackOverflow forums are really good to learn from other mistakes. The official Sauce Labs Blog is also really interesting for their real life experiences.

Use jre 1.6

If while starting your integration tests you encounter :

java.lang.NoSuchFieldError: HOURS
at org.openqa.selenium.remote.internal.HttpClientFactory.(HttpClientFactory.java:44)

or

java.lang.NoSuchFieldError: java/util/concurrent/TimeUnit.HOURS

Run your test with a jre 1.6

How to close Firebug startpage

FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("extensions.firebug.currentVersion", "1.8.2");
driver = new FirefoxDriver(profile);

note that you may need to adapt the version.

IE not working

check

  • The browser zoom level must be set to 100% so that the native mouse events can be set to the correct coordinates.
  • On IE 7 or higher on Windows Vista or Windows 7, you must set the Protected Mode settings for each zone to be the same value. The value can be on or off, as long as it is the same for every zone. To set the Protected Mode settings, choose “Internet Options…” from the Tools menu, and click on the Security tab. For each zone, there will be a check box at the bottom of the tab labeled “Enable Protected Mode”.

Possible workaround for Protected Mode

DesiredCapabilities ieCapabilities = DesiredCapabilities.internetExplorer();
ieCapabilities.setCapability(InternetExplorerDriver.INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS, true);
ieCapabilities.setCapability("ensureCleanSession", true);
driver = new InternetExplorerDriver(ieCapabilities);</pre>

How to deal with ui elements like

File upload

Selenium Tips: Uploading Files in Remote WebDriver

Single and multi select

One option

public class AbstractPage {
...
   protected void selectRadio(String name, String code) {
    WebElement select = this.webDriver.findElement(By.xpath("//input[@value='" + code+ "'and @name='" + name + "' ]"));
    select.click();
   }
...

Maybe a better option is using org.openqa.selenium.support.ui.Select

    Select singleSelection = new Select(getDriver().findElement(By.id("single-selection")));
    singleSelection.selectByVisibleText("July");
    Select mulitpleSelection = new Select(getDriver().findElement(By.xpath("//select[@multiple='multiple' and @size=12]")));
    mulitpleSelection.deselectAll();
    // Select February, August and November using different functions.
    mulitpleSelection.selectByIndex(1); // February
    mulitpleSelection.selectByValue("Aug"); // Select ...
    mulitpleSelection.selectByVisibleText("November"); // Select November

DatePicker

public class CampaignSearchPage extends AbstractPage {
  ...
    sendDate(creationDateFrom, parameter.getCreationDateFrom());
  ...</pre>

sendDate is defined in AbstractPage

   protected void sendDate(WebElement e, DateMidnight dm) {
    if (dm != null) {
        sendString(e, dm.toString("dd/MM/YYYY"));
    }
   }

AjaxTableElement

If you use table elemenst with paging, sorting,… you can perhaps create a component AjaxTableElement
add in pageObject :

public class CampaignSearchPage extends AbstractPage {
          ....
         public AjaxTableElement getTableResult() {
        return new AjaxTableElement(getDriver(),"results");
    }
}

Then you can use the AjaxTableElement

page.searchFor(param);
page.getTableResult().hasResult();
page.getTableResult().clickOnResult(1);

Detect when selenium isn’t the good tool for the job

Selenium can’t deal with OS level dialogs so first avoid them as much as possible…
For eg don’t try to download a pdf through selenium and open acrobat reader.
You may be tempted to use tool like : autoit or sikuli (his java api) but you will loose portability and may be they will only offer you a false sense of quality.

Don’t be afraid to hack around selenium

A lot of people are using it but also trying new stuff like :
– collecting performance statistics.
– creating a full acceptance framework
– implementing a new webdriver

, ,

41 Comments

Mockito Best Practices

Posted by mestachs in java on July 9, 2012

You have picked up Mockito and you are interested in some good practices to avoid hangovers.

Don’t mix with Spring Integration Test

Mixing spring integration test and mockito can produce strange border effects like : singleton replaced by mocks,…
and will also lose the main benefit of mockito tests : speed !

Avoid abstract testcases

When something breaks it simply takes too long to diagnose.. An alternative is to create custom assertion and factory methods that can be reused.
-> prefer composition over inheritance.

Don’t mock your model

Prefer simply using your model

Campaign campaign = new Campaign();
campaign.setAssignee(office);
campaign.setType(PersonTypeCode.MEMBER);
campaign.setCreationUserLocation(office);
campaign.setOrganization("google");
campaign.calculateEndDate();

Over mocking it

Campaign mockCampaign = mock(Campaign.class);
when(mockCampaign.getAssignee()).thenReturn(office);
when(mockCampaign.getType()).thenReturn(PersonTypeCode.MEMBER);
when(mockCampaign.getCreationUserLocation()).thenReturn(office);
when(mockCampaign.getOrganization()).thenReturn("google");
when(mockCampaign.getCalculatedEndDate()).thenReturn(new DateMidnight());

Easier to read and you will may be add convenient contructor/factory methods to your production or test codebase… remember to put the code where it belongs.

Don’t abuse mocks

Mockito isn’t a Panacea. If all you have is a hammer, everything looks like a nail…

For example testing an xstream converter DescriptionTargetConverter

Don’t use mock, it requires more code… and you don’t really verify the output xml format (id,type,label)

    mockWriter = mock(HierarchicalStreamWriter.class);
    mockMarshalContext = mock(MarshallingContext.class);
    discriptionTargetConverter.marshal(mockTarget, mockWriter,mockMarshalContext);
    verify(mockWriter, times(3)).endNode();
    verify(mockWriter, times(3)).startNode(anyString());
    verify(mockWriter, times(3)).setValue(anyString());

It’s much more easy using xstream “as-is”

    comparaisonString= "<Target>  <id>3</id>  <type>" + type+ "</type>  <label>firstname lastname</label></Target>";
    XStream xstream = new XStream();
    xstream.registerConverter(discriptionTargetConverter);
    assertEquals(comparaisonString, StringUtils.replace(StringUtils.replace(xstream.toXML(target), "\n", ""), "\r", ""));

For more advanced xml assertion… check xmlunit

Don’t mock servlet api

The org.springframework.mock.web package contains a comprehensive set of Servlet API mock objects, targeted at usage with Spring’s Web MVC framework, which are useful for testing web contexts and controllers. These mock objects are generally more convenient to use than dynamic mock objects such as mockito.

Don’t replace asserts with verify

Taken from this article
The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked.
In contrast, mock objects make it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method’s purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug.

If your test code contains assert methods then you have a good test. See for example the pmd rule for junit.

If it doesn’t contain asserts, and instead contains a long list of verify() calls, then you’re relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, “My test method needs to do several things: x, y, and z.” The charter and responsibility of the method is no longer clear. This is a candidate for refactoring.

Finally “if it’s hard to…”

Junit and Mockito drive a fair amount of refactoring for me.

If it’s hard to read… it’s hard to test.
If it’s hard to test… it’s hard to use.
If it’s hard to test… you’re probably doing it wrong.
If it’s hard to test… take a look at the design and test it again !

Other mocking frameworks

Most of these rules should apply to other mocking frameworks like
Mockito,, Mockachino, , , JMockit , Unitils,…

, ,

6 Comments

Personal taste for test and quality toolbox

Posted by mestachs in java, jenkins, sonar on June 16, 2012

TL;DR

* Software Quality Axes
  - Focus on test
  - Quality of a good test
* Libraries to write maintainable tests
  - Junit, fest, mockito, openpojo, spring-mock
  - Build forge with jenkins and sonar
* IDE features and plugins
  - Eclipse : shortcuts, static imports, eclipse template for junit4, eclipse save actions
  - Plug-ins : more unit, Eclemma codecoverage, infinitest, sonar ide
* Coding style
  - Test as running specifications
  - Write testable code
  - Junit best practices
* Conclusion

Software Quality Axes

Sonar is defining software quality through these 7 axes. I will mainly focus on unit tests and their code coverage, maintainability and make them fun to write.

As Java developers we are really lucky,

  • there’s a plethora of libraries to write easily, maintainable tests and
  • we have also a great IDE with a lot of plug-ins and built-in feature to maximize our productivity !

Quality of a good test from Kent Beck :

  • isolated (unaffected by the presence, absence, or results of other tests)
  • automated
  • easy to write
  • easy to read :  expressive -> maintainable (fix or add new ones)
  • easy to run
  • unique : don’t overlap with others -> code coverage

I picked up some plugins and libraries to help on these areas.

Libraries to write maintainable tests

JUnit (easy to run)

JUnit3’s last release was in 2002. So it’s time to move to junit4 ! All the tooling around junit is available : ide, maven, jenkins, sonar,…
Don’t forget to use the ‘new’ features like

I know testNg do this better… but junit is so widespread in my development environment.

Fest Assert(easy to read)

Default junit assertion are not really expressive, custom errors messages are often required to make failure more understandable to the developer. Fest-Assert is a library that let you as a developer write assertion fluently like assertThat(frodo.getAge()).isEqualTo(33); or assertThat(fellowshipOfTheRing).hasSize(9);. More samples in this snippet. After reading this article you will understand why I prefer fest-assert over hamcrest. Note also that festAssert can be extended to support your model : find a sample for jodatime assertions

(easy to write)

Mockito is a mocking framework that tastes really good. It lets you write beautiful tests with clean & simple API. Mockito doesn’t give you hangover because the tests are very readable and they produce clean verification errors. Mockito will help you testing nominal use cases but also extreme situation like unexpected/rare/hard to reproduce Exception (like db connection issues, corrupted database,…).

org.springframework.mock.web (easy to write)

The org.springframework.mock.web package contains a comprehensive set of Servlet API mock objects, targeted at usage with Spring’s Web MVC framework, which are useful for testing web contexts and controllers. These mock objects are generally more convenient to use than dynamic mock objects such as mockito.

Openpojo (easy to write)

Trivializing POJO testing and Identity.

It’s perhaps no a great idea to test getters/setters. The main point with this library is in fact to let you focus on the non trivial code by covering the trivial one. You will be surprised how easy it is when you use openpojo.You can also easily add your own testers for example verifying equals/hashCode transitity, reflectivity, nullsafety,..

Build forge based on

(easy to run)
Getting fast feedback from your commit or the one from your coworker is what jenkins provides you.

Sonar will follow the evolution of your project taking a look at potential bugs, code duplications, uncovered code, fragile test, integration test,…

 IDE features and plugins

Eclipse shortcuts (easy to write)

To be a productive programmer, don’t use the mouse 😉 Eclipse offers already a lot of shortcuts.My favorites : ctrl-shift-T or ctrl-shift-R for a goto type or resources
If you don’t know them well, some articles here and here and a great plugin to learn them.

Static imports (easy to write)

Open Preferences (Window -> Preferences)
Browse to Java -> Editor -> Content Assist -> Favorites
Click 'New Type...'
enter org.junit.Assert.*
enter org.mockito.Mockito.*
enter org.mockito.Matchers.*
enter org.fest.assertions.Assertions.*

If you type mock and then press Ctrl + Space, the static method Mockito.mock is proposed. The static import is added.
see it in action here

Eclipse template for junit4 (easy to write)

These templates once imported will help you to produces junit4 test method via content assist.

enter test
Ctrl + Space and pick test4
your test method is generated just fill the name of the testcase

Eclipse save actions

Sonar is checking unused import, and the systematic use of braces. The good news is that you can configure eclipse to fix these 2 plagues automatically with save-actions

Under "Preferences": Java > Editor > Save Actions
Check "Additional actions"
Click "Configure…"
Go to the "Code Style" tab
Check "Use blocks in if/while/for/do statements" and configure to your preferences

plugin (easy to write and run)

MoreUnit assist you in writing more unit tests. With feature like easy navigation between code and unit test, skeleton generation,…

ctrl-j jump to test or code
ctrl-r run associated test

If you don’t really follow the same naming convention as proposed by the plugin you can modify the preference ‘enable extended search for test methods:’
If needed there’s a similar plugin : fast code

Eclemma codecoverage plugin (unique and isolated)

Detecting uncovered code allows you to easily identify missing unit tests.
Eclemma brings code coverage analysis directly into the Eclipse workbench:

  • Fast develop/test cycle:Launches from within the workbench like JUnit test runs can directly be analyzed for code coverage.
  • Rich coverage analysis:Coverage results are immediately summarized and highlighted in the Java source code editors.
  • Non-invasive: EclEmma does not require modifying your projects or performing any other setup.

(easy to run)

Infinitest is a never ending unit tests runner, displays assertions failure or errors as problem just like compilation errors ! To make infinitest your new best friend you need a fast test suites or filter out the slow ones.

Let’s see fest and infinitest in action : errors displayed in front of contains and the fest message shows the actual content of the fellowshipOfTheRing

plugin

Setting up checkstyle, findbugs, pmd in all your developers workspaces, keeping all the plugins and rules up to date is quite hard. This last plugin will cover all the other axes. A good habit is to launch it when you think you are done… you will perhaps discover that you are not done yet 😉

The Sonar Eclipse Plugin provides a comprehensive integration of Sonar in Eclipse for Java projects. The objective of this integration is that developers do not leave their favorite IDE anymore to manage the quality of their source code. Most information displayed in the Sonar Web interface is now available in Eclipse. You can run local analysis based on the ruleset configured in your central sonar server.

Coding style

Writing tests is always easier if your code is designed to be testable and your test shows the intent of the test.

writing unit test as running specifications :

A good naming convention make it clear what you expect from the MailInboxSyncer implementation.

Class MailInboxSyncerTest {
     @Test
     itShouldSyncMessagesFromPopSinceLastSync() {...}
     @Test
     itShouldCloseConnectionEvenWhenExceptionThrown() {...}
     @Test
     itShouldSyncMessagesOnlyWhenNotAlreadyInInbox() {...}
     @Test
     itShouldIgnoreMessagesOlderThenLastSync() {...}
     @Test
     itShouldIgnoreMessagesFailingFilter() {...}
     @Test
     itShouldDefaultToDefaultTimeWhenNeverSynced() {...}
}

Write testable code

In general, don’t follow any rules from this article on How To Write Unmaintainable Code and Ensure a job for life ;-).
Global state is undesirable (statics, singleton,…) see testability-explorer and then injectability is good : dataSource=new org.apache.commons.dbcp.BasicDataSource() vs setDataSource(Datasource datasource).

Code Design principles that could help having a code base more testable

Layered design
Code against interfaces
Put the code where it belongs
Prefer composition over inheritance.
Favor polymorphism over conditionals
Avoid static methods and the Singleton pattern.
Isolate dependencies.
Inject dependencies

Code practices that don’t help

Mixing of concerns (business logic, caching, transaction,...)
Mixing service objects with value objects
Kilometric classes/methods
A lot a dependency between your classes
Do complicated creation work in objects
Avoid interface usage, depends on concrete
Static methods/fields
Satefull singletons
A lot of global attributes
Make the scope of variable broader then required

Junit Best practices

donts

Don't write unit test without any assert
Don't log… but assert
Don't write complex condition logic : something you are not testing or something that is too complex that should be re-factored.
Avoid hard-coding dataset
Avoid over complex assert

do

Small and fast test
No side effect
A bug means a new unit test
Externalize your dataset if possible (xml,csv,...)
Avoid time dependency via jodatime

For spring integration test : @RunWith(SpringJUnit4ClassRunner.class) Assert result (not null, empty,…) Delta assertion : The pattern is like this: 1.measure 2.exercise 3.measure second times and 4.assert on deltas Guard Assertion : You assert some state before exercising the SUT. For ex an empty case for current dataset Create custom assertions if needed : When complex assertion are “copy-pasted” over the test code or when the assertion code “hides” the goals of the test you can “extract the code” in assertion method.

Conclusion

No excuse ! You have a now my personal toolbox to write more (maintainable) tests.
I’d like to thank all these developers contributing to a better quality, more productivity, gain in visibility via their tools and libraries !

Anything missing from this list ?
Do you have experience with bdd frameworks ?
Share your own testing pearls ?

, , , , , ,

7 Comments