Automated Testing for wooCommerce – intro

Testing Introduction

WooCommerce is one of the largest eCommerce platforms, with over 10+ million downloads to date, over 350 contributors and claims to power around 30% of all online stores.

In these sessions we are going to set up the basic test harnesses for wooCommerce and examine their practices and then extend these to test plugin custom functionality.

eCommerce tools provide a lot of different options and different ways of managing and configuring inventory, tax tables, shipping tables, coupon rules, product variations etc. When these are extended further with different plugin extensions the scenarios for testing become very extensive and costly to test manually.

If these aren’t configured right or simply don’t work in the way the user expects (both for the shop manager and the purchaser) then it’s going to affect sales and customer service so you have to test it.  It’s too costly to test manually so it needs to be automated.

Unit Testing Basic Setup

Unit Testing via PHPUnit exists for both WordPress and wooCommerce.

Start with:

• WordPress PHPUnit Introduction – you will need to start here and get the wordpress-develop repository, since the WooCommerce unit tests require components from here.
• WooCommerce Unit Tests  – the guide suggests you canSimply change to the plugin root directory and type: $ phpunit
  However to get these tests working there are a few additional points to consider.

Firstly consider your software installation versions:  if you are running MAMP or similar installation, be aware that the testing tools running from command-line tests might use different versions from the local website:  the MAMP web will pick up the MAMP version of PHP etc, whereas the command-line will pick up whichever version is configured at the OS level, which may be different unless you adjust your PATH to allow the MAMP version to take precedence.

Also if you don’t have wp-cli or phpunit, go and install them:

• wp-cli
• PHPUnit

And make sure you are aware of which versions you are running eg:

which php
php --version

which phpunit
phpunit --version

which mysql
mysql --version

which wp
wp --version

This is just to ensure you have the tools installed and are aware of the versions as they will impact on later error messages – later when we integrate a Travis CI build we will set it up to test against the different versions.

The examples in these articles use:

• PHP 7.0.15  (some fairly widely used components are not yet compatible with PHP7.1.x.  WooCommerce 3 should be nevertheless let’s take it out of the equation for initial testing)
• PHPUnit 6.1.2 (this did cause some issue, see below).

• WP-CLI 1.1.0

Next if you use the woocommerce/tests/bin/install.sh to install the tests, note that if the install fails, you may need to clean up (delete) the associated directories eg /tmp/wordpress/ and /tmp/wordpress-tests-lib as the script will not re-attempt download if the directories already exist.

Once install.sh has run successfully you also need to get the core test files:

svn co https://develop.svn.wordpress.org/trunk/ wordpress-develop

then copy over the /data and /includes directories from there to the wordpress-tests-lib directory that the WooCommerce test installer created (eg /private/tmp/wordpress-tests-lib/).  Unfortunately the WooCommerce test installer omits the data directory and installs an outdated version of the /includes directory that doesn’t included fixes for recent versions of phpunit.   (If you see errors like Fatal error: Class ‘PHPUnit_Framework_TestCase’ not found in /private/tmp/wordpress-tests-lib/includes/testcase.php that’s the PHPUnit version issue).

So, now the setup is done, navigate to the wp-content/plugins/woocommerce/ directory and run:

phpunit

This will then execute 816 tests and you’ll be presented with a summary in the command line. Code coverage documentation is automatically generated as HTML in the tmp/coverage directory.

The coverage files can be browsed directly from disk without going through a webserver, depending on your setup this will be like:

file:///Applications/MAMP/htdocs/woo3test/wp-content/plugins/woocommerce/tmp/coverage/index.html

or if mapped to a website as eg:

http://localhost:8888/woo3test/wp-content/plugins/woocommerce/tmp/coverage/

You can also run a single test file, for example:

phpunit tests_unit-tests_product_product-simple

run from the wp-content/plugins/woocommerce directory will run the phpunit tests in the file wp-content/plugins/woocommerce/tests/unit-tests/product/product-simple.php

End-to-End Testing

An End-to-End testing suite is now available which uses Mocha, Selenium and ChromeDriver to automate the tests via Chrome as a website user. To get started, see:

1. this woocommerce blog post
2. the End-to-end Testing documentation start page

There are a few points to note on the Getting Started instructions on the End to End testing page,

try checking the following points between points 4 and 5:

4. If you followed the wooCommmerce 3 setup wizard to create the clean install, rename the ‘basket’ page slug to ‘cart’ since the e2e tests are hardcoded to test /cart url.

5. If you installed from the plugin distribution, you need to add files from GitHub which are not included in the distribution such as the .json files and the /test directory from the plugin.   The easiest is to clone https://github.com/woocommerce/woocommerce/ and copy it to your wp-content/plugins/woocommerce then you know you have everything…

Then the npm install step helpfully installs selenium and the 480 other components necessary.  So now you are done you can run a sample test:

grunt e2e-test --file="checkout-page.js"

Cool to see Chrome running by itself and going through all the tests, but it takes a while.
And unfortunately frequently results in:

Error: Timeout of 30000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

What??…. wait these are quite long tests and it did take 30seconds, so what happened there?

If you look at the Gruntfile.js you can see how the e2e commands are defined:

e2e_test: {
 command: 'npm run --silent test:single tests/e2e-tests/' + grunt.option( 'file' )
 },
 e2e_tests: {
 command: 'npm run --silent test'
 }

We can then find the definition for npm in package.json:

 "scripts": {
 "test": "cross-env NODE_CONFIG_DIR='./tests/e2e-tests/config' BABEL_ENV=commonjs mocha \"tests/e2e-tests\" --compilers js:babel-register --recursive",
 "test:single": "cross-env NODE_CONFIG_DIR='./tests/e2e-tests/config' BABEL_ENV=commonjs mocha --compilers js:babel-register"
 },

and in the ./tests/e2e-tests/config directory default.json there is the entry:

 "startBrowserTimeoutMs": 30000,

So now the tests can successfully run to completion!

Except that the result is now:

 1) Checkout Page should displays cart items in order review:

     AssertionError: Could not find order item "Happy Ninja" with qty 1 and total $18.00

Original reason: [Error: Timed out waiting for element with xpath of '//tr[@class="cart_item" and .//td[contains(., "Happy Ninja") and contains(., "× 1")] and .//td[contains(., "$18.00")]]' to be present and displayed

So what happened?  It could be:

1. a difference in the theme which is not returning the html which is being tested for
2. an error in the test which is not constructed correctly
3. an error in the code, the app is not working properly

The test browser window is closed which makes it more difficult to determine exactly what the test saw at that point, however the data generated remains, and in this case the test made an order for “Happy Ninja” at $35.00 however was testing for a “Happy Ninja” at $18.00.   Well in the WooCommerce dummy-data.xml imported as part of the test setup, there are in fact two products called “Happy Ninja” with different prices….

So let’s look at the test setup:

guest.fromShopAddProductsToCart( 'Flying Ninja', 'Happy Ninja' );

The function fromShopAddProductsToCart can be found in woocommerce/node_modules/wc-e2e-page-objects/src/pages/shop-page.js and simply does:

 products.forEach( ( product ) => {
 shop.addProductToCart( product );
 } );

If we actually look at the addToCart which is eventually called in: woocommerce/node_modules/wc-e2e-page-objects/src/components/component-product-card.js

we can see that this is just selecting product on the screen by title, so since there are two with the same title, we can put this down to a problem with the test case.

So, should I be doing Unit Testing or End-to-End testing?

Easy to answer – both.

• End-to-end: mimics the user testing that you would do, makes it repeatable and really gives you the assurance that the whole process is working.
• Unit Testing: allows testing of specific functionality with a lot more variables, errors detected here are precise and can be rapidly debugged and retested.

 

Next time – extending the test suites to plugin which extends the existing functionality.