Magento TAF is a pretty good thing to automize your Magento store testing. When you start working with Magento TAF, you have many tests, which are ready for use, so this framework can help to develop all kinds of tests without building an automatic infrastructure. Since a TAF users guide is a bit outdated, and some things in the framework have been already changed we’ve decided to write an article on this topic.
- At the base level, Magento TAF will require the following software:
- PHP: 5.2.0 or later
- PHPUnit: 3.5 or later
- Java SE Development Kit
- Apache
- MySQL Server
- Selenium RC 2.0.3
Furthermore, Magento TAF contains the following two configuration files:
1. config.yml – contains the settings of the Selenium Server.
It is located in:
../magento-afw-x.x.x/config/config.yml
Here is a default content:
## Set up browsers for running tests ##
// List of browsers which can be used
browsers:
googlechrome: &googlechrome
name: 'Google Chrome'
// Browser name
browser: '*googlechrome'
// Host where it is installed and will be run
host: 'localhost'
// Port on the Host where this browser will be available
port: 4444
timeout: 60
firefox: &firefox
name: 'Firefox'
browser: '*chrome'
host: 'localhost'
port: 4444
timeout: 60
iexplorer: &iexplorer
name: 'Internet Explorer'
browser: '*iexplore'
host: 'localhost'
port: 4444
timeout: 60
// Default browser for run tests
default: *firefox
## Framework configuration ##
framework:
// Base pathes to the fixture and testsuite
fixture_base_path: 'fixture'
testsuite_base_path: 'testsuite'
// work only if tests are run for single browser
shareSession: true
coverageScriptUrl: ''
// Capture screenshot on failure works only for browsers that support it: Firefox/Chrome, IE with snapsIE tool
captureScreenshotOnFailure: true
// Parameter sets up the saves HTML pages on failure
saveHtmlPageOnFailure: false
load_all_uimaps: true
load_all_data: true
cache:
frontend:
name: 'core'
options:
caching: false
cache_id_prefix: 'selenium_'
automatic_serialization: true
backend:
name: 'file'
options:
cache_dir: 'var/cache'
## Applications for running tests ##
//List of Applications which can be tested
applications:
//Settings of Magento application
// Name of Magento application
mage: &mage
fallbackOrderFixture: 'default'
fallbackOrderHelper: 'Core'
areas:
// General link, base UImap, login and password to the backend
admin:
url: 'http://www.localhost.com/magento/admin/'
base_page_uimap: 'log_in_to_admin'
uimap_path: 'admin'
login: 'admin_login'
password: 'admin_password'
// General link, base UImap, login and password to the frontend
frontend:
url: 'http://www.localhost.com/magento/'
base_page_uimap: 'home_page'
uimap_path: 'frontend'
login: ''
password: ''
// General link, base UImap, login and password to the paypal sandbox
paypal_sandbox: &paypalSandbox
url: 'https://www.sandbox.paypal.com/'
base_page_uimap: 'paypal_sandbox'
uimap_path: 'paypal_sandbox'
login: 'paypal_sandbox_login'
password: 'paypal_sandbox_password'
// General link, base UImap, login and password to the paypal developer
paypal_developer: &paypalDeveloper
url: 'https://developer.paypal.com/'
base_page_uimap: 'paypal_developer_home'
uimap_path: 'paypal_developer'
login: 'paypal_developer_login'
password: 'paypal_developer_password'
// Default Magento application which can be used
default: *mage
2. phpunit.xml – contains the list of the test-scripts (test cases) to be run, and it is located in:
../magento-afw-x.x.x/phpunit.xml.
Default content:
<phpunit backupGlobals="true"
backupStaticAttributes="false"
bootstrap="bootstrap.php"
cacheTokens="true"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
forceCoversAnnotation="false"
mapTestClassNameToCoveredClassName="false"
printerClass="PHPUnit_TextUI_ResultPrinter"
processIsolation="false"
stopOnError="false"
stopOnFailure="false"
stopOnIncomplete="false"
stopOnSkipped="false"
testSuiteLoaderClass="PHPUnit_Runner_StandardTestSuiteLoader"
strict="false"
verbose="true">
<testsuites>
<testsuite name="Test Suite">
<directory suffix=".php">tests/Customer/</directory>
</testsuite>
</testsuites>
<logging>
<log type="json" target="./var/logs/logfile.json"/>
<log type="tap" target="./var/logs/logfile.tap"/>
<log type="junit" target="./var/logs/logfile.xml" logIncompleteSkipped="false"/>
<log type="testdox-html" target="./var/logs/testdox.html"/>
<log type="testdox-text" target="./var/logs/testdox.txt"/>
</logging>
</phpunit>
Then, for executing all tests, you should change the test suite name to All Tests, choose directory with all tests, and leave empty test name in the phpunit.xml file, like on the following example:
<testsuites>
<testsuite name="All Tests">
<directory suffix=".php">testsuite</directory>
</testsuite>
</testsuites>
Moreover, for executing one or few test suites, you should change a test suite name to Test Suite, choose the directories with test suites and leave empty test name. If you want to execute all tests from the suite, it is necessary to enter the name of a single test from the chosen directory:
<testsuites>
<testsuite name="Test Suite">
<directory suffix="RegisterTest.php">testsuite/core/Mage/Customer/</directory>
<directory suffix=".php">testsuite/core/mage/Newsletter/</directory>
</testsuite>
</testsuites>
For executing a single test, you should change a test suite name to Single Test, enter full name of the test file and full path to this file:
<testsuites>
<testsuite name="Single Test">
<directory suffix="SimpleTest.php">testsuite/core/Mage/Product/Create/</directory>
</testsuite>
</testsuites>
In case of executing Magento TAF – follow these steps:
1. Run Selenium Server.
$ java -jar BaseDir/selenium-server-standalone-x.xx.x.jar
2. Execute runtest.bat (in Windows) through the command prompt for test running.
3. Execute runtest.sh for Linux/*NIX OS through the terminal for the running.
$ sh BaseDir/runtests.sh
After executing runtests.sh in the terminal, you will be able to see the current progress.
When all the tests have been executed, you’ll see a description of all failures, errors, incompletes, skips.
Legend:
- “.” – test PASSed
- “S” – test SKIPped
- “I” – test INCOMPLETE
- “F” – test FAILed
- “E” – error appears during test running
Configuration read from /Users/admin/taf/magento-afw/phpunit.xml
..FF.........................................F................. 63 / 241 ( 26%)
....................F.............F............................ 126 / 241 ( 52%)
..........F...............F.................................F.F 189 / 241 ( 78%)
..............................................FFFFFE
Time: 20.13 hours, Memory: 36.00Mb
There was 1 error:
1) Core_Mage_Product_DuplicateTest::duplicateConfigurable
OutOfRangeException: Cannot find page with mca "catalog_product/edit/attributes/NjQ1/set/4/type/configurable" in "admin" area
/Users/admin/taf/magento-afw/framework/Mage/Selenium/Helper/Uimap.php:301
/Users/admin/taf/magento-afw/framework/Mage/Selenium/TestCase.php:1697
/Users/admin/taf/magento-afw/framework/Mage/Selenium/TestCase.php:1733
/Users/admin/taf/magento-afw/framework/Mage/Selenium/TestCase.php:2876
/Users/admin/taf/magento-afw/framework/Mage/Selenium/TestCase.php:2857
/Users/admin/taf/magento-afw/tests/Product/Helper.php:429
/Users/admin/taf/magento-afw/tests/Product/DuplicateTest.php:306
--
There were 14 failures:
1) Core_Mage_Product_Create_BundleTest::allFieldsForDynamic
general_news_from: The stored value is not equal to specified: ('1/8/14' != '1/8/2014')
general_news_to: The stored value is not equal to specified: ('1/8/14' != '1/8/2014')
prices_special_price_from: The stored value is not equal to specified: ('1/8/14' != '1/8/2014')
prices_special_price_to: The stored value is not equal to specified: ('1/8/14' != '1/8/2014')
design_active_from: The stored value is not equal to specified: ('1/8/14' != '1/8/2014')
design_active_to: The stored value is not equal to specified: ('1/8/14' != '1/8/2014')
/Users/admin/taf/magento-afw/framework/Mage/Selenium/TestCase.php:1384
/Users/admin/taf/magento-afw/tests/Product/Helper.php:618
/Users/admin/taf/magento-afw/tests/Product/Create/BundleTest.php:123
That’s all for now. We hope this article will help you to start working with the test automation framework and you are welcome to share your experience and thoughts about it in the comments.