Automation Testing

By KaustubhIn ArticlesPosted

What is Automation Testing?

Automation testing is the application of tools and technology to testing software with the goal of reducing testing efforts, delivering capability faster and more affordably. Software is complicated and prone to regressions. That is, adding new features or fixing defects can cause existing features to stop working. That’s why it is so essential to have comprehensive automated testing because completely testing the application, manually, after each tiny change is extremely impractical.

Benefits of Automation Testing

Test Automation is a great way to increase the effectiveness, efficiency and coverage of software testing. It is most preferable for large projects that require testing the same areas over and over and projects that have already been through an initial manual testing process. Some of the benefits of automation testing are,

  1. Faster Feedback Cycle
  2. Saves Time
  3. Reduce Code
  4. Higher Test Coverage
  5. Reusability of Test Suite
  6. Better Insights
  7. Improved Accuracy
  8. Reliability
  9. Less Stress on QA Team

Automation Testing Approach

Test automation approach requires careful planning and design work. The following steps are followed in an automation process:

  1. Planning automation test strategy
  2. Selection of right automation testing framework based on project requirements
  3. Defining scope of automation
  4. Planning, Designing and Development, creating automated tests more resistant to UI changes
  5. Working on quality test data sets
  6. Test execution – early and often in CI/CD pattern and report generation
  7. Maintaining scripts

Automation Testing Framework : CodeceptJS vs Selenium

CodeceptJS

CodeceptJS is an open source, modern end to end testing framework with a special BDD-style syntax. The test is written as a linear scenario of a user’s action on a site. Each test is described inside a “Scenario” function with an “I” object passed into it through which helper methods can be accessed. Tests are written from a user’s perspective.

Selenium Webdriver

Selenium WebDriver is a collection of open source APIs which are used to automate the testing of a web application across browsers and platforms in multiple programming languages. Each test is described inside an “it” function. 

Comparison of CodeceptJS and Selenium Webdriver

FeatureSelenium WebdriverCodeceptJS
Language AgnosticYesYes
Cross BrowserYesYes
Cross PlatformYesYes
Community SupportYes – LargeYes – Growing
Parallel Test ExecutionYesYes
Test MaintenanceHighModerate – Low
Learning CurveHardEasy
Browser SupportMinimal All
Flexible DriverNoYes
Continuous TestingYesYes
Test Approach (Default)FunctionalUser
Browser HandlingExplicitBuilt In
HTML Reporting SupportNoYes

Sample Code for Google Search

With Selenium Webdriver

Libraries need to be explicitly imported to access helpers, every browser driver needs to be installed, built and launched separately, using the library methods, browser drivers need to be explicitly closed.

With CodeceptJS

Each test is described inside a “Scenario” function with an “I” object passed into it through which helper methods can be accessed. Tests are written from a user’s perspective. Handles opening and closing of browsers. 

Reporting with CodeceptJS

CLI

By default CodeceptJS provides cli reporter with console output. Test names and failures will be printed to screen along with time taken

Testmat.io

It is a modern  test management tool focused on CodeceptJS and created by the CodeceptJS team. Testomat.io is a commercial SaaS service that can receive run reports from local runs or CI. Out of box Testomat.io supports parallel runs, uploading of screenshots and videos.

Allure

Allure reporter is a tool to store and display test reports. It provides a web UI which contains all important information on test execution. CodeceptJS has built-in support for Allure reports. Inside reports you will have all steps, substeps and screenshots.

XML

Use default xunit reporter of Mocha to print xml reports.

Multi-report

Mocha-multi reporter helps you to generate multiple reports in the same time.

Testrail

Testrail integration with CodeceptJS is now so seamless. The test run is created automatically afterwards. The screenshots of failed tests are also attached to test results.

HTML

Detailed HTML reports could be produced with Mochawesome reporter. Report will have the following information 

  1. Total time taken to execute entire suite
  2. Number of modules executed
  3. Total number of tests executed
  4. Total number of Pass, Failed and Skipped tests
  5. Module wise reporting in similar manner
  6. Additionally for every test case
    1. Test case name with input data passed
    2. Time taken for execution
    3. Steps executed
    4. In case of failure,
      1. Stack Trace
      2. Screenshot where the test failed

Summary

We at OpenTurf prefer CodeceptJS for Front End Automation for,

  1. CodeceptJS is a powerful, yet easy to learn JavaScript based framework. 
  2. Test cases are written from a user (behavior) perspective.
  3. Since written from the user’s perspective in a synchronous way, it improves readability and maintainability of tests, even for non-tech resources.
  4. Multiple helper support – Webdriver, Puppeteer, Playwright, etc. , making underlying driver binding easy to change, if needed, with minimal code change.
  5. Built-in support for multiple detailed reporting formats.
  6. Comes with all the power that is required for automation testing of most web applications
  7. Maintained and supported by an active community of developers.
kkadmin

Kaustubh