Automation Testing with CI/CD

In this blog post, I have taken Playwright tests to integrate with CI process using Bitbucket and GitLab. A typical diagram of CI/CD from a QA stand point below

When we say Continuous Integration, It means an Automated process which easily manages frequent releases. It is a process which kicks off the pipeline by detecting, pulling, building the changed source code and automation testing on the fly for the changed source code of a product.

From the above diagram I have taken a typical monolithic project where automation testing is performed here on both Localhost/Dev environment and staging environment.

In some projects we directly target the Staging/Pre-Prod environment only as the developers perform unit testing on their DEV environments. The idea here is to create an automation test framework and create a repository for the test code. Run these tests on headless mode on your CI tool like Jenkins, GitLab, Bitbucket etc.. as a part of continuous integration.

When it comes to Continuous Delivery, It is a set of automatic processes which will get the source code changes and runs them through different operations like build, packaging and related operations to deploy the software automatically without any human interference.

You will need the help of your DevOps and SRE team to manage these complex pipelines and releases.

Coming back to our CI, I have taken Playwright tests to show the tests run on CI with Bitbucket and GitLab.

You can also run tests with Cypress, WebdriverIO or Test Cafe according to your choice.

The automation tests are usually environment agnostic meaning you have to just change the Env url or handle the url's with ENV Variables. The tests will be same across all the environments DEV, Staging and Prod.

Integrate Playwright Tests with Bitbucket

With Bitbucket, First you need to sign up and configure your SSH keys.

Make sure you add your ssh keys at the account level and not at the repository level. Adding the ssh keys at account level is enough for now to run the tests.

https://bitbucket.org/account/settings/ssh-keys/

I have pushed my code to Bitbucket using "Git" command line.

you have to add this file "bitbucket-pipelines.yml" to your VS code project folder

image: node:10.15.0

pipelines:
  default:
    - step:
        name: Playwright-Ui-Automation-Tests
        image: mcr.microsoft.com/playwright:focal
        script:
          - npm install
          - npx playwright install chrome
          - npm run test:chrome

The same will be seen in Bitbucket like this below

You need to install "Microsoft" or "Google" Authenticator on your phone for 2FA (2 Factor Authentication) to run the pipeline in Bitbucket for the very first time. Just scan the QR code and you are ready.

You have to manually click the "Run Pipeline" button for first time only. From the second runs whenever you push the code changes to Bitbucket automatically the pipeline runs and you can check the status under the "Pipelines" section.

For the first time click on "Pipelines" on the left side nav menu and click on the "Run pipeline" button on the top right.

From the next runs when you push the code from VS code, automatically you see the pipeline runs.

I have created a branch named as "ui-test" and pushed the code to this branch as a real time practice. Once you have the code on the branch you have to raise a "Pull request" on Bibucket to merge your test code with the master branch.

Pull request on Bitbucket is same as Merge request from GitLab.

Click on the first job for example ran on chrome headless "Run Chrome Headless on CI"

Note: The best part of Playwright is that you can run tests not only on Headless mode with Chrome but also you can run Headless mode with Safari and Firefox browsers which is cool isn't it :) !

Playwright on Headless Safari browser:

Playwright on Headless Firefox browser:

Integrate Playwright Tests with GitLab

With GitLab, First you need to Sign up and configure your SSH keys.

On your VS code add this file ".gitlab-ci.yml" to your project folder

stages:
  - test

tests:
  stage: test
  image: mcr.microsoft.com/playwright:focal
  script:
    - npm install
    - npx playwright install webkit
    - npm run test:headless:safari