Test Automation Report with Allure and TestNG

It is time to deliver a test report to all the stakeholders who are interested in the quality of the application. After the completion of tests, report should be generated. The test automation report should be as clear as possible so that to be understood by everyone. So this blog will help you to have a Test Automation Report with Allure and TestNG.

What is Allure Report?

• Allure Report tool is a flexible lightweight test report tool. It allows everyone participating in the development process to extract the maximum of useful information from everyday execution of tests. Allure Report provides a nice big picture of which features have been covered, where defects are clustered and also it’s help to create Test Automation Report with Allure and TestNG.

Prerequisites

• Java as the programming language
• Maven as the build tool
• Web Driver as the browser automation tool
• Allure & TestNG as the assertion framework
• Intellij as the IDE

Installation of Allure

1. Linux

npm install -g allure-commandline

2. Window

• To install Allure, download and install Scoop and then execute in the Powershell:

scoop install allure

• Also Scoop is capable of updating Allure distribution installations. To do so navigate to the Scoop installation directory and execute.

\bin\checkver.ps1 allure -u

Add maven dependency for Allure Report

    <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-testng</artifactId>
            <version>2.13.0</version>
        </dependency>

• We can use any version dependency for suitable code. Go to “https://mvnrepository.com/artifact/io.qameta.allure/allure-maven“ and select latest version dependency.

Allure Report using Annotations

• Epic
• Features
• Story
• Severity level
• Description
• Step
• Attachment

These annotations are used in the same way like we use TestNG annotations.

• Epic – Epic are useful as placeholders for large requirements for your project. In agile, tests can also be classified by Epic, stories and features.
  Example –
  @Epic(“E01”)
  @Story(“Story : Verify the login functionality with password”)
  @Feature(“Feature1 : PRO-64647”)

• Severity – we can define @Severity annotation with any of these values like BLOCKER, CRITICAL, NORMAL, MINOR, TRIVIAL. By looking at this, we can understand the Severity of test if Failed.
  Example –
  @Severity(SeverityLevel.BLOCKER)

• Attachment – An attachment is simply a method annotated with Attachment that returns either a String or byte[], which should be added to the report. We can have failure screenshot as attachment.

• Description – We can add detailed description for each test.
  Example –
  @Description(“Verify the null values in Text Field”).

• Step – We can annotate any method with any visibility modifier (public, private, protected).
  Example –
  @Step (“Verify the login form with username{0} and password{1}”).

@Severity(SeverityLevel.BLOCKER)	
@Test(priority=2)
@Description("Verify login with different Credentials")
@Epic("E01")
@Feature("Feature2: Login")
@Story("Story:PRO-45262")
@Step("Verify login with different password")
public void loginTest1() throws InterruptedException
{
	driver.findElement(By.linkText("Log in")).click();
driver.findElement(By.id("Email")).sendKeys("prajjawalkansal@gmail.com");
driver.findElement(By.id("Password")).sendKeys("Test@123");
driver.findElement(By.xpath("//button[@class='button-1 login-button']")).submit();
Thread.sleep(3000);
Assert.assertEquals(driver.getTitle(), "nopCommerce demo store1");

}

Capture Screenshots

• By capturing screenshots, testers can better identify what went wrong when the software acted erroneously during a test. Capture screenshots only when a test fails, since they consume a lot of memory.

@Attachment
public byte[] saveFailureScreenShot(WebDriver driver) {
		return ((TakesScreenshot)driver).getScreenshotAs(OutputType.BYTES);
	}

• Adding the Failed Screenshot in Allure Report.

@Override
public void onTestFailure(ITestResult iTestResult) {
		System.out.println("I am in onTestFailure method " + getTestMethodName(iTestResult) + " failed");
		WebDriver driver = BaseClass.getDriver();
		// Allure ScreenShot and SaveTestLog
		if (driver instanceof WebDriver) {
			System.out.println("Screenshot captured for test case:" + getTestMethodName(iTestResult));
			saveFailureScreenShot(driver);
		}		
	}

• Adding the text for failed test cases.

saveTextLog(getTestMethodName(iTestResult) + " failed test case");

How will we generate the Allure HTML Report ?

• Run the Framework.
• Refresh the project.
• Open the Terminal and go to project directory.
• Run the below command.

Allure Serve

Test Summary Report

Suites

Graphs

Behaviors

Packages

• 

Techhub Template Reference

https://github.com/knoldus/test-reporting-with-testng-allure

Reference

https://docs.qameta.io/allure/#:~:text=Allure%20Framework%20is%20a%20flexible,from%20everyday%20execution%20of%20tests.