Introducing Katalon Analytics for Test Automation Analysis & Quality Visualization

Test execution reports are not only one of the most important feature of any automation framework but a valuable deliverable to track dynamic changes in the project timeline. To add value to the stakeholders, automation teams usually rely on third-party libraries to generate visual reports & dashboard which are time-consuming and unreliable sometimes.

Unlike other automation tools, Katalon Studio provides a rich UI for viewing test execution reports as a trusted, built-in feature without any configuration of third parties. The native reports provide the “snapshot” information of each test execution but limited at analyzing data to help make smart decisions about the quality of the application under test.

Offering a solution for this problem, Katalon Studio is going far beyond the basic by introducing Katalon Analytics that brings users to the next level of test reporting with analytics and visualization on aggregated execution historical data.

What is Katalon Analytics?

Katalon Analytics is an intelligent analytics platform with visual and filtering tools to give you a better understanding of your test executions of multiple projects in real time. The Analytics translates test execution results into actionable data which empowers you with the knowledge to pivot your automation strategy, focus on the high-risk areas of your product, detect flakiness tests and to have a better control on the quality.

A product owner could utilize the test results to the fullest only when it is structured and categorized such that the information facilitates decision making and improve the quality of the product. For the purpose of effective analysis, we need to have all the test execution results in one central repository to form a single source of truth for the test. With Katalon Studio v5.0, the test results can be uploaded automatically to Katalon Analytics server to form a seamless solution to provide the benefits of having the full history at your fingertip to be analyzed and filtered in any way you need it.


How Katalon Analytics helps your Test Automation Process?

Flaky tests are noteworthy, sometimes your tests fail unexpectedly for various reasons which could be data synchronization, improper test data management or having the dependency on other tests. As your test suites and functionalities of application grow, flaky tests become harder to debug. There are cases that tests that run perfectly fine in series might get flaky when executed concurrently. Automation test is a vital component to detect new errors introduced as a result of new features development but if tests failure rates are high, time and effort spent in the analysis and validation process lead to unacceptable time loss which deceives the purpose of automated regression tests. Katalon Analytics helps you solve this problem by providing flaky analysis on your test executions with visualization to easily identify this particular problem.

Katalon Analytics also helps you to optimize your code and reduce the regression rate. Execution time mainly depends on how you have designed & configured your tests, Katalon Analytics helps you to identify and benchmark the slowest performing functions by providing an analysis to optimize your test executions.

Benefits of Katalon Analytics:

  • Katalon Analytics is free to use with intuitive charts & reports to interactively navigate through vast amounts of log data
  • Eliminates the need for manually generating reports, reducing the possibility of data
  • Using Katalon Analytics pre-built aggregations and filters, you can easily run a variety of analytics
  • Updating and Collaborating test results in Real Time
  • Easy Distribution, Dashboards can easily share dashboards and reports with anyone in the team
  • The calculated execution time of each test cases
  • Time Series Analytics, the Total execution history of the test case and many more features
  • Log Analytics can easily gain valuable insights about the tests executed provided with screenshots and videos

How to start with Katalon Analytics

No code, No installations, it is quite simple to synchronize your test results from Katalon Studio to Analytics server. Setting up your project settings with Katalon Studio v5.0 and you and your team members can view all of the project test executions by access Katalon Analytics using your registered Katalon account.

Whenever you execute your test suites, Katalon Studio automatically uploads the test results to the server along with logs, screenshots and execution videos. Katalon allows users to create multiple projects inside the Analytics dashboard and also from Katalon Studio. It also allows users to share the dashboard with the team members just by sending an invitation. Dashboard not only provides us charts but also gives detailed information of the entire test case history such as a number of times executed, the performance of the test case, console logs, status of the tests. Using the dashboard one can easily analyze the results from test suite level to test case level.

Katalon allows users to upload their test results to the dashboard in several ways. If the user does not want to sync the results automatically, then they can upload the results from the Reports section. By clicking ‘Upload‘ button under Katalon Analytics label, results will be uploaded to Katalon Analytics dashboard. The other way to upload the results is to execute a simple command by providing the project ID mapped in the dashboard. For more information, refer to the documentation.

You can give a try by downloading the latest version of Katalon v5.0.  Katalon Analytics in current in Beta stage and the team is working on additional quality intelligent reports that will be available in future releases. Please submit your feedback at the community forum to make the tool more helpful.

Advertisements

Broken Authentication and Session Management – part Ⅱ

Introduction
HTTP is a stateless protocol, hence web server does not maintain any track of user activity. To track user activity we generally use Sessions. There are various ways of session management where the server generates a session identifier (ID) initially and ensure that the same ID will be sent back by the browser along with each subsequent request. This helps us to maintain a record of user. Improper handling of these session variables could be a serious threat and allows attackers to gain access to the system. This article illustrates session fixation considering ASP.NET web application. For better understanding I have created a simple ASP.NET application. You can download the project from here. This project has two folders ‘SecureLoginFunc’ & ‘InsecureLogin‘ which contains login & logout mechanism of the application. You need to import the downloaded project to Visual Studio or create a virtual directory in IIS and add this project to it.

As you know a Session is used to track the user activity using a Cookie. In ASP.NET, server creates a cookie named as ‘ASP.NET_SessionId‘ on the client. This ‘ASP.NET_SessionId’ cookie value will be checked for every request to ensure the authenticity & Identity. ASP.NET has two ways of transmitting session IDs back and forth to the browser, either embedded in the URL or through a session cookie. You can easily spot the session ID when it’s embedded in the URL, for example ‘www.abc.com/(S(dacaanfdgasdfdadfghq))’. Anyway this is not recommended solution.

Continue reading

Broken Authentication and Session Management – part Ⅰ

According to OWASP, Broken Authentication and Session Management was defined as  ‘Application functions related to authentication and session management are often not implemented correctly, allowing attackers to compromise passwords, keys, or session tokens, or to exploit other implementation flaws to assume other users’ identities.’ In other words, an attacker can get unauthorized access of the user due to the flaw in the implementation. Before exploiting this vulnerability you need to know few concepts

  1. What is a Session and why do we need a Session
  2. What is a Cookie
  3. What is an Authentication

Continue reading

What makes Katalon Studio more powerful than any other Selenium based Frameworks?

As a test automation engineer, we should always consider a few cornerstones before employing a tool to automate our products. As you know the primary benefits of test automation are reusability, scalability, improving test coverage, time and effort saving. Quite many of us say, using Selenium integrated with other third party tools would achieve all the above cornerstones. But still, we have a few drawbacks. Let’s see how Katalon Studio turns to a magic wand solving test automation difficulties.

Challenge to Selenium/Appium based Frameworks

Let’s say we need to automate a modern application which has web UI, a mobile app and web services. Applying test automation for this application takes much time and effort for coding to automate test scenarios, publishing content from the web UI and verifying the same content in the database & from web services, the same process is required for the mobile app on Android and iOS platforms. Making this more challenging, open-source automation solutions like Selenium require various third party tools needed to be integrated and configured. Continue reading

Katalon Studio for Test Automation – Simple & Powerful

Recently I came to know a Tester/Developer friendly Test Automation tool Katalon Studio. I am really impressed with the provided features of Katalon Studio 4.5. I have seen all features which are required for Test Automation. I can say this is a comprehensive Test Automation Tool, a cluster of Selenium, Appium, QTP, SOAPUI for both Web & Mobile Apps testing.

Continue reading

SQL Injections – Part 2

As you already familiar with the subject SQL injections from the previous article part 1, we will quickly dive into exploitation with SQLi. Login to your bWAPP and select vulnerability SQL Injection (Login Form/Hero). As stated in previous post, we need to do some manual analysis to know the functionality and it’s implementation. Try to login with your some random text (test, test). Now let’s do some dynamic analysis by reviewing source code of the functionality.

Continue reading

SQL Injections – Part 1

Though there are many vulnerabilities, SQL injection (SQLi) has it’s own significance. This is the most prevalent and most dangerous of web application vulnerabilities. Having this SQLi vulnerability in the application, an attacker may cause severe damage such as bypassing logins, retrieving sensitive information, modifying, deleting data. Sometimes this costs life when it comes to Healthcare, Banking domains. Okay introduction apart, the objective of this article is to exploit and read some sensitive data from the database. If you don’t know what exactly is this SQLi then read my other article which may throw some light. I am splitting the subject into two parts, having everything in one might throw you out of interest.

Continue reading

Server side include (SSI) Injection

What is Server side include

Before knowing what exactly it is, I would ask you a simple question. Let’s assume that you need to develop an application of 100 pages with dynamic content. And each page must have a Header, Footer, Logo. What would be your answer? How much time does it take to add header and footer in all the pages?

Continue reading

HTML Injection – Stored

Compared with other types of HTML injections, this would be quite interesting. We can easily trick others with this injection. You can create duplicate login screen, you may inject the code to trick users to click on it. Basically, this HTML stored injection will be stored in the database and retrieved later as per the need.
Continue reading

HTML Injection Reflected – POST

From the previous article we came to know how to find and exploit HTML injection with HTTP verb ‘GET‘. Now we will inject with method ‘POST‘. Pass some values in first name and last name and click on ‘Go’. There’s no much difference in the exploits but notice the URL here, there are no parameters being passed in URL. Whereas in GET method we could see parameters with values in the URL. You can try the examples shown in article HTML Injection with GET.
Continue reading