LEARNING CENTER > INTEGRATION
TFS Integration

Automatic creation of bugs in TFS

An important part of running automated test cases is to monitor, inspect and react on the results. This is best done in the test management and bug tracking system that you use in your daily work.

In this article we describe 3 different scenarios of how to trigger and create/update bugs in Team Foundation Server (TFS) based on the result of test cases running in Leaptest.

Using a plugin for VSTS or TFS

One of the easiest ways to integrate Leaptest with Visual Studio Team Services or Team Foundation Server 2015 and later, is to use the following plugin, which is community developed:

Please refer to the documentation on the GitHub page for installation instructions.

Creating bugs from Schedules

To run a collection of test cases on one or more environments (machines, devices etc.) on a scheduled basis or by triggering it through the Leaptest REST API, you define a Schedule. In a Schedule you can select one or more test cases from a Leaptest project and pair them with one or more environments. Further you set at what time and how often the schedule should run.

As part of a Schedule you can define one or more Actions. An example of an action is to send an e-mail, when the schedule has ended. Another is to call a specified web service if more than X number of test cases in the schedule has failed.

In this case we will trigger a Powershell action when the schedule is done. We will use this action to call a local Powershell script and pass the result of the schedule to the Powershell script. An example of how to specify this action is shown in the figure below:

tfs_action

The command tells Leaptest to run a Powershell script: “c:\scripts\createTFSbugs.ps1”. Download the script (zip) by clicking here: CreateTFSbugs and extract it to the location where you want to execute the script from.

Further we have added a few parameters:

  • -result: This parameter is read by the Powershell script and is the result of the entire schedule. To add the result data right click in the command field and select “Insert token -> RESULTS-JSON”.It is VERY important that the inserted token is surrounded by
    @”
    [

    before the token, and
    ]
    “@
    after the token- including the line breaks. This structure ensures that powershell transfers the JSON result in the right format.
  • -rootPath: This parameter is read by the Powershell script and is the file path to the folder containing the videos and screenshots recorded during the execution of the test cases.

The Powershell script, “CreateTFSbugs.ps1” will connect to the TFS REST API and create or update bugs based on the failed test cases in the Schedule.

In order to use the Powershell script you need to specify a few settings about you TFS installation:

tfs_ps_settings

domain: If you are using Visual Studio Online (VSO), insert your visualstudio.com domain name, e.g. “https://leaptest.visualstudio.com”

tfsPath: The path to your project.

pat: Personal Access Token. You can find more information about setting up a Personal Access Token here.

logfile: The path to the integration log file. All log entries are appended to this file.

The script has been tested against Visual Studio Online, but it’s the same API used for the OnPremise installations of TFS. There might be differences in the version of the REST API. Please consult the MS documentation for further info about this: https://www.visualstudio.com/en-us/docs/integrate/api/overview

Triggering schedule from API and create bugs in TFS

Using four simple steps to integrate Leaptest with any third-party system through the REST API, automatically creating or updating bugs in TFS when test automation cases are failed in Leaptest is very easy.

The following is a Windows Powershell script that runs a pre-defined Leaptest schedule, polls for the results until they are available and then loops through all failed cases and creates or updates bugs in Visual Studio Online (Microsoft’s cloud TFS solution) as appropriate.

Please note that the script contains no error handling or logging mechanisms. It is meant only to demonstrate the core functionality of integrating Leaptest with TFS.

After running, cases will be created in TFS and can be managed for instance through Visual Studio:

tfs

The above script uses Personal Access Tokens (PATs) to authorize access to TFS (Visual Studio Online). For information about how to setup PATs and how to perform TFS REST calls, please see the following links:

  1. https://www.visualstudio.com/en-us/docs/setup-admin/team-services/use-personal-access-tokens-to-authenticate
  2. https://www.visualstudio.com/en-us/docs/integrate/get-started/rest/basics

You can explore the Leaptest endpoints mentioned above by going to this url: http://localhost:9000/help/index#!/RunSchedules if you have the Controller installed on your own computer.

If you would like to use the REST API to integrate Leaptest with your own systems, feel free to contact prioritysupport@leaptest.comand we’ll do our best to help you get up and running in no time.