Search This Blog

Wednesday, August 11, 2021

Test base – test you most critical apps automatically against insider builds

image

Borrowed from Microsoft

 

Introduction

Test Base for Microsoft 365 is a validation service made for Software Vendors and System Integrators.

Why should that be interesting for you as a Configuration Manager or Intune admin?

Because most businesses does have critical applications that they like to test properly and to make sure the applications work on the next Windows feature level.

Test Base is a service in Azure that will help you test your applications against a subset of OS versions after your choice. That could be the insider of the next Windows 10 build or even Windows 11, to make sure you are all set for next version of Windows feature rollout.

In the end this will give you a good indication of how your app stack will behave, save you a lot of time and make you sleep better!


Important question: How much does it cost?

At least while in preview it is free. Customers will be charged once it go GA

clip_image002


Requirements

  • Azure Subscription


How to set it up

Browse to https://portal.azure.com/

Search for “test base” and click on “Test Base for Microsoft 365”

clip_image004


Press Create

clip_image006


Tick “I confirm I have read and acknowledged the terms of use above”

Choose your subscription

And choose “Create new” on the resource group

clip_image008


Give it a name and press ok

clip_image010


Choose Resource Group Location and create a Test Base Account Name

clip_image012


Validation passed and we can press Create

clip_image014


Test Base was deployed and ready to be used.

NOTE: Before you proceed from here you should already know what applications you would like this service to test.

Click “Go to resource”

clip_image016


Press “Upload new package” and let’s start adding details to the work we would like Test Base to go through.

clip_image018


I’ve used 7-zip in this example:

Add package name

Add package version

Choose the test type

NOTE: the Out-of-Box (OOB) test performs install, launch, closed and uninstall of your package. After the install, the service will launch and close the application 30 times before continuing to uninstall the application. Scripts will run for 80 minutes at the most.

clip_image020


OS update type I’ve selected Security updates and feature updates.

NOTE: The Security updates enables your package to be tested against Windows pre-release monthly security updates
The Feature updates enables your package to be tested against Windows pre-release bi-annual feature updates builds from the Windows Insider Program.

clip_image022


Mark the Windows builds that you like to test against

clip_image024


Select “Insider Beta Channel”

Select “Windows 10 21H1”

Press “Next”

clip_image026


Before we can proceed with this step, we need to create a ZIP file with a certain format.

TIP: There is no limited number of ZIP files you can uploade, but the size is limited to 2GB per ZIP.

clip_image028


Go to your application(s) you want Test Base to go through.

clip_image030


Inside the application folder we need certain binaries to be present.

First of all, we need the application. This is because the test base will install the product inside a virtual machine.

Second of all, we need a script that can open the applications executable for test base to open op the program.

Third of all, we need a script to close the process that the second script opens. This is because test base performs the “launch action” 30 times and then the “close action” 30 times.

And finally, we need a script to uninstall the application, to make sure the machine can be cleaned up.

clip_image032


All the scripts are very basic and does not contain error handling:

TIP: If your script exists with other than 0 the operation will fail, and the sequence stops.


Script Argument
Install.ps1 Start-Process -FilePath ".\7z1805-x64.msi" -ArgumentList "/QB ALLUSERS=1"
Open.ps1 Start-Process -FilePath "C:\Program Files\7-Zip\7zFM.exe"
Close.ps1 Stop-Process -Name "7zFM" -Force
Uninstall.ps1 Msiexec /x 7z1805-x64.msi /QB


Once the scripts have been made and saved inside the application folder - ZIP the folder.

clip_image034

clip_image036


Back to the instructions of test base

Browse for the newly created ZIP.

clip_image038


Open it

clip_image040


Uploaded successfully

Click Next

clip_image042


Insert all of the scripts that will do the jobs. If you need the virtual machine to be rebooted after the application install, tick the “Reboot after execution”

clip_image044


NOTE: The script path should be like this app/script, so when having the scripts in the root of your ZIP, then the name for the folder + the script e.g., 7-zip/install.ps1

clip_image046


Once you added all the information, click Next

clip_image048


We did not add “Functional test”, so nothing to do here

NOTE: Functional test is a more a custom test method for Software Vendors. If you need more than the Out of box test can deliver you can use this feature and describe your flow Functional testing on Test Base | Microsoft Docs


Press Review

clip_image050


Press Create

clip_image052


Now be patient. The test process will take some time. But once it completed you will be able to see the result on the “Test Summary” page

clip_image054

clip_image056


You get a lot of data that you can pull out for documentation purposes.

NOTE: you can download a video of the test process, but you cannot get access to the actual virtual machine.

clip_image058


Summary

This was Test base and how you set it up to test your most critical applications towards the newest Windows builds before security patches and feature updates apply to your production environment. I thrilled about the idea and makes me sleep better when Windows evergreen is doing its thing.

Happy testing!


Source: Test Base for M365 documentation | Microsoft Docs

No comments:

Post a Comment