Test base – test you most critical apps automatically against insider builds

Test base – test you most critical apps automatically against insider builds

image

Borrowed from Microsoft

 

Introduction

Test Base for Microsoft 365 is a validation service made for Software Vendors and System Integrators.

Why should that be interesting for you as a Configuration Manager or Intune admin?

Because most businesses does have critical applications that they like to test properly and to make sure the applications work on the next Windows feature level.

Test Base is a service in Azure that will help you test your applications against a subset of OS versions after your choice. That could be the insider of the next Windows 10 build or even Windows 11, to make sure you are all set for next version of Windows feature rollout.

In the end this will give you a good indication of how your app stack will behave, save you a lot of time and make you sleep better!

Important question: How much does it cost?

At least while in preview it is free. Customers will be charged once it go GA

clip_image002


Requirements

  • Azure Subscription

How to set it up

Browse to https://portal.azure.com/

Search for “test base” and click on “Test Base for Microsoft 365”

clip_image004

Press Create

clip_image006

Tick “I confirm I have read and acknowledged the terms of use above”

Choose your subscription

And choose “Create new” on the resource group

clip_image008

Give it a name and press ok

clip_image010

Choose Resource Group Location and create a Test Base Account Name

clip_image012

Validation passed and we can press Create

clip_image014

Test Base was deployed and ready to be used.

NOTE: Before you proceed from here you should already know what applications you would like this service to test.

Click “Go to resource”

clip_image016

Press “Upload new package” and let’s start adding details to the work we would like Test Base to go through.

clip_image018

I’ve used 7-zip in this example:

Add package name

Add package version

Choose the test type

NOTE: the Out-of-Box (OOB) test performs install, launch, closed and uninstall of your package. After the install, the service will launch and close the application 30 times before continuing to uninstall the application. Scripts will run for 80 minutes at the most.

clip_image020

OS update type I’ve selected Security updates and feature updates.

NOTE: The Security updates enables your package to be tested against Windows pre-release monthly security updates
The Feature updates enables your package to be tested against Windows pre-release bi-annual feature updates builds from the Windows Insider Program.

clip_image022

Mark the Windows builds that you like to test against

clip_image024

Select “Insider Beta Channel”

Select “Windows 10 21H1”

Press “Next”

clip_image026

Before we can proceed with this step, we need to create a ZIP file with a certain format.

TIP: There is no limited number of ZIP files you can uploade, but the size is limited to 2GB per ZIP.

clip_image028

Go to your application(s) you want Test Base to go through.

clip_image030

Inside the application folder we need certain binaries to be present.

First of all, we need the application. This is because the test base will install the product inside a virtual machine.

Second of all, we need a script that can open the applications executable for test base to open op the program.

Third of all, we need a script to close the process that the second script opens. This is because test base performs the “launch action” 30 times and then the “close action” 30 times.

And finally, we need a script to uninstall the application, to make sure the machine can be cleaned up.

clip_image032

All the scripts are very basic and does not contain error handling:

TIP: If your script exists with other than 0 the operation will fail, and the sequence stops.


Script Argument
Install.ps1 Start-Process -FilePath “.7z1805-x64.msi” -ArgumentList “/QB ALLUSERS=1”
Open.ps1 Start-Process -FilePath “C:Program Files7-Zip7zFM.exe”
Close.ps1 Stop-Process -Name “7zFM” -Force
Uninstall.ps1 Msiexec /x 7z1805-x64.msi /QB

Once the scripts have been made and saved inside the application folder – ZIP the folder.

clip_image034

clip_image036

Back to the instructions of test base

Browse for the newly created ZIP.

clip_image038

Open it

clip_image040

Uploaded successfully

Click Next

clip_image042

Insert all of the scripts that will do the jobs. If you need the virtual machine to be rebooted after the application install, tick the “Reboot after execution”

clip_image044

NOTE: The script path should be like this app/script, so when having the scripts in the root of your ZIP, then the name for the folder + the script e.g., 7-zip/install.ps1

clip_image046

Once you added all the information, click Next

clip_image048

We did not add “Functional test”, so nothing to do here

NOTE: Functional test is a more a custom test method for Software Vendors. If you need more than the Out of box test can deliver you can use this feature and describe your flow Functional testing on Test Base | Microsoft Docs

Press Review

clip_image050

Press Create

clip_image052

Now be patient. The test process will take some time. But once it completed you will be able to see the result on the “Test Summary” page

clip_image054

clip_image056

You get a lot of data that you can pull out for documentation purposes.

NOTE: you can download a video of the test process, but you cannot get access to the actual virtual machine.

clip_image058


Summary

This was Test base and how you set it up to test your most critical applications towards the newest Windows builds before security patches and feature updates apply to your production environment. I thrilled about the idea and makes me sleep better when Windows evergreen is doing its thing.

Happy testing!

Source: Test Base for M365 documentation | Microsoft Docs

Table of Contents

Share this post
Search blog posts
Authors
Modern Workplace consultant and a Microsoft MVP in Enterprise Mobility.
Modern Workplace consultant and a Microsoft MVP in Windows and Devices for IT.

Infrastructure architect with focus on Windows Client management & security.

Cloud & security specialist with focus on Microsoft backend products and cloud technologies.

Infrastructure architect with focus on design, implementation, migration and consolidation.

Infrastructure consultant with focus on cloud solutions in Office365 and Azure.

follow us in feedly
Categories

Follow on SoMe