Test base – test you most critical apps automatically against insider builds

Test base – test you most critical apps automatically against insider builds


Borrowed from Microsoft



Test Base for Microsoft 365 is a validation service made for Software Vendors and System Integrators.

Why should that be interesting for you as a Configuration Manager or Intune admin?

Because most businesses does have critical applications that they like to test properly and to make sure the applications work on the next Windows feature level.

Test Base is a service in Azure that will help you test your applications against a subset of OS versions after your choice. That could be the insider of the next Windows 10 build or even Windows 11, to make sure you are all set for next version of Windows feature rollout.

In the end this will give you a good indication of how your app stack will behave, save you a lot of time and make you sleep better!

Important question: How much does it cost?

At least while in preview it is free. Customers will be charged once it go GA



  • Azure Subscription

How to set it up

Browse to https://portal.azure.com/

Search for “test base” and click on “Test Base for Microsoft 365”


Press Create


Tick “I confirm I have read and acknowledged the terms of use above”

Choose your subscription

And choose “Create new” on the resource group


Give it a name and press ok


Choose Resource Group Location and create a Test Base Account Name


Validation passed and we can press Create


Test Base was deployed and ready to be used.

NOTE: Before you proceed from here you should already know what applications you would like this service to test.

Click “Go to resource”


Press “Upload new package” and let’s start adding details to the work we would like Test Base to go through.


I’ve used 7-zip in this example:

Add package name

Add package version

Choose the test type

NOTE: the Out-of-Box (OOB) test performs install, launch, closed and uninstall of your package. After the install, the service will launch and close the application 30 times before continuing to uninstall the application. Scripts will run for 80 minutes at the most.


OS update type I’ve selected Security updates and feature updates.

NOTE: The Security updates enables your package to be tested against Windows pre-release monthly security updates
The Feature updates enables your package to be tested against Windows pre-release bi-annual feature updates builds from the Windows Insider Program.


Mark the Windows builds that you like to test against


Select “Insider Beta Channel”

Select “Windows 10 21H1”

Press “Next”


Before we can proceed with this step, we need to create a ZIP file with a certain format.

TIP: There is no limited number of ZIP files you can uploade, but the size is limited to 2GB per ZIP.


Go to your application(s) you want Test Base to go through.


Inside the application folder we need certain binaries to be present.

First of all, we need the application. This is because the test base will install the product inside a virtual machine.

Second of all, we need a script that can open the applications executable for test base to open op the program.

Third of all, we need a script to close the process that the second script opens. This is because test base performs the “launch action” 30 times and then the “close action” 30 times.

And finally, we need a script to uninstall the application, to make sure the machine can be cleaned up.


All the scripts are very basic and does not contain error handling:

TIP: If your script exists with other than 0 the operation will fail, and the sequence stops.

Script Argument
Install.ps1 Start-Process -FilePath “.7z1805-x64.msi” -ArgumentList “/QB ALLUSERS=1”
Open.ps1 Start-Process -FilePath “C:Program Files7-Zip7zFM.exe”
Close.ps1 Stop-Process -Name “7zFM” -Force
Uninstall.ps1 Msiexec /x 7z1805-x64.msi /QB

Once the scripts have been made and saved inside the application folder – ZIP the folder.



Back to the instructions of test base

Browse for the newly created ZIP.


Open it


Uploaded successfully

Click Next


Insert all of the scripts that will do the jobs. If you need the virtual machine to be rebooted after the application install, tick the “Reboot after execution”


NOTE: The script path should be like this app/script, so when having the scripts in the root of your ZIP, then the name for the folder + the script e.g., 7-zip/install.ps1


Once you added all the information, click Next


We did not add “Functional test”, so nothing to do here

NOTE: Functional test is a more a custom test method for Software Vendors. If you need more than the Out of box test can deliver you can use this feature and describe your flow Functional testing on Test Base | Microsoft Docs

Press Review


Press Create


Now be patient. The test process will take some time. But once it completed you will be able to see the result on the “Test Summary” page



You get a lot of data that you can pull out for documentation purposes.

NOTE: you can download a video of the test process, but you cannot get access to the actual virtual machine.



This was Test base and how you set it up to test your most critical applications towards the newest Windows builds before security patches and feature updates apply to your production environment. I thrilled about the idea and makes me sleep better when Windows evergreen is doing its thing.

Happy testing!

Source: Test Base for M365 documentation | Microsoft Docs

Table of Contents

Share this post
Search blog posts
Modern Workplace consultant and a Microsoft MVP in Enterprise Mobility.
Modern Workplace consultant and a Microsoft MVP in Windows and Devices for IT.

Infrastructure architect with focus on Modern Workplace and Microsoft 365 security.

Cloud & security specialist with focus on Microsoft backend products and cloud technologies.

Cloud & security specialist with focus on Microsoft 365.

Cloud & Security Specialist, with a passion for all things Cybersecurity

Cloud and infrastructure security specialist with background in networking.

Infrastructure architect with focus on design, implementation, migration and consolidation.

Infrastructure consultant with focus on cloud solutions in Office365 and Azure.

follow us in feedly

Follow on SoMe