Borrowed from Microsoft
Test Base for Microsoft 365 is a validation service made for Software Vendors and System Integrators.
Why should that be interesting for you as a Configuration Manager or Intune admin?
Because most businesses does have critical applications that they like to test properly and to make sure the applications work on the next Windows feature level.
Test Base is a service in Azure that will help you test your applications against a subset of OS versions after your choice. That could be the insider of the next Windows 10 build or even Windows 11, to make sure you are all set for next version of Windows feature rollout.
In the end this will give you a good indication of how your app stack will behave, save you a lot of time and make you sleep better!
Important question: How much does it cost?
At least while in preview it is free. Customers will be charged once it go GA
- Azure Subscription
How to set it up
Browse to https://portal.azure.com/
Search for “test base” and click on “Test Base for Microsoft 365”
Choose your subscription
And choose “Create new” on the resource group
Give it a name and press ok
Choose Resource Group Location and create a Test Base Account Name
Validation passed and we can press Create
Test Base was deployed and ready to be used.
Click “Go to resource”
Press “Upload new package” and let’s start adding details to the work we would like Test Base to go through.
I’ve used 7-zip in this example:
Add package name
Add package version
Choose the test type
OS update type I’ve selected Security updates and feature updates.
The Feature updates enables your package to be tested against Windows pre-release bi-annual feature updates builds from the Windows Insider Program.
Mark the Windows builds that you like to test against
Select “Insider Beta Channel”
Select “Windows 10 21H1”
Before we can proceed with this step, we need to create a ZIP file with a certain format.
TIP: There is no limited number of ZIP files you can uploade, but the size is limited to 2GB per ZIP.
Go to your application(s) you want Test Base to go through.
Inside the application folder we need certain binaries to be present.
First of all, we need the application. This is because the test base will install the product inside a virtual machine.
Second of all, we need a script that can open the applications executable for test base to open op the program.
Third of all, we need a script to close the process that the second script opens. This is because test base performs the “launch action” 30 times and then the “close action” 30 times.
And finally, we need a script to uninstall the application, to make sure the machine can be cleaned up.
All the scripts are very basic and does not contain error handling:
TIP: If your script exists with other than 0 the operation will fail, and the sequence stops.
|Install.ps1||Start-Process -FilePath “.7z1805-x64.msi” -ArgumentList “/QB ALLUSERS=1”|
|Open.ps1||Start-Process -FilePath “C:Program Files7-Zip7zFM.exe”|
|Close.ps1||Stop-Process -Name “7zFM” -Force|
|Uninstall.ps1||Msiexec /x 7z1805-x64.msi /QB|
Once the scripts have been made and saved inside the application folder – ZIP the folder.
Back to the instructions of test base
Browse for the newly created ZIP.
Insert all of the scripts that will do the jobs. If you need the virtual machine to be rebooted after the application install, tick the “Reboot after execution”
NOTE: The script path should be like this app/script, so when having the scripts in the root of your ZIP, then the name for the folder + the script e.g., 7-zip/install.ps1
Once you added all the information, click Next
We did not add “Functional test”, so nothing to do here
NOTE: Functional test is a more a custom test method for Software Vendors. If you need more than the Out of box test can deliver you can use this feature and describe your flow Functional testing on Test Base | Microsoft Docs
Now be patient. The test process will take some time. But once it completed you will be able to see the result on the “Test Summary” page
You get a lot of data that you can pull out for documentation purposes.
NOTE: you can download a video of the test process, but you cannot get access to the actual virtual machine.
This was Test base and how you set it up to test your most critical applications towards the newest Windows builds before security patches and feature updates apply to your production environment. I thrilled about the idea and makes me sleep better when Windows evergreen is doing its thing.