Testing…123…Testing

[pamlefkowitz]

There are days where I miss IT work and days I couldn’t be happier to be out of day-to-day IT. This time of year makes me happy that I’m no longer in day-to-day IT work. Truthfully, I loved it. It was some of the best time of my life and I made a number of very dear friends. But there were times when my clients just drove me bats*$^% crazy. Anyone else relate? Anyone? 

The toughest of these times were when new tech was released or announced. The hours or days right after WWDC was always one of those times that I’d see an uptick in obsessive behavior. I’m writing this shortly after WWDC ended to memorialize some of those tech support days.

I cannot count the times I had to tell users (who somehow got dev accounts) that I would NOT support betas. And, I also had to tell them (repeatedly - you know how that goes) that they should not install new software without my approval. Yes, all that is solved with MDM now, but that was 1) before MDM was affordable for a small business and 2) not everyone wanted to be managed like that. There’s a reason I grew tired of the tech support biz, right? 

But how did I decide when a tool was ok for my client to use? Suffice it to say, when I gave the OK it was because I had tested the OS or app to a point that I could say it was solid enough for daily use and I could support it sufficiently. That is today’s topic - testing.

Why Test?

Most people I know take a car (or many cars) for a test drive before buying one. Buying a car is more than just buying a slick style at a fair price. It’s about the stats and reputation of the car. It’s the mileage, the engine, the brakes, the smoothness of the drive, the responsiveness of the steering, the interior, the amenities, the visibility, the comfort of the driver’s seat, the trunk space, the leg space in the back, and all the other parts of the driving experience that you deem important. It’s also important to look at the dealer’s service agreement, what it takes to get service, what the manufacturer’s policies are regarding service, and what participation you are required to do before requesting service. You may drive 3 cars and find your match, you might drive 20 cars and decide to wait till the next version of your favorite. 

It’s the same with software. Before spending good money on an app or a SaaS, you want to take it for a test drive too. 

You might want to see if a developer has implemented a long-promised (or long-wished-for) feature. You might want to learn how a new feature works or assess if it would be right for your organization. You might also want to get a leg up on what’s coming from a beta release. 

If you’re an MSP, you might want to test out software before you decide to resell a product. Or your users have requested that you find them “a perfect solution” to a problem…so you set up a testing environment to check out a variety of candidates. Or you want to get ahead of your clients who, you know, will install an update before you’ve approved it (that you should be able to control with your MDM). As you can see, there are a host of reasons to test software.

And, truth be told, sometimes it’s just fun to live on the bleeding edge. You develop other circles of colleagues when you’re all in the beta-OS trenches together.

When the User Gets Different Results

For folks on the front lines of tech support (I recognize that you, dear readers, are not all in HelpDesk roles) a test environment can help you look like a hero. And as you are aware by now,  I’m all about being a hero with clients. When you have a properly designed test process in place, you can solve many of those extra-troublesome issues without having to be at the user’s desk. 

You can run experiments on your test system to see where conflicts might be, where configurations could be off, or whether a well-written step-by-step instruction might just solve the user’s problem. All of that can be done in your test environment, in the privacy of your office, without the user standing over you impatiently waiting for their problem to be fixed. They’ll still be impatient… you just won’t have to hear about it. 

How To Test

One way to test the desktop OS or app is to have another physical computer available. While isolated hardware is great, it can be expensive and is self-limiting. A more efficient solution is to use virtualization software. Virtual machines allow the sysadmin to set up multiple virtual computers in a variety of configurations. And virtualization software (specifically Parallels and VMWare Fusion) allows you to clone a machine so creating a new virtualized computer can be super fast.

I’m not about to give a tutorial on how to create and use VMs. But we can certainly talk about strategies for using them effectively in your IT Admin work. My preference for managing this task was to create a new virtual machine for each operating system upgrade including each major update. 

When a new app showed up for testing, I would clone the vm and install the app. I could set up a clean system at any time so there was no conflict with other apps. I could install demo versions of other software to mimic a client’s environment. I was able to run through some pretty extensive testing to make sure that a piece of software would not interfere with a client’s work or that it would meet their needs.

With virtualization, I could pause a vm to go do other things and come back to it days/weeks/months later and pick up right where I left off. And If I didn’t like the results of a test it was no big deal to wipe the vm and start fresh to run the experiment again with a different configuration. Better yet, create snapshots along the way and if something blows up on you, back out to the previous working snapshot.

I would also use vm’s to test OS upgrades and major updates to see where they break (let’s be honest…major changes don’t always go as smoothly as we wish they would). And, for the most part, my clients learned not to update before I update (because who wants to pay their consultant to learn on their dime?). And I would - on occasion - jump into the Beta fray and, for that, I absolutely used a vm for testing. OS beta testing is a fun topic. 

A Word About Beta OSes

It is important to remember that features in a beta OS do not necessarily become features in a final release. And more than once we’ve seen that features missing from a beta OS become a surprise feature in a released product. But more than that, beta operating systems are, generally-speaking, unstable. Software hasn’t been fully vetted on a beta OS, hardware hasn’t been fully tested on a beta OS, and just the beta OS itself could be unstable no matter what hardware it’s running on. For that reason, I recommend that you advise your clients against installing any beta OS. 

But you know, you just know, that you’re going to have a couple of users who feel they are better equipped to test a beta OS than you think they are. For those users, there’s valium. No. Wait. For those users there are contracts and device controls. You use your Hold Harmless Agreement as a rider on your contract to allow you to bill for any overages incurred because of users installing a beta OS. And, you put blocking scripts in place that won’t allow users to install a beta OS.

In case it needs to be said as a reminder: JumpCloud DOES NOT support Betas in production.

Things I Learned While Setting Up a VM Last Week

Presently, I have no real reason to set up virtual machines since I’m no longer doing regular tech support. But because I want to experience what y’all experience, I went ahead and stood up both a Parallels and a VMWare Fusion instance. I have a Macbook Air (M1) and am running the most recent OS. Here’s what I learned.

I started out with Fusion - it’s the last virtualization software I used in my MSP so I went with what was familiar. Well, I forgot about the M1 issue when creating a new Windows vm. So, I installed Parallels and tried to install Windows again. Whoops. My iso wasn’t going to work here either because of the ARM processor. But Parallels, at least, has a download for an ARM-compatible version of Windows 11 so that’s what I used. I didn’t activate it because it was just there for a couple--day test of the JumpCloud onboarding process while I was doing some exercises in JumpCloud University. Here’s what I learned from those couple of days, ymmv:

• As with pretty much all of the best laid plans, initially, everything takes longer than you think it should take.
• Presently, hosts with ARM processors make things a bit more complicated (at least on the Mac they do; for now). 
• You can’t run powershell scripts in a non-activated version of Windows. 
• Work in a clone, not the original vm. Keep the clones single use and toss them when you’re done. 
• Forgot Mistyped your password? It doesn’t matter since you’re working in a clone. Simply wipe and create a new clone. So long as everything important is either in the cloud or on the host itself, you’re safe.
• Don’t like your result? Delete and create a new clone. 
• Build time into your week to test new software, new configurations, or new policies.

Going Forward

It’s been a long time since I had to set up a testing lab here. I realized a few things while doing it. I realized how much I’ve forgotten by not being in the daily IT support fray - use it or lose it, friends. I realized how important having a lab was to me and to my clients and users. There is so much value in having something tested before going live. I realized that in order to maintain a great support experience for users, one must always be trialing. Build time into your week to test new software, new configurations, new policies, and new possibilities. 

Testing may take time, but it saves you time in the long run. And time saved in testing will benefit you and your users in the long run. Take the time to do the up-front work.