Here’s what dry run testing means:
A dry run test is a way to check whether or not software works while eliminating the prevalence of outside variables.
At the most basic level, a dry run test involves a programmer reading the code to make sure it works without actually running.
There are many other forms of dry run testing in software development.
So if you want to learn all about how dry run software testing works, then this article is for you.
Let’s jump into it!
What Is a Dry Run Test?
The easiest way to understand the concept of a dry run test is with a short history.
I promise this won’t be bad.
The term actually comes from firefighters.
A long time ago firefighters realized that practicing their techniques and responses could help them do the job better.
So, they would stage mock fires where they could go through procedures and iron out any problems.
In these mock fires, there wouldn’t be any actual smoke or flames.
So, they didn’t need to pump water through the hoses. And, that’s the origin of the name.
A dry run was a practice run where they didn’t actually spray water on anything.
Running with that idea, a dry run is a test where possible problems are deliberately mitigated.
Dry runs are done with all kinds of things, from military procedures to hardware development.
Today, we’re talking specifically about dry runs in software.
So, a dry run test is something that is going to simplify the environment to get a better feel for how the software works in a vacuum.
How Does Dry Run Testing Work? (2 Things)
There’s a bit of history behind dry run software testing, and I’ll get to that next.
Before that, we can talk about how to implement the idea of a dry run test.
The key is adapting to the environment.
The point of a dry run test is to keep things simple so you can test specific elements of the software.
In order to do that, you have to adapt the test according to your purpose.
So, if you want to see how netcode works at reducing ping in a video game, then a dry run test would probably look at two instances playing on the same device, rather than across a network.
It removes the uncertainty introduced by the network environment.
That’s a specific example, and we’ll cover more general kinds of dry run tests later, but the point is simple.
The dry run test is built around whatever it is that you are trying to figure out.
#1 Traditional Dry Run Tests in Software
This might make more sense when we dig into the history of dry run tests in software.
Traditionally, dry run tests have been an integral part of computer programming and development.
The idea is to check the logic of the computer code before you actually try to run it at all.
How do you do that?
Well, you carry out the code by hand.
This can be a little complicated to think about at a theoretical level, so let’s start with a simplified example.
Let’s say we’re going to write a calculator program, and we’ve finished the section on addition.
In reality, this is so simple that you wouldn’t have to write any code anymore, but let’s stick with the simplified idea for now.
So, we’ve written however many lines of code it takes to tell a computer to add two numbers.
We want to make sure the logic is good. So, we use a test case that we can check by hand.
We choose 5+5. It’s easy to do that in your head, so you know that the computer code should spit out an answer of 10.
To test the code, you actually read through it.
You can use a pencil and paper if you need to, and you, the human, follow the instructions in each line of code (so naturally, you need to understand what the code is saying).
When the code asks you to input numbers or a formula, you use 5+5, and after you follow all of the steps, you’ll get an answer.
If the answer isn’t 10, then you know there is a logic problem with the code.
The point of this traditional dry run test is that you’re inspecting the logic of the algorithm without actually running the software.
When you run the software, you introduce the possibility of what is called execution errors.
That complicates your check. So, you follow through with the code by hand.
By removing the computer itself from the equation, you remove computer-based problems, and your dry run test is able to examine something very specific with the software.
#2 Evolving Dry Run Tests
The idea of a traditional dry run test is pretty easy to follow, but in modern programming, it’s not always viable.
A modern software package includes millions of lines of code.
As an example, QuickBooks was launched in 1992.
Even then, it already had more than 10 million lines of code.
You’re not going to work through that by hand, and software gets way more complicated than this.
So, as programming has evolved, so too have dry run tests.
There are now a number of different ways to go about dry run testing, and I’ll be showing you prominent examples next.
Even then, testing by hand isn’t completely gone.
Instead of testing the whole program by hand, software developers can do a traditional dry run test with a single portion.
That’s how the software is written anyway.
Any single function in a software package is written separately and then incorporated into the greater whole.
So, when a developer finishes their small chunk, they can perform a dry run test to see if the logic works.
Instead of going through millions of lines of code, they’re probably only dealing with hundreds for that little piece of software.
It’s still a meticulous job, but it’s actually doable.
What Are Some Good Examples of Dry Run Testing in Software? (6 Approaches)
But, testing each software component isn’t enough for the entire development process.
Eventually, you have to make sure that the different segments of code work well together.
As you add more and more pieces, the idea of testing by hand starts to go out the window.
So, how do you set up and perform dry run tests that still keep environments simple?
As I mentioned before, the test has to be tailored to its purpose.
But, the sections below will cover common approaches to dry run tests.
In each case, it’s easy to see how variables are removed from the equation to simplify what you’re looking at.
#1 Single-User Tests
One of the most obvious ways to design a dry run test is to simplify how many things are happening at once.
In a lot of cases (including any that use the cloud), the software is designed to serve multiple users at the same time.
In fact, it might even be designed to serve multiple thousands of people at the same time.
So, how do you test it in a simplified environment?
Make sure it’s not actually online, and only has one user.
In that single-user environment, the developers can go through any list of actions they want to test.
This is important for simplifying hardware issues.
If cloud servers are dealing with thousands of people at the same time, there are a lot of opportunities for additional errors to corrupt the test.
By running everything in an environment that can’t be accessed by multiple people, you eliminate all of the problems related to server stress and multiple users.
You can see how the software runs by itself.
#2 Fault Monitoring
Fault monitoring tests are a great way to see how dry run testing has evolved and can be applied outside of obvious conditions.
Fault monitoring is software that tries to identify problems as soon as they arise.
There are countless kinds of fault monitoring, but an easy one to understand is network monitoring.
You can make software that can identify any time a computer in a network goes offline.
The software knows how many computers should be online, so it sends out network signals to make sure everything is connected.
If the number of responses is ever less than it should be, the software knows that there is a problem, and it reports the fault.
This is another super simplified example just to make the point.
A dry run test of this kind of software inverts the problem.
Instead of trying to test the software in a blank environment with nothing going on, you’ll test it in an environment where you introduce problems on purpose.
In the simple example, we’re using, you would unplug one of the computers in the network to see if the software notices.
With more sophisticated fault monitoring, you would introduce deliberate problems into the system to make sure the fault monitoring software successfully identifies them.
#3 Clean Operating Environment Testing
When you’re trying to do more holistic dry run tests, then it gets harder to eliminate variables.
We already talked about how complicated modern software can be.
How do you perform a dry run test when you finally put it all together?
There are two keys to this type of test.
The first is sequential testing.
Hopefully, the software components were all able to pass dry run and compiling (where you actually run it in a computer) tests before this point, so you know each component works.
The second key is to use a clean operating environment.
Basically, wherever you use your computer, you introduce new data to the system.
This can come from installing software, changing settings, and all of the general use stuff that you do.
Your presence on the computer changes things.
As a result of those changes, there are countless opportunities for bugs to pop up.
Every time you install a new program, it has to interact with all of the other software on the computer (at least to some extent).
So, as the system grows more complicated, you’re going to see more random problems.
A dry run test can sidestep all of this by running the software on a new, clean build.
You install a fresh operating system and don’t use it except to test the software.
It minimizes all of these other problems, and it lets you see the software perform in its totality.
#4 Hardware-Free Testing
We’re actually circling back to traditional dry run tests.
Hardware-free testing is another name for running through code manually to check for logical errors.
As I already said, this still applies to modern software, as long as you break it into manageable pieces.
But, there is an important concept that I left out earlier.
Basically, there are two philosophies for approaching a hardware-free dry run test.
The more common approach is to have the person who wrote the code do the test.
Since they wrote the code, they definitely know what every line means (or is supposed to mean).
They can also go through the logic very quickly and efficiently.
The problem is that they actually know the code too well.
They understand how the logic is supposed to work, and that introduces a bias that can actually harm the dry run test.
Have you ever had to proofread your own writing?
Did you miss small mistakes?
That happens because your brain automatically corrects what you actually see to what you expect to see, and the same thing can happen to developers with dry run tests.
So, the competing philosophy is to have developers perform these tests blindly.
You bring in a person (or team) who understands enough about code to go through the algorithm.
They then do a standard dry run test to make sure the logic works.
This can help eliminate biases, but it usually takes more time, and you have to pay the tester, so it’s more expensive.
But, these competing philosophies point out some of the challenges of dry run tests and what software developers have to do to make products that work reliably.
#5 Test Scripts
Lastly, we get to test scripts.
Test scripts follow the same general idea as a traditional dry run test.
You’re plugging known values into your code to make sure it spits out the right answer.
But, with test scripts, you’re automating things to make them a lot faster.
For those unfamiliar, a script is a small bit of code that tells another program what to do (technically, scripts can be long, but we’re focused on simplified tests today).
So, you could write a script that tells your calculator software to punch in 5+5 and run.
Obviously, you don’t need a script for such a simple example, but when dealing with more complicated software, scripts can run through a bunch of actions within the software very quickly.
You can check the outcomes of the scripts against what you are supposed to see, and in that way, you can check the logic of the software.
So, in a lot of ways, script testing is a modern version of traditional dry run tests.
You can write a script to test any element of the software as desired.
You run the script, and you see if the software works.
This does run the risk of hitting execution errors, but when testing millions of lines of code, it’s a much more efficient way to check everything.
An Easy Example
Let’s look at a quick example of test scripts to really see how they work.
Imagine you’re on a team working on QuickBooks (since we already talked about it).
You’ve built your little segment of software, and you want to check it.
But, even though it’s “little,” it’s still a few thousand lines long.
You can use a dry run script.
Instead of going through everything by hand, you’ll let the script tell the computer what to do.
t will carry out the actions, and you’ll see if your section worked.
So, in this instance, you might make a script that tells the software to look up tax deductions.
Your script will include input information that the program needs.
When it runs, the software will tell you which deductions are viable, and you can then check the answers by hand.
Was the software correct?
The answer to that question gives you the logic check you are seeking.
#6 Dry Run Checks of Tests
This might feel like we’re off in the weeds, but it becomes very important for testing full software packages.
Traditionally, a dry run is trying to test the logic behind the code in a piece of software.
Yet, modern variations just outright test the software, such as in the script tests above.
You can actually introduce an intermediate step between these two where you apply dry run test principles to figure out what exactly the script should be testing.
Is that a little weird?
Basically, you want to come up with a test for the software.
You then dry run the test itself to make sure it will actually tell you what you want to know about the software.
So, in a previous example, we looked at running a script to check if tax software was finding the right deductions.
You can perform a dry run test on the script itself to make sure it really is checking that part of the software.
It’s a logic test within a logic test, but this kind of thoroughness ensures that your test results mean what you think they mean.
