You’ve probably read about HBO’s ‘Integration Test’ Email. Just what is an “Integration test”?
In software development there are many testing strategies. Two of them are called “unit tests” and “integration test”.
A “Unit Test” tests a component “in isolation”. An “Integration Test” tests how all those components work together. (My mnemonic device: You are testing a unit by itself or how they work when integrated.)
Suppose you were building a car. You’re going to need headlights, wheels, an engine, a body, seats, etc. If you want to ship a car without bugs… I mean defects… you want to test the individual components and the car itself. Testing either is not enough.
For example you might test the headlight in a “unit test”: You take the lamp out of the box, connect a power source, and make sure that it lights up. That’s a unit test. The test is “done in isolation” i.e. it doesn’t involve the car itself. The power source you connected it to isn’t the real car, it is a “fake” (that’s the real industry term) i.e. something that simulates just enough of the car to do the test. In this case, the “fake” provides voltage and nothing else.
Once you’ve assembled all the parts of the car, you do an “integration test”. That tests that everything is working together. You might start the car, run the engine, take it on the road, and so on.
Software engineering is similar. You have many smaller components that combine and work as one big unit: “Autobots, Transform And Roll Out!”
For example Facebook Messenger has many components: the visible user interface, the component that sends your message to Facebook, the component that figures out where to send the message (which may be one or many friends), and the component that examines what you wrote and decides that in the future they should target you for adverts based on what you wrote. Each of those components are tested “in isolation”. For example, the visible user interface might be tested on various browsers to make sure it works on each of them. The testing isn’t sending real messages, in fact it may be talking to a fake Facebook, just like the “fake” used for testing the headlamp.
A common mistake is to accidentally run a test in the real production environment instead of the test environment. What you saw with the test email was just that.
There are a lot of fancy ways to prevent such tests. I prefer simple things like using a different color in the test environment. You get used to seeing (for example) red in tests and blue in production. When you see blue while testing you know something is wrong.
Similarly the data that you use in testing should be obviously fake. Instead of English words, use gibberish. Use prices that are so obviously wrong everyone will know it is test data if it leaks. Consistency is important: if your strategy is that test data always mentions Britney Spears, don’t have one dataset that is not about Britney.
Or, if you are the New York Times, make sure your test articles are about Watermelons On Mars so that when that gets leaked, it doesn’t nearly start a nuclear war.
Unit testing and integration testing are relatively new concepts. In the old days software was only tested “at the end”. That would be like not checking a car’s headlamps until just before you sell the car to the customer. It is most embarrassing to have a bug at that point, and more expensive to fix it.
Congrats to HBOMAX for having integration testing on your email system. Many companies don’t!
