Isn’t it time we “IoTified” our Test Strategies?

We interviewed lead author Tom van de Ven to discover his 5 step approach and the building blocks required to implement a successful test strategy for a multi-layered IoT solution.

What would be your elevator pitch for your new book?

The book outlines a future-proof, 5 step approach to IoT testing that demonstrates that there are 3 key areas of focus required for success:

1. Testing IoT solutions is all about testing the Customer Experience (CX) and the building blocks of a strong test strategy must be focused on the end user by bringing together business intelligence, high tech and mobile testing expertise.
2. Ensuring quality, security and an excellent CX requires a shift from relying on functional unit and regression testing, to focusing on non-functional quality attributes through, for example, interoperability, performance, recovery, reliability, security and usability testing.
3. Crowdtesting and storytelling are an excellent way to understand the needs and habits of the end user and keep pace with rapid releases.

The book uses real life examples and clear jargon-free descriptions to illustrate that if you understand the 4 layer IoT model and apply the 5 step approach with CX, non-functional testing and crowdtesting as your focus, you can create a powerful and comprehensive IoT test strategy.

How is the test strategy in the book future-proof?

The 4 layer IoT model I use as a basis for the test strategy can be dismantled, adapted and repurposed to embrace new research by technology leaders such as SogetiLabs, so that the 5 step test approach will remain relevant to solving IoT challenges even as they evolve and grow in complexity.

1. The “Thing” layer comprising the device sensors, hardware, electronics and mechatronics
2. The “Bridge” connecting the thing to a platform for data storage and wifi, bluetooth or wired connection.
3. The “App Data” platform layer where data gets stored such as Azure or Bluemix
4. The “Application” layer where the level of end user interaction is at its peak whether you are dealing with a touchscreen for a smartphone app or navigation device or a web interface.

These layers can all be combined when dealing with the types of IoT objects we are seeing today, but can also be viewed as separate elements and combined with new elements that arise as IoT solutions and technology develop in the future. The book is also future proof in the sense that there are chapters devoted to exciting, cutting edge ideas, such as using Artificial Intelligence in your test strategy.

Which industries are going to be impacted first?

Banking and Financial Services particularly Insurance, will be heavily impacted.  Currently insurance companies distribute risk across a huge number of people, but the IoT will enable companies to mine a vast range of highly accurate data, very fast, in real time, giving a much clearer individual picture. Insurers will need to rethink how they will calculate our insurance premiums!

What are the challenges testing in an IoT environment?

Determining how to test in an IoT environment means considering what will be tested, how to test it and crucially, as the potential levels of connectivity are almost infinite, when to stop. My mantra for decision-making is Connect, Talk, Think, Act.  Are we testing the entire device or the component parts? Do we need to simulate the whole IoT environment such as a connected home or just simulate the immediate application environment? Does the environment require extremes in temperature, terrain or speed? If the device is a smart fridge, is the end user likely to connect it only to their mobile phone or is it also going to be talking to a smart watch, fitness device and separate freezer unit? It’s important to know at what point in the possible chain of connections we can safely and sensibly stop testing, as after 3-4 connections there are so many variables that testing becomes extremely complex and increasingly expensive.

The fact that we are dealing with a physical object comprised of complex technology is the biggest challenge in IoT testing and of course security is one of the most important considerations. It should be built-in but IoT objects are composed of several component parts and different developers and vendors have hugely varying security policies. As testers we need to look beyond the usual security tests and assess the device from the perspective of whether we can break into it remotely and in person, exploring all the ways it can be misused and putting ourselves into the head of a new kind of IoT Cybercriminal.

Another challenge is the potential skills gap in IoT testing. As end-to-end IoT testing requires virtually every area of testing expertise, even companies with a dedicated test team will not have the resources to test effectively. The answer here is greater levels of collaboration both with dedicated test partners and also with businesses in other industries which have the required expertise and access to the necessary information. For example insurance companies are now working with mobile phone providers to access data about car drivers and their vehicles. These practical challenges are overcome by my 5 step strategy and tooling approach, which is described in detail in the book and which we have already used at Sogeti on several client projects, to great effect. However the IoT also demands a new way of thinking and a cultural shift. To ensure that connected Things are safe and secure and provide a brilliant CX, we all need to take inspiration from services like Amazon Dash and switch our thinking to be more like a collaborative and nimble start-up.

IoTMap – Testing in an IoT Environment is now available in print and downloadable versions here.

*Gartner: http://www.gartner.com/newsroom/id/2636073

**HPE: http://go.saas.hpe.com/fod/internet-of-things

About the Author 

Tom van de Ven has been active in the field of High Tech testing for over 12 years. As a High Tech test expert he is a frequently asked sparring partner for Sogeti High Tech customers with regard to starting/professionalizing test projects. Besides a multitude of test assignments (eg. in the field of healthcare, semiconductors, agriculture and automotive) he is an active member of the Sogeti High Tech Test Competence Centre and a speaker for High Tech seminars.

Tom uses his experience in a role as a coach for (beginner) High Tech test engineers and is constantly looking for improvements in High Tech test methodologies. He completed his new book that combines Internet of Things and TMap: IoTMap in April 2016. He also teaches and develops several testing courses in the embedded and high tech domain. If not teaching, testing a tunnel or promoting “Quick Tech Testing” you can find him setting up a high tech test automation framework for the odd customer.