When we examine COTS IoT devices with various kinds of hardware architectures, we need to connect the IoT devices with a particular security tester physically using debug interface, which is inconvenient, unscalable, and inefficient. To resolve this issue, we build an automatic IoT security testbed, which operates firmware of COTS IoT devices in the compatible system with emulated IoT architectures. 

We utilize the opensource QEMU to emulate the hardware architectures, such as ARM, X86, and MIPS, for various kinds of COTS IoT firmware. Currently we can successfully operate COTS IoT firmware from Zyxel, CISCO, Dlink, and Netgear on top of our testbed and exploit the developed vulnerability testing tool to examine those softwarized COTS IoT firmware simultaneously. The testbed could discover many potential vulnerabilities from the firmwares and generate a testing report for researchers and engineers to evaluate the effectiveness of their proposed IoT operations and products.

The existing commercial security testbeds, such as very famous IoT Inspector, are focus on the static analysis of the source code of the IoT devices. However, for those companies who want to test their product, revealing source code is risky and offering firmware binary is more reasonable. Moreover, by performing the firmware on the emulated system, our testbed could precisely trigger the suspicious file, capture the malicious behavior of the operation, and discover logic flaws, unknown and zero-day threats. The main breakthrough of this testbed is that we successfully operate 661 COTS firmwares (with brand Dlink, Zyxel, Netgear) on the emulated hardware architectures, such as ARM, X86, and MIPS. By exploiting public vulnerability testing tools, we found 115 vulnerable COTS IoT firmwares.

For those IoT developing company without stronger security team, our testbed could offer an extensive vulnerability testing to their developed product and a corresponding testing report. The development teams could patch the vulnerabilities accordingly and make sure the product on the market is not vulnerable. For the company with individual security team, the testbed could be cooperated in the produce development cycle where the engineers could leverage this tool to examine the prototype of the product directly and fix the potential bugs in the next version. In both cases, the testbed improves the development quality from the perspective of security and significantly reduce the high cost of handling hacked or intruded IoT devices.