The Internet of Things is going to solve climate change, fix our political system, and ensure that you can always find a parking spot. Some see a future of 15 billion connected devices.

Now, just the tiny matter of deploying them. There's a long way between all IoT's utopian promises and the reality. We've never attempted anything like this before.

The challenges are immense. How do those devices work autonomously? How do they work together? How do you balance the energy overhead of sending data from a low-powered sensor against processing it locally? What's the best format for your data and how can you use it when it arrives back at base?

Data is seen as one of IoT's biggest payoffs – generating and gathering it to help your business. But get IoT wrong, and you stand to be overwhelmed by that data wave. Cisco estimates IoT will generate 500 zetabytes of data by the end of 2019 while Gartner has warned of a data tsunami at the network's edge. Security, meanwhile, has consistently been one of the largest inhibitors to rollouts. The use of open systems running often low-priced chips coupled with sheer proliferation has made IoT vulnerable to hackers.

So what's the best way to deploy IoT and stay secure?

The biggest mistake is to become simply enamoured with data. It's a problem Ben Howes, founder of IoT specialist Zoetrope, sees all too often. People often skip the part where they think about the data that they need and what they will use it for, he told us. "There's a perception that you can get any and all data and then think about it later. You need use cases."

Richard Soley, chair and CEO of the Object Management Group, warns against looking for oft-touted idea of big-data magic. His advice is to design systems that are constrained to produce good results. Start small and with clearly defined data points and build from there, he says. "If you feed it into a machine learning algorithm, those programs can't explain how they decide things," Soley told The Register. "You'll screw it up eventually because you'll feed it data that you won't understand later."

Security and protection of data consistently top the reasons why companies have yet to roll out an IoT strategy. It's not difficult to see why: in 2016, websites including Twitter were taken down by a DDoS attack on DNS provider Dyn launched from millions of devices compromised using the Mirai malware.

Designers must therefore think about the security early on. After all, we're meant to be doing security by design and default, right? All the usual security thinking and practices apply, like encryption and use of passwords, but you'd be wrong to think you can forklift existing traditional security policies into an IoT environment.

John Moor, managing director of the IoT Security Foundation, explains: "If you talk to people who have a passion for data security they tend to be traditional infosec types that concentrate on the CIA triangle." That's confidentiality, integrity, and availability. "In IoT, it's more the 'I' and the 'A' that become dominant."

Ensuring that no one has tampered with your data – either incoming data for analytics or outgoing data for controlling devices in the field – should be a top priority for IoT teams. What kinds of things could attackers do to mess with your system? Imagine a factory that uses IoT data to report on production. Moor says competitors could – and have – interfered with that data to alter things just enough that it throws off a company's yield and inventory reporting. It's an excellent way to snarl up a supply chain and annoy your competitors' customers.

Encrypting traffic en route helps to stop attackers. One way to do that is to use digital keys so that they can sign their communications. Unfortunately, we're talking about hordes of devices here, which presents a fundamental management problem. Some vendors solve it by sharing private keys across thousands of devices. That is the wrong answer.

"You need unique keys on each device," says Howes. Even large clients have had trouble grasping how to do that, though, he admits. Responsible manufacturers will help with this process.

One promising development is physically unclonable functions (PUFs) that are being realised as hardware security modules (HSMs). PUFs analyse the random electrical properties of individual integrated circuits to create a unique key on demand without having to store it on the device. A major challenge in IoT is that hackers have been able to bypass the crypto keys.

Traditionally, if power to HSMs is cut, data is lost. Researchers in Germany, however, claim to have developed HSM technology that is self-powered so not dependant upon a battery. HSMs do, however, remain relatively expensive to implement.

In many ways, though, security will come not purely from technology but from the processes and practices you install. That should include ongoing risk assessment and threat detection and response. That ongoing aspect is important, because the nature of IoT means the threats and attacks are constantly evolving while you will often be acting without all the information.

But then there's transit, too

Two types of data are open to attack: the device's log data, which shows how well it's working, and the production data it creates to serve a business need. One of Howes' common complaints is that deployment teams often want to leave the log data on the device itself, rather than send it back to headquarters. When the data does move, organisations will SSH into a device – often only when something has gone wrong. "That's a big attack vector, and you can only get logs when the device is on," he points out. "There is no trend analysis, and they can't set up alerting when something goes wrong. There's no proactivity."

Configuring devices to communicate their log data regularly can help to flag problems in the infrastructure, he suggests.

What about that other data – the data supposed to let your organisation do all that useful, digital business stuff? This is the data you'll want to crunch and analyse. The big question here is: do you send it all home in its raw state, or do you process it locally first? Is your IoT data to be harvested centrally, or treated at the edge? Or somewhere in the middle, via some aggregator device? And how do you make that decision?

Sometimes, your answer will be driven by practical considerations like the volume of data and network capacity.

"Sometimes it's just a question of low bandwidth. So how much data am I generating and do I have the bandwidth to move it elsewhere," Soley says. 'If I can, then I will do it because then I can correlate that data."

There are times, though, when you won't have the bandwidth or you may lose connectivity altogether. In mining or on an oil rig, wearables might need to keep track of where personnel are and what they're doing but might not have connectivity. You could document that data on the device or maybe aggregate it somewhere in a local mesh network, and then upload a small, processed file at the end of each day for reporting.

You may not have the latency to communicate data for actionable decisions, either. Autonomous vehicles don't have the time to talk to the cloud when deciding how to drive, so the time-critical data is handled on board, while the non-time-sensitive information used for AI training and analytics can be processed and uploaded later.

The Industrial Internet Consortium has been looking at this. It has been operating more than two-dozen testbeds, in which it assesses real-world IoT deployments in industrial settings. The idea is to learn more about what works in industrial IoT settings.

One of these testbeds tracked the usage of tools at Bosch and at a large, unnamed aircraft manufacturer. Smart tools containing sensors could feed back data including their location and current usage patterns. Tools might stop working until a user was holding them correctly and in the right part of the factory for that task, for example.

"We had to write device drivers for every tool. That would have been easier if there was a semantic framework for them," Soley says. "Now, Object Management Group is working on that."

Your choice of where and how to process data could, then, contain hidden complications. Standardising interfaces at a low enough level, though, and applying them across multiple devices and applications, could mean companies in the future can deploy their IoT without having to reinvent the wheel at a deep level.

Devices in the workplace are not a new phenomenon. They've existed in sectors such as oil and gas and healthcare for decades. What is new is the fact devices are now being made of industry-standard hardware and software, connected to the internet, and rolled out on such a massive scale – including to the home.

Managing the devices and the data in these deployments is more difficult than it looks, and the pioneers are still navigating uncharted territory.

Getting real on the data myth and also tackling security fundamentals remain two of the biggest common issues in IoT – regardless of sector.

Armed with some basic best practices and knowledge of the common pitfalls, though, you can at least proceed with a rough map – even if you do find yourself filling in some gaps yourself along the way. ®