Reflection in water of world

Digital Twins, the Internet of Things, and Machine Learning

In a world where so many things are Internet connected, how is machine learning playing a role? Bruce Sinclair speaks with us about the intersection of IoT, AI/ML, and the digital twin.

Bruce: Where AI, and in particular machine learning, and then in particular neural networks, and then in particular deep learning neural networks, where they apply is mostly in this model making, so with IoT, there are two types of models for the digital twin: we have the analytical model that’s created through more analytical techniques, and then we have the cognitive models that are being created through a machine learning and artificial intelligence techniques. I kind of like to separate the two, but the the impact in both cases are profound.

Ginette: I’m Ginette.

Curtis: And I’m Curtis.

Ginette: And you are listening to Data Crunch.

Curtis: A podcast about how data and prediction shape our world.

Ginette: A Vault Analytics production.

Today, if you haven’t guessed already, we’re talking about the intersection of data, artificial intelligence, and the internet of things, or IoT. So we’re talking to an expert well versed in this topic. A little bit about his background: Among many other things he’s done, like found and head companies, he’s authored a book on the Internet of Things, created a certification program for people who want to become certified IoT professionals, and he explains all things IoT on his podcast called “The Internet of Things Business Show.” Today, we’ll learn about AI in the IoT world and more specifically digital twins—a concept named by Gartner two years in a row now as one of the top ten strategic technology trends for both 2017 and 2018. Let’s dive into this topic with our guest.

Bruce Sinclair: My name is Bruce Sinclair. I am the president of IoT Inc. We consult for brands, manufacturers, and vendors and help them with their IoT strategies, both on the business side and on the product side, and we produce content, so part of the content is the podcast, and we do trainings, so we’re training executives on how to introduce IoT within their business, and how to—most importantly—be profitable with IoT, and the reason I started IoT Inc. was that I saw pretty quickly that there was a lot of hype around the Internet of Things, and this hype was all around the shiny new things, in particular the technology, but as most technologies, they run out of steam if they can’t make any money. And so I was very deliberate in focusing on the business aspect of IoT to try to help executives and managers to understand how to apply this technology.

Curtis: One of the most important concepts in IoT is the digital twin, which is a virtual reflection of a physical object. One major use of the digital twin is taking the virtual reflection of an object and virtually change it before actually changing things in the physical object in the real world. Today a digital twin is generated from data coming from sensors embedded in a physical object.

Bruce: So the Internet of Things, for everyone that’s listening, is really just the Internet being put into physical objects. The Internet being networking, things being the device. That’s really, at least when you look at it from a business perspective, that’s not where the action’s at, and not coincidentally, where the action’s at is in data analytics, data science, and a subset of that being AI, and the purpose of putting the Internet in the physical objects at the highest level is to capture data. So we capture data in our sensors, which is more of our internal data sources, and we capture data on the Internet, and that is using business systems, that is using microservices, and coincidentally or interestingly, it’s also other products, and this leads us to the most important technology for the Internet of things and this is the digital twin, and the digital twin is the virtualization of the physical into the digital, so this is where it kind of allows us to take the physical world and start applying analytical techniques to the data that we gather from it so that we can create value.

Ginette: While the concept of a digital twin is a hot topic right now, the term stems back to 2002 and the concept in action stems back even earlier. Some link it back to testing space technology on a simulation before applying changes to the real thing, and Bruce can see the beginning of it in his early career.

Bruce: So the history of the digital twin goes back to engineering, and specifically where my career started and that is the graphical representation of physical objects, so my original career started in computer animation. Our claim to fame was that we created, and this goes back a few years, but we created the, or at least animated, the dinosaur for the first Jurassic Park movies, and in fact, I think all the Jurassic Parks now, and we also used our software, and this is computer modeling software, but what we’re modeling is the geometric representation of the objects whether, they are a dinosaur, and then we also, our company also animated engineering models, so that would be automobiles and different types of machinery. So the digital twin originally came from that idea, but we needed to extend it to be able to use data science within the physical world, and so the digital twin in the form of the Internet of Things does not represent what it looks like graphically or what it looks like geometrically. It represents the value that we’re trying to create with that physical thing, and I call them products, and then I overlook the term to mean product, system, or environment, so an Internet of Things product in our vernacular, is a . . . it’s either is a discrete product, so a discreet product like let’s say an IoT dryer.

Ginette: Think of a clothes dryer that communicates with your other house appliances to bid for a time to operate .

Bruce: It could be a system, so like a telematic system, or it can be a smart city, entire smart city, which is more of an infrastructure or an environment, so a digital twin in all those cases don’t represent what it looks like. It represents what they do to create value for, for the user.

Ginette: So a digital twin in the IoT world is one that recreates the most important business-driving information from a product, system, or environment, and then from that, business owners can derive important insights to improve their product’s performance, which ultimately drives product value. Bruce gives us a few examples.

Bruce: When we’re trying to capture value from The Internet of Things, the best way to approach the problem is to think in terms of outcomes. So for a smart city, there’s going to be different subsystems. Let’s think about the lighting, and I’m just going to use lighting because I do have one client, one consulting client, that is in the process of producing smart lamp posts for campuses and for smaller sized cities. It’s not necessarily what you think it is. Again, the digital twin to go a little bit further into it technically, it represents this value creation. In the case of the light post or the lamp-post, the value or the outcome that we’re trying to create is safety, and so everything that we do when we designed the light post, which is part . . . or the lamp-post, which is part of the smart city is thinking in terms of Illumination for pedestrians, illuminations for vehicles, putting just enough illumination on the ground cuz we can measure the luxes, and that’s part of the computer modeling, so we can measure how bright it is with respect to the external environment, whether that’s the time of day or whether those are cars, but again, we look at the lighting of a smart city, and we have to give it a certain type of outcome, so we would have a digital twin, which again consists of the raw data from those light ,light post, lamps, and then the computer models that we produced from that original . . . from that raw data.

Now each product, system, or environment can consist of multiple digital twins, so there’s different outcomes that you’d want for the lamp post—one outcome would be safety, and that’s one that we mostly focus on, but the other one could be just strict illumination. Another one could be, let’s say, providing other services within the city because you have the real estate, so providing other types of services such as free wifi.

Curtis: And by real estate, Bruce means being able to use the space on the pole for adding wifi components.

Bruce: So we always start with a value proposition. We then quantify that value proposition with a data model, and then we, from that data model, we understand the data that we need to capture, and then from understanding the data we need to capture, we understand the technology that we need to put behind it to be able to capture that data to create that information, to create that value for the product, system, or environment.

Ginette: For another familiar example of the digital twin concept, we can turn to Elon Musk.

Bruce: It’s a relatively new and I would say cutting edge concept, but at its highest level is really just a data construct. A good one that most people would be familiar with is Tesla, so Tesla uses a digital twin, and in fact, if we go up one level, a digital twin plus an application, the executable application is called a software defined product, so a Tesla is really a software defined vehicle, and I’ll give you two examples. In 2015 the US traffic and safety administration did a recall for both Teslas and for GM trucks, and in both cases it was due to sparks that were being created when you were doing charging, and since charging station where sometimes near gas stations, petrol stations, then that was an issue. So GM, they had a recall 360,000 pickup trucks, and if anyone listening has received one of those dreaded notices in the mail like I have, that’s like such a super nightmare to have to go through. Plus for GM, it’s so costly. Just think about that.

Tesla, because they have a software defined car, which consists of this digital twin in the application code that executes it, they just made a modification to the software, and in doing that, they actually redefined the definition of a recall, and now recalls of physical objects can be just software updates.

Curtis: And if you think about, if recalling a product from your company means physically retrieving hundreds of thousands of cars, that’s going to cost orders of magnitude more than fixing a software problem on a digital twin and then deploying that change to a car through airwaves.

Bruce: And so Tesla, that’s one example, then another example is how they continuously add new features. If it was a hard-wired car, if it was just a bent metal product, you wouldn’t be able to update the features in it, so autopilot, for anyone that’s heard of it, this is the precursor to their fully autonomous driving vehicle technology, and what autopilot does is it it takes data—it first drives from the the digital twin that’s been created, and then it augments that data or improves the model, I should say, from the reactions of drivers who theoretically are supposed to be holding on to the steering wheel, and it looks at the situations, and if the driver intervenes, then it uses that data to improve the digital twin, so using a digital twin, this really is the source of all competitive advantages of a Tesla over their competitors, and the reason is they can update it. This is done through something called a over-the-air update mechanism, and they can use that from a security point of view, so that was the case of the recall, but they can also use it for features and functions, and just as an antidote, Tesla’s, just to kind of reinforce the notion of a software-defined vehicle or digital-twin-based vehicle, they’re all they’re all software version numbers, so it was 7.1, I believe, when the recall happened, and we know all the way now we’re into the 8.x versions of the model S is what I’m talking about in this case. They’re numbered as software because that’s really how they think of the cars. They’re software, they’re analytics, they’re artificial intelligence, and it just happened to be encase by a physical object. So that’s a great example of a of an IoT product that uses a digital twin.

Ginette: Tesla’s just one of many, many examples of how digital twins are being used by companies today with IoT products. In the consumer space there are smart doorbells; speakers, like Alexa; thermostats; lights; window shades, and these can all work together as in-home systems get more complex.

Bruce: Now where AI, and in particular machine learning, and then in particular neural networks, and then in particular deep learning neural networks, where they apply is mostly in this in this model making, so with IoT there are two types of models for the digital twin: we have the analytical model that’s created through more analytical techniques, and then we have the cognitive models that are being created through machine learning and artificial intelligence techniques. I kind of like to separate the two, but the the impact in both cases are profound, so for IoT and AI, we can look at pretty much any device because, again, it’s like software so whether just using two devices that we talked about already in our discussion would be the Tesla, and they’re using artificial intelligence in terms of deep learning to understand how a drivers are reacting to the original models and so this is the way of improving those models, and then with Alexa, again, it being being more of an input device, it’s using machine learning to understand in in the form of natural language processing to understand what people are shouting at at it in terms of questions, and also being able to try to decipher or tease apart the vocal of the human from the ambient noise in the background, and it does an amazing job, and again these are two examples where artificial intelligence again mostly machine learning and again more of these deep neural networks are being used in IoT to create very strong, or I should say very, very accurate models because all the value and IoT comes from these models and interrogating the model, so all incremental value from an IoT product comes from transforming the data into useful information and that transformation starts at the analytical and at the more and more at the AI base in transforming that data into useful information.

Curtis: Beyond items we buy for ourselves as consumers, which are the things we hear about the most because they impact our personal lives and are very visible to us, one of the biggest sectors benefiting from this technology is industrial IoT, or IIOT. But even beyond IIoT, there are tons of innovations happening right now across almost every sector.

Bruce: One area where combining AI and IoT has made a big difference is in the area of surgical tool creation, and when we’re looking at surgical tools, we can use, again, it all comes down to the same thing, we can create value by creating models. Now the different types of models that we create from a surgical tool point of view can be as simple as battery life, so we don’t want to go into a surgery without having a good idea of whether our tools are completely charged, but then it gets more interesting, and one area that I find very interesting, or one one particular use case I find very interesting, is in the area of hip replacement, and there’s work being done today where models are being developed to understand how the tools affect the bones, the cells of the bones. So in the case of a hip replacement surgery, there needs to be a semispherical hole that’s drilled out of the pelvis to put a cup, and that’s one half of the artificial joint, to attach to the femur, which is going to be a ball joint that’ll be another metal, which would be the second half of the artificial joint. Well, it turns out one of the biggest problems in performing this type of surgery and any type of surgery when your heating up bone cells, whether you’re drilling them or whether your sawing them is a process called or phenomena necrosis, and necrosis means that just like when your drilling a piece of wood, if you let the drill drill for long enough, you’re going to see some smoking, and it’s going to burn. Well, that’s what happens with these tools when they’re drilling or they’re sawing on on human bones, and so we can model with an analytical model, and in some cases, an AI model or machine learning model how these bone cells burn out, and if we can accurately model how these bone cells burn out, then we can make modifications on the fly in real time to the surgical tool to prevent that from happening. So the surgeon, no matter how skilled they are or how unskilled they are, they would in this case be able to use a tool but never be able to burn out the bone cells. Now the issue of burning out the bone cells, it creates an irregular surface when you put that that semispherical cup into the pelvis, and if it’s . . . if some of the bone cells are dead and some of them are alive, you can see some of them become a little rough, and this is the biggest problem for redress surgery,

Ginette: That’s a surgery to attempt to fix the problem again with another surgery.

Bruce: so by using an IoT, and the tool is called an acetabular reamer, by using IoT for the acetabular reamer again we are quantifying the value that is producing and in this case, it is to prevent necrosis and so that way we can modify the speed of the drill head. It’s kind of like a drill with a cheese grater kind of drill bit on it, and we can modify the speed so that no matter what the surgeon does how hard they push on it and for how long it spins, we can control the speed of how fast it rotates so that necrosis never happens, and so that’s a good example of kind of tying together the digital twin with cognitive models with analytical models to produce something that at least in my view is pretty is pretty cool.

Curtis: And as with any analytical model, there are a lot of thing we can learn from this use case over time.

Bruce: It’s just like any analytical model. We’re now, again, because we’ve virtualized the problem, we can treat it . . . we have a certain configuration for our tool, and in this case it would be rotational speed, it would be pressure and time. Those would be the main variables and then the the one that we’re measuring is temperature, so those are, those are the three variables that affect the temperature and by cause and effect by looking at the different considerations, then we have to then see well the effect being, okay, one day after the surgery, what’s the pain level? Two days after surgery? Seven days? One month? One year? Two years? Five years? So this isn’t a quick process, but just like in any analytical or AI based dataset, the more data we get, the more valuable that the model is, and then the more accurate the model is the more value that it produces for its owner, and so yeah, it’s measuring cause and effect, the different configurations, and then using it to learn what are the best configurations for and it may not just be, again, the pressure, the rotational speed, and the time, it could also be then bring a demographics like the age of the patient, the sex of the patient, do they smoke? Do they not smoke? Are they fit? Are they not fit? Where do they live? And as you know, I can just go on and on.

Ginette: If you want to delve more into IoT, Bruce is a fantastic resource, and here’s some information that’ll point you to his resources.  

Bruce: If anyone’s interested to understand how the Internet of Things works and how to apply to business, a good place to do is you’re listening to a podcast now is to check out my podcast. Just do a search on IoT business, and it should come up. It’s the IoT business show, and if you want to have all the information in one place you can go to IoT-Inc.com where I have all the podcasts, where I have a lot of videos, where there’s a number of articles, and all that information is available for free. If you want to go a little bit further, you can go on Amazon, you can search for my book—it, too, is called IoT Inc., so that’s easy to find, and if you want to get certified as an IoT a certified professional, a certified IoT professional, then you can check out the ICIP certification program, also can be found on www.iot-inc.com.

Curtis: For any of these links, you can go to datacrunchpodcst.com, and you’ll find the links at the bottom of the show transcript, along with the music attribution to Kevin McLeaod. And a huge thank you to Bruce Sinclair for being on our podcast. If you like what you’re learning here with us, please share our podcast with your coworkers and friends and go to iTunes or your favorite podcast playing platform and leave us a review.

Bruce’s Resources

Website: www.iot-inc.com

Podcast: The Internet of Things Business Show

Book: IoT Inc.: How Your Company Can Use the Internet of Things to Win in the Outcome Economy

Music Attribution

“Cold Funk” by Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
http://creativecommons.org/licenses/by/3.0/

Picture Attribution

Photo by Marc-Olivier Jodoin on Unsplash