The Hidden World of Data Science in Utilities

David Millar is a man bringing analytical solutions to an industry that historically has had little data. But with the explosion of smart devices, that is all changing, and the way utilities operate is as well.

David Millar: The way that electricity markets work is that you have what’s called the day ahead market. And so the day before, let’s say one o’clock tomorrow, markets run, and this is a big optimization problem.

Ginette Methot: I’m Ginette

Curtis Seare: And I’m Curtis

Ginette: And you are listening to Data Crunch,

Curtis: A podcast about how applied data science, machine learning and artificial intelligence are changing the world.

Ginette: Data Crunch is produced by the Data Crunch Corporation and analytics training and consulting company.

Ginette: The father of lean startup methodology once said “There are no facts inside the building so get the heck outside.”

The utilities industry is no different. Sometimes the facts that’ll make your machine learning career are waiting just outside your office.

Read more at mode.com/MLutilities. m o d e dot com slash M L utilities.

Ginette: David Millar is a man bringing analytical solutions to an industry that historically has had little data. But with the explosion of smart devices, that’s all changing, and the way utilities operate is as well. Let’s get into it.

David: I’m, ah, Dave Millar. I am the director of resource planning consulting at Ascend Analytics where I lead the research client consulting team. And so my team and I work with utilities primarily to help them make decisions using analytics, regarding their longterm power portfolio. So primarily I read looking at we’ll say we’re retiring coal plants or retired, retired gas plant. What would we replace it with? Renewable energy. We need batteries. How do we approach these questions using analytics in order to help us come up with the best solution going forward.

Curtis: You had talked a little bit about, you sent me some notes about how the, the sector that you’re in, the power sector, you know, is kind of slow moving, right? It’s not known for these quick changes and innovations, but you are starting to see some things that, that’s gonna change this fundamentally. And so if we could jump into that and, and then get your perspective, I’d love to hear about it.

David: Yeah, the power sector basically didn’t change from the time of once they figured out that we’re going to use alternating current that it didn’t really change much in the past hundred years, that the model is essentially the same. You have big power stations that are far away from the load centers and then you have this transition network and flow of electricity is really one direction, right, from, from the big power plants to your home. And technology is rapidly changing that and it creates a space to becoming both more digital and more decentralized.

So, on the digital front, we, we actually have generation technologies, that don’t use anything, any spinning parts, right? so you have solar, solar power, and you have, now we’re seeing more and more batteries being connected to solar. And so those are both digital technologies that are increasingly becoming this default, energy source, wind or solar and batteries and and just because the cost of the signals is have, dramatically over the past 10, 10. It’s really happened over the past 10 years. And so now renewables are at parity with the more conventional sources of electricity. So gas, power and natural gas power, coal power.

Curtis: Is that in terms of like how much energy they’re currently producing parity or just effectiveness or efficiency. What is that parity?

David: Parity in terms of costs. So, you know, as renewables drop in costs, especially as batteries drop in costs, that means that when, when I look at a problem with my clients, we’re comparing, technologies that essentially have the ability, similar attributes, but the renewables are now becoming the least cost resource.

So the, you know, the problem with renewables of course, is that they only generate when the sun shines, the wind blows. And so, you know, you have to be able to integrate those renewables into electricity system that really demands sort of a, an exact match, of generation and load at all times. And so this is, this is why electricity is a very special commodity. there’s really very little storage on the system. So that’s the challenge of integrating renewables is that, you know, we need to, we need the power when we need the power. So we need to add a lot more storage on another system

Curtis: Just to touch on that point. So this problem of it’s intermittent, right? It’s like you don’t know when you’re going to get the power from wind or solar necessarily. and maybe you were about to go here, but that brings up an interesting data science question of how well can you forecast that?

David: Yeah, that’s right. Right. So there’s the, the way that electricity markets work is that you have what’s called the day ahead market. And so the day before, the day before let’s say one o’clock tomorrow, there’s a market markets run and this is a big optimization problem. And this, this is self, this is again, sort of a newer development in power markets because of the computing and the ability to, to run these optimizations or the, an entire electric system. And so that’s sort of a, something that’s kind of out of the last 10 years as well. But same as you have this day ahead market. And then we have to use, basically forecast for the next day in order to figure out what we think the load is going to be and what we think the amount of generation is going to be from the renewables, which is, you know, again, a function of weather, right?

And so the ability to better predict the weather and then how the weather affects the generation of renewables will help the system operate more efficiently. And so that’s what we, what we ended up seeing is, is that we miss a lot. So weather forecasting and power production forecasting from, ah, renewables is still where we’re a place where we have a lot. We can get a lot better and sometimes we miss by a very large amounts. And then what ends up happening is that in, in real time markets, so markets are run 15 minutes prior to, you know, prior to the operating interval, will have to pick up the slack. And so you’ll have to have you know large, ramps from, resources that are dispatchable. And that’s, but it’s basically a lot of value that can be captured by better forecasting. And we’ll help, folks who are maybe operating batteries, better capture the price spikes that come from Israel time markets.

The, the volatility that’s driven by renewables is renewables already intermittent that volatility gets manifested in the realtime markets. And so, which what that turns into is basically you have these periods where you get these spikes of prices. So you’ll get, normally prices are going at $20, $30 a megawatt hour, but then some, when you have the say let the renewables are generating less than you think. And obviously the system needs a lot more energy. I don’t get a accusation, we’ll make a price by a for $1,000 to $2,000. And so if you’re, if you better able to predict those price spikes, then you can do, you know, have your battery fully charge be ready for that price spike. And then when it comes, you deliver. And then you capture all that value. And so they are, that’s something that we’re, we’re actually working on in order to, you know, try to better predict the probability of a price spikes or to capture that value.

Curtis: And maybe we can dig in a little bit into the data science here before we move on to a lot of other points you have. But I’m curious, how do you approach solving this problem, right. You’ve, you’ve mentioned the forecasts aren’t, they’re not as good as they should be or need to be. How are you approaching getting this done?

David: Yeah, we can, we can look at the data. We know that there are certain patterns we can find, like in the summertime, in the afternoon as the system is at its max in terms of load, but then the solar, it at the end of the day the solar drops off. We know that there is going to be a cluster of price spikes in the hour. And so it’s, it’s really trying to use these data science techniques to, in order to understand patterns and in these price behavior. And in order to create basically a bidding strategy with the ISO in order to try and maximize that value.

Curtis: So we’ve talked a little bit about the supply side and the economics there and kind of what you’re doing. You also mentioned that data is changing the demand side, which is an interesting thing to think about. What, what’s going on there?

David: So it used to be that you’d have, you know, a meter reader would come to your house and then with your meter in the main, you know, measuring concentricity used and you would have 12 data points per customer per year. And then with smart meters, basically you can get that to a meter read every hour. You can, sometimes it goes down to every 15 minutes. And so we just have this, you know, big data, this, this proliferation of data now that we can work with that wasn’t available 10 years ago. And so, you know, utilities have taken awhile to figure out how to use this, but there’s a lot of interesting things that are, that are starting to come out now. So, you know, one is do you look at what is sort of known as the grid edge. So this is putting smart appliances into their homes.

Smart thermostats, you know, could be smart hot water heaters. There’s going to be smart washing machines and smart, you know, everything is smart refrigerators. So this is going to give the ability for the utility to actually send a signal to your house and manage the demand, your demand for electricity. So this is, this is basically another resource that they’ve added to their toolkit in order to better manage the system, reduce costs, cause most, you know, most of the costs for electricity come from the very peak, you know, the peak output of your load, of your demand. So if they can go in and you turn down your thermostat, just for a couple hours, and then they can actually save the system a lot, like a lot of money, and then pass that savings on to you. There’s some research going on in something called transactive energy, which is basically, you know, look, thinking about how to use, you know, blockchain, in order to, compensate homeowners, or businesses that, you know, may have their, they have solar panels on the roof, maybe they would have a battery, maybe they have, you know, these smart appliances and can we have them produce electricity or, or reduce electricity, at the certain times, and then compensate them based on what the actual conditions of the grid are at that time.

So, you know, this is kind of a interesting, space that’s starting to develop, you know, the challenge in the industry and it industry is that, you know, press, unlike tech, it’s, it’s very heavily regulated, and so it’s to, to get things to change, quickly. It’s, it’s tough to do, but there’s something like, you know, transactive energy, you know, could have the potential to really disrupt, the business model over and over, you know, in the next 10 years I’d say.

Curtis: That’s interesting. And you had mentioned a little bit in your notes about the challenge of trying to get some of this new approaches pushed through past regulators, so typically, you know, the, these regulatory bodies aren’t staffed by data scientists and so it’s kind of hard to, to speak the same language. Right. And so how do you, how do you overcome that?

David: Yeah, yeah. No, especially in electricity, yeah, I mean, I would, you know, encourage folks interested in data science to look at working at a regulator. You know, California, there’s the, you know, the CPUC and you know, every state has as a regulatory body and they more and more are going to need, folks of that expertise. And you know, working at a utility is actually a really great option as well. And cause utilities are good at keeping the lights on, but they aren’t, you know, have deep expertise in data science. And they’ve really been relying on outside firms to, to help them. But they absolutely need that expertise in house.

Curtis: No shortage of data science problems to solve there, it sounds like. Are most of them actively hiring for this kind of talent? Like is it hit yet that this is going to be a big transformation and they’re looking for that or are they a little bit still hesitant to do that?

David: No. I’d say, well it somewhat depends on where you are. But I certainly with my experience with PG&E that they were looking for those sorts of folks. Absolutely. And I think most any major utility across the country is gonna be looking to have this expertise. And just because there’s so much, again, you know, the grid is becoming much more digital, more big data to manage and to use to provide value. What’s been the bottleneck is not that we don’t have data, it’s that we just haven’t known what to do with it yet. And that’s really kind of, I think the challenge of the next five to 10 years is to figure out how to leverage this to make a more efficient system that’s integrates, you know, that renewables and empowers the customer in order to have better control over their energy use and contribute to the energy system and be compensated for it.

Curtis: A big part of data science is, you know, there’s obviously the technical side, but also just the creativity of, of trying to figure out here we have all this information, all this data, like where’s the value and how do we get value out of this? And you’re working, like you said, in an industry where there’s a lot of unknowns, right? And so how do you approach the creativity aspect of coming up with, well, what should we work on? Where is the value in all of this data?

David: You know, I, I, from my perspective, I just try to understand the macro trends and try to think about where do we need to be? You know, five years down the road, 10 years down the road, and where, where is technology change? How’s technology changing? And that’s part of the challenge of being at this particular juncture and time is that the technology is changing so quickly. We, we’re in an industry that usually is used to planning 30, 40 years in the future. And so we’re always asking, you know, where are we gonna be in 2030, where are we gonna be in 2040? And, you know, it’s, it’s tough when, when we’re like, I don’t even know what the primary dominant chemistry in the battery’s going to be. oday’s lithium ion, but it might be something else. And in near future, yeah, we’re really embarking on this great unknown of this transformation from a, you know, fossil fuel based electricity to renewables down electricity system. And so this is, you know, I try to look at these macro trends and say, okay, how, how can we use data? How can we understand the dynamics in the market? How is it changing and what, you know, what can we use to help us, tackle some of these issues that we see coming up.

Today’s lithium ion, but it might be something else. And in near future, yeah, we’re really embarking on this great unknown of this transformation from a, you know, fossil fuel based electricity to renewables down electricity system. And so this is, you know, I try to look at these macro trends and say, okay, how, how can we use data? How can we understand the dynamics in the market? How is it changing and what, you know, what can we use to help us tackle some of these issues that we see coming up.

Curtis: So instead of operating maybe on a, on 10 year scales, it should more be maybe two to three year scales or something like that. Given the speed at which technology is changing.

David: At this point, it’s actually, if you’re a utility, it’s typically best not to be a first mover.

Curtis: Oh, that’s interesting. So is there, is there some economic pressure to not be the innovator then?

David: Yes, absolutely. I mean there’s, there’s a lot of pressure to, well, we should wait until 2025 when the batteries are half as much as they are today.

Curtis: And speaking of batteries, you would also mentioned an interesting point about electric vehicles and how those might be able to be hooked into the grid. Can you talk about that a little bit?

David: So the electric vehicles are interesting because they are basically a distributed battery and, for, if people are gonna be buying these for their transportation needs, most of the time they’re sitting plugged in and not being used. Well that is a potential resource that again the infrastructure and the algorithms to manage a distributed resource like these batteries ah to maybe maybe charge or discharge. In order to follow the renewables, is a really great, is a really good opportunity. In so far there’s, it’s sort of experimental, a lot of manufacturers of vehicles, aren’t that psyched on it. Because when you cycle batteries you are, you know, wear them down faster. And so that is, you know, that that is an issue that you know, is, is, is slowing that idea down a bit. But after, if you get a certain penetration of these electric vehicles, this really starts to be of a realistic option.

Curtis: How would you approach that from a data science perspective? Trying to manage all these distributed resources and figuring out, you know, when to discharge and when to not, and prices. How would you approach that?

David: Yeah, I mean that’s, that’s like the big data science question, I think, for the next five to 10 years is how do we get more insight into this distribution grid? And in terms of the, you know, let’s say we’re trying to understand where are the power flows happening? Where is the congestion? What’s going on with the energy supply? You know, what’s going on with the rooftop solar in this neighborhood? Being able to capture that data is, is gonna be a process of, go the, the distribution grid does need to get smarter. And that’s something a lot of folks are working on is being able to capture the, that data from the distribution grid.

So, so the first part is it’s just getting the data, getting that the next part is, okay, so how do we optimize, all of these, you know, instead of, you know, just a few resources, it’s hundreds of resources or thousands of resources. And again, that’s, that’s going to be a function of, the data availability and the computing power. And then the algorithm are to be able to sort of orchestrate all of these resources together that is the really the least cost, and the most of them, you know, and reduces greenhouse gas emissions the most.

Ginette: A big thank you to Dave Millar for being on the show. As always head to datacrunchcorp.com to see our shownotes and attributions.

And remember, the father of lean startup methodology once said “There are no facts inside the building so get the heck outside.”

The utilities industry is no different. Sometimes the facts that’ll make your machine learning career are waiting just outside your office.

Read more at mode.com/MLutilities. m o d e dot com slash M L utilities.