The Good Fight against Shadow IT

Simeon Schwarz has been walking the data management tightrope for years. In this episode, he helps us see the hidden organizational and economic impacts that come from leading a data management initiative, and how to understand and overcome the inertia, fears, and status quo that hold good data management back.

Simeon Schwarz: Fighting against shadow IT . . . you have to find a way to adopt it, you have to find a way to incorporate it, and you have to find a way to leverage it. You will never be able to completely eliminate it.

Ginette Methot: I’m Ginette.

Curtis Seare: And I’m Curtis.

Ginette: And you are listening to Data Crunch,

Curtis: A podcast about how applied data science, machine learning and artificial intelligence are changing the world.

Ginette: This might come as a surprise to some, but……tools won’t build a data-driven culture.

The right people will.

Read more at mode.com/datadrivenculture. m o d e dot com slash data driven culture.

Ginette: Today we speak with Simeon Schwarz. He’s been working in data management for over twenty years and owns his own consultancy, Data Management Solutions.

Simeon: Being in the data management function, you’re de facto seeing the life blood of how the business flows, how the uh, where the information goes, how the decision are made.

Curtis: So have you been focused mainly in a, in a specific industry or have you spend a lot in your career?

Simeon: I’ve started in telecom. I’ve built first cell phone carrier back in my home country. I worked in academia, in a retail, ecommerce, and then 10 years in financial services, most recently, and now I do insurance. So a lot of different fields.

Curtis: So you’ve run the gamut. That’s interesting. And now that you’ve done this in several different fields, do you find that the principles and your approach is basically the same or or is it different depending on the problems that you’re trying to solve?

Simeon: The approach is the same, and there are two parts to this. We’ll talk about what’s difficult in this role a little bit further in this conversation. The second part is you really need to understand the domain you’re dealing with because, one, if we, if we’re talking about data management in general, one of the key functions, one of the key challenges that you’re going to be facing is establishing and building your credibility. Without knowledge of the domain. B insurance or financial services or manufacturing or any other field, you simply can’t have intelligent conversations with your stakeholders in a way that would lead to good conclusions. So you will absolutely have to know the domain, which is large portion, of your value.

Curtis: So as you’ve gotten into a domain that maybe you weren’t as familiar with in a data role, how did you overcome this need to understand the domain better?

Simeon: Let’s step back and talk about what a data genuinely is right now and specifically talk about data management. You are running a data function or sometimes called data services because what used to be DBA teams or data analysts or various forms is really becoming a practice and looking at it as a practice. You have a certain set of clients, the are paying you for the services, you have certain amount of resources and you trying to optimize those resources to serve your clients better. So what are the challenges that you’re going to face in any data management role? So you’re in this interesting balance between moving forward very rapidly as well as not destroying what already exists, not destroying the services that are already provided. People have to breath, people have to be able to, to leave. You can’t disrupt too much the services that already exist, your reports, your, you know, our auditing work your work with, you know, regulatory agencies. Anything else that the business needs to produce has to continue to happen. The people who are doing their jobs in the current way similarly have to continue to be able to do it in some way. So it’s a very tough challenge in a way that when you are doing that role and data leadership role, you’re finding how how much can you put the slider and you knowing very well that both sides are going to be unhappy with you, and you haven’t really replaced all the pains, all the challenges, all of the issues and both sides: are you moving too fast, you’re not moving enough and you just need to know that heading in.

Curtis: From your experience, how did you balance this and overcome this and then be successful with that dynamic in something that you’ve done?

Simeon: It comes to credibility. It comes to being able to influence. It also comes in a lot of ways. It’s management. So, yes, there is technology management, there is data management, but there’s also management management portion of it. People who are working in your accounting and finance and other secret groups who are de facto doing data management work as analysts or as accountants or as you know, finance specialist who gets who, who face data questions, very frequently, their job is to take one Excel spreadsheet that you produce and another and the third one and merge them together and pivot them out and come up with an business answer to a business question you, which you may never actually see. You see a request coming in, “please give me this information.” What you’re seeing a portion of this picture, you don’t know what they’re doing with it or what question they truly answering and one of the first things you’re going to do when we talk about shadow IT, you have to earn their trust and you have to get over their fear that, “hey, for the last five years I’ve been producing this quarterly file for my CFO and here is this hot shot coming out of college, frequently junior to me in age, who says that they can automate all of it and they are going to automate me out of a job.” Let’s, let’s be very kind of upfront and honest with it. Almost always the problems you’re facing are not technology problems. So here is the most basic example that I would face coming into. Let’s pick on financial services and let’s, let’s say it’s a brokerage and let’s say it’s a very, very simple brokerage that that does nothing but open accounts and you know, runs transactions, collects condition. A person comes in and says, we absolutely need to understand how we getting accounts, how much we’re spending on their accounts, what those accounts are doing, who should we keep, who should we not keep? And you sitting at the table with all of the key stakeholders and you ask them a basic question, “okay, let’s talk about an account. What is an account?” And the marketing person says, “well, it’s very simple and account is a converted lead.”

Say, “thank you. Finance. What’s an account?” And they’re going to say, “Well, an account is, it’s very simple. It’s a person who can trade. If they can trade they’re not an account.” So we converted lead in account. Now they have to fund. If you don’t have the money to pay commission had to buy a products, you’re not an account.” And you’re going to get many different definitions and all of those definitions are correct and all of those definition reflect a viewpoint over a specific business set of business processes within their organization. And then you go into turn to your stakeholder and you’re going to say, “I can build you any sort of data warehouse and analytics and model, but we have to have one common definition. When you go into ask me, hey, um, tell me how many accounts we’ve opened, which account definition want me to respond with?”

And the major challenge is not in just definitions is the longer you delay this, the tough other reconciliation is going to be, the more you have to restate, the more risk you actually taking on. And the toughest things to do is not at this point, is not to sweep things under the rug, is not to tell me well prepare me a system or a model or a data warehouse or a solution that will have four definition of the accounts, which is the typical response you get. So that’s a very basic example, but almost always when you look at the major data challenges, your challenge is not going to be technology. Your challenge is going to be facing these things and saying, “Marketing, I understand you want to track your ROI that way, but it’s really meaningless because we can’t count the people who can really trade and yes, that lowers your numbers and yes, that decreases your bonus.

Let’s be honest, and yes, that makes you look quote unquote worse, but that’s not, you know, that may not be the right definition.” So you actually in the middle of massive amount of information flowing that you are helping leadership reconcile and very, very, very quickly, almost instantly, you either become a trusted advisor or you find that you are in the wrong role.

Curtis: What do you mean by that?

Simeon: If you don’t have credibility, if you can not explain to them you and convince them that this viewpoint is worth considering, you will not succeed regardless of how well you know algorithms and technology of what great program you’ve, you’ve listened to, or how good are you with a tool? If you’re going back to the account definition, if your definitions from the start have contradictions and you’re going to get two different numbers to the same questions, two different answers, no amount of technology will help you. You will fail.

Curtis: Time to find a new position in that case.

Simeon: Time to find the new position. You have to win them over. You have to convince that you have to make sure it makes sense for them. In a lot of ways you have to deal with fears which are there, which you will never, they will never actually raise, but they’re still present. If, if my job as a financial analyst is to prepare five Excel files a month and here is a person that’s coming in and saying Power BI or Tableau or whatever presentation layer and automation can completely replace this and they would be available at whim to my executives who have delivered this fast previously, what will I do as I’m sitting in this meeting and listening to this data, data, person data that is depending on . . . regardless of what the name of the role is that that will always go for the back of my head.

Curtis: Do you in that case try to say, Hey, we’ll train you to do other things or what do you, how do you approach that?

Simeon: You talk about different, different value and different levels of value that is provided, and yes, the training is a component of it and being able to focus on on on different solution and there is untold amount of challenges that the person can go to. In the back of all of this, iIn all honesty, very frequently when you look at this, you will see that specific groups are likely overstaffed, and you will see in a lot of, it’s not your fault, it’s not your credit, it’s not good. It’s not bad. It’s just how things work really. And in a lot of ways the decision is not in your hand if you have 15 people who have all been produced, this one file because the data was so extraordinarily dirty that every row in the data had to be manually verified.

And the company decided that, “you know, we heard this cool thing called data quality and data quality management” and now it actually comes in a lot of ways built in and introducing data quality management increases the, the value and the quality of this data. You may not need those people and, and that’s the reality. So what do you do then and how do you deal with this and how do you find, you know, graceful transitions or find some way to dealing with it that that’s a lot of data management challenges, which are not usually described in the books of algorithms or you know, how do you build the system or the right data warehouse design pattern,

Curtis: Right. Right. What happens when you build a system, people lose their jobs. All these kinds of the politics, the economics,

Simeon: The politics and the economics. And in a lot of ways what you are doing is you’re solving this complex multidimensional puzzle that has technology as a dimension. It has timelines, it has risks, it has costs, it has resources, you know, people, their skills. None of your dimensions are fixed in any way. People Change, technology grows, risk risks also change. Timelines always change. Your funding could be increased or removed and new things come out on the market. So you, you’re working with this puzzle making and you’re trying to find the right combination for this moment knowing very well that um, three months, six months, 12 months from now, that may not be the best solution. So one of the ways, you know, I look for people, I recruit people and similarly in this field, we’ve started talking about willing solve problems. The other major part of your role, are you willing to be wrong? Are you willing to make decisions knowing that they could be wrong or they could be viewed as wrong because if you’re not, this isn’t role for you. If you’re thinking clear caught yes, no black, white, you know, absolute certainty, I will not make a mistake if you’re afraid of it, data management and data management leadership is the wrong role for you.

Curtis: You’ve mentioned shadow IT. And I really wanted to, to get your opinions and thoughts on that as well.

Simeon: So shadow IT is going to be in every organization you are going to join and visit, and it’s defined multiple ways but it’s it’s activities that are happening defacto in organization to support various needs primarily in the data space. It impacts you as a data management data manager or you know data management leader more than anyone else because analysts can go out and write c sharp code, but they can very easily grab a piece of data and throw it into Excel as one of the examples and manipulated and come up with quote unquote solutions. Fighting against shadow it . . . you have to find the way to adopted, you have to find a way to incorporate it and you have to find a way to leverage it. You will never be able to completely eliminate it. Just and it all comes to time to market. No amount of information, no IT solution, no IT methodology can give you instant results.

Yes, we are going to talk about self service. We’re going to talk about other solutions but very frequently business need is a business need, and “we have a request. It’s for the meeting tomorrow. We need to know how many . . .” the rest of the question follows. This is not something you can spin in their sprint. It’s not something you can launch a project for. You just need to get it done and business typically gets this done and gets this done faster. What happens next is one-time need becomes a recurring need. Hey, you’ve produced this numbers before. It would be good for us to know what it is right now so we can do comparison. Sure. Oh, you’ve done the twice. How about now get it weekly. Hey I’m, you’ve done it weekly but this week is missing. Oh I’m sorry. You got married, you had, you had, you had the child.

You’re not allowed to have any of that, and typically when you talk to a, to your, to the analyst who are quote unquote on the other side, you very quickly show them that progression that they’ve seen and you keep, you are what you are telling them is, look, I’m not your enemy. This is how it’s going to go and you know how it’s going to go. It’s going to be a manual operation that you’re going to do once, then it’s going to repeat, then it would be expected that you will add it to any other workflow that you’re going to do forever, and it’s actually, it’s actually a piece that you will now own and you can’t have time off. You can’t have any other activity because you are busy running those spreadsheets and you’re busy putting them together and the fact that you did 51 of them last year without any issues, nobody’s going to see but one you paste in the wrong area and it created the wrong results.

Everybody’s going to point to. Do you want to be in that world? Do you want to be working over Christmas break? Do you want to come here on January 1st because this is when you have to pull the data and the things. Think about it, and usually you have heads nodding. The other thing you keep very directly tell to people is if you cannot be replaced, you cannot be promoted. If you are sitting there. When you curating those spreadsheets and you have any career aspirations, you actually sabotaging yourself. So let me give you an environment. Let me give you tooling . Let me give you a process to let you play in that area. We can call it prototype, we can call it rapid development. We can call it self-service, we can call it any any way you want, but you as an analyst will always have ability to do what you need to do with some help from me.

I, then, am going to be here to help you when you’re ready to take quote unquote prototype to the next level, your requirements may change. You may produce something for your stakeholder a and they’re going to say, you know, that’s really nice, but I’m looking for this. Or can you do it this way or that way? After a couple of iterations, guess what? You just developed requirements. Congratulations. Thank you. Let me take this off your hand. Let me automate it. Let me still keep you as a SME subject matter expert and make it easy for you to to get this results and take the brunt so you can go to your son or daughter’s soccer game. You can actually take vacations. You don’t have to wake up at 4:00 AM every Sunday because you need to produce all of this. That is how you can, one of the ways you can embed shadow IT and how people understand that they can let go of quote unquote job security and use the tooling, still do their job and then transfer it to me for quote unquote automation.

What actually happens behind the scenes would be the actual implementation. It may not be automating what they do. It would be thinking of the process that they are following and then thinking of how do I do it at scale or how do I, I’ve heard from five different people in five different groups but actually working on the same problem. Can I solve the problem for all five of them? But they don’t know each other. They they way with each other and they meet at the, at the kitchen. But they don’t know they actually doing in the same thing. So how do I implement that? How do I incorporate it all requiring domain knowledge? If you don’t understand the business question they’re trying to answer, you have no hope of of helping them. You will find data management roles to be incredibly rewarding because you can solve real things. You can solve real problem. You get a lot of respect from all throughout organization. You becoming the trusted advisor that is providing answers.

Ginette: A huge thank you to Simeon Schwarz for chatting with us. For attributions, you can head over to datacrunchcorp.com to our shownotes.

Also, this might come as a surprise to some, but……tools won’t build a data-driven culture.

The right people will.

Read more at mode.com/datadrivenculture. m o d e dot com slash data driven culture.