Listen to this episode:
About this episode:
In this episode of Behind the Data, Matthew Stibbe interviews Alan Moss, Director of Data Strategy at GrowthZone. They discuss the evolving landscape of data storage, the challenges of data wrangling, and the role of CloverDX in managing data effectively. Alan shares his personal journey in the data field, the importance of understanding membership data for associations, and how technology has enabled better insights and reporting. The conversation highlights the significance of adapting to new technologies and the continuous learning required in the data management field.AI-generated transcript
Matthew Stibbe (00:01) Hello, welcome to Behind the Data with CloverDX. I'm your host, Matthew Stibbe, and I'm here with Alan Moss, who is Director of Data Strategy at GrowthZone. Great to you on the show, Alan.
Alan Moss (00:13) Thank you. Glad to be here.
Matthew Stibbe (00:15) So before we dive into your world, let's start with a really practical question. What's happening in the world of data right now? What emerging trends or technologies have got you excited?
Alan Moss (00:27) Well, I think the biggest thing for me right now is just the expansion of where you can store data, how you can store data. We've grown from just being, having just a SQL database that had one database to multiple databases on a cluster to now data lakes inside of Snowflake. I think the biggest thing is the ease of storage, how much you can store and where you can store it and just how easily it's available to you. But I think the fact that all of that has just grown so much by leaps and bounds over the last few years.
Matthew Stibbe (01:06) Yeah, it's extraordinary, isn't it? I remember the story about Bill Gates when he was working out the original sort of operating system for MS-DOS. And it had a limit at that point of 640 kilobytes, like that was the maximum addressable memory. And he got to that number by going, right, 64 kilobytes is the most anyone could ever possibly need. And I'll multiply that 10 to give a fudge factor. And now it's just sort of gone bazillion times that.
Alan Moss (01:17) Right. Yeah.
Matthew Stibbe (01:36) But that's been in our lifetime. yeah, and you've told me a little bit about moving to Snowflake. And is this sort of where you store data and the volumes of data? Has there also been a transition from on-prem in the cloud hybrid? Is that a thing that's evolving for you?
Alan Moss (01:56) It is. With GrowthZone and MemberSuite we've always been in the cloud, but we've seen the size and the scope of our data expand. And then that's part of the reason we, you know, we've gone from, you know, we use AWS and we've used their version of Redshift, which is a Postgres database. But we've also moved to Snowflake.
Mainly because of the fact that it is so dynamic and allows us to store so much. And that became our data warehouse. We have our transactional database that handles our day-to-day data, but then we're moving data off to Snowflake, and that's where we're storing it for our reporting and our analysis and our BI and just the amount of data we can push that doesn't affect our customers on a day-to-day basis. And that's why we did that.
Matthew Stibbe (02:49) So tell me a little bit about GrowthZone and the market you serve.
Alan Moss (02:54) Okay. So, GrowthZone is an association management software that, you know, an association management software we're a SaaS product in the cloud. We have multiple products. We have what we call Chambermaster, which handles, it's an older, it's been around longer. I shouldn't say older, that sounds negative, but it's been around longer and it serves Chamber of Commerces throughout the US. It's one of the largest chamber software. We also have Growth Zone and we also have MemberSuite. MemberSuite is the one I've worked with the longest. I've been with them 13 years. And what that does is we handle enterprise size associations, mostly nonprofits that need to store their data, whether it's a software that handles CRM, whether it's individuals, organizations, memberships.
It handles all the financial data they would handle. Events, event registrations, committees, anything an association would do. We're their product so they can go in and whether it's through a portal, their members can come in and work with their, have memberships or whatever, register for events. And they also, we have a console that allows their users and their staff to go in and manage all of their data and all of their people as well.
Matthew Stibbe (04:20) Amazing. And what sort of volumes of data are we talking about?
Alan Moss (04:26) You know, on scale, I wouldn't say it's huge. It's large in the sense that, you know, we have members that could be anywhere from, you know, and the volume I would say is based on the context, let's say within the system, whether it's individuals or memberships or organizations. you know, it may be anywhere from 10,000 people within the system to 200,000, you know, with the mid and enterprise, per association. Yeah.
Alan Moss (04:52) So we're talking on average, on a database, we're talking hundreds of gigs, not gigabytes, not terabytes for us, depending on that, but it is a good bit of data, especially when you're talking about just an association and how many people they track. That's a good bit of data for us on scale. It's not a huge, massive retailer or anything like that, but it is a good bit of data.
Matthew Stibbe (05:17) Hmm. And important, important to them, important to members. Sure. so tell me about your, your journey into data, your personal journey, you mentioned to me that you've been at this game for 30 years. So how did you get started?
Alan Moss (05:20) Absolutely, yeah.
I got started. Yes, 30 years this this year in, I started just because I wanted to make my job a little bit easier at the time. I didn't like working with big stacks of paper as an auditor and I wanted to to make it a little easier. So I figured out how to get into the back end data and started writing dBase code to make the with a little bit of where clauses and stuff and making my job a little bit easier and I ended up getting into programming back on an AS 400 writing writing COBOL code.
Matthew Stibbe (06:09) there's like a massive demand. Right now, COBOL coders are like rarer than rocking horses, rarer than unicorns. You could make a million of them.
Alan Moss (06:15) It really is.
And I, and I absolutely, I loved it. I had never, you know, I, the first day I started on that job was the first day I'd ever written a COBOL program. So I had to, I had to go out and I didn't have any, I, luckily I had a good mentor that really taught me well and taught me how to do some good, you know, good data analysis and work with it. And as over the years I grew and went from COBOL to RPG and then, to SQL.
And, and so SQL's where I've been. SQL is my, I would call it where I love to be the most is in, you know, working with T-SQL and SQL Server. And started with that probably way back at SQL 6.5, 6.0 and 6.5 way back in the early 2000s, late 1990s. So, and then it's just grown over the years, whether it's in data analysis, data engineering, you know, getting into ETL work.
And I've just really in, like I said, in 30 years, I I absolutely love working with data.
Matthew Stibbe (07:21) And you described your job at GrowthZone as sort of wrangling data. And I love that word wrangling. What's the biggest challenge with the data that you're wrangling there? What do you have to work hardest at to sort of solve it?
Alan Moss (07:29) Right.
Well, I think there's probably two or three phases, I would say, within how we handle data. So the one part is where I started with MemberSuite is in importing legacy data from an association's current system into MemberSuite or our product. And I think the biggest thing with that was learning how, I think that's where the data wrangling really started for me, because learning someone else's old system to figure out where it needs to go inside of our system. And so that would be the first phase. We do a lot of legacy migrations into MemberSuite or into our product. And then from there, it's a matter of, now we already have in the system, what are we going to do with it? We've got tons of data in our transactional SQL database. How do we make it more manageable? How do we report on it? How do we create dashboards on it?
And we tried to do that on top of our SQL Server, it didn't work real well. Our SQL Server was mainly set up to manage the day-to-day transactions. So we had to create the data warehouse to move data over. So that's wrangling the data out of our system into a new system and then work figuring out what we wanted to do with it there. So yeah, there's multiple phases on how we push data around, work with it, massage it and everything else. And I think that that's the challenge. And I think that's what... you know, makes it interesting and fun as well.
Matthew Stibbe (09:06) Yeah, and where does CloverDX sit in that sort of landscape?
Alan Moss (09:09) So for us, Clover kind of sits in multiple areas, to be honest with you. So where we first started with Clover was we have our database that has 4- or 500 plus tables sitting in it that we work with and have data in. And the biggest challenge we had initially was, is how do we create a sandbox? How do we take a client's data and create a sandbox they can work with?
But that means replicating those 500 tables worth of data and we're a multi-tenant. we have, you know, ours is based on the fact that, you know, each association isn't on their own database. They're mixed in with others within that multi-tenant. So we had to get that data from here over to a sandbox server that they could test and play with. And so, you know, that was our first big challenge we had with that Clover solved for us is the ability to create that workflow and that job flow to say, I can read all these tables, I can make a copy of it, and I can push it into this database over here.
And so that's where they got started with us. And then after that, we started using them for, you know, if I need to get data out of SQL Server over to Snowflake, and I can write code to do pull certain parts of it and push it to different tables and creating those multiple complex job flows.
And they're also involved with us in, you know, when we need to move in our financial data, if we're moving subledger entries around, that, you know, if somebody needs to close a batch, Clover's doing that for us. The client doesn't know it. They click a button in our console, but on the backend Clover's doing a lot. We're coding within Clover to be able to do all that.
Matthew Stibbe (10:55) Right, wow. So we were discussing earlier a particular project, what this, you mentioned this sort of migration or moving data over to Snowflake. Can you tell me what the goal or the intention of that project was? Let's explore that.
Alan Moss (11:09) So I think one of the big things about having data is knowing what to do with it and what story you want to tell with it. And I think the biggest thing that we needed was, is we have our clients, the majority of what they make their money on is memberships. So they can have events and everything else, but they have their memberships. And it's also the lifeblood of an association that they want the members engaged and they want them to be members of their association, pay those membership dues, and then be involved in other things within that association.
One of the things that we were not able to do is tell that journey of a person being a member. When did they, you know, if they joined, we had the join date, we had when they left, but we didn't know what they did in between because of how we were storing the data in the transactional database. So we needed to create a warehouse that was constantly updating a snapshot of that membership data that told their journey.
So if somebody came in and joined and then a year later they said, okay, I'm not going to rejoin and they, their membership expired. And then six months later, they said, well, if I go to this event and I'm a member, I, if I join, I can, you know, I can have a discount. So they would join again. So we needed to have that snapshot. That's a constant growing, constant growing story of how the, keeping track of that membership for that person.
Matthew Stibbe (12:37) The interactions that each individual member had with their associations, such event attendance or signing up or renewing or whatever, so that you could, right, that's a fascinating thing and it could be quite a lot of data going back over the years.
Alan Moss (12:37) Oh it can. Yeah. I mean, we're really kind of, it's a cumulative effect of that, especially the membership data and because of how we needed to store it. So we're constantly moving a membership file over, you know, on a weekly, monthly, you know, quarterly basis to make sure it's there. And then we report off of that. And it kind of led us to the biggest thing out of that was, is Clover is the mechanism we use to go grab the data, know what we're grabbing, pull it over, put it in Snowflake and have it available to us.
And then it allowed us to go in and, you know, we could create a retention report off of it in dashboard so people could see how long people were staying in the retainment and everything else. And then that led to other stories we could tell around the engagement.
We could then create scoring off of that and show, you know, if a person's a member, are they also going to events? Are they also joining committees? Are they also...going and get certifications or CEU credits and stuff like that. So you could see the lifeline of that member as opposed to just one aspect.
Matthew Stibbe (13:57) And I'm interested in this because it feels like something you did with the technology unlocked some capability for the application, the service that you were providing and you describing there some scenarios that I would imagine if I was a membership secretary of an association or something, I'd be really interested in it. Not just what did Matthew do, but actually on aggregate, how long do we keep people as members and how engaged are they and is this person at risk of leaving? Those are powerful insights.
Alan Moss (14:21) Absolutely.
Matthew Stibbe (14:27) So how did you, were those coming out of your team and your, the data and being offered to the business or was the business asking for them where customers are asking for them? How did that interact with the real world?
Alan Moss (14:40) It was a kind of a mix. We knew we needed it. It was one of those things we were kind of waiting for the technology to catch up to the questions. And I think that once we realized we had the technology with Clover and a couple of other things that we were working with, and then we had Snowflake and we were able to get into Snowflake and figure and really be robust in how we would set up that database in Snowflake.
We started being able to answer the questions. We had a lot of the questions. People wanted the engagements. They wanted the retainment. We just didn't have the mechanism and the ability at that point and kind of the infrastructure for that. And I think what we did was is we took a step back and kind of disengage from the day to day stuff and started thinking, okay, what would this look like in a year? What would this look like in two years? And then we started setting that up with the ETL process and those data pipelines to move that data out of SQL into Snowflake to be able to report on it and make it actionable data.
Matthew Stibbe (15:42) Help me understand the obstacles to doing that in SQL. Was it a data volume issue or why did you have to go to Snowflake to do it?
Alan Moss (15:51) It was, I think the data volume was one also is, you know, our database was tuned to the day to day work, you know, and it was whether it's indexing or how the data was stored or, cause one of the things we built out of that, and I call it a membership snapshot because it is the membership data, but we also went to, and if I'm remembering correctly we probably, to create one membership table, we were joining probably 10 or 11 tables on the backend of SQL to create that data for us. So it was, if we had been querying off of that in our transactional database, our customers would have been affected performance wise. And we didn't want to affect that. And we didn't want to make changes just to, just so we could have a good reporting experience and then not affect. we needed to kind of isolate that data and a data warehouse is where you do that.
You want that to be, because that data is not changing constantly. It's dynamic in the sense you're pushing data to it and having data there, but it's not something that's changing constantly.
Matthew Stibbe (16:57) Got it. That's really interesting. To what extent was the ability to do the ETL pipelines out to Snowflake a barrier? And tell me a little bit about kind of the work you had to do to get all those connections working.
Alan Moss (17:15) I think it was more, it was time, you know, you know, having the ability to have the time to work on that. The other is, you know, I think, you know, you and I talked a little bit about it that we've been around with Clover long enough to see their evolution over the past few years of how much they've grown. It was, okay, as, as Clover's grown, we've had to do a lot to keep up with them, to be honest with you and the technology they have.
So, you know, there was certain times where, this may not have been feasible two years ago because we weren't at a place where we felt comfortable using it. But now as Clover's grown, we've grown with them a little bit. so they've made it easier for us with how they do their job flows, how things connect, you know, the fact that they had their own connector to Snowflake. I didn't have to write a connector to it. I just needed theirs and it works fine. So I think the barrier technology wise was more us in terms of just, you know, getting to a point where we had the time and the knowledge to use it was the big thing.
Matthew Stibbe (18:16) Yeah, it's often the case, isn't it? I had an acquaintance who worked at DARPA, the defense research thing, huge budgets. But the challenge for her and for the team she worked with was it wasn't so much the money, it was just how much attention they could give to different projects. And she told me once, the phrase they worked with was, if anything's possible, what's important.
And it's... triaging or allocating time between different projects is the hardest thing that she had to do. And I think that's often an obstacle. But I love the idea that Clover has unlocked that capability. I think it was Marshall McLuhan who said, first we make the tools and then the tools make us.
Alan Moss (19:01) That's very true.
Matthew Stibbe (19:03) So imagine you could go back in time to the beginning of that Snowflake sort of process. What advice would you give old you, knowing what you know now?
Alan Moss (19:06) Okay.
I think there's a couple of things. One is be willing to take a short term hit for a long term gain. And I think that's where I that's kind of been my mantra for a while, you know, for years is, you know, if we want to be really good at something or we want to be bold in what we're trying to do, we're, know, there's going to be things in the short term that are going to get, I don't want to say missed or are not done, but it's going to be a little bit of hit if we're willing to look into the future a little bit and say, if I'm willing to take this, this is where we're going to get, and this is what we're going to do.
I would say to myself a few years ago, be a little bit more bold and a little bit more, you know, stand up for the fact that this is really going to be good if you're willing to give me the chance to get it done or the team. You know, because we are a smaller company and sometimes, you you have people that can kind of stretch to, you know, out a little bit. And I think the biggest thing was is, and we were able to do that. But I think I would say to myself, make sure you really push to be willing to stand on the fact of where you could be if you would go through this.
Matthew Stibbe (20:30) And you mentioned earlier that you had sort of done a one and a two year roadmap or a sort of an envisioning exercise. How did that help?
Alan Moss (20:38) It helped us because I think it gave, it was good for us because it gave us kind of a plan. We knew if we started with membership that we could go to events and we could go to this and we could, you we could go to multiple areas and create that overall dashboard and what it looked like. but I think it was good for us because it let us lay out a plan of what was most important at that time.
But I think the other part that was really good for was the company because not everybody sees data the same way. and they say, all they have in their mind is what this screen needs to look like. You know, when you're looking at data or you're looking at a report, you see the data and that's it. You see the finished product and they can, they kind of could see where we were going with it. They just didn't know the work that was going to go into it. And so I think laying that out for people really helped them understand one, where we were going, but two, the amount of work it was going to take to get there and, let us, let us do it.
Matthew Stibbe (21:30) Yeah. So somebody goes, I want to, I want to a member retention report and you go, you know, and they think you can click a button and it's going to appear and you have to do all this. Yeah. Well, it's heavy, heavy engineering to get it, get it to deliver that. Yeah. So we're almost out of time. And before we bring the conversation to a close, know, you talked about the start of your career and, and, and sort of self teaching, COBOL, and data queries and things. How do you keep your enthusiasm and your knowledge and your skills sharp over time? And what are you excited to learn next?
Alan Moss (22:10) I think the fact that the technology constantly changes. and I think the ability in technology to self teach and to learn is what keeps me excited and keeps me and the seeing how things grow. You know, we used Snowflake in a very rudimentary way when we started and seeing how we can expand outside of that and how it can interact with other things. The same with Clover.
And I think just the fact, that it's constantly changing and there's not, you know, people may think data is boring, but the fact of what you can do with it and how much you can do with it today is what excites me. And I'm just as excited today as I was 30 years ago when I got started.
Matthew Stibbe (22:56) I, it's a very exciting, positive place to finish. And I'm reminded of some famous person who said that poets are the unacknowledged legislators of the world. Well, I think these days data engineers are the unacknowledged legislators of the world. You you guys make all this happen for us. Anyway, that's on that on that bombshell. I'm going to finish the conversation. Alan, it's been an absolute delight talking to you. Thank you very much.
Alan Moss (23:23) You too. Thank you very much. I appreciate it.
Matthew Stibbe (23:26) And if you're listening to this, if you'd like to get more practical data insights and learn more about CloverDX, please visit https://www.cloverdx.com/behind-the-data. Thank you for listening and goodbye.