00:00 The Foundation of Data Quality
04:56 Explicit vs Implicit Tracking
11:10 Building Trust in Data
14:37 Data Quality in CRM and AI
19:06 Proactive Data Insights
19:36 Harnessing Generative AI for Data Analysis
21:03 Innovative Engineering Practices at Clarify
24:25 The Evolution of Analytics Roles
25:19 User Expectations in the Age of AI
27:59 Transforming Sales Processes with AI
29:53 The Future of Analyst Skills
32:59 The Importance of Context in Data Analysis
35:33 The Role of Prompt Engineering in AI
38:45 Embracing Innovation in Analytics
Katrin (00:00)
Welcome to a knowledge distillation where we explore the rise of the AI analyst. I'm your host, Katrin Ribant, CEO and founder of Ask-Y. This is episode five and today we're talking with Patrick Thompson.
CEO and co-founder of Clarify about why data quality is the unglamorous foundation that makes everything else possible and why AI doesn't fix bad foundations, it just builds houses of cards faster. Patrick, I usually do the intro, but let's break that pattern. Please tell us about your journey as a founder, your experience at Amplitude and what made you want to build again.
Patrick Thompson (00:37)
I'm excited to be here and great to chat with you about all the things here related to data. For folks that don't know me, my name is Patrick. I'm based in Seattle. I'm the co-founder and CEO of Community College Clarify, but previously was the co-founder and CEO of a community called Iteratively, the amplitude acquired back in 2021, which was a customer data platform solving a lot of the data quality challenges that teams faced. And before that, I worked on growth at Atlassian.
Katrin (01:01)
Great, so let's sell some knowledge. Patrick, when you co-founded iteratively, ⁓ you saw the same pattern as hundreds of companies. We all know the pattern. It's a pattern where analysts spending a lot of time questioning their data, more time actually questioning the data than analyzing it. Teams build dashboards, knowledge experiments, make decisions, and then you discover that the tracking was wrong, the data quality doesn't support what you want to do. We've all been there.
hands, if not hundreds of times. ⁓ And obviously nobody ever knows that the data is wrong until it's too late because nobody really had a chance to look at it in that way until then. And then they've done all of this work and you realize some basic stuff isn't working. So can you tell us what principles did you build iteratively with? What were we solving for? What was your key differentiation, your key insight there?
Patrick Thompson (01:58)
Yeah, well, it's funny. Actually, we working on growth at Atlassian, we ended up using data a lot for experimentation, for personalization, and ended up having to build a lot of the tooling in-house. So when deciding to actually leave Atlassian and go start a company, you're right, I interviewed hundreds of people doing customer discovery, trying to figure out what problems they had, and then realized that nobody really had the tooling that we had ended up building in-house at Atlassian. So I ended up building a solution similar to what we had called Data Portal, which allowed
folks to do accurate instrumentation and effectively as kind of a mini data dictionary, an event pipeline that did quality monitoring. And what we saw from all the interviews that we did is that typically, know, analytics is very much a team sport. It requires collaboration across engineering and product and growth and other stakeholders that use the data. And so what we wanted to do is how do we build a system more akin to GitHub where folks can collaborate around their data?
really provide the definition of what's needed, be able to define the metrics that matter to the organization, instrument them reliably, and then be able to analyze it long-term as well. This was a fairly quick journey for us. ended up raising about five and a half million in venture and then got acquired within about two years from Amplitude. We were working with a bunch of different companies like Box.com, Canal Plus, Freshly, Thrive Market, you name it, who really wanted to be able to understand their customers. And the biggest gap
was just the amount of time it took to instrument and this back and forth loop across the data producer and the data consumer. this, you know, I still say it's probably not a solved challenge today, as you allude to. There's many organizations, I still still face this problem. And it's funny because it's still equally as relevant even in the context of CRM, we're focusing on it clarified today when it comes to kind of garbage in garbage out problems.
The principles that we really focus on is how do we think about data as a team sport and involve the right stakeholders? How do we build such an easy to use product that anybody can do it? It didn't require kind of a PhD. And then I'd say third is that understanding that data in general is iterative, right? So it's not like you're going to get it right the first time. The reason we called the company iteratively is that ⁓ very much you probably define what was important.
you try to instrument it, you would get some analysis and you find that you were missing something. There was some key piece of customer data or information that you wanted to capture and you have to go re-instrument, re-gather the data. And you know, I think this is where companies like KEEP try to do implicit tracking and there's companies like Amplitude that try to do explicit tracking. We're a little bit more, I'd say in the form of the explicit tracking camp for what it's worth, because we just found that the data quality was higher there.
But now with AI, it feels like there's a lot of signal you can get, generally, from the noise of implicit. And so what we're seeing.
Katrin (04:56)
Just for the
audience, could you explain explicit tracking versus implicit tracking?
Patrick Thompson (05:01)
Yeah, 100%. So explicit tracking would be ⁓ an engineer actually instrumenting an event. So there's a, you know, a dot tracker event where somebody's writing code in their code base to actually capture a particular event. Implicit is you instrument or you add an SDK to your product that then tries to capture a bunch of different signals, right? I think this is more where RUM and ⁓ other performance monitoring tools have really kind of worked well in the past.
And then people really tried to apply that to more behavioral analytics. And I'd say up until recently, it's been really hard to sift through the noise because what you're doing with implicit tracking is you're basically making the haystack bigger, but the needle kind of stays the same. Whereas previously you wanted to do explicit tracking where you think about analytics as code and it's testable and it's deterministic. ⁓ And I still say there's value in both.
here for Ditsworth and even a company like Amplitude has since added implicit tracking, what we would call auto capture to their product. Now with CRM, it's something that's very similar for us. CRM is kind of a garbage in garbage out problem, whereas people typically require a lot of manual effort to capture data. And now we're working on tools to make that auto capture of information in CRM just much easier for teams.
Katrin (06:21)
So you've, yeah, about the implicit tracking, just something that you're saying that makes the haystack bigger. I would argue sometimes it actually makes the haystack just being a stack of needles. And you're trying to find a needle in the stack of needles, which that really creates, it can really create a lot of noise when it's badly applied. But no, it absolutely has its use cases. So. ⁓
Patrick Thompson (06:35)
Ha ha ha.
Yeah.
Katrin (06:48)
You're saying this is a team sport, right? So you and I could not agree more. So iteratively was ⁓ working across data engineers, data analysts, stakeholders. And so your product was working across all of those. suppose that obviously you worked with a lot of companies and.
What would you say within that process, having been able to sort of work with so many people as a team, what are the sales signs that tell you this company has a data trust problem? This company has some things in place that make it possible for them to actually build data trust relatively fast, iteratively, but at a certain pace.
Patrick Thompson (07:33)
Yeah, I think the challenge with most organizations is most organizations have a data trust problem. It's not always the data. Right. And so when you think about being data driven or data informed, however you want to talk about it, there are folks in organizations that have an implicit bias to that their belief system is the right one. Right. And they're looking for data to either confirm it, but they're not looking at data to prove it. Right. And that's the, or disprove it I'll say. Right. And so when I think about organizations that
Understand data. They're not looking to prove out their hypotheses They're looking to disprove them and because you can torture data to say whatever you want I've seen that in many other organizations I've worked at and many of the companies that I've worked with and so quite often when it comes to being get informed or being able to use data You can put the same data set in front of two different individuals within the same organization get two wildly different ⁓ outcomes of the analysis
So this is the hard part where it's not always just a instrumentation problem. There's the data literacy challenge of how easy is it for people to analyze the data. like, you know, there's organizations that invest quite heavily in ⁓ upscaling people on how to write SQL and buying better BI tooling. ⁓ There's organizations that embed analysts into two different product teams because it's kind of the fourth leg in the stool, similar to
design, product and engineering. And then there's organizations that centralize analysis because they want to, I wouldn't say gatekeep, but they want to ensure that the quality of analysis is at a high level. And so you see this like distributed or federated model or centralized model quite often with organizations for analytics teams. And I tend to like a federated model where you're trying to teach folks how to use data, but it is, it is hard.
when people are naturally finding their own biases. Because they don't want be proven that they're wrong, right? Like people like, like they're right more often than not. But something else that you were mentioning earlier is probably worth chatting about just quickly, which is like, what are the systems of reward that these organizations have as well, right? Because depending on the...
operational and management or leadership within the organization and how they think about data and how they think about making decisions that will definitely influence people below them, right? And, uh, influence the behaviors that are being driven. So there's a lot of companies, I think that will play lip service to be like, yeah, we're data informed as organization. And the first question that I typically would ask them is, okay, well, tell me about a time where you recently used data to make a decision and they can't, they literally can't remember.
an example of that. And that's like the easiest indicator for me to understand like, are you actually data informed? Are you just driving based off your gut and intuition? And to be clear, I actually think gut and intuition is a form of data, right? Like you're the more, the more experience you have, the better you're fine tuning your gut and intuition. And so when I think about more senior people, they tend to have, if they have more experience, they tend to rely on their gut and intuition more because I've seen a lot of the qualitative quantitative data.
Katrin (10:34)
close.
Absolutely.
Patrick Thompson (10:51)
over their career to actually inform their future belief system. ⁓ So not to say that that's not a good form of decision making. I it really is, especially with absence of data. But if people don't have that experience and they're not relying on the qualitative or quantitative data that's available to them, then it's like they're just shooting in the dark.
Katrin (11:10)
Yeah, think that data, ultimately decision making, which is the goal, right? The goal of data analysis is to support somebody who makes a decision. So that decision making ⁓ and the whole process that leads to it, it's very much a human process. AI or not AI, ultimately it is very much a human process and a human process that involves trust. Trust in the data, trust in the analyst, trust in the entire process. And if that trust doesn't exist,
Patrick Thompson (11:20)
Yeah.
Katrin (11:39)
the data will not be used to make the decision. And data quality is an absolutely essential part of obviously building that trust. It's not the only part, but it is an absolutely essential part of building that trust. And if you don't have that, everything else ultimately is a house of cards, right? So in one of your blog posts, I think this was a blog post that you wrote when you were at Amplitude.
It was titled prioritized outcomes over outputs. I mean, that's basically struck me as being context engineering before Jehuba coined the term really. So how do you think about that relationship between data quality context and ultimately trust?
Patrick Thompson (12:27)
Yeah, think the outcomes over outputs things is an interesting one. And I think a lot of organizations end up operating as kind of feature factories where they just ship feature after feature after feature, right? And specifically talking about product organizations here. ⁓ But they're not necessarily measuring, like are these things actually laddering up? In absence of data, velocity is a very good indicator, right? So like shipping is a good input metric.
Katrin (12:51)
But that can
be true for an analytics team as well. Tickets closed, you know, like you can translate that to a lot of contexts.
Patrick Thompson (12:59)
Yeah. And then, but there's always the five wise, right? Like, you know, when we, let's assume you generate an insight, let's say, so let's assume you have an embedded or an embedded analytics model, where you have an analyst who's working on sharing insights, right? And there's hopefully some, uh, business unit that's requested information or requested some analysis. the analyst goes and actually performs some analysis, the generated insight. Well, how's that insight used, right? And what decisions were made by that business unit?
that hopefully impacted and what was the impact of the business by those decisions. Right. So this is the downstream impact of analysis, which is, you actually measure the outcome of the work that you're doing as an organization? There's a lot of people that talked about this, ⁓ as well, but it's the goal of analysis is hopefully better decision making over time. And this is why I do think it's still even in the context of building a muscle, right? Like you have to train it, right? People are going to be fairly bad once they start going to the gym.
Right. then, you know, the first three or four weeks of going to the gym suck. No one likes it. Like it hurts. But like over time, you're like, okay, cool. You developed the muscle memory. You get better at working out. becomes more of a, of a habit than, more of a one-off. And so when you develop that habit of good analysis and actually understanding how to use the data and then also harrowing
internally, like, you know, being able to stand up at an all hands and be like, Hey, this is some analysis that we did, and this is the outcome of that analysis. Like you're showing that you care as an organization and you're developing that muscle. And this is what's really hard is I don't think people do a good job of that, ⁓ within many businesses today.
Katrin (14:37)
And how are you thinking about that problem, the context quality foundation problem ⁓ at Clarify in a completely different domain really?
Patrick Thompson (14:46)
I mean, yeah, we're, we're an early stage startup, so we're only about 25 people today. And so we talk about metrics that we all hands, right. And then we try to go into them and we try to analyze them, but like we're still operating quite heavily on qualitative data and gotten intuition over quantitative data. And we have the quantitative data and we've done the instrumentation. do the analysis there, but we have very, I'd say fairly clear signals and indicators on what we need to go do as a business to, I'd say like hit the benchmarks that we're trying to go after.
Katrin (15:15)
And within your product, how does that translate into thinking of maintaining this context and this data quality in your product, which I'm sure you're obviously dogfooding and using your product as well, right?
Patrick Thompson (15:23)
Yeah, yeah.
Yeah. We're
the number one customer at Clarify is Clarify, right? We dog food everything and use it every single day, hours a day to power our go-to-market motion. think the thing that I think about for data quality in the context of CRM for us, where I think like, see, like you think about quality and data quality. And so when we did this discovery back in 2019, you know, quality came up in regards to data that's in your warehouse. came up in regards to data that's in your CRM. It came up in regards to the
experimentation or marketing campaigns that you're running on top of the data as well. So quality comes up across the entire spectrum, but a lot of the focus that we have both at the last startup and this startup is around customer data. ⁓ And so customer data being, know, CRM being a core aspect of customer data. And so for us, when we think about that, the standard CRM challenge around data quality is like ⁓ identity resolution.
Right. So you have multiple of the same people, right? Like in amplitude Salesforce CRM, there was six Patrick Thompson's, there was an Atlassian Patrick Thompson amplitude to clarify and iteratively personal email and then my venture fund email as well. So you had all.
Katrin (16:22)
Yes.
I mean, I remember
clients who had like 17 instances of Salesforce.
Patrick Thompson (16:41)
Yeah. And so when I think about AI and what we're trying to do is like, we're trying to build a better entity resolution model for like how to one normalize the data and then to be able to stitch together. Right. And like we did some actually cool stuff at amplitude around building kind of a, ⁓ identity graph across these and be able to alias identities together. And we're trying to apply some of that same learning insight into the context of CRM.
⁓ That's standard data challenge that you have, which is like, what person do I reach out to if I'm running any analysis on top of this? Like, how do I stitch together the timeline of customer communication across multiple different cities? And that's just one example of how we think about building a better product that can solve some of the data challenges. Additionally, just like managing fields and field usage, we amplitude built what we call kind of the data portal.
which inside of amplitude data portal, we had things like a health score, like how healthy is your data? And then you can actually see the entire data dictionary of all the different events that you were, ⁓ that you had as well and how they were used downstream. So what was the analysis? Like what analysis was built on top of these events inside of amplitude? And so we're thinking about ways to kind of incorporate that same level of thinking into cool. have
All these fields on these core objects and you have custom objects in a field but like how are these being used and queried and analyzed? Are there any downstream? ⁓ Campaigns that are being built on top of this data and being able to show utilization is a great way of at least understanding like where should I be going spending my time if I'm an analyst actually validating data as well and then using things like anomaly detection as just a ⁓
I'll call it more machine learning than AI. I think that term is pretty overloaded these days to understand like, are there things that we should be looking for, signals that we should be looking for to alert people? I think the hardest part about data quality is you don't want it to be a passive thing. You want it to be kind of an active thing where the systems that you're using should actually help inform you if there's an error or a challenge. Whereas,
This is actually the same for most analytics tools today, for what they're worth, you're having to pull information out of them instead of them pushing information to you. We're really trying to think about building a system that's pushing you information and insights instead of just being an analytics platform or a data platform.
Katrin (19:06)
It
makes sense, right? Because one of the main challenges a lot of data teams have in certainly in larger organizations is ultimately they kind of like send dashboards across the ocean, sometimes literally, and they don't really know what is being used, what is being looked at in all of their tables, data models, etc. It's very hard to know whether something is being utilized because it's not because you are sending a dashboard that is being opened and that anything is being done with it.
Patrick Thompson (19:36)
correct.
Katrin (19:36)
So that's
really something that that signal is really hard to get. I feel like we have a chance with ⁓ generative AI and potentially some sort of like evolve self-serve BI where once you have a really clean data model and you have like, know, good data quality, et cetera, you can have some level of self self-serve conversational interrogating of the data by the people who need to make decisions that gives you a lot of signal.
Patrick Thompson (20:03)
Yeah, we recently launched our MCB connector to Clarify and we're about to launch Rep, which is our agent built into Clarify as well, which has like tool calling and much of other things, but it's really, really good at the analysis side of the data. The hard part is then reconnecting the loop into like, how is this analysis actually changing how we think and operate as a company? And ⁓ because it's pretty much disconnected from the other tools and systems that we use, which we're hoping to plug the app on, which is tools like Linear for...
just yeah, software engineering or the SDLC and the notion for lever documentation. But the goal being that like, once you actually have an agent that can help do the analysis is can it actually learn from that analysis of is it changing behavior internally? And like we're, I'd say we're a little bit far away even ourselves from being able to close the loop there, but it is, it does definitely unlock a way of operating that I think will be super beneficial for teams in the future.
Katrin (21:03)
So one of the ways that we kind of like look at AI analysts ⁓ is by looking at AI engineers, because we can sort of look at our engineering cousins and see what's part of what's probably going to happen to our roles. So how is your engineering product organization different at Clarify than it was at Amplitude or at Iteratively?
Patrick Thompson (21:12)
Mm-hmm.
There's a few things that we do differently this time around. We hire a lot more product engineers, like people that can actually think like a PM or designer who can understand the customer problem and then figure out what the potential solutions look like. And then actually be able to have good product sense and being able to like, or, you know, build it. And that's something that, you know, I wouldn't say it's, I'd say has been a trend over the last like five or six years within the industry. And so we hire a lot more folks at.
You know, have been PMs that are also come from an engineering background and want to actually focus on this particular skill set. It means that, generally they're wearing many, many hats throughout the day versus just one hat. So I would say it doesn't, it's not a good fit for everybody. I'd say the other thing is that, um, we're trying to really build, uh, agency and autonomy within what we call pods. have a pod model internally at Clarify. So we're trying to organize.
these pods around particular missions and focus areas that we want to invest in and then push the decision making as far down to the team as we can. You know, I'd say this is working fairly well for us, honestly, because we're able to tackle multiple priorities at any point in time without necessarily being bottlenecked by resources or having to figure out how to best organize or self organize the teams. I'd say that's something that's very different. The other thing is
⁓ we ended up investing fairly heavily in clarifies data architecture early on. So we're a multi-tenant single tenant architecture, which means that every customer has their own unique warehouse of data. ⁓ that's purely separated and isolated for every other customer. ⁓ and then we actually built clarify to be extensible and you can connect it to any BI solution you want. Right. So for us, like we're because we're warehouse native, like
You actually have the ability to go connect directly into Looker or Mode or Hex or whatever other tool you want. Hopefully ask-y as well. Specifically to go run the analysis. And we have a lot of
Katrin (23:34)
Yeah,
it makes a lot of sense. You don't want to reinvent that layer.
Patrick Thompson (23:38)
Yeah, exactly. like the thing about reporting and what we've seen for pretty much every organization we talked to, that's a Salesforce company, they end up exporting the data out of Salesforce into Snowflake or some other warehouse anyways, and then building reporting specifically when it comes to time series reporting, because Salesforce doesn't really shine there on the time series side. And because they don't do full activity capture on all the field level updates as well, it's really hard to use it for modeling, I'd say, for forecasting.
which is why companies like Clary exist to begin with. And so for us, we just kind of embraced it from the beginning, which is that we're definitely taking more of a modern data stack approach to how to build a company and just embracing the fact that companies are going to want to bring their own BI solution and reporting solution to clarify.
Katrin (24:25)
And what you described with your ⁓ product engineering and pod structure is very much what I hear a lot around analytics, ⁓ changes in analytics teams and roles, where we see because AI is really powering this full stack extension of analytics skill sets, you see people embracing a lot more of the analytics lifecycle.
and ⁓ a lot more of this sort of like part, know, sort of probably something of what you were fostering at iteratively where you have people collaborating more across the board as opposed to just like staying in strictly analysts ⁓ team. What do you see your users do differently or how do you feel ⁓ their expectations are changing as to how they sort of think about using systems today?
Patrick Thompson (25:19)
I I think a lot about agents. And there's ambient AI, and then there's more like agent AI. So in Clarify, we pulled a lot of ambient AI. So AI just runs in the background and doesn't look for you. But we have our agent as well that you can just query information from. And I see the normal default expectation that's primarily been driven by the user behavioral change from tools like ChatGPT and Cloud is that people just expect to do lot more natural language queries on top of their data instead of having to go
build reports via like a point and click editor. And, you know, even the context of like this one-off analysis has in my mind, like what we've seen both internally for our own organization, as well as what we've seen within customers, especially when it to like MCP, our MCP connector, is that they just like, they just let the analysis rip. They just like ask question after question after question, they try to slice and dice the data in many, different ways. This still all assumes that people know how to ask good questions, which is a different topic.
⁓ or the right questions. but it is, it basically has unlocked superpowers for a large group of people that potentially were hamstrung or resource constrained previously. ⁓ because what we found is like, I'm sure Philly got it right in SQL still. And like, if you provided a relatively constrained schema data model, like it can give you an answer. And you know, it's the, still think the it's worth fact checking and making sure that you're validating the data or at least the analysis.
Katrin (26:20)
Yes it is.
This is really where your data quality needs to be there, right? Because if you're opening that door and your data quality isn't there, I mean...
Patrick Thompson (26:51)
higher. Yeah.
Yeah, I always think about like chaining or cascading events, right? So we use AI to automate a lot of the manual data entry, and then we use AI to analyze a lot of the data inside of Clarify, right? And so if the data that we're automating as far as manual entry into the system is bad, then the analysis is going to be bad, which means that we have to have fairly high confidence in our ability to, I'd say like auto-fill the data in the CRM. And then we also have fairly good confidence that we're validating the data that's being auto-filled.
Katrin (27:28)
Yes.
Patrick Thompson (27:28)
for the analysis to be accurate.
Katrin (27:29)
And from that perspective, obviously one of the main pain points of CRM is compliance, data entry compliance. All those are like that aspect is every sales leaders nightmare, right? And I'd imagine that you can see already across your client base that's organizational changes are taking place as a result of.
Patrick Thompson (27:38)
Yeah.
Yeah.
Katrin (27:56)
using a tool like yours. Can you talk a little bit about that?
Patrick Thompson (27:59)
mean, the metrics that we hope to move for our customers that we're getting some good signals on our ability for people to just spend more time working with customers. Right. So because we have this autonomous CRM running in the background that just works for them, like they're able to take on higher quotas, they're able to work with more customers, they're to accelerate other sales things because we just automate a lot of the manual work, right? Whether or not it's like email follow-up or scheduling or pipeline management, we just take off.
which is take off the burden of that from them. So they're able to just do more with less. Now the things that are interesting that I hope to move to a world of is right now we're very much kind of a consumption based business model, right? So the more our AI does work, the more you pay us. I actually think we're only about a year or two away from more of an outcome based model where like we should actually be helping drive revenue for your business and actually showing the value that we're able to actually have there.
Katrin (28:50)
Mm-hmm.
Patrick Thompson (28:55)
I think we're probably one of the only few CRMs that are thinking about that.
Katrin (28:58)
And so if you think about your experience, you've been working with analysts for 15 to 20 years. There is a point where we call it 20 plus years and we just, that's it. And we never ask about the plus. And, know, so obviously you've seen a lot of
Patrick Thompson (29:05)
whole life. mean, whole academic career. Yeah. ⁓
Yeah.
There is definitely gray in my hair now.
Katrin (29:23)
hype cycles, tool evolutions and revolutions, roles rising, falling, know, changes, etc. What in your opinion is different this time around? And more specifically, if you're like you were advising an analyst today about what to invest in in terms of skills to stay relevant in the foreseeable future, which the future is not very foreseeable these days, but still.
in the hypothetically foreseeable future, what would you say?
Patrick Thompson (29:58)
I mean, context matters so much, right? In the con like, so as an analyst in a business, your context, your understanding of the business, your relationships that you have with the different stakeholders is so much more valuable, right? Cause what we're seeing right now is like the floor for analysis is being lowered. Right? So the ability for data democratization to happen within an organization is much, much better. Right? Like you have better tools, you have just better systems record that you can query and analyze. You have better and that's
means that there's more analysis happening generally speaking. Now the hard part is there's the same thing with five coding. There's the difference between five coded analysis and actually really, really good analysis, right? There's difference between five code and good code that's written by an engineer. And so I think the folks that are able to still tell a really, really good story with the data, be able to be, have managed relationships and have the understanding of the context to be able to insight change or organizational change are going to be much, much more valuable.
So this is the same thing when I have conversations with people like, what should my, you know, my kids study in university now, should they actually learn how to do software engineering? Like, yeah, I should still learn how to do software engineering. But I think an engineer who understands business or understands the context of why the work that they do matters is way more valuable than somebody who doesn't, who's just taking tickets off of a queue. It's the same for somebody in support. It's the same for somebody in sales. It's the same for somebody.
in a data team. like understanding organizational context and what matters and then be able to distill your analysis into things that are actionable recommendations for the business. That's still a very valuable skill.
Katrin (31:42)
Yeah, when you say soft, like I had those questions as well, should my kids still study software engineering? And I don't understand how the answer could possibly be no. Be it simply because, you do need to look at what, as an analyst, right? As a software engineer, certainly, but as an analyst as well, you do need to look at the output of the AI.
what it generates for you. You have to understand that code. You have to read that code. You have to understand what logic is being applied. How do you do that if you don't know how to read code? Maybe you're not going to write that much code anymore. Sure, you're still going to analyze it, understand it, architect processes, and certainly as much as you start having agents that are being able to do more and more of the work for you.
Well, you're going to have to architect and plan what those agents are doing for you, right? Yeah, I really don't see how the answer to should I still study software engineering could possibly be no. But I think, you know, I'm thinking of it in terms of you have this whole area around analytics is a human process. And obviously, business is a human process.
So you do need to have all those skills around the context. What is the business doing? How am I contributing? Who does what? Who sort of works with which pieces? And how does that reality get modeled into the data? So how do I create that data model that is modeling that reality? Consciousness of that model is a reduction of reality. It is compressing information. I will always have to add.
information on context on top of it and I will always have to do the translation between that compressed information and actual the actual reality and that's on me as an analyst to do that that translation because I know what was compressed in the data model I know how much of the reality is not there and I think that that aspect is extremely undervalued in our profession it's kind of like this translation of
What is exactly missing in this data in order to be able to transform this analysis into an actionable recommendation? What do you think about that aspect?
Patrick Thompson (34:02)
Yeah, it's also a
memory challenge as well. Right. So when you think about an agent, right, like you're giving it memory, you're providing context, but there's also memory that's valuable. Right. And like, you're an analyst that understands the business and you've done a bunch of, know, where the skeletons are in the data, it's a lot easier for you to be able to like navigate that.
Katrin (34:10)
Mm-hmm.
And there are always
skeletons in the data.
Patrick Thompson (34:20)
Correct. Yeah. And so like also working with a stakeholder, right? The interpersonal skills matter way more now because it's like, know, kind of what they're looking for or not. And you're able to better massage the deliverable, whether or not it's a report ⁓ to this particular person. like, I'd say memory is the thing that is still a challenge, right? You just have like a BI tool with like it's generating SQL based on natural language input. Like memory is still a challenge. And, ⁓
Context is still a challenge. So I think that's an area that an analyst, you know, can still outperform some of these tools. And then the question is like, do those things get solved over time? I think the answer is yeah, those get improved over time, but similar to like every other function, right? Like we're not going to like see the disappearance of analysis. I think the people, yeah, the people that can do more with less are going to be way more valuable.
Katrin (35:07)
No, definitely not.
Patrick Thompson (35:14)
And so this is the same thing for engineers at Clarify, really. We expect them to do product work and design work and analytics work. We're expecting people to be able to take on more disciplines than what they pretty historically would have been able to do potentially ⁓ because they have better tools available to them. And then they can context switch better.
Katrin (35:33)
Yes, and on that note, one of the things you touched on is it's a question as to whether people can ask good questions. And for that, if you think about natural language as your interface to a number of tools these days, asking questions and prompting basically is a real skill, right? And it's true that that got a bit of a micro hype cycle maybe a year ago where
Patrick Thompson (35:46)
you
for sure.
Katrin (36:01)
prompting, being a prompt engineer was something that was sought after and maybe that has disappeared a little bit. But I think that that's because that has been diffused in everybody's skill set. Like you are, I think today, expected to know at least the basics of prompt engineering and the people who are really good at it differentiate. Do you see that in your team? Do you see that in your clients?
Patrick Thompson (36:26)
Yeah, definitely. think the one thing that you actually mentioned that I love to touch on a little bit is like, it's one of the challenges we had recently is ⁓ we, someone on the team was presenting data at Clarify. It was very easy for me to see the data and know that it was wrong, Because I just know the business fairly well. But then it wasn't easy for them to look at the data and know it was wrong. And that's like the gut intuition where you're like, how good are you at gut checking?
some of the numbers and the analysis that you've done. Are you using these tools as a crutch? So using them as a tool or using them as a crutch? If you don't understand the business, that's really hard to know that. But I kind of, I kind of expect people at Clarify to know the business so well that when they look at something, they can gut check it and be like, Hey, is this right or is this directionally right? Or is it directionally wrong? And then when you dig in, you're like, Hey, this is an analysis issue. Like the data that you looked at wasn't correct.
that's, that's something that, you know, AI is very good at trying to be people-placed, right? And like you ask it a question, I might give you an answer. You're like, oh, here you go. And you're like, no, you have to double check it. And, I can still say that's a challenge that we face today. The question, that you asked earlier as well, which is like, tend to think of the evolution here of what's going to happen over time. like,
These tools are only going to get better and better, right? We're not good. I don't think we're going to hit this like a plateau effect here. think we're still, you know, I don't know if we're going to be accelerated in the same pace. I still think they're only going to get better and better from here. So trying to train people to think of these as tools in their arsenal, right? The battering on their tool belt ⁓ is valuable. And so what we tend to try to look for in folks is like,
how AI literate are they or how adaptable are they to using new tools? How quickly do they adopt things? And that's something that we tend to try to screen for a little bit more when it comes to hiring. It's been amazing to see folks internally pick up tools, especially as they iterate or try to test the new models because we are still in this kind of, belief is that we're still in this like rapid pace of acceleration and innovation.
with them, our language models and like even the things that you see improved in Gemini 3.0, right? It was amazing, right? And it's like, yeah, yeah. And so this is a great time to be a builder, right? And like the thought process of being like, I don't know, for me, like the thought process of being like locked away in a bigger stodgy company and not having access to these things and not being able to experiment, that that's actually what terrifies me with a lot of the people today.
Katrin (38:53)
It's so much fun. It's really great.
Yes.
I agree, I think that this is really the time to build an experiment.
Well, Patrick, this has been amazing. We're coming up on time. I could do this for another hour easily. But for people who want to talk to you, I know you're hiring at Clarify. How do people get in touch with you? Where can people find Clarify? We'll put all of that in the show notes, so you can just tell people.
Patrick Thompson (39:34)
Feel free to reach out to me via email at patrick at clarify.ai or find me on LinkedIn. Yeah, I'm happy to help anybody and I really appreciate you on the pod.
Katrin (39:46)
Thank you so much, Patrick. We'll make sure all this is in the show notes. That's it for episode five of Knowledge Distillation. If today's conversation made you want to experiment with AI for analytics, visit us at ask-y.ai and try Prism. Thanks for listening and remember bots won't analysts will, but only if they're working with data they can trust. Thank you.
Patrick Thompson (40:11)
was a great wrap.
