Figuring Out Fabric: Learn Fabric in 30 minutes.
Each week I’ll be interviewing experts and users alike on their experience with Fabric, warts and all. I can guarantee that we’ll have voices you aren’t used to and perspectives you won’t expect.
Each episode will be 30 minutes long with a single topic, so you can listen during your commute or while you exercise. Skip the topics you aren’t interested in. This will be a podcast that respects your time and your intelligence. No 2 hour BS sessions.
Figuring Out Fabric: Learn Fabric in 30 minutes.
Ep 3. Extracting Data from Legacy Systems
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
In this episode, Kellyn Gorman talks about the challenges of getting data out of legacy systems (i.e. relational data bases) into Fabric. She explains that whoever hosts the data wins. She talks about often content talks about the golden path or focuses on the marketing content, but it's much rarer to see content that deals with the difficult edge cases.
We talk about how despite being Software as a Service, in order to learn Fabric you will need to learn networking, authentication, and infrastructure more broadly. We talk about how it's impossible to completely get away from hardware and infrastructure.
00:00:04:18 - 00:00:23:04
Unknown
Hey everyone, welcome to the Freaking Out Fabric podcast where I ask dumb questions and you get smart answers. And I have a very smart person with me. I have Kellyn, who's been working with databases of all sorts, starting with Oracle for the past two decades. She's been, director.
00:00:23:06 - 00:00:41:08
Unknown
I'm sure you've done a bunch of consulting, and now you're at Redgate, so you're just all over the place and some of these things, like, I can never manage being a director. So very impressive. And I couldn't be there. Don't worry about it. Now, let's.
00:00:41:10 - 00:01:02:12
Unknown
Fair enough. All good. And, I think you're going to talk to us a little bit about data movement from these kind of legacy or on prem relational databases, and maybe we'll talk a little about some of the benchmarking I've been doing is I've been trying to like, get my arms around fabric. I think that's an awesome idea.
00:01:02:14 - 00:01:36:22
Unknown
I know I went through those challenges in my last role, that director role. Yeah, yeah, yeah. But I'm now a part time consultant for I know I every time I go to a Microsoft event, folks come up to me and go, where do you work now? Because I'm seeing like multiple places on your LinkedIn. Yeah. So I'm still part of the data and AI team where I used to be the director of data and AI at silk, and my lead role, the one that I spend most of my time on, is, of course, the engineer and advocacy role for Oracle and MongoDB and MySQL.
00:01:37:02 - 00:02:00:04
Unknown
Redgate. Yeah. So what? So what have you been dealing with, with the whole moving data from these, from these existing systems and the fabric and all of that? It's rather interesting. As we were trying to find out if these relational workloads, because we have to admit, they who get the data wins. It's all about getting the data into fabric, right?
00:02:00:06 - 00:02:22:19
Unknown
Right. It's also it reminds me the the way I think about it is like the like narrative arc of Linux and Microsoft, like Ballmer literally called Linux a cancer because they were still against that, like copyleft kind of licenses and all that kind of stuff in the in charge of a football team. So now and something like that, he's probably much happier.
00:02:22:21 - 00:02:53:11
Unknown
Yeah. And so, it's it's all right. I said that out loud. We got to talk about that. But yeah, but I have that challenge now. As you know, I, I'm at Redgate and a lot of the folks are like, I spent the last couple of weeks migrating and upgrading Oracle environments on windows, and I even put a blog post on it because I was like, this is an island unto itself.
00:02:53:12 - 00:03:15:06
Unknown
There's no like someone running SQL Linux. Like God said. Yeah, yes, it's it's painful. I am aware there's there may not be any teams supporting much of it at all. I know the Fail-Safe group is gone. I've looked for them. They're they're missing like the rest of the people at Oracle. May have hunted them down and killed them.
00:03:15:06 - 00:03:39:01
Unknown
I don't, I don't know, but, you know, it was painful. There was so many bugs. It's so unsupported. All the documentation is for Linux. So you recognize that, that there is a preferred platform. Yeah. And so running Linux, specially in-house and then folks go so we're going to write a PowerShell script to support this Oracle endeavor with our product.
00:03:39:01 - 00:04:05:04
Unknown
It's like no you're not the the Oracle people will actually bring out pitchforks at this point right? Yeah. So you have to use the right tool for the job for the platform that you're working in. And there's a huge percentage of workloads that are sitting in Oracle. We had a massive migration into MongoDB. There are another 30% that are sitting in MySQL that nobody knows what they're doing.
00:04:05:06 - 00:04:29:18
Unknown
They got HPE admin behind them, but they're all sitting out there. How are we going to get that data into fabric? And if you look at the documentation, it's a pretty complex process. Yeah. Yeah. No it's not well so there's a couple things. First, when you said MySQL, it's very much the SQL Server Express of the open source relational databases, right.
00:04:29:18 - 00:04:53:06
Unknown
Like because you run blogs or PHP be born like everybody spun up, like I was running lamp on a thumb drive in high school, right? Because that was the stack. But yeah, it's it's not easy to get data into fabric. But the other problem is there's like ten different ways to do it. There was a there are storage engines, everything.
00:04:53:08 - 00:05:11:13
Unknown
There was a blog. There was a blog post by Jane Sara that I was working off of for my benchmarking, where he listed like ten different ways to get it in there. And it's like, well, which 1 a.m. I supposed to use? So not only is it challenging, but you don't even know which which way is the right way.
00:05:11:15 - 00:05:34:06
Unknown
And I went through that even when I was trying to do a white paper for SQL server with silk, which is my previous, you know, employer, my really my employer, part time, where I was on your side has a side gig, my side gig, where we had SQL server and I was trying to bring that data simply into fabric.
00:05:34:06 - 00:05:56:10
Unknown
I wanted to start with the native database platform. As I configured it, I was looking through all of the different ways that I could bring this into fabric. I followed everybody's recommended practices, and the one thing that I found that I discovered is that all of the blog posts, when I got to the networking component, the end points.
00:05:56:15 - 00:06:19:06
Unknown
Yeah, it will state this is outside of my blog post, but here is the links for the recommended, you know, practices. Follow this document and each of them referred to a different document, but all of them use the same statement. That made me feel that they were using kind of a marketing post that was stated, that was said, you know, this is outside of us.
00:06:19:06 - 00:06:41:06
Unknown
And I wasn't surprised because I was at Microsoft for over five years. You had the data and AI team. You had the infrastructure team. I had moved my entire team over to infrastructure because we had to learn networking. It was so crucial, right? And if you're in fabric, you actually have to understand networking. If you're going to grab those relational systems outside of there.
00:06:41:06 - 00:07:09:00
Unknown
And that is such a different world, it really is challenging or even just off like so whenever I was just trying to move CSV files, you know, I'm not I'm not even trying to change data capture on Oracle or any of that kind of stuff. It's just CSV files. And I struggled a lot because. So if I'm pulling like some of the things were simple, the UI stuff made to be friendly to like the power BI normies.
00:07:09:00 - 00:07:34:07
Unknown
Super simple. You can upload it cost a lot of CPU or CPUs, but that's fine. But like okay, I'm trying to read from blob storage. Well, I had to learn there's like four different ways to authenticate with blob storage. You've got your account keys, your SRS, all this kind of stuff. And then there's the URL pattern isn't even consistent because you've got the blob URL and then you've got the like, Azure Data Lake, which is a completely different structure.
00:07:34:09 - 00:07:55:16
Unknown
And then you have to figure out how it maps to one lake. And then you find out, like all this stuff inside of one lake because they announced, SQL, DB and fabric, everything is like off. So it seems to target at the same time you're configuring. Yeah. Yeah. It's well it's just like it's it's like the it's like knockoff brand ice cream.
00:07:55:16 - 00:08:14:17
Unknown
Like. Yes. This tastes like vanilla, but it's not like the the Brewers kind of because like there's a, there's for all this stuff I mean you know this but like Azure Data Factory slightly different feature set SQL DB and fabric slightly different feature set. Yeah. And so a lot of your experience transfers. But then there's places where it doesn't.
00:08:14:19 - 00:08:41:00
Unknown
And they have to make these massive documentation articles of like here's what's in Azure SQL and here's what's in fabric DB. And like it's it's hard. It really is hard. And then if you're doing something like myself where you have multi-platform database, those data sets that you want to bring over from these relational systems, and they need to use historical Azure Data Factory now fabric data.
00:08:41:01 - 00:09:10:06
Unknown
Right. We're right. Yep. The OGS no connectors because the connectors don't. They don't exist yet. You have to like do this multi-step process. Kind of this multi hop process between the actual services to connect in that can be really complex for most customers. They're like well everybody's telling me I need to use Fabric Data Factory. Well you can, but you need to have Azure Data Factory still for the connectors as the connectors need to be built.
00:09:10:06 - 00:09:37:03
Unknown
Many of them are doing it for the popular things. The popular kids on the block, I like to call them. I'm going to do snowflake. I'm going to do Databricks. And it's like, that's not where all the data is. The data is still sitting in SQL server, in Oracle, in MySQL. That's where all the data is. So if you're ignoring two of those because only SQL server is getting the love, all of a sudden you're not getting all of that data and you're not going to be the winner.
00:09:37:05 - 00:09:58:17
Unknown
Well, and part of that is like, you're right, like Databricks and Snowflake is getting all love. But that's those are analytical databases for the endpoint. Like no one in their right mind is going to be running an accounting system on snowflake. That's not what it's for, right? Like no one's going to run their business on. They'll analyze their business.
00:09:58:19 - 00:10:31:21
Unknown
But like unless you talk to the snowflake guys, they'll tell you, yes, they do. So look, I do support dynamics great planes as a DBA for like a number of years, okay? I had to migrate data from SQL server to MySQL. Well, okay. It is not an easy task. It is not a simple task. No no no no no I do have a question about this because, I know, this has been like, you know, the past 5 or 10 years of your career, but I thought Oracle and Microsoft products were oil and water.
00:10:31:21 - 00:10:56:11
Unknown
So why are people moving data from Oracle to Fabric? I thought like, you know, West Side Story, like they're not allowed to touch. I have preached this forever. All the data is an ecosystem. It interacts constantly. I have always been multi-platform. If you go over to the Oracle side, they'll say, Kellen is she's betrayed us. She's always done SQL server, she's always done MySQL.
00:10:56:13 - 00:11:26:05
Unknown
I built ELT and migrated my first Oracle database to a sharded MySQL, and I say MySQL or MySQL because that tells how old school you are when you say MySQL. By the way. I created and worked on a sharded MySQL system way back in 2010, 2011. Yeah, from Oracle. From an Oracle Data Mart. We you know, I've worked on SQL server since it was SQL server 6.5.
00:11:26:10 - 00:11:43:04
Unknown
So I've always been multi-platform. When we were working at Oracle, I had only been there for about four months and I just came to my boss. I said, you need me to move them. And he says, what do you mean? I said, if the Oracle databases don't move to Azure, none of us are going to have Azure projects because all of it has to move together.
00:11:43:06 - 00:12:12:06
Unknown
The network is our last bottleneck. It is a painful, painful one. Yes. Route fast. Connect whatever you're calling it for, whatever cloud vendor and everything else, whatever they want to call it, it helps, but it's expensive and multi-cloud. If you're not doing it right, it can be really painful. I know of customers that were shifting around seven terabytes of data every single day between cloud, and they came to a screeching halt.
00:12:12:09 - 00:12:32:10
Unknown
It does not work. So put your data where you're working on it well, and the movements and the movements faster. Like I'm so naive because. So when I did the benchmarking, I made sure to generate the data in a VM in the east US to, data center, because that's where my fabric tenant was. Right. But I your fabric.
00:12:32:12 - 00:12:50:20
Unknown
Oh, you're doing better. Mine was in Europe when I was working with it. Oh, you're doing good. The joys of being like a one man shop. And I was like, okay, I need to generate there because my, I, you know, I pay for fiber, but it's home, so I get like 50 megabits or whatever. I'm happy with that.
00:12:50:22 - 00:13:12:22
Unknown
But whenever I was doing like easy copy, it was like five gigabits and I'm like, oh, this took 45 seconds to move 195GB of data. I see why they do this now. I see, you know, it's like you underestimated like the orders of magnitude from being co-located. This goes back to old school when we were doing ESP on files between Linux servers.
00:13:12:22 - 00:13:35:07
Unknown
You know, there's sometimes the oldest way of doing things is the best. AC copy has a lot of power, a lot of power. I mean, the AC commands that we were doing with bash scripts when we were at Microsoft, that's the way we we live. Yeah, that's the way we lived. And, again, when we start doing all these transformations and using Azure Data Factory, that can be painfully slow.
00:13:35:07 - 00:13:58:21
Unknown
Two customers were coming to us. They were saying, how can I even get my data in fabric if I have petabytes sitting inside an Oracle database? Oh, can you expect me to to actually have this perform, you know, how do I do this in a way that's going to make sense. I had somebody come to me from Microsoft just within the last week, ask me if they could use Goldengate.
00:13:58:23 - 00:14:27:23
Unknown
And I actually moved in the Azure Data Factory because the network latency that they had from where their golden Goldengate instance was and their Oracle database to Azure was not going to fly, they could not get it in fabric. No way, no how. Yeah. So you have to always look at that. Some of the biggest latency issues where we were brought in for performance challenges were often dependent on where the service sat within the Azure data centers and where the VM sat and where their actual data was sitting.
00:14:28:01 - 00:14:55:17
Unknown
You had to look at it all. Yeah. So I a learning question for you. So I've because I've been starting to dig into this stuff. I'm starting to like some of these pieces about data engineering is starting to click right. I'm starting to actually understand entering the lake house and database and all that kind of stuff. But what you're describing very much sounds like it with a data accent, as opposed to necessarily what we think about when we hear data engineering.
00:14:55:18 - 00:15:20:13
Unknown
And so like, can you talk about that a little bit because it seems like you people who have to do this sort of stuff are going to have to deal with a whole set of skills that they have, maybe even atrophied. Like for me, it's been 3 or 4 years since I've done anything in the Azure portal meaningfully, because I'm all power BI software as a service and fabric supposed to be the same way, but you have to get the stuff in there.
00:15:20:13 - 00:15:38:15
Unknown
And so suddenly I'm spinning up a VM or I. I set up Azure Data Factory for the first time. Right? I didn't end up testing it because at a certain point it felt like three card monte, like, where are we moving the compute costs? But other people are going to have to for reasons that you said. So what is what is this role or this learning look like?
00:15:38:15 - 00:16:11:04
Unknown
Because I think people have been weaned off having to worry about hardware. Right? I don't think you can get rid of the hardware ever. You're going to have something in your environment that sits outside of a service capability that's within that cloud. The cloud is there to serve the the standards. And every place that I've worked, there's been something that somebody first started designing underneath their desk that ended up in the data center that now needs to go to the cloud, and you need to find a way to support it because it's mission critical.
00:16:11:06 - 00:16:33:04
Unknown
I've made a career that lets you know, emergency production is Asian. I saw it a lot at my last job. Right. Yeah. Hardware gets field promoted. Yeah. And what we were considered as Oracle on Azure, we were data infra specialists. When people would bring us in, we actually knew how to troubleshoot the infrastructure as well as the data layer.
00:16:33:06 - 00:17:00:10
Unknown
It was everything. And it gave me a lot of great experience and allowed me to pretty much troubleshoot. Anything you threw me into other than networking was still a hurdle. And even with the endpoints that I dealt with when we were setting up fabric, and that was really challenging for me, but I had still enough experience to know how to do network sniffing across to the service and back, and found out where my connections were even breaking.
00:17:00:12 - 00:17:21:01
Unknown
That's a lot of expertise that you're expecting from individuals that many times are saying, anybody can connect fabric, anybody can use it. There's a lot there, end to end. You need to know. Yeah, I mean, even the stuff that's supposed to be simple, like setting up an on premises data gateway that's supposed to handle a lot of that stuff for you.
00:17:21:03 - 00:17:39:19
Unknown
Well, the problem is, whenever you can't connect to a source, a lot of people don't have those troubleshooting skills. Yeah. Like I, I worked more on it at my last job or to a certain degree. And so I was used to okay, we set something up on this server. It's not connecting. Well, I'm going to go on and I don't have a Sims.
00:17:39:19 - 00:18:03:21
Unknown
So I'm going to make a UDL file and save it as a text file and then like edit it. So I can try testing a connection. Or I don't know if you ever had the misery of dealing with SharePoint on prem, but it did help one. But go ahead. Well then you probably know this, but like, I forget the exact internals, but it saves the SQL server name internally.
00:18:03:21 - 00:18:25:21
Unknown
It hard codes it. And so if you want to do a migration yeah you don't get to just change the setting. You go on to the SharePoint server, open up SQL alias and do basically a DNS redirect. You lied to it about where the server is. There's Oracle software that does the same thing. Yes. Yeah. And so like you you kind of it's kind of has been talks about like these layers of abstraction.
00:18:26:02 - 00:18:45:02
Unknown
Right. And you you learn that way okay. Let's go below the layer of extraction. Or you know, I supported enough weird stuff that I'd have to, like, open up proxy on or on or whatever to see what a piece of software was doing because we didn't have documentation or we didn't have support for it. And so we'd see, like, what is it doing?
00:18:45:02 - 00:19:03:12
Unknown
You open a Wireshark and say, where is it talking to? And yeah, I worry that a lot of these skills have atrophied for folks because, like, I love power BI, but you get used to software as a service until it's suddenly not software as a service anymore. And now you have to learn infrastructure as a service and you're in trouble.
00:19:03:14 - 00:19:42:12
Unknown
This last week, I was talking about working with Oracle with windows, and I haven't worked with Oracle on Windows for years. Yeah, and I had to go into the registry to do a comparison of two Oracle installations to figure out what hadn't registered in the Oracle inventory, and then trace it back to find out that the My sequel installation on the same box had grabbed a hold of the Java files and was holding them hostage, but it took me a while to bring all that back into my brain and understand that and accept it, because all of a sudden on windows, everything's running as services.
00:19:42:12 - 00:20:08:18
Unknown
We don't have services, you know, we're not using services the same way. I'm Linux, I know, and everything's process based, you know, and you don't have a registry. There's all these things that you have to think about. But that's a really powerful point of realizing that OS levels these services cloud versus on premise. All of that changes everything that you're working with and being able to switch your brain to manage that and really troubleshoot it.
00:20:08:18 - 00:20:28:14
Unknown
Nothing can be off the table. You must be able to kind of I'm going to use this word that I used for my podcast early in the week. You have to be able to pivot. We have to be able to just adjust, pivot, turn the corner and be able to pick up what we're dealing with at that moment and figure out what is broke.
00:20:28:16 - 00:20:45:04
Unknown
Yeah, I mean, I think have you ever heard the wolf and wolf in Alaska analogy for for troubleshooting, for debugging? No. So I in my course I talk about a little bit, but it's like from 1982. But someone described the way that you troubleshoot code and it's true for this other stuff is the same way you find a wolf in Alaska.
00:20:45:07 - 00:21:04:14
Unknown
Alaska is a big state. It's six times the size of the UK, where you build a fence down the middle, and then you wait to hear which side the wolf is howling on. And then you do it again. You and you. You bisect the problem, but you have to learn to think that way. How do you know which side of the fence is it on the VM or is it a network or is in the service?
00:21:04:16 - 00:21:26:22
Unknown
But then also, you know, you have to get really, really good at saying, well, how do we know? How do we know for sure? And like I'm not assume take the data. You have a step forward and just yeah, I need to prove that this is the problem. And how can I do that? Because I'm sure you've dealt with too many people with too much bravado in your life.
00:21:27:00 - 00:21:53:00
Unknown
But it folks, a lot of them, especially like consultants, they're like a bulldog they get hooked on. This is the root cause. It's his code. His code is why it's performing poorly. And I'm like, if you don't have the data to prove that we're not even going to. I need a log. Show me the numbers. Right. Yeah. Well, because like, I'm trying to pivot into performance tuning and performance tuning is all like I describe it like isolate, measure and modify where you have to measure.
00:21:53:02 - 00:22:14:04
Unknown
You don't know if you made it faster. Like sometimes you do stuff and it makes it worse. So the only way you know is if you have a baseline like that's the that's the only way I have come into situations, even at Microsoft, where they were like, we shut off what we call automatic workload repository. We have no historical performance data in the Oracle database.
00:22:14:06 - 00:22:33:23
Unknown
It's running on Azure. We shut everything off. Something happened, it's performing poorly. And and people of course, in the room is like this changed that changed. How do we prove that we can't then we're starting from ground zero. Yeah, right. You start from brought over, you bought a lemon, and now you need to fix it because you don't know who owned it before.
00:22:34:01 - 00:23:08:02
Unknown
Yeah. So we put into place recommended practices. You check everything and you say, did this help? And it was funny because it was a missing recommended practice. Was the challenge that caused in the performance issue? It was the network too, was that the VM went up on the opposite side of the data center from where their application server was, that them close and proximity placement groups, everything performed well again, but because I had optimized with the recommended practices and I had optimize their workload, they got a much better performing system when I was done.
00:23:08:04 - 00:23:29:15
Unknown
And yeah, you can only hope for that in the end. Well, and I worry about this too, because like I talked about, okay, we've got it, folks like myself where their skills of atrophy because we moved to the cloud moves our servers. Yes. But then like I, I was I did a ask me Anything on the power bi subreddit yesterday and it was funny because someone's like, what's your off the record opinion on fabric?
00:23:29:15 - 00:23:48:17
Unknown
And I'm like, well, it's a good thing there's no Microsoft employees actively watching this because like, you know, OGS policy is great. But I said, like, they've got this grand ambition. Like, you are really hoping that you can take people using power BI desktop and get them using data flows and visual SQL queries on hundreds of gigabytes of parquet files.
00:23:48:21 - 00:24:12:13
Unknown
They might do it. It's possible, but I can promise you with that. Here's a question for you. Who is going to create those parquet files from all of those relational databases where there's a ton of data because I'm like, am I just BCP seeing all this data out? Am I doing Sequel Loader and and getting it out? Am I dumping it in a dump files and then turning it into parquet files?
00:24:12:15 - 00:24:35:12
Unknown
This is the thing I don't understand with the lack of connectors right now in fabric, that they are so isolated and so many of them are marked as nope, don't exist in fabric, you see. Did you see the announcement about the whole open mirroring thing? I do, but unfortunately you still have a wider utilization of systems that I working on.
00:24:35:18 - 00:25:08:08
Unknown
No, I got it. Yeah. It doesn't you it's just interesting. But yeah. No I and that's well and that's always been the challenge with with Microsoft is like sometimes they pretend they're the only game in town and it we. So I spent five years at that. Yes. Yeah. We saw this on the power BI side because early on you had things like direct query and it was mostly just it was it was Microsoft sources.
00:25:08:10 - 00:25:28:00
Unknown
And then whatever cloud databases were intimidating enough, they felt they had the support. So like snowflake and I think like sap Hana and stuff and like over time on the power BI side, the connector supports gotten better, the direct query supports gotten better. But I mean, I tell people, if you want the best experience, it should be something owned by and made by Microsoft.
00:25:28:00 - 00:26:07:19
Unknown
Because integration is easy. When it's vertical integration, you control all the parts. But like you said, like, I don't know if there ever was a time where everyone was just running on one RDM, one database system, right? Like and my last job we were mostly SQL server, but again, sometimes you need MySQL or something else to to run it either vendor choice because vendors come in and say I only run an Oracle, only run on my cycle, or I only run and or blandly run on SQL server, or they may come in these days and go Postgres, and I'm only running on these flavors of PostgreSQL.
00:26:07:21 - 00:26:35:06
Unknown
You feel that pressure that you want to be on a supported platform for that vendor product. So that's going to drive that decision we now have where I joke about everybody's team. Kellen, everybody's multi-platform because you now not only have your open source databases with your enterprise level paid license databases inside their data centers, but also usually 1 or 2 flavors of cloud databases included as well.
00:26:35:07 - 00:27:05:00
Unknown
It is become multi-platform everywhere. I am not a rarity anymore. I am everywhere I was, I was dealing with this, with some consulting I was doing because for a while I was able to stick to mostly just for power. I consulting Microsoft owned data sources, and I did a performance tuning, gig for people where their data was in Postgres, like, AWS Aurora.
00:27:05:02 - 00:27:30:05
Unknown
And you talk about weirdness. So one, I had to learn some Postgres, which was challenging because it's like learning, Portuguese. If you know, Spanish, the word sounds similar, but you're like, wait, there's no clustered index. Wait, the wait, a table scan is good, right? Just all this stuff, you're like, that was the bad word. You used the bad word.
00:27:30:06 - 00:27:53:08
Unknown
But then you talk about the integration and the support. So something I learned the hard way is if you use direct query against Snowflake and Postgres and you read the logs, so you hook a profiler, hook up Dax Studio, and you read the SQL that it emits in the event log. It is not PostgreSQL, it is T-sql.
00:27:53:10 - 00:28:18:23
Unknown
So me thinking I'm clever, I'm like, oh, Dax studio says it's running this code. Let me run this SQL against the source. Why is this working? Well, because it was emitting T-sql and then rewriting it somewhere in the Postgres, and it was awful because I didn't have access to Aurora. So literally, like, and I didn't know the best way to set up logging for that kind of stuff.
00:28:19:01 - 00:28:42:05
Unknown
So literally I had two options and they were both miserable. But this is more of a me problem. I would either run the slow queries from power BI and then scatter over to my script to run, see what processes were running. I was basically doing like SB who is active, but for Postgres, or I got a local copy, a dev copy of the database set up locally, and then I had full access.
00:28:42:07 - 00:29:04:08
Unknown
But it's it's rough and you find these edge cases that no one, no one talks about Postgres for power BI. And so who knows what edge cases you run into. It's it's very challenging. It's a bit blogpost now. It's on the backlog. It's on the backlog. All right. So I think quite a good spot to kind of kind of wrap up here.
00:29:04:08 - 00:29:31:21
Unknown
This is been awesome. This is been fun. So thank you. What is, so going through the questions, we'll start with what is your favorite fabric feature? My favorite fabric feature, I have to admit, is getting, what they're calling Azure databases in their SQL. Yeah, yeah. Azure SQL, SQL server 2025 I keep hearing people referred to it different ways because, you know, it's not really out yet.
00:29:31:21 - 00:29:52:22
Unknown
It's, you know, I like to call it the, it's not GA it's private preview right now. Yeah, yeah. But I really love that idea. I've been saying for, you know, over a while since it was fabric, wasn't GA that they needed a relational database as part of this, they couldn't just have a data warehouse and they've now brought that about.
00:29:52:22 - 00:30:19:16
Unknown
So I'm very happy to see that I don't think I had SQL server on my bingo card, but I'm all about that. I'm good. Yeah. So I that that's my favorite for the announcement. So Roon confirmed at ignite with Reza that they're working on cosmos DB and Postgres as well. Postgres is an excellent choice. Cosmos DB I'm going to be totally honest here because we're allowed to be totally honest here.
00:30:19:18 - 00:30:43:03
Unknown
Yes, it's a safe space. It's cosmos DB is just I know they want globally distributed databases and they want a solution for it, but cosmos DB does not impress me in one shape. It's the one they want. They want their mongo DB drop in. Yeah they do, they do than just have Mongo DB you know this idea that you're going to be able to corner this market on that.
00:30:43:03 - 00:31:06:12
Unknown
Just pull in Mongo DB it was the same way with Azure event hubs like I did a I did a Pluralsight course on it, and you could tell they're like, we need our Kafka competitor any time that they have, like Over the Wire, like compatibility. But it's very limited, both with like the Mongo and the Kafka stuff. You're like they start to create something and the actual enhancements just come to a stop.
00:31:06:14 - 00:31:36:09
Unknown
Yeah. Stop it. Just either buy they got the money, either buy the product that's already built that has all the features in it, stop creating something that you never invest in that stalls and that's the problem, right? It's like I said, it's it's those knockoff brand ice cream of of data technologies. And that's the that's the weird thing is Microsoft has this crown jewel of SQL server.
00:31:36:09 - 00:31:58:17
Unknown
They've got stuff, they've got good stuff. But then they make all these weird seasonal flavors of their data technology and you're like, do we do we need kql? Is there a reason we made a whole new querying language? Maybe we already has a globally distributed database that was their sharded system. They are going full force into this. They're saying we're going to move rack.
00:31:58:17 - 00:32:18:15
Unknown
We're going to move data guard into this. This is our next level. That makes total sense. Why can't you do that with SQL server? Create a started system, build it into a globally distributed database. You're done. Do not create another database platform and call it cosmos DB. Just stop. I'll get off my soapbox. Well, you're going to fall back on real quick.
00:32:18:15 - 00:32:48:10
Unknown
What's your least favorite feature or fabric? Least favorite feature of fabric is that it is so disjointed right now and disconnected. Many of the features do not work. Yeah, I, I do not envy the team. There's there's rough edges to work through. There's a lot of stuff in preview. But even as an educator, they've got such a training left because they're trying to cover such a broad base of people like how in the world can you make a training that is both applicable to someone in data science and accounting?
00:32:48:16 - 00:33:04:13
Unknown
Like, how are you supposed to do it? I do not I do not envy the people who have to manage Microsoft learning and all that stuff. For this. It's hard. It's really hard. Yes it is, it is. I also want to see better data governance in it, and I want some control over my data. I don't want it to become a swamp.
00:33:04:15 - 00:33:19:16
Unknown
Yeah, a data lake to stay a data lake. Yeah, yeah. The the work workspace approach worked when it was just power BI stuff. But now that we're putting like the whole kin caboodle, like we need some more management. So that makes sense there. At least on top of that I believe they are. They see that. They see that.
00:33:19:16 - 00:33:39:09
Unknown
And I you know, I believe it. Yeah. I don't think there's anything new to them. So where can people find you? I know there's like 12 different social media networks these days. I'm always DBA Kevlar. Yeah. I may maybe go through a rebrand. Okay. We'll see, we'll see. I've been DBA Kevlar since 2008, so it's been a long time coming.
00:33:39:09 - 00:33:58:06
Unknown
Patrick still. Patrick DBA so yeah, like it feels like the whole thing. Like whenever you make a username when you're 12 and you're stuck with it. Yeah. Well, it's funny too, because like, outside of work, no one calls me Gene. No one. But I'm like, oh, we're doing cool handles and I'll make a pun with my name.
00:33:58:11 - 00:34:20:16
Unknown
And then I just didn't really think too hard about like, oh, Gene is a name. And so yeah, but I, I made an LLC. It's too late for the rebrand. Yeah. People still call me Kevlar in multiple platforms, multiple communities and everything. And, I'm DBA Kevlar. No matter. I still am on Twitter. I keep saying I'm going to leave and somebody drags me back.
00:34:20:18 - 00:34:43:13
Unknown
Yeah, but, on LinkedIn is probably my most powerful platform for social media. I'm on blue Sky. I don't do much out there, but on DBA, Kevlar everywhere. Very cool. And then lastly, do you have anything to promote? I mean, I know you work at Redgate and work in every single product team. Yeah, but we're janitor two from what sounds like I am busy at Redgate.
00:34:43:13 - 00:35:08:13
Unknown
I'm having so much fun there because they really, they. I feel needed, and I believe that. Yeah. No, I think when we talked at summit, I could hear it in your voice like, I'm happy for you. Since the amount of products involvement that I have with their Oracle product, the the stuff that we're doing now with we're getting into MongoDB and MySQL, I stuff it just really it just keeps me engaged.
00:35:08:13 - 00:35:25:03
Unknown
I, I setting time to be around for sure. Yeah. Yeah. They're they're awesome. It's an awesome set of teams that I work with. I mean just from end to end. Love it. And of course the advocates I mean, it's Steve Jones and Grant Fritsch and rainbows. How can you be upset about that? Excuse me. Too many massages without asking.
00:35:25:03 - 00:35:33:16
Unknown
I'm a worry about Steve, but otherwise he's a good guy. Very cool. Well, this has been a lot of fun. Thank you for your time, Kellen. Thank you. It's been great. Thanks.
00:35:33:16 - 00:35:38:16