Monday, July 22, 2024
No menu items!
HomeData IntegrationSources and Destinations Podcast Episode #5 Lorna Mitchell

Sources and Destinations Podcast Episode #5 Lorna Mitchell



Putting garbage in gets you garbage out. No one understands this better than developers. No one understands developers like Lorna Mitchell. 

Lorna is an author, developer advocate, and prolific technical writer. She is currently the lead developer advocate at Aiven. Lorna has also authored several books on PHP, Git Fundamentals for Web Developers and Dzone Ref Card on Redis. In her own words she is a “technology addict and incurable blogger.” Lorna chats with us about open source, API’s, under-appreciated coding paradigms and working with engineers to become better communicators. As a developer advocate, Lorna works with engineers to better navigate and understand the choices they have when being successful with data projects. Lorna does this all with the flair and passion of a great story teller, we hope you enjoy her as much as we did.

Introducing Lorna Mitchell

Sean Anderson  0:49  

Hello and welcome to another exciting episode of the sources and destinations podcast where we talk about everything data and data engineering. We’ve got a great guest for you guys today. I think you will find her very illuminating. So we’re going to get to that here very shortly. But some exciting news at The Sources and Destinations Podcast here. In April, we had our largest volume of listeners. So thank you from everybody that checked out the podcast, all of our friends from the Reddit community, we had over 60 plays in the early part of April. And we are very humbled by you guys stopping by and checking out what we’re all about. 

Also very exciting is we have just opened up the call for speakers for our DataOps Summit on September 28 of this year. If you have an exciting topic around data, data engineering, or you want to take the conversation further, I would invite you to stop by and check out the call for papers. Send over your best topic to us and we hope to see you at the event. So that’s enough of the fluff and I’m going to send it over to my faithful co-host Dash to introduce our guest.

Dash Desai  2:01  

Thank you, Sean. So this week, we’re super excited to host Lorna Mitchell. She’s joining us all the way from the UK. She is the developer advocate at Aiven and Lorna has also authored several books on PHP, Git Fundamentals for Web Developers and Dzone Ref Card on Redis. That’s pretty awesome. And in her own words, she is a technology addict and incurable blogger. Hello, and welcome to the sources and destinations podcast. To kick it off. Would you like to say a few words about yourself?

Lorna Mitchell 2:38  

Hello, I’m Lorna. I’m a software engineer, a writer, a speaker, a teacher, an open source project maintainer. How am I doing? I don’t really know how to introduce myself. But yeah, I’m a developer advocate with Aivan. And I am super excited to be here.

Dash Desai  2:56  

That’s awesome. We’re so happy to have you join us today. So the first time we spoke, you talked a lot about the passion you have for API’s and open source projects. Would you like to share some of that with our audience today?

Lorna Mitchell  3:13  

Sure. There’s two big topics there and I guess I’ll take the API’s first. I used to be a fairly full stack developer. But the truth is, I’m a terrible front end developer. And so I became API Specialist kind of by default. API’s are really all about data, if the data exists somewhere, and I want it in another system, or I want another system to know about it. I don’t want to be moving it over waiting for a batch export or typing data from one place to another. Teaching the machines to communicate for themselves with API’s is like magic. I have so enjoyed doing that for such a big part of my career. On the open source side, I am not sure how to tell this story. Like it doesn’t have a beginning. When I was learning to code, I’ve been doing this a while so I had a book. Do people still learn to program from books? I’m not sure. Anyway, I had a book and I wrote some code. I was building odds and ends for different student societies when I was at university, and then I just never looked back. You know, we talk about open sources in business terms as being about total cost of ownership and stuff. Open source for me is about not really about free, like not paying for it, but about freedom. I don’t mind paying for things. And I actually have some money now, unlike when I was a student, but I don’t know why I would want something that I couldn’t fix or couldn’t adapt if it didn’t do what I needed. If I need a new feature. I want to add that and that just seems wild. So, um, yeah, open source is the community where I learned everything. And as I described now, I would categorize open source as a way of life.

How to Foster an Open Source Community

Sean Anderson 5:04  

So that’s great. I love hearing your story on how you landed in this open source ecosystem. I think we’d probably be remiss in talking about open source if we didn’t mention your current project at Aivan. At Aiven, we hear that you’re tasked with fostering the open source community. What does it mean on a day to day or week by week basis to foster an open source community?

Lorna Mitchell  5:28  

Aiven is a company built entirely on open source, like our community is the open source community we are a company which was founded by four tech founders. And they themselves are, I don’t wanna say they’re a bunch of geeks. But seriously, they’ve committed to Linux to Postgres and Python. They’re engineers who built something that they enjoyed using that they saw that their customers needed, and made it into a platform. We only offer open source tools on our platform at Aiven. And in fact, I have a Fedora laptop, which is our standard issue. It’s not the only reason I work there, but I do really like it. And my role is, or part of it, is in support of the company’s open source efforts, I spend time with Aiven’s own open source projects. So we have a couple of fun ones. If you’re in the Postgres space, maybe, you know Pghoard, or in the Kafka space, we published Karapace, which is a schema registry REST API interface. I spent time with our open source projects, looking at the issues that are coming in and maybe doing some work on the repos. Sometimes I even fix stuff. And then, as a developer advocate, I often published open source demo code. I’m active in the open source communities. I mean, Aiven only has open source data technology. So by definition, we’re in the open source space, stuff you can do with Aiven, you can do yourself if you run Postgres locally or use another provider. 

So it’s, it’s very open. And that’s really nice, because it kind of brings me into contact with different people using other platforms, other tools. And you know, when you are a developer advocate for a company that has open source projects, you are an advocate for those projects themselves. So I’m often creating content that enables users to get the best out of Postgres or Kafka, or Redis, whatever it is, and I almost don’t need to mention Aiven. I love that.

Dash Desai  7:31  

That’s awesome. Thanks for sharing that. So what are your thoughts on how data engineers perceive open source tools and technologies these days?

Lorna Mitchell  7:41  

I suspect that I’m seeing kind of a small segment or maybe just not the whole industry in the areas of data engineering that I come into contact with. I’m seeing the kind of Python enabled crowd. And those data engineers expect to be able to work with familiar tools and to be able to access that crowd wisdom of all of us using the same tools, sharing ideas, sharing expertise, maybe sharing extensions, or I don’t know, approaches. Again, I feel I’ve returned to my theme of open source. But, you know, that’s my world. And that’s what I’m coming into contact with. I do see some pretty slick other offerings. But I also enjoyed that interaction with the standards with being able to create your own connectors, or anything. So I just don’t think open source is going anywhere in this space.

Sean Anderson  8:42  

Lorna, I think I’d have to agree with you there. I did want to ask you real quickly, and this kind of brought up a thought in my head as you were talking. What do you think in your community or in open source communities that you have observed people value most about open source? Is it really committing to the code? Is it the crowdsourcing like you talked about? Is it actually getting your hands on the code and creating that extensibility? When you talk to these people in the open source ecosystem, what are they most excited about?

Lorna Mitchell 9:14  

I think the most exciting things in open source are everything that you said, and maybe more. The familiarity, because we’re all building on the same stuff. All our efforts go in the same place, and you take those skills and that reputation with you from place to place. You know, if you want to change jobs, or you know, start something else, you have that knowledge, and we’re probably all using the same tools. 

I think security is also an important one. And that’s part of the transparency piece, right? And there’s kind of this gray middle ground where the sources may be available, but you can’t offer it as a service or whatever. But I think the security, the accessibility, the openness of the code, even if you’re not going to contribute. Knowing that you are not locked in is a really important part of this story from the people that I come into contact with.

The Importance of Communication Skills

Sean Anderson  10:07  

That’s great, and I love how you preface the no lock-in really about security because it’s not only business security, but at the end of the day, it’s developer security, right? You guys are out building things and you want to have that peace of mind that you’re building things that are really reusable and evergreen in a sense as time goes on. Now, I wanted to touch on this because I thought it was really great. We had a short conversation before this podcast. And I know that you’ve talked in previous forums about the idea that data scientists have developed these communication skills where they can really relay what they’re trying to do to somebody on the business side of the fence. But you say that oftentimes, engineering departments don’t have that abundance of clear vernacular. So I’d love for you to touch on that a little bit more.

Lorna Mitchell  11:10  

I don’t intend that as an insult to engineers anywhere. But, I do think that certainly the way that I think about data scientists work, and I’m not a data scientist. Is that they, you know, they find the data, they clean it, and then they do their actual data science work. Then they do the final step, which is to go ahead and explain to me what that is that they did, and what it means and what I can do with this information that will change my life. Humans are innately bad at math, and especially things like probabilities, or risk and good data scientists bring those findings and turn them into something that I can use. And it really strikes me as a difference between that and maybe the software engineers that I’m more commonly mixed with, where they don’t always take that last step. They might have made something amazing, you will either never know about it, or you’ll never know how to use it. And especially at the more junior levels of the software development profession, there just isn’t a good understanding that documentation is part of the job. I think it’s secretly a superpower. I don’t know any amazing developers that are not also great communicators, especially in written form. And now I work with some really excellent and high level engineers, and they all communicate pretty well.

Tools and APIs

Dash Desai  12:38  

Yeah, thanks. That brings up to my next question, what are some of your top technologies these days that you love working with?

Lorna Mitchell  12:46  

I have the whole of the Aiven platform to play with. So I don’t know how you think I’m going to choose my favorite technologies. Let me think about that. It’s been really nice to spend some time with Kafka, in particular, and building some streaming demos. I’ve been away from that sort of technology for a little while, like just a couple of years, doing very much API specialism. And so kind of coming back to this, and the tooling has come on a bit, I’m really having the chance to play. I have enjoyed that. You know, I think that’s been really nice. And the other thing is, everyone loves the new shiny, right? And at Aiven, we’ve recently released M3, which is a distributed time series database. So that’s a huge amount of fun, getting up to speed with that, or getting to play with that seeing the community start to get to grips with it, seeing what questions they have, what their building. Most people are seeing it for the first time and I am just greatly, greatly enjoying that.

Dash Desai  13:57  

Yeah, so I have a follow up question. Since you talked about API’s and how much you love them. Maybe the right question to ask is, what are your top API you love playing with?

Lorna Mitchell 14:08  

I have a few favorites. I have to call out the Vonage API’s, they do telecommunications and I was working there until December. So I obviously know and love theirs and have built a lot of integrations with their various offerings the last few years. I’m teaching API’s, there’s a brilliant Pokemon example API that has all the different endpoints in it. So I’ll often use that, just think it’s a lot of fun. Whether you are familiar or friends with those Pokemon characters or not. There’s something for everybody. And it has all different types of fields and stuff. So it’s a silly example. But in a way that makes it something we can all relate to. It means that you remember it. 

Also in the API space, I think the way the tooling is improving with the open API standards and some of the stuff that Postman in particular are doing. It’s just a super exciting place to work. So I don’t want to call out individual favorites, but there’s as a whole sort of space. It’s a very, very exciting time.

Dash Desai 15:14  

Yeah, I totally agree. I love API’s and I used to go to this conference that was called Unconference. It was all about taking different API’s, and creating these mashups of applications. I met a lot of different developers and engineers through those conferences. 

Under-appreciated Software  

Sean Anderson  15:38  

And as we know, with Pokemon, you have to catch them all. So it gives you plenty of variability there. So I think that’s a really cool example. We talked about what technologies you’re in the middle of and that you’re dealing with. Are there certain technologies that you feel either have a bad misconception? We obviously know that the data ecosystem is completely devoid of opinions. But are there ones that you feel are extremely powerful, that maybe get, you know, criticized or shunned in a way?

Lorna Mitchell  16:13  

I’m not sure there’s anything that’s really got a bad reputation but there’s a bunch of things that are just really under appreciated. And it’s things that you know, so well, you’ve forgotten, that it’s even a tool or that someone had to make it. And for the tools that I use every day, that stuff like JQ, it’s Kafkacat, even PSQL actually has some really cool features if you go digging, and I forgotten that because my fingers have the muscle memory. And I’m not discovering an amazing new tool. And I just mentioned Postman. But today I had someone who was documenting an API for the first time asking questions, and then, they had like a Curl cheat sheet, then they were introduced to Postman, I don’t need to paste the variables every time. 

And it’s like, I didn’t even think to mention this to you. And it’s those utilities that we  can’t even see anymore. Because we’ve forgotten that they could ever not exist. And typically, for me, those tools are not the GUI tools, because I have some accessibility needs. But stuff that we use every day, especially the kind of utility diagnostic tools, they do not get the credit that they deserve.

Sean Anderson 17:32  

I agree with you, it’s often the kind of unsexy side. But realistically, you can’t be successful without a lot of those things. I think a lot of developers and engineers really kind of hang their hat on that. So Lorna, I wanted to ask you a quick question. You are a developer advocate, which means you’re advocating on the part of developers in your ecosystem and everywhere, but you’re also a very established author, and writer, right? As well as being technical. So I’m curious if you had to have a conversation with your engineering self as a writer. What would you encourage engineers and developers to do more of? You’ve kind of seen both sides of the fence. I’m curious what advice that you have for a developer or a data engineer, that maybe wants to up-level their skills and start doing a little bit more writing and advocating on behalf of their company?

All Developers Need to Write More

Lorna Mitchell  18:29  

All developers, everywhere, need to write more. I think if I could give advice to every engineer, it’s probably that they’re overthinking it. If you’ve read my blog, I am not overthinking it. Right? I’m just drinking coffee and typing words, it’s fine. I think I just wish that engineers understood how important it is to share things. Whether it’s feeling grumbly about documenting a feature. But why build it, if you’re not going to tell everyone how they can use it and enable hundreds of people? 1000s of people or whatever, to make use of your feature? It’s like not writing tests, why would you build it if you’re not gonna not make sure it’s working or not make sure your intended audience can use it. I think there’s a barrier there. I think there’s something that we think that writing is so other from code, and it isn’t developers, if for the technical facing tools, your audience is a developer, just imagine that I was beside you and you could tell me what you made. Okay, now write that down because I’m not actually beside you right now. But I might need to know this someday.

It’s that showing up and just finishing it off. I don’t know any really great engineers that are not really great communicators, but it doesn’t come overnight. My blog is 15 years old and I wasn’t a published author at the beginning. So this is about wanting to share, understanding who you’re talking to, and wanting to empower others. And then I think the words come, and you know, the grammar, it really doesn’t matter if you can be understood, you’re doing great!

Dash Desai  20:24  

And I totally agree. That’s how I transitioned myself from a full stack software engineer/data engineer to being an evangelist. Because I wanted to share what I was learning with other developers. That’s how I started writing, speaking at conferences and things like that. So I totally share the sentiment. 

That One Weird Thing

Dash Desai  20:24

Okay, so I believe we are on our last segment which is called that one weird thing. As we work on data engineering projects, or software engineering projects, there’s always that one weird thing you run into all the time. A great example is working with time zones and data formats, between different data sources, destinations. Is there something similar that stands out for you that you want to share with the audience today?

Lorna Mitchell  21:19  

I think sometimes we don’t think enough about the source and the destination when we’re debugging a problem. I had a really good one recently, I spent a really long time debugging the output of something. And actually, it was the input. That was the problem. So I was actually sending nonsense. But I couldn’t understand why the tool was showing me nonsense, because I didn’t debug every link along that chain. You know, when you work in API’s, you’re making a request, and the response is garbage. Huh, Did you check whether the request was garbage? It happens more often than you think, you know what came back but do you know what you sent in the first place? So debugging every possible link in the chain and trying to keep that operational, so you’re looking at what it looks like before it leaves the building, as well as what it looks like when it arrives there. I was trying to fight with my code that was sending something to behave better, but actually it was on the other end. It was the thing that I was using to display that was actually causing the problem. So I think understanding how else could I get insight into this? How else could I approach and observe this problem? That’s the key. That’s the key skill.

Dash Desai  22:41  

Absolutely. Basically, garbage in garbage out. Right.


Sean Anderson  22:45  

So Lorna, I know we’re rounding up on our time here. If you could give our listeners a little information on how they can find out more information about what you are doing and more information about Aiven. That’ll help everybody make sure that they can navigate and continue the learning process.

Lorna Mitchell  23:02  

Okay, well, let me start with Aiven. So we are We have a free trial. So go and kick the tires on whatever open source data offering we have that you need. I think for me as a reasonably new developer advocate, I joined this company earlier this year. It’s always amazing to hear your first impressions. We can deploy to whichever cloud you’re using, and there’s a button to move from one cloud to another, which I will never get tired of pressing. So check that out and let us know what you think. For me personally, I’m Lorna Jane, almost everywhere. So is my website, l am LornaJane on Twitter and on GitHub, so you can reach me however you want to. And yeah, I’d love to hear from your audience and hear what you’re building is always great to hear other people’s stories.

Sean Anderson  23:54  

Fantastic Lorna, you’ve been a great guest for us here on the Sources and Destination Podcast. So I definitely encourage our listeners to find out more about what you’re doing and continue the conversation and the laughs from today’s today’s show. So that’s it for today’s episode of the sources and destinations podcast. We will be back with you guys in roughly two weeks with another great guest. If you’re liking the conversations you want to hear more you want to make sure you’re aware of the next episode, just go to your favorite platform that you listen to us on and hit that follow button and then you’ll be sure to catch all of the great episodes on the sources and destinations podcast for myself for Dash. Our guest today Lorna, thank you all for joining us and we hope to see

If you would like to find out more about Lorna find her on Twitter and at

To find out more about the podcast follow us on your favorite platform.

The post Sources and Destinations Podcast Episode #5 Lorna Mitchell appeared first on StreamSets.

Read MoreStreamSets



Please enter your comment!
Please enter your name here

Most Popular

Recent Comments