Machine Learning Zoomcamp: Free ML Engineering course. Register here!

DataTalks.Club

Articles Slack Events Podcast Books Courses

Freelancing and Consulting with Data Engineering

Season 9, episode 4 of the DataTalks.Club podcast with Adrian Brudaru

Links:

Did you like this episode? Check other episodes of the podcast, and register for new events.

Transcript

The transcripts are edited for clarity, sometimes with AI. If you notice any incorrect information, let us know.

Alexey: This week we'll talk about freelancing in data engineering. We have a special guest today, Adrian. Adrian worked as a data engineer, but four years ago (if I'm right, he will correct me if I'm wrong) he decided that office life is not for him and that he wants to be a freelancer. So, welcome. (1:40)

Adrian: Thank you. It's actually five years now. Yes, five years – since two days ago. (1:59)

Alexey: Wow! [chuckles] (2:07)

Adrian: So thank you for having me. Yeah. (2:09)

Adrian’s background

Alexey: Thanks for joining. Before we go into our main topic of freelancing in data engineering, let's start with your background. Can you tell us about your career journey so far? (2:12)

Adrian: I studied economics with a specialty in marketing in Romania. I thought it would be a platform or a method to statistically analyze people's behavior. But I discovered that the marketing industry was mostly people that wanted to be creative, and not so much look at the results. We even had a joke about PR that's basically when you throw the money out the window and you don't look at the results. (2:22)

Adrian: That aside, I came to Berlin at age 25. I wanted to do things more mathematical and more logical, so I started a business analyst role. Five years later, I started working in a corporation. I didn't like it, so I switched to freelancing. I have to say, I really love the journey. Given the choice to make those choices again, I would choose freelancing again, but sooner. (2:22)

Alexey: So you would keep the part where you work in a corporation? (3:24)

Adrian: I wouldn't, but you know… (3:30)

Alexey: [laughs] Okay. Can you tell us about this story? How did you become a freelancer? (3:34)

Adrian: Well, in all honesty, I hadn’t worked in startups for quite a while. Startups are known, especially like 10 years ago, they were known for being very poorly paying. I joined the corporation about five years into my career and I found that, despite the better level of pay, the level of challenge was much lower – both the standards and the opportunities to do things. I didn't like it. I actually couldn't quite adjust to the environment. I couldn't make it work for myself. I wasn't able to actually perform my job duties and I was let go. I was actually too demotivated at the time to look for a new job. I literally couldn't see the good thing about joining another company. So what happened was, I just told the recruiters “Yeah, if you want to place me – place me on a project.” (3:40)

Alexey: That's the end of the story? How did it go after that? (4:47)

Adrian: The recruiter that I was talking to got back to me with a couple of projects. Within a couple of conversations, I was already signing the contract. My first project was helping a team clean up a lot of legacy that had been left behind over the years and implement Airflow. That was pretty nice. It was actually the first time I was doing that. I guess I was worried in the beginning that work would be difficult to come along, but actually, the projects just kept coming. I didn't even take vacation for the first year and a half. (4:50)

Freelancing vs Employment

Alexey: It seems like it was quite easy for you. A recruiter just reached out to you and you said, “Hey, you know what? I'm more interested in project-based work rather than a full-time job.” And then the recruiter got back to you, right? Sounds quite easy. Was it like that? (5:33)

Adrian: It is actually like that. When it comes to the fit between yourself and the job, you normally have a lot more steps involved. You want to look for good company, you want to look for a good team, you have maybe four, maybe six interviews. With freelancing, it's not like that. It's more or less like they look at your history, they ask you if you can do it, what you think about it. And you should be honest because, frankly, if you can't do it, that's just going to follow your reputation. If everybody has a good feeling, then it usually takes only one or two talks. There is no job security, so if they don't like working with you, you could get fired within a day or five. (5:46)

Alexey: So that's why the process is much easier, right? Because to get a job at a corporation, you need, as you said, four to six interviews. And there are all sorts of interviews – behavioral interviews, coding exercises, take-home assignments, and so on. But here you just talk – they tell you what to do, you say if you can or cannot do this and that's pretty much it. That sounds way easier. [chuckles] Why isn’t everyone doing this? (6:38)

Risk and occupancy rate in freelancing

Adrian: Yeah, I think the reason why not a lot of people are doing this is because nobody likes risk. And there is an element of risk involved. Ultimately, you need to calculate with some kind of occupancy rate. I think if you calculate with it at around 75%, then you might be on the same level as salary, depending on how well you negotiate your salary. (7:06)

Alexey: What is occupancy rate? (7:33)

Adrian: Excuse me? (7:37)

Alexey: What is occupancy rate? You said it should be 75% or something. (7:39)

Adrian: There are hours you're going to manage to occupy. You have about 2000 a year, maybe less. So if you can calculate with occupying 1500, that would be good. (7:43)

Alexey: So this is the risk – not being able to occupy all the hours. Correct? (7:58)

Adrian: Yes. Frankly, the risk is real. For a lot of people, there is a lot of risk. If you take a salary that's equivalent to 75% of what you could make as a freelancer and people's expectation of you might also be different, the process with feedback and adjustment might be different. Some people say that's false security. I would agree since I've worked in my share of startups, and I've been on the list, and I've seen other people on the list – and I keep seeing them. This keeps happening. (8:03)

Alexey: The list? What is the list? (8:43)

Adrian: [inaudible] I think, only in the last couple of months, we've seen a few. Right? (8:44)

Alexey: Ah, these kinds of lists. Okay. But still, a full-time job is probably more secure, in a way, in that it's not easy to let people go. Maybe? (8:51)

Adrian: Ultimately, you can have unemployment insurance. So in the worst case, you're going to receive unemployment pay as a freelancer. You can optionally pay the insurance but at least in Germany, the payout is kept at such a low amount, it wouldn't be worth it really because you'd be paying a very large percentage of a large sum of money, which you wouldn't ever be getting back. (9:04)

The scariest part of freelancing

Alexey: What was the scariest part for you when you were transitioning to freelancing? Okay, you told the recruiter that you were more interested in projects and then the recruiter told you “Okay, we have these projects.” Were you scared at this point? Did you worry about this? (9:27)

Adrian: I was more worried about impostor syndrome in the beginning. Also, let's say on my second project, we didn't have very good chemistry or “team fit,” so I didn't enjoy working in that team. I wanted to end it early, which I did. It was the right decision – everybody was happy with it. I was initially worried, “Am I going to find work?” But I did. (9:43)

Alexey: Okay. So you're saying that something like that might happen. Since there are just two interviews, and during these interviews you don't know if you're going to like the company, and it might turn out that you don't. But, it's fine – there's nothing to worry about. You can just say, “Okay, looks like it's not working for us. Let's part ways.” Right? (10:20)

Adrian: Yes. I think the biggest, scariest thing in freelancing is that you're not so much in control over your flow of projects. If you want to get a job, you know what to do. You can make a list of 50 jobs, you start applying to them in order, or five of the time – eventually, you're gonna get one. It's in your power. But when it comes to freelancing, it's not like that. Usually, the set of projects is kind of limited over a short period of time. So if you look out there, you'll see all of them, or a recruiter will tell you all of them. But if you check back next week, it might be a different set or not. But that's kind of it. Unless you have your own personal network that is coming to you, which again, they still come to you so it's not so much in your control. You might start feeling a little scared as a result. (10:41)

Adrian’s first projects

Alexey: Probably we should talk more about the personal network, but a bit later. Right now, I want to ask you about your first project. I think you talked a little bit about that – you said that your first project was to clean the legacy setup. Were these the kind of services you provided? Like building data pipelines, setting up Airflow, removing legacy? Do you remember what exactly the services were, the skills, the things you worked with? (11:36)

Adrian: So I developed quite a diverse skill set before because I did end-to-end in startups. When you do that, you have to do everything. So I was pretty flexible, actually. I was more concerned about being a freelancer than the specific type of work, so I have to say I was quite flexible. The first work was essentially engineering/programming. The second project was data science. The third project was – actually, it was a funny one – it was helping with an ICO. (12:05)

Adrian: I went into an office in some strange buildings to be alright with the data privacy laws. It was interesting. I charged double my rate because it was on short notice, on a Sunday. That aside, I think my third project was actually one that took about a year. It was building a data warehouse and later creating the team for an urban sports club. [cross-talk] (12:05)

Alexey: But for this one – the one that took one year – where you were building a data warehouse and then the team, isn't this kind of stuff that companies usually have a full-time employee to take care of? Like head data or whatever. (13:27)

Adrian: I think a lot of people believe the same thing. This is why I think it's not a great idea to be actively looking for work as a freelancer – some people already know that you're not the right solution, just because you're not employed or something like that, but other people just want to solve a problem. This is how it comes to be. Ultimately, if you are reliable, if you are able to deliver independently, if you're proactive – you will see a lot of interesting work. (13:44)

Alexey: So I guess they didn't hire you to build the team, but eventually you just proved yourself to be reliable and then one project after another with this company, you ended up building a team for them. Is that right? (14:18)

Adrian: I pushed them, right? Because I built a data warehouse, which took two weeks. Then it took like three months to actually properly define what we should be looking at and to get everyone to agree. And then came the point “Okay, well now you're going to want to have someone in-house taking care of this.” So we hired the first people, we got some embedded analysts because now the things are consuming the data, training some people, and finally hired a replacement. [inaudible] (14:31)

Freelancing 5 years later

Alexey: How did your services evolve over these five years? What do you do now? Is it much different from what you did? (15:08)

Adrian: Right now it’s actually quite different. I would say my last major project was building a prototype – something that’s not quite production ready, I think it's just getting polished for production right now – of a loading framework, kind of like Fivetran but for a specific client (for a specific space). Now, I actually founded a company with two other people, one of whom I met on my previous project and I still do a little bit of consulting on the side. But I have a max capacity of one day, so I usually try to do more fun things like a little coaching. (15:16)

Alexey: Okay, so you don't actively freelance or consult right now, it only takes 20% of your time, but you still do this. And you spent the rest of your time working on a product prototype, as you said, similar to Fivetran, and you also have a company. What does the company do? (16:07)

Adrian: This company that we started didn't have a clear objective in the beginning. I think when it comes to starting companies, one of the most important things is to have a very high-functioning team. This is what we had, and we didn't have a clear idea of what we were gonna do, but we decided to do it. We ended up building a data pipeline for a company that one of my co-founders is invested in, which is Rasa. (16:24)

Adrian: After doing this data pipeline, we realized that the core component that we used for building this data pipeline is essentially something that we have – all the data engineers have written it so many times, over and over. So what we're doing now is, we're actually going to release it as an open source component. I would guess it's about a week away from version 0.1. So don't consider that a launch just something we’re doing. (16:24)

Alexey: Okay. So you have a company, but at the beginning, it was more like a consultancy company, even though maybe you weren't doing the actual groundwork. But there were people who would go and do this. You are one of the founders of this company and one of the things you do at the company is this open source project? (17:39)

Adrian: Yeah, right now at this company, we're just building this stuff. We don't have employees. We don't even have support for it. Really early. (18:02)

Pay rates in freelancing

Alexey: Okay. [chuckle] Yeah, interesting. But let's say now, you're pretty advanced in your freelancing/consulting career, but back when you started, I guess one of the difficult things (maybe it wasn't difficult for you, I don't know, but it looks pretty difficult to me) is deciding how much to ask the clients. Like, should it be an hourly rate, should you ask for per-project pay? If it's an hourly rate, how much to ask for? How did you solve this problem? (18:12)

Adrian: For me, it was pretty easy to go hourly, partly because it was offered to me that way on the first contract. Then I quickly realized how amazing it is to work hourly, because this means you don't actually need to sit for eight hours in the office. Also, if you want to go after seven hours, you can go after seven hours – if you're fine with taking the pay cut. If you have somewhere to be for a few hours in a day, for whatever reason – as a freelancer, you dictate your availability. You're not necessarily expected to be there all the time, so it's way more flexible. I find that to be a better quality of life. And if I do want to do overtime, I get paid for it. (18:40)

Alexey: Okay. And how did you select the rate? Did they just come to you and say, “Hey, this is the rate that we can offer. Take it or leave it.” Or how did that come about? (19:26)

Adrian: This was the first project, and I have to say the rate was really low. I didn't know that at the time. So I would say the rate that you should be charging should be the best one that you can get. I say that because depending on how you position yourself and what kind of clients you have and what kind of skill set you have, you might be taking as low as 60 per hour if you’re working not in Germany, let's say – so not for German clients. (19:37)

Alexey: 60 dollars? (20:06)

Adrian: Yes – or euro or pounds or… it depends. But if you're a little more senior, if you're going on-site, if you know the people, if you don’t have a middleman, if it's not a corporation that has strict budget ranges and they just want to get stuff done and they want good talent – then you might be charging as much as 150 per hour. (20:07)

Alexey: One hundred and fifty. Okay. (20:36)

Adrian: The rate also very much depends on what you're brave enough to ask for. For a lot of people, it reflects more from their own opinion of their own value rather than the company. (20:37)

Alexey: Because I can ask for, I don't know – 500. But how many people will actually be willing to give me 500? I imagined that not so many, right? [chuckles] (20:49)

Adrian: Right. Part of the reason is simply because they think that 500 per hour is just unreasonable for a person, but other people just think about the value. If you are doing something that is special and unique, you can get paid for that. (21:01)

Acquiring skills while freelancing

Alexey: A couple of questions ago, you mentioned that in startups you learned how to do things end-to-end. So you were fine doing pretty much everything. Your first job was engineering and programming, data engineering, and then it was data science and it was building a data warehouse – how did you acquire this versatile skill set? (21:17)

Adrian: By working a lot, I would say. I think it's good to always find work that challenges you at least a little bit. When I look for job fit in a project, I look to have, let's say, somewhere between 80 and 60% match on skills. This allows me to learn on every project. The thing is, when it comes to freelancing, people expect that you will not be skilled or a specialist with a specific thing that they're working with. So usually they do accept that you might not be great at the specific technology or whatever, therefore, you will have time to learn on the job and get paid for it. That's quite nice. (21:41)

Alexey: But why would they hire a freelancer to learn on the job if they can hire a junior to learn on the job? (22:29)

Adrian: I think a freelancer that has years of experience in both doing things and in learning things is going to be able to learn much quicker and apply way more context, and also handle things more maturely than an employee. To give you an idea, I view myself as an entrepreneur in what I do, not just a freelancer, because I need to make sure that I am getting paid, that I am getting clients, and so on. Sorry – I got anxiety and lost my track. (22:39)

Working with recruitment agencies and networking

Alexey: [chuckles] Yeah, I also don't remember. I was checking questions on YouTube and I don't remember. But maybe the other thing I wanted to ask you – I know, since I got contacted by agencies as well, and the way these agencies work is: they have some clients, they have some work, they contact me and say, “Hey, there is some work. Please work for our client,” and they take a cut. Right? They offer you X amount for an hourly rate, but they charge the client X plus 50%, for example. Is this how they work? As freelancers, should we work with them? Should we try to find clients ourselves? Is it worth talking to them? (23:19)

Adrian: That's really up to you and how you like to work. To give you an example, there are multiple types of intermediaries. There could be a really big staffing agency like Hayes or Modis – essentially, what these guys do is just intermediate a transaction and you're interfacing with the client directly. This means that you can actually create relationships with the client and if you don't have your own relationships or other avenues – this is okay. It's probably one of the worst earning possibilities simply because these agencies often take a third of the project value and they do not negotiate good rates for you because they just want to sell the project. They don't care about your bottom line. So I would say that's fine if you don't have your own network – 80 per hour is still a lot. (24:10)

Alexey: Would you suggest to somebody who is just getting started in freelancing to get in touch with Hayes or some other agency and work with them first? (25:10)

Adrian: Absolutely. I think, if you want to be autonomous, that is a good way. An alternative for if you're just getting started would be to get in touch with smaller agencies as well. When it comes to smaller agencies, usually, they are expected to do project management as well, since they are the one selling a piece of work to the client. That means you need to sync with the agency and with the client. (25:21)

Adrian: In my experience, this is harder because now you have two different personas that you need to make sure you understand well, and synchronize them. It's actually not my favorite way. If it was up to me, I would either go with something like Hayes because you're completely relaxed – they do the finding for you. Or with my personal clients. (25:21)

Alexey: Your first project – was it from an agency? (26:09)

Adrian: It was from such an agency, yes. (26:14)

Alexey: And the second one? (26:18)

Adrian: It was from the same agency. (26:20)

Alexey: Okay, and at which point was it not from an agency, but from your own network? (26:22)

Adrian: Well, I can give you a pretty simple history. This agency was actually paying me 60 to 70 per hour, which was quite low – even for the time. So I started looking on my own. At this time I had started to create a network – talk to people, to other freelancers, to potential clients. Essentially, at that point I was able to just answer to people coming back to me and ask for a higher rate. That’s when I stopped working with those agencies. (26:27)

Alexey: Okay. So now you find new customers and projects without these agencies, right? (27:05)

Adrian: Yes. But I will say that now, the market has also changed a little bit. If you have ‘freelance data engineer’ in your title on LinkedIn, you will get contacted a lot. (27:11)

Alexey: So it changed in a good way. Right? (27:22)

Adrian: It changed in a good way, yes. I think all these data scientists that have been in training for the last years have finally matured as professionals on the job market. And there is a lot of missing engineering to support. For someone who is a bit more generalist, it's a wonderful time. (27:24)

Looking for projects and getting clients

Alexey: So how do you find new customers and projects now? Through your network? (27:45)

Adrian: Now, I actually don't look for projects so much. Right now, for example, I was looking for what we call a ‘design partner,’ which is somebody that works closely with us and with this new tool that we're building in order to help us find bugs and use cases. For those people, yeah, through my network. Sometimes it’s asking people that I've worked with before, or even if I see like a new fitting project. (27:51)

Alexey: But you said you still spend 20% of your time on consulting. How do you fill this 20%? Where do customers come from? (28:20)

Adrian: The last customer actually came from my network and it was the third time around. He previously came as a customer on two separate projects. It's good to have a network, what can I say? When you network with other freelancers, it happens very often. If you have a good relationship (and by good relationship, I mean be a decent person, I don't mean sucking up) if you're a decent person and you can communicate well, then you will get follow-up work quite often. (28:31)

Alexey: Okay. That sounds simple, right? Be a decent person. (29:17)

Adrian: Be honest and communicate directly. I think the biggest fear for people is that you will create problems when they bring you in. One way to create problems is by not communicating directly and honestly. If you do communicate, usually everything can be resolved. (29:22)

Alexey: How do you tell the clients that you will not create new problems for them? (29:44)

Adrian: You listen. If you see that there are certain expectations and if you see that those expectations are at risk of not being met – I don't know, if your work is too slow due to whatever reasons, maybe it's you, maybe it's something else – communicate that ASAP. Manage expectations. (29:52)

Freelancing vs consulting

Alexey: Another thing I wanted to ask – you started as a freelancer, but now you say you do consulting for 20% of the work? What is the difference? In my opinion, from what I understood, freelancing is more often when you do the actual work yourself, but consulting is when you tell others what to do? Is this a correct explanation? (30:16)

Adrian: In a way, yes. Practically, my experience has been that, by this definition, you're only purely a freelancer when you're coming through an agency. In any other situation, you're also doing the sale – So you’re consulting people, like “If you're in this stage, who's taking care of your stack? Then maybe you could use this or that.” It eventually ends up – well, now that you've told them what could be a solution that works, you could actually implement it for them or find something that can help them. (30:36)

Alexey: So the lines are blurry, right? You can call yourself a consultant, but do everything as well. (31:15)

Adrian: I guess the distinction is that – for me right now, due to the limited amount of time that I can spend with a topic, I'm not going to spin up your repository and do any programming, because it's just not feasible for me to change between yet another [inaudible]. (31:22)

Clarity in clients’ expectations (scope of work)

Alexey: Yeah. We have quite a few questions. One of the questions is, “When you talk to a company – to a potential customer – and they tell you about the problems and about the use cases, do you usually get enough clarity of what they expect from you before taking on a project? Or is it usually more vague like “Something is broken, and we don't know what to do.”?” (31:43)

Adrian: It depends. “Something is broken, we don't know what to do,” is also clarity. Then you usually can try to create a scope of work document before starting the work because this helps you define and manage expectations. That scope of work might be that we do a spike for two weeks and identify all the problems and what we need to do as the next steps and we agree on that. Then we reassess after two weeks whether we're going to continue. (32:10)

Alexey: What does the scope of work document look like? Is it a Google document with a description of a problem or something like that? (32:45)

Adrian: I mean you can Google it, but the way that I do it is quite personal. I've gotten the feedback that it's quite specific. I like to put in as much detail of what goes into scope, what is not in scope, things about expectations, what they can expect also about working style and things like this – timelines as well. (32:55)

Alexey: Is it something that you can share as an example? Or maybe there are examples online? (33:20)

Adrian: I can share an example afterwards. I will need to analyze it. (33:27)

Alexey: Okay, I’ll take a note to ask you about that because it looks quite interesting. You also said you do a spike for two weeks – for those of us who don't know what a spike is, can you tell us about the process? (33:32)

Adrian: It's just taking a defined amount of time to refine a problem that is not defined. When you have a problem and you don't know what it is, then you can't say if it's worth solving – it might take too long, for example. So the first thing you do is actually define it. (33:48)

Alexey: I guess here, the main message is working through iterations, right? You don't know what to do, you don't know what the deliverables are, what is expected of you, what the scope is, so you take an iteration to figure that out. Then after each iteration, you can see if it still makes sense to continue, to what extent to continue, what the scope of the next iteration will be, what the milestones and projects will be, and so on. Right? (34:09)

Adrian: With the scope of work documents, I don't know if it's worth overthinking it simply because my experience has been that different agencies have asked me for different formats. Whatever works – ultimately, it's a document that you use for managing expectations and one that you can refer to on a contract. (34:38)

Building your network

Alexey: By now, I’ve understood that having a network is very important when freelancing. How do I go about building a network? Do I do this online and show this work online? Or nobody cares about online and you need to network offline? (35:01)

Adrian: I think when it comes to actually building a network – you're not really building a network, you're building relationships with individuals. It goes without saying that the best way to build a relationship with someone is actually being present and being there and having the opportunity to communicate in all the nonverbal ways. Ultimately, what needs to happen is that, at the end of the conversation, you know what this person needs and this person knows what you need, and they will remember you. Maybe two years later, they might get back to you and say, “Hey, you're that guy that was doing that thing? Are you available?” (35:20)

Alexey: So the best way to build this relationship is to show how well you work and then they will contact you, or maybe recommend you to their friends who need help. Correct? (35:59)

Adrian: The best thing to do is to be proactive. What I did in my first year of freelancing was try to have a target of meeting at least two people per week. I would write to people on LinkedIn and I would schedule breakfasts with them – just before going to work, I would ask them where they're working and find some kind of midway or something that works. That was a pretty quick way. (36:13)

Alexey: What does the message look like when you write? “Hey, let's catch up. I'm interested in what you're doing. I can come to the place where you live and let's have coffee.”? (36:39)

Adrian: At least from my experience, it's a good idea to express your goals – why do you want to meet this person? For example, I think it's important in the beginning to meet a lot of people that are doing the same thing that you're doing, but have been doing it for a year or two longer than you. This way, they have already gone through the steps that you will be going through and they have already established some sort of network. And they might actually be able to forward you some plans as well and help you out with advice. (36:50)

Adrian: The thing is, it's a win-win situation, because if you tell them what you need – remember, I was telling you about those percentages or fees. Freelancers will often charge each other a small fee, if it makes sense. For example, if it's my client and I sold them something, and I would like you to help me on the project, I might take 5 or 10%. If I'm actually managing you on the project, and I have to check your work, tell you what to do, and stuff like that, then I will take maybe 20%. (36:50)

Alexey: Okay, so you work for a client – let's say you're building a data warehouse – and you see that you need help with some part of the pipeline, and you know somebody that you’ve met before, who has the kind of skills you're looking for. You just write them on LinkedIn saying, “Hey, I have this project. I need this kind of help. Are you free now, right?” (38:00)

Adrian: Yeah, exactly. Personally, I keep a small core of people that I've worked with before and enjoyed it close. But this is how you start in the beginning. (38:23)

Alexey: Then in the same way, when you meet people (other freelancers) they might contact you. They kind of act like an agency in this case, because they will take a small part of your fee. But still, it's worth it. Right? (28:34)

Adrian: In a way. First of all, this doesn't need to always happen. Second of all, the relationship between freelancers and clients is usually a little different. When you bring another freelancer onto a project, you are kind of responsible in the face of the client for the outcomes. Therefore, it's okay that you do charge a buffer, and you might actually need to use that up if the person you hired is not performing. (38:50)

Adrian: I think it's also beneficial because a lot of new freelancers are actually smart, experienced people that are too afraid to ask for what they're worth. Say they got offers for 60 per hour from some British agency, when you could be putting them on a project where they will be making 100 per hour or something. (38:50)

Freelancing platforms

Alexey: What do you think about platforms like Upwork? (39:45)

Adrian: I think they're a waste of time, but other people have had their own experiences. It looks to me like the biggest pro problem, really, when you're hiring a freelancer is that you usually don't know what a good professional looks like. If you have a good relationship with a person, you will try them out and see if they're helping you, and if not – then not. But when you're doing this online, there is no way you will know. Frankly, we'll have a lot of people that are writing bullshit and bidding really extremely long projects and stuff for like $5 per hour for development. (39:49)

Adrian: Let's get real, it's not gonna work out – but the client doesn't know that. Because of that, you're wasting a lot of time. So I think you should always consider what your time is worth. Let's say you set yourself a goal for the month and say, “Okay, I have 170 hours. I want 100 per hour on average, so I need to make 17k.” Give it a shot. If you think this platform is making you that money, then you know that you can keep using it. (39:49)

Alexey: When you work with your clients, do you normally choose if you want to work remotely or on-site, or is it up to the client? (40:57)

Adrian: There are many projects where the client has an expectation. It's up to you whether you take the project or not. In my experience, if you have developed a network enough that you can refuse projects, you will find a remote project. All you need to do is say ‘no’ to the ones that want otherwise. (41:10)

Adrian’s data loading prototype

Alexey: Yeah, thanks. You said you're working on something right now – on a prototype. Can you tell us more about this, or it's a bit too early? (41:32)

Adrian: I can tell you a little bit. But if you stay tuned, I think within a week, I will have this first version. The problem that I'm trying to solve is that data loading is hard. Why is it hard? Because it takes some engineering knowledge, understanding how to design good pipelines, but then you also have the problems that are unrelated to engineering such as data typing, unpacking data, schema migrations. (41:41)

Adrian: None of the frameworks that we currently have out there have actually solved this problem gracefully. What's worse, the loading frameworks that exist out there have a very tight coupling between the source concept and the target concept. This means that you need to learn the entire framework, you need to understand it in depth, you need to learn object-oriented programming – you need to be a really good software developer to be able to do a simple source and use one of these frameworks. So what I've been working on is a framework that makes this very easy. (41:41)

Adrian: You give it a JSON list or a JSON generator and it automatically unpacks it. Optionally, it types it – it generates a YAML schema that you can then manually edit and reuse. It uses a loading manifest, so like the merge columns, if you want to load incrementally. Basically, it decouples the source from the loading. If you are able to produce a JSON object or a Pandas data frame – it doesn't matter – if it's nested or whatever, this unpacker is gonna manage it, type it, rename the columns to something that works in the database, and then you can access it with SQL. (41:41)

Alexey: Looks pretty useful. Is it something that you saw that many clients have problems with? And that’s why you ended up implementing this? (43:31)

Adrian: It's something that we ended up creating because of this Rasa data pipeline project that we did. What happened there is – essentially the data is available for consumption as a conversation object. And this conversation object can be extremely complex because it has the entire conversation and all the technical actions that happen – like all the guesses the bot might make and stuff like that. (43:37)

Alexey: Rasa is a chatbot company, right? (44:04)

Adrian: Yeah, open source chatbot company. Essentially, you don't know what you will find in the subject because it depends on implementation. Even the schema is not really fixed. So we needed a way to manage a very volatile schema, but also be able to freeze it if we wanted to. (44:06)

Going from freelancing to making your own product (and other investments)

Alexey: I heard that many people want to go to consulting, to freelancing, but they don't want to do it forever. Because here, you exchange time for money, but at some point, maybe you just want to get money without spending time. [chuckles] Is this usually what freelancers do at the end? They see a problem that is repeated many times, so they end up packaging it as a product and then sell it as a product? (44:28)

Adrian: Some people, yes – because they get bored. But each person has their own ambitions and approaches. Some people simply go three days a week and enjoy a richer life. Other people start companies, other people don't care, and just work like that. Other people put all their money in Bitcoin and cash out enough to retire. No joke. One of my mentees in freelancing did that. Yeah, every person has their own dreams and ambitions. (45:03)

Alexey: When did he invest in Bitcoin? Like in 2012? (45:41)

Adrian: I don't really want to talk about his finances, but he took it when he was a freelancer and I think he invested in January before last year. [cross-talk] (45:48)

Alexey: It was already expensive then, right? (46:01)

Adrian: Yeah, so I think he went like 3 or 5X or something and he cashed it out in Portugal, where he didn't have to pay tax. So he moved smart. As a freelancer you can do that. (46:04)

The usefulness of a portfolio

Alexey: [chuckles] Okay, thanks. We have a bunch of other questions. Would you recommend building a portfolio to showcase your skill and for reaching out to new customers? (46:17)

Adrian: I recommend building a portfolio of products you can reuse, and basically do it again for other customers. That's going to be efficient. Because in my experience, if customers like you and you tell them that you can do it, then they will give you a shot. [cross-talk] (46:29)

Alexey: This tool that you're developing is one of such things, right? If somebody has a problem, then you can say, “Hey, you know what? I actually have a solution for you. Let's give it a try. This is the solution. I can implement it for you, if you want.” And typically, they will agree. Right? (46:51)

Adrian: Exactly. For example, this tool that I'm talking about for data loading – I'm personally using it to create really good pipelines. But it's also usable by someone that doesn't want to create really good pipelines, but just wants to throw some stuff in a database. If a client wants it done commercial-grade, and with a nice DBT package on top of that, it has iterative [cross-talk]. (47:13)

Adrian: It’s cross-database compatible and all the nice stuff with rebuilding states, having a nice schema. For example, for us, we even inject templates into my database through the API. So yeah, if you want something like that, of course, it's not open source anymore. [chuckles] [cross-talk] (47:13)

Alexey: But still, you want to show them that you can solve their problems as quickly as possible, right? (48:07)

Adrian: Yeah. (48:14)

Alexey: Okay. But I guess a portfolio is a good start anyways, right? If you don't have a portfolio of reusable things, a portfolio with projects you solved is already good enough. (48:16)

Adrian: The question is “Who are you talking to? I think if you go through agencies like Hayes, then you will probably be talking to a technical person that is hiring you – then maybe a portfolio is useful. If you are talking directly to your clients, then chances are that those might be business people or people that already know you, and then they won't care about your portfolio. Personally, I talk to the second type of people. (48:30)

Introverts in freelancing

Alexey: Okay. Got it. There’s another question, “Is freelancing better suited for extroverts or can introverts also do this?” (48:57)

Adrian: I'm an introvert, so I would say if you're an extrovert – that is a nice bonus. What you need to do is just not get in your own way. Ultimately, you're the only person who can stop you. (49:12)

Is it possible to work for 3 months a year in freelancing?

Alexey: When we were talking about exchanging time for money, you said that not everyone wants the same thing. Some people want to work less. Is it feasible to, let's say, work for three months per year or is it too little? Is it even possible? Are there companies who are interested in this kind of stuff? [cross-talk] (49:33)

Adrian: I would say that I've had projects that I could run either between one and three days a week, or one and four – four days is already considered full time. Some projects might run a week, if it's just a small project helping someone with something. It might run for a year… Ultimately, you can also say you've had enough after three months, or you can keep going until it’s over. (50:00)

Alexey: But would it be enough money, income-wise, if I just worked three months per year? Or probably not? (50:30)

Adrian: If you’re working three months per year and only taking the projects that are paying you a decent enough rate – so let's say 120 multiplied by eight, that's like 1000 per day, multiplied by 60 for three months. So I would say 60k. If you live in Germany, you probably have to pay about a third of that as taxes, subtract health insurance, or about 10k. But that's the max if you earn 55k. You're left with I think 30,000, which is actually very comfortable. (50:37)

Alexey: Outside of Berlin, right? If you live in Berlin, then you spend most of the money on rent. (51:12)

Adrian: I bought my apartment almost eight years ago, so I'm not entirely sure. (51:18)

Choosing projects and skill-building strategy (focusing on interests)

Alexey: [laughs] Oh, okay. You’re lucky. You said that now you're not actively looking for new projects. But at some point, you were looking for new projects and you said you always wanted to leave some room for learning. In this case, do you have a strategy between deciding which project to take and which skills to learn next? (51:27)

Adrian: I would say I don't necessarily have a skill strategy. It's also very much about what I like to do. Because by being a generalist, I already have such a broad array of skills that finding work is not a problem. Being a specialist doesn't necessarily help me either find more work or do that work much better. So I really go by interest. (51:53)

Alexey: So interest can be – interest in technology, or interest in a domain, or interest in working with specific people, right? It doesn't matter. Okay. How can I check if freelancing is for me? Let's say I have a full time job and I hear this conversation – it all sounds nice, like working three months per year, doing all that, developing the product, and then selling it for millions. This is all good in my imagination, but how can I check if it's actually for me? (52:22)

Adrian: I can tell you about the people who probably managed freelancing successfully and why they failed. I think if you're that kind of person, maybe it's not the best option for you. There are two types of people that fail, the ones that don't put themselves out there because they think “Oh, it's fine, I will find the work when I need it.” (53:02)

Adrian: But you need to build a network and then this is not a deterministic action, like “If you do this, this will happen.” It might happen and it might not, so you need to keep at it. The second thing is – you definitely need to ask for a decent rate. If you're too afraid to ask for a good rate, if you are asking for the equivalent of a salary or something like that, then you will fail because you won't be able to make ends meet due to the added risk and occupancy rate and all that. (53:02)

Alexey: You said a good occupancy rate is 75%? (53:59)

Adrian: Yeah. (54:02)

Alexey: So it means that you work only 75% of the year? (54:05)

Adrian: It means you will be working 1700 hours. (54:09)

Alexey: And you don't control when you work and when you don't, right? A project might end, it might take two weeks to find a new project. So during these two weeks, you're not working. (54:18)

Adrian: Yes. In all honesty, I've only really had downtime when we had the first Corona wave and everybody panicked and they cut all the contracts. [cross-talk] (54:30)

Freelancing in Berlin

Alexey: Yeah, I think you said that the market is different right now. On the topic of Berlin, there is a question about Berlin – Is the market in Berlin currently good for freelancers in terms of pay opportunities and everything? (54:43)

Adrian: Yes, but you need to put yourself out there and make your own opportunities. If you're going to wait for the few projects that come at you, then you will compete with the other people who are doing the same. So unless you're as good or better than them, it's going to be hard. But I would say that one of the most important things when taking a freelancing job is to be available. I think you will find something. (54:57)

Clients’ expectations for freelancers vs employees

Alexey: Okay. In terms of expectations from clients – let's say if we compare a full time employee from a company and a freelancer, would a company expect more from a freelancer in terms of being more proactive, driving decision making, coming up with architectural proposals, or not? Or is all of that in this ‘scope of work document’ that you mentioned? (55:30)

Adrian: I would say that people are people, so it really depends. I'm the kind of person that does get involved – I just can't let it go. Other people, even as employees, you have to beg them to do their thing. They don't get involved, they're not proactive, and so on. I would say if you are a proactive person that wants to get stuff done, you will get access to the best clients and the best rates. (55:58)

Alexey: Again, just to make it clear, by proactive, you mean what? You said “putting yourself out there,” and by that you mean – networking with people, keeping in touch with past clients or what exactly? (56:32)

Adrian: I mean, you don't necessarily need to keep in touch with past clients. But yes, keep networking, maintain a healthy network, when you work, make sure you care. Actually care about the outcome of the project, care that it's a good outcome for the client – a good outcome for the client sometimes might be that the client is happy and not necessarily what you think is a good outcome. But if you actually do what people want, help them, and advise them, then they will usually find this so invaluable that they will want to keep working with you all the time. (56:47)

Working with more than one client at the same time

Alexey: What do you think about working for two clients simultaneously? Does it make sense? Does it just happen? (57:23)

Adrian: Some people advise you to do that simply to not put all your eggs in one basket. I would say – yeah, don't put all your eggs in one basket and have some savings. Focus is important, and if you want to do a good job, I think it's best that you do focus. It might also depend on you, if you have the capacity to do multiple clients. (57:29)

Adrian: I like to usually have one main client and one side client that is small enough, just for some extra learning or networking. Sometimes I take these small projects just for enlarging the network. But otherwise, I would say just focus on one. If you want to make the best money, just focus on one. (57:29)

Alexey: How does this look in practice? Let's say you spend around three days on your main client, one day on your other client, and one day chilling on the beach? (58:14)

Adrian: It could be like that. I'm not a ‘chilling on the beach’ kind of guy. If I go to the beach, then I'm going to be walking around for the whole day. But yeah. (58:26)

Alexey: Okay. [chuckles] In terms of managing expectations, how do you manage expectations of this second client? I guess you need to tell them, “Hey, I actually have a primary client so everything I do for you will be a background activity for me. Sorry about that.”? (58:36)

Adrian: Yes, of course. You need to tell them ahead of time, and you need to be very clear. Otherwise, they will have expectations for you to answer and communicate in times when you won't be able to. If that happens, then you will have a problem. (58:53)

Adrian’s freelance cooperative on Slack

Alexey: Okay. I know that we should be wrapping up. I remember now that we actually talked in person a month ago, I think – and I was asking you about that. You said there is some cooperative – a freelance cooperative. Can you tell us more about this? What is this and how does one get in? (59:09)

Adrian: To get in, I will send a link to you and then you can send it further. Essentially, it's a Slack group that is a bunch of data freelancers that loosely cooperate. Some of them know each other well. We do you know this thing where we help each other with clients, we tell each other the projects that we find. We do take fees, but it's usually much smaller than what will happen outside of the group. There is also a little bit of a mastermind network that allows you to cooperate. (59:28)

Alexey: I see. Is it Germany-specific or worldwide? (1:00:13)

Adrian: I started with people from Berlin. I started inviting some further people, but I have not advertised it broadly. It's just kind of organically grown to 20-something people. There is more work there than we can take, usually. Definitely a good place for people who want to start freelancing. (1:00:17)

Other advice for novice freelancers (networking)

Alexey: Would you say… [cross-talk] Sorry. You want to say something? Would you suggest that people start a similar cooperative, let's say, if they are not from Berlin but from some other geographical area? Would you suggest that they start like that? (1:00:41)

Adrian: If you are starting freelancing, I suggest starting broader. Rather than creating a very narrow interest group like a freelance cooperative, I would say to create a broader interest group – something like a BI group or a data group for your hometown. Meet these people, create personal relationships, and they will be your network. (1:00:59)

Alexey: Are there any networking events in Berlin that you would recommend for freelancers? (1:01:21)

Adrian: I wouldn't know off the top of my head, but in this group, sometimes we meet. (1:01:29)

Finding Adrian online

Alexey: Well, I think that's all we have time for today. There are still quite a few questions, both in live chat and Slido – sorry for not being able to cover all of them. There is a lot of interest in the topic. Thanks, Adrian, for coming today, for answering all these questions, for sharing your experience. And thanks, everyone, for joining us today and for asking questions. Maybe one last thing, actually. I forgot to ask, usually I ask at the end – what's the best way to get in touch with you? Is it LinkedIn? Is it this freelance cooperative? Email? (1:01:40)

Adrian: Yeah. If you’re in this co-op and you want to Slack me – Slack is going to be faster. But I'm checking LinkedIn quite regularly, when I can. (1:02:15)

Alexey: Okay, thanks. I accidentally forgot to ask this at the end. But again, thanks for joining us today. I guess that will be it. (1:02:25)

Adrian: Thank you. Have a nice weekend. Maybe keep in mind too – if you go to my LinkedIn, there's actually a link to the GitHub account. If all goes well, in one week, you will find this loading tool. (1:02:35)

Alexey: Cool. I'm looking forward to seeing it. (1:02:52)

Adrian: I hope it'll be fun. (1:02:56)

Subscribe to our weekly newsletter and join our Slack.
We'll keep you informed about our events, articles, courses, and everything else happening in the Club.

DataTalks.Club. Hosted on GitHub Pages. We use cookies.