Name: From bioinformatics scripts to nf-core pipelines: my Nextflow transition
Uploaded: 2026-10-13T14:32:55.250Z
Duration: 14 min 52 s
Description: From bioinformatics scripts to nf-core pipelines: my Nextflow transition

Transcript for "From bioinformatics scripts to nf-core pipelines: my Nextflow transition": Alright. Alright, everyone. We are continuing with our training and community, track this afternoon or morning or evening, depending on where you are in the world. For this one, we have with us, Harris Spireidis, who is going to tell us about his journey from scripts to n f core pipelines. I think that's going to be a journey that's going to feel fairly familiar or or potentially something that a lot of people are interested in. So, without further ado, we are going to roll the recording, and we'll have our, live q and a after the talk. Thank you. Good morning, everyone. My name is Harish Piritis. And today, my talk is, from bio... Bioinformatics scripts to n f core pipelines and how was, my Nextflow transition. And just to give you an overview what I will, talk in this presentation, I will start the how... Where I was before Nextflow, my hackathon experience, and how I chose the pipeline. My first attempt to start contributing, things, what kept me going and because I was about, starting to quit and what led to this contribution led to and, a road map for people, considering to make this transition, from from my side. So I have a master in fractional genome from Hackneygen. I have worked in, whole genome sequencing for, variant discovery. I have done also an ASEC analysis and as well as comparative genomic pipeline for, identifying integration from one species. And all those, pipelines have been done in BaaS and Snakemake. I didn't have any exposure, in Nextflow, and, education system in Wageningen mainly uses Snakemake because it's already in Python. And I only knew that Nextflow was complimentary tool. I didn't have, work on that. So, after my master, graduation and while I was searching for a research position, I thought, yeah, it's it's a nice opportunity to join the hackathon in the previous March. So, I chose a pipeline to start working on. I chose tax profiler, which is a metagenomic tax profile... Taxonomic profiler profiling and classification for short and long reads from shotgun metagenomics. It runs 14 profilers in parallel from from one input, and the pipeline for low, it's pretty straightforward. It does quality control pre processing, classification and profiling, post processing, as well as reporting those results from the various tools. It has modular structures like the all the other NF core pipelines, which I was really surprised in how all these things works. It worked for me before three reasons because, firstly, it overlapped with my metagenomic interest. I didn't work, in my research project, in metagenomics, so I always wanted to work on that. There was a clear spacing for for space for adding new tools. And, thirdly, there was there was actively developed and the community welcoming new members to contribute. So I chose, this pipeline. During the hackathon... Before the hackathon, I started to prepare myself, with the talk and the training series giving give, taught by Phil. And, I chose the YouTube series because Phil was make very clear, how things work. And he he teaches very nicely the fundamental, the concepts, everything about Nextflow. And, there were things and that I thought, okay. Two days, I will join the hackathon. It might take a bit of time, but it's fine. And there were things that, carry out from, snake snake snake experience. I had the workflow mindset. There were also the input out declarations as well as, the container conda base rebutability. However, there were things that they're new, for me, like the d c l, two, syntax, the reusable modules and the groovy. But where the groovy, I had really difficult to to adjust because I I came from Python and think most of the computational biology come from. Standards and tunnels operators are a nice thing about Nf core pipelines where... As well as the Nf... Nf core module structure and the testing con... Convention about where you validate, the pipeline. And, specifically about the third one, I was really excited how things work. And, to be short, I... When I was, trying to make the... To contribute, during, the hackathon, I was about to quit because it was two days and it it take... Took more than two days and weeks and all... Almost a month until I I get, to integrate it into the pipeline. And there was a friction about it from my side, and I almost was about to quit, contributing. But there was, there were things that, kept me back from quitting. And, I would like to talk about those things, here. So what kept me going on this contribution was, the community made continuing possible because specifically James and Sofia were the developers of, tax profiler were patient, giving me patient PR reviews. They were unblocking when I was stuck. Also, then, of course, Slack is very, very nice, opportunity in the unblock when you ask questions are very, quickly answered those questions. The commitment to the specific module, it was not about, learning next flow anymore. It was about integrating the tool. And, yeah, I think that gave me also the confidence to continue. And the moment that things clicked, like the first integration test in GitHub, also gave me the confidence, to continue on that contribution. And from my side, the community community was a tremendous force to to continue. After the first contribution, I started the second contribution. I added another profiler, which was faster because I knew the pattern. Developers already provide a suggest me if I wanted to pro to part to be part of the manuscript, which is submitted to microbial genomics. And then the previous June, I also joined the next flow ambassador program. What... That's why I'm presenting here. And, it's a very nice opportunity to foster collaborations and, promote Nextflow, with... Within your environment. And I think it's very nice opportunity to attend, to return back the knowledge to people that, they, they need it. I have made a roadmap from people that thinking, make this transition, from scripts, either coming from snake make or, other, language or even from web lab experience to first next flow contribution. So the first thing that I would like to tell them is, ground yourself to specific training resources. Do not dive into all the dogs. Do not... Because you can easily get lost in... Inside the dogs. There are lots of many, and also they're groovy. It's a lot of things. So I started with Phil, a video series from Nextflow. I think it's very illustrative way to understand the concept, and, this can be your vocabulary. When you are stuck, go back and see the fundamentals. Because sometimes when you're working on a pipeline, it's easily to expose the gap of, of, of, tutorial because there's a lot of things going on. Secondly, to pick a pipeline that matches your interest is, I chose metagenomics, and it's easy... It's easier to understand concepts where you are fascinating working for something. So, so when you understand cost, you can easily implement those concepts. Thirdly, to, make a public commitment, hackathon or open PR. I think it's a very nice way because it puts you in a in a way... In a mental mode, that you contribute to something that's meaningful. For example, if I have started by myself alone, if I was not in hackathon, probably I will I will not, have continued that, that way and I will have quit it. So, yeah, in the next two days of the hackathon, take your time and enjoy enjoy that and ask questions as you can. And, it's a very nice opportunity. There are a lot of existing code also in, in the GitHub. There are a lot of, like, 2,000 modules already in the in the GitHub account from an end of core. Sometimes when you are creating a code, it's it's it's easier, to see a similar, module or what other people have done. And this this, code can be your your template, and you can copy or see what other people have done. I think already professional of Nextflow sometimes if they are stuck to see existing code as well. And ask early. And last but not least is to ask early and post in Slack because a very active community if you have question. Because sometimes our researchers, we have our tendency to to figure out ourselves, things how how they work. And sometimes it may take time, even days, for a thing that can be answered, within an hour. For example, I had an error, and I was trying to to find or asking AI, but I could not I could not find the answer. And when I ask, like, when I asked James to give me a direct answer, on my problem, and it it unblocked me immediately. So ask question in Slack or, people, from Nextflow. In a... So if I had to put all these five in in, one sentence, I will say you don't need to master, next level before contributing. You can... Your contribution can be your way how you're learning, and, don't hesitate to to con... To start on that journey. So if I can do it, everyone can do it. Thank you. Alright. Well, that was a wonderful talk, Harris. Thank you so much. Thanks for sharing your journey. I I love a good migration story. I'm really glad that you didn't quit, so thanks for sticking with it. I'm glad the the the tutorial resources were helpful. I will point... Okay. Since you went through all that pain, we we do have some new tutorials focused on NF core, which might have saved you some trouble at the time. I don't know. Yeah. I we're always developing new things. did. But, Indeed. yeah, thank you so much for sharing. We do have a a couple of questions. There is one, question I'm going to punt to to one of our other speakers. There's a question about. the usual tasks of ambassadors. But we have Marcel who runs the ambassador program is going to be, talking detail about that, at the half hour. Yeah. So there'll be a lot about that. Although if you have any particular things you wanna say about what you you enjoy the most about the the ambassador program and and, specifically, like, do you have opportunity? Do you enjoy the opportunity to supervise or assist other scientists working on NF core or Nextflow? Is that something you... You're able to, to do? Yeah. Thank you very much for the opportunity and, the questions. So currently, because I'm not affiliated to university or something, I have not the opportunity to, supervise or or assist other scientists. So I don't think either it's like, mandatory to do stuff like that. It's more the master program to to promote next law within your research environment, create sessions and, other stuff. And Marshall will explain you more about it. It's a very nice talk, I think. So... Yeah. Well, thank you. And one other question is, Matthias asks how much LLM assistance you were using while contributing, and how has it or would it have influenced your path, beyond just asking questions for support but, like, Yeah. for for developing and so on. Yeah. Yeah. Yeah. Yeah. Exactly. That's a wonderful question. I also wanted to, include it in my presentation, but I didn't. I think, you have to be careful with, airlines because sometimes it gives you, let's say, hallucinations and stuff like that, and you can really, get lost, specifically with Nexpo. I used it, of course, mainly for, understanding some concept and and and policing, found some error. Sometimes it worked, but sometimes it really give me very wrong ideas. So I think it's better to stick, to the fundamentals and also, of course, ask if you got... If you have question, but be careful with, the lens. Yeah. Yeah. That makes a lot of sense to me. We are out of time, but thank you so much for joining us. And you. looking forward to seeing you online. And for everybody. else, we'll. see you in the next talk. Bye. Yep. Mate.