Hello community, absolute prefect noob here, so far i like all i have seen! kudos for the team! one silly questions; i have a python script i use to fetch data from an api and save in a remote storage, the script has a lot of recurrency and takes about 1 hour to complete i took the same script added the necessary decorators and minimal adjustment to fit prefect pattern, and when i run the script in prefect it takes 11 hours to complete is there any red flag related to performance i should be aware of that i may have completely ignored? the script is too simply for that amount of time its literally request api -> convert json to pandas -> calculate one column — repeat until its over --> save to storage as csv

If it isn't too complex, consider sharing some code.

In case it helps, I had a similar issue with a script taking way longer after adding the decorators. For me, the problem was that I had a function, which I designated as a @task, that I iterated through 50+ times in a loop. I was told this will be very slow as a task. I just removed the @task decorator from that specific function and the script went back to running in a normal time.

Hi @Nicolas Gastaldi - depending on how your data is passed around your workflow, it can make sense to define certain bits of work as a task or a plain python function. If you're able to share your flow code, we can see if there are some optimizations we could make!

It's worth noting that in a case Ashley's, you can still run your code as a task, but depending on what functionality you're looking for, it may make more sense to put the entire loop inside of a task

def func_to_loop():
  do_stuff()

@task
def loop_task():
  for i in range(1000):
    func_to_loop()

Hey folks, first of all thank you all for the replies, i tried the removing decorator suggestion and i had a very small improvement, so i went digging into the code to try to isolate the loops as suggested by Peyton and i ended up finding a indentation error on an error code retry which was the responsible for the hanging on the interaction. final veredict, i am not only and absolute noob in prefect, i am also noob in python 😅 thanks again for all the input