Folks i am porting some of my Airflow jobs to prefect. And wanted to know, is FunctionTask behaviour is same as PythonOperator (Airflow operator) behaviour?
k
Kevin Kho
09/09/2021, 1:50 PM
Hey @Abhishek, yes but from my understanding but Operators seem to be both source and sink in one. MSSql to S3. Prefect will just be one (MSSqlQuery and then UploadtoS3)
a
Abhishek
09/09/2021, 2:06 PM
Thanks @Kevin Kho, got it.
My current code in Airflow way uses PythonOperator to call a function to do x whenever a a dynamic task is getting created.
Hence wanted to exact replacement of it :-)
Anyways, i got your point w.r.t “Prefect way” :)
k
Kevin Kho
09/09/2021, 2:16 PM
I think in Prefect you would just directly call the Python callable. Like wrap that callable in the
@task
decorator. If you get stuck and have a concrete code snippet, I can certainly help with that.
🙏 1
a
Abhishek
09/09/2021, 2:25 PM
let me try that approach - i had a thought of that though.
Let me try, if i get stuck, will ping you.
thanks, appreciate your help 🙂