https://prefect.io logo
Title
d

David Frischer

03/05/2023, 3:58 PM
[Kedro Pipeline, deployment with Prefect] Hey everyone, has anyone been able to successfully create a Prefect 2 deployment from a Kedro project? If so, could you share some insights or tips on how to go about it? I'm currently working on a project and would love to integrate Prefect into my Kedro pipeline, but I'm not sure where to start. Any help would be greatly appreciated! I am trying to convert the
register_prefect_flow.py
script given by Kedro documentation. Unfortunately, I don't seem to succeed because Prefect 1.0 API is different from Prefect 2.0. Thanks in advance !
:upvote: 3
f

Frits De Roos

03/07/2023, 7:45 AM
Hi David, I'm trying to do this as well, but I haven't found the perfect solution yet. Currently our Prefect flows just execute command line commands to run Kedro pipelines, but this is not the best route for debugging. A better solution would be to just call the Kedro runner within a flow. Example on how to use this runner (without Prefect, but probably easy to integrate): https://dev.to/waylonwalker/using-kedro-in-scripts-6l2 . It still feels a bit patchy though. So, if you have any better ideas for this, then I would love to hear! Another of my concerns the combination of Prefect and Kedro is that in the future package conflicts could arise, because there is quite some overlap between these two.