https://prefect.io logo
Title
f

Faheem Khan

06/15/2022, 12:02 AM
Hello Everyone, is it possible with prefect to run python script i.e. (abc.py) instead of refactoring my code to prefect tasks. I have already written my python script with multiprocessing functionality and would like to schedule it prefect.
1
a

Anna Geller

06/15/2022, 1:18 AM
You came to the right place, that's what Prefect 2.0 is for - check the docs here orion-docs.prefect.io
k

Kevin Kho

06/15/2022, 1:22 AM
If you already wrote your code with multiprocessing functionality, just make sure to use the
SequentialTaskRunner
by default because we also use a
ConcurrentTaskRunner
by default
f

Faheem Khan

06/15/2022, 2:52 AM
should I still be using SequentialTaskRunner if I only need to schedule the python script? As in Airflow I can run bash script.
k

Kevin Kho

06/15/2022, 2:53 AM
Ah I know what you are saying. Prefect tasks are run concurrently but if you don’t use task and just decorate your Python code with
@flow
, there is no need
f

Faheem Khan

06/15/2022, 3:21 AM
@flow decorator is available only in prefect 2.0. I am using prefect 1.2.0
k

Kevin Kho

06/15/2022, 3:24 AM
2.0 might be a lot easier for your use case because Prefect 1 requires everything in the Flow to be a task, but for 2.0, you just need to decorate your
main
function
f

Faheem Khan

06/15/2022, 4:03 AM
I am running prefect 1.2.0 server in a docker container using docker compose "https://github.com/PrefectHQ/prefect/blob/master/src/prefect/cli/docker-compose.yml" . Is there a similar docker compose for prefect 2.0?
In prefect 1.2.0 I am running server, agent and client in 3 separate containers, can I do the same for prefect 2.0.
k

Kevin Kho

06/15/2022, 4:10 AM
You can do:
docker run -it --rm \
    -p 4200:4200 \
    prefecthq/prefect:2.0b6-python3.8 \
        prefect orion start --host=0.0.0.0
because you can spin everything up in the same container, but if you want compose specifically, you can check this community contributed project. we don’t have an official one yet