Hi everyone ! I discovered Prefect a few weeks ago and i’ve been struggling since a few days with the saving of my csv files. I’m doing everything locally for the moment and this line of code does not work when i launch my program.

<http://DATA_FULL.to|DATA_FULL.to>_csv("csv_files/DATA_FULL.csv")

But when i run it without prefect it works fine. does anyone has an idea of what i have to change ?

I'm trying to setup Prefect using the following docker-compose. Everything seems to work except one important thing: being able to run a flow, which I've deployed with a DockerContainer infrastructure block and Github as storage block. Traceback from the agent (I've removed the url in the second line) :

18:41:58.602 | INFO    | prefect.agent - Submitting flow run '5b4470d3-8cd0-4073-8c03-be70eac1deff'
18:41:59.077 | INFO    | prefect.infrastructure.docker-container - Pulling image URL...
18:41:59.722 | INFO    | prefect.infrastructure.docker-container - Creating Docker container 'invaluable-hyena'...
18:41:59.743 | INFO    | prefect.infrastructure.docker-container - Docker container 'invaluable-hyena' has status 'created'
18:42:00.303 | INFO    | prefect.agent - Completed submission of flow run '5b4470d3-8cd0-4073-8c03-be70eac1deff'
18:42:00.307 | INFO    | prefect.infrastructure.docker-container - Docker container 'invaluable-hyena' has status 'running'
18:42:02.695 | INFO    | prefect.infrastructure.docker-container - Docker container 'invaluable-hyena' has status 'exited'
18:42:02.697 | INFO    | prefect.infrastructure.docker-container - Docker container 'invaluable-hyena' has status 'exited'

Traceback from the container:

Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 260, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/usr/local/lib/python3.8/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 167, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/usr/local/lib/python3.8/site-packages/prefect/filesystems.py", line 908, in get_directory
    process = await run_process(cmd, stream_output=(out_stream, err_stream))
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/processutils.py", line 73, in run_process
    async with open_process(
  File "/usr/local/lib/python3.8/contextlib.py", line 171, in __aenter__
    return await self.gen.__anext__()
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/processutils.py", line 35, in open_process
    process = await anyio.open_process(command, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/anyio/_core/_subprocesses.py", line 127, in open_process
    return await get_asynclib().open_process(
  File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 1105, in open_process
    process = await asyncio.create_subprocess_exec(
  File "/usr/local/lib/python3.8/asyncio/subprocess.py", line 236, in create_subprocess_exec
    transport, protocol = await loop.subprocess_exec(
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 1630, in subprocess_exec
    transport = await self._make_subprocess_transport(
  File "/usr/local/lib/python3.8/asyncio/unix_events.py", line 197, in _make_subprocess_transport
    transp = _UnixSubprocessTransport(self, protocol, args, shell,
  File "/usr/local/lib/python3.8/asyncio/base_subprocess.py", line 36, in __init__
    self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
  File "/usr/local/lib/python3.8/asyncio/unix_events.py", line 789, in _start
    self._proc = subprocess.Popen(
  File "/usr/local/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'git'

Need help on building a deployment from the flow. I have created a GitHub storage block without reference to any branch. Can I fetch/use files from any branch through build_from_flow function?

Is there a set upstream with prefect 2?

youtube Here's a great by @Bianca Hoch

I have another issue in 1.0 that I am not sure how to classify. I have a download zip file task at the beginning of my flow. I have used BeautifulSoup to grab the dynamic zip file urls from the landing page and convert them into class called Page(). Then these links are exposed as a list of links. The first line of my Flow() is page = Page(). The problem is it seems this "page" object is cached as the newly posted links are not processed at all. It appears this is default behavior but How do I turn it off or bypass it? I need a new page = Page() object every time the flow runs... The flow is created with the following: result=PrefectResult(),storage=Local() Should I remove those?

Hello guys, I discovered Prefect a few days ago. I do have a few questions. I'm planning to use the Prefect Cloud as UI, and install the agents in different cloud providers on their always free tier. My main question is related to egress usage, are the agent polling periodically for work?

Hey all, I have a quick qs that can hopefully be easily answered - my flow/task descriptions don't seem to show up in the Orion UI. Is this a bug or am I missing something? PS let me know if this is not the right place to ask!

Hi, can we add @flow to existing class?

class API:

@flow

def start(self):

pass

I am getting TypeError: missing a required argument: 'self'

When I import function for

pytest

which has perfect

decorator

, I got the following error (screenshot bellow). If I use comment #flow decorator, pytest runs without errors.

Hello, I was hoping to submit an issue for a small typo in the docs (

task

decorator should be above function definition), but not sure which issue template to use. Bug template seems like overkill...help?

When using

from prefect.tasks.shell import ShellTask

it throws out and error

ModuleNotFoundError: No module named 'prefect.tasks.shell'; 'prefect.tasks' is not a package

I know i can use

prefect-shell

to overcome this and run python scripts but i couldn't find any resolution on the above error

I tried to run prefect Orion server inside a docker on port 8000 but the server was not loading. The port was open, when i ran the airflow server on the same port i was able to access the airflow server. Is there some further configuration required to be done. I ran the server with the following command

prefect orion start --port 8000

My port is open to communicate with the host.

I have some questions regarding the pricing and cost associated with prefect. Who can i reach out to?

Deployments are created successfully but not showing on the Cloud.

I believe you need to either add

--apply

to your build command, or run

prefect deployment apply

on the YAML file generated by the build command.

Hello guys, recently I tried to run simple Prefect flow script, as well trying out deployment on Prefect Orion service, but I got

httpx.HTTPStatusError: Server error '500 Internal Server Error' for url '<http://127.0.0.1:4200/api/flow_runs/>'

all the way. I got no idea why is this happening, is any one encounter this before?

Hello guys, recently I tried to run simple Prefect flow script, as well trying out deployment on Prefect Orion service in Windows 10, but I got

httpx.HTTPStatusError: Server error '500 Internal Server Error' for url '<http://127.0.0.1:4200/api/flow_runs/>'

all the way. I got no idea why is this happening, is any one encounter this before? Below are my steps and some snapshot of Prefect version, command prompt that executed the code and the code itself. Here is my steps to get this error: 1. Create python virtual environment using this command

python -m venv prefect

2. Activated the

prefect

environment 3. Install Prefect

pip3 install -U prefect

4. Start the Prefect Orion Server

prefect orion start

5. Write a sample python code and execute the code

python test.py

6. Error happened.

Hi there, I want to use prefect for a small-scale personal project to automate a few python scripts. I was hoping to use a docker container with everything completely self-hosted (I am on Unraid OS) and see that prefect and prefect server are on dockerhub, but I see no actual documentation for using the dockerized version anywhere. When I pull the container down, as is, it fails to start and the only information I get in the log is the following:

|  __ \|  __ \|  ____|  ____|  ____/ ____|__   __|
           | |__) | |__) | |__  | |__  | |__ | |       | |
           |  ___/|  _  /|  __| |  __| |  __|| |       | |
           | |    | | \ \| |____| |    | |___| |____   | |
           |_|    |_|  \_\______|_|    |______\_____|  |_|

Thanks for using Prefect!!!

This is the official docker image for Prefect Core, intended for executing
Prefect Flows. For more information, please see the docs:
<https://docs.prefect.io/core/getting_started/quick-start.html>

Anyone have any idea of what I'm doing wrong? Should I be starting with prefect server instead?

Hello, I am trying out prefect for the first time and am running it on my local machine. The prefect orion start was running fine, until I created a flow that does a parallel flow execution. Will appreciate any guidance on how to resolve the error.

how to fail a task / flow when there's error without raising exception. tq...

Hello, Any guide on how to setup prefect for production which has best practices and scalability in mind. I have an ec2 instance where i will run the polling agent. Currently i am running the polling agent inside a docker in that server for testing purposes. We are confused between airflow and prefect. I am pitching in for prefect, i just can't find many differences.

Hello, I currently have a deployment set up that uses GCP Compute Engine for the polling agent and GCP Cloud Run Jobs for the execution of the flows. Currently I am using Prefect to run NLP processes on large batches of documents.

Hello, I currently have a deployment set up that uses GCP Compute Engine for the polling agent and GCP Cloud Run Jobs for the execution of the flows. Currently I am using Prefect to run NLP processes on large batches of documents. To do this, I am loading in a set of 1000 documents, then splitting into batches of 50 documents and using

.map

to run a task on each batch of 50 documents. It appears that when I use

.map

, it is running all of the tasks within a single GCP Cloud Run Job. I was wondering, is it possible to configure Prefect to spin up new Cloud Run Jobs for each of these mapped tasks? Thanks in advance for any help!

Hello, I am currently doing a PoC of prefect for my company to compare it to airflow (which we currently use on AWS). I have a bit of trouble when connecting with prefect cloud login through a web browser. Any hints ?

Hi all. I have some flows running in production that I want to migrate to Prefect 2. I have been reading the migration docs and my flows are quite simple so I don't think I have too many code changes but I am looking for guidance on updating my development and test environment. I am using Prefect Cloud, running agents on a windows server, development environment is windows. I am a developer but do not usually work with python and I'm not sure how updating the prefect package will affect my ability to keep working on my prefect 1 flows if I need to. Thanks in advance, Greg

Hi, is there a way to disable the Parameters>Default selection when creating a new run in Prefect UI? Or at least have it always selected as Custom? Most of our flows have required parameters without a default value, so it does not make sense. Also for those that can run only with defaults you still always want to check what they are so the Default section in general seems not helpful.

I have a question quite similar to the one asked here: https://prefect-community.slack.com/archives/C048ZHT5U3U/p1670522837872239 If I define a prefect flow inside a Jupyter Notebook and I re-evaluate the cell, I get an UserWarning