Hello All , I got weird problem with Prefect 1.0 flow run , I have other flows running fine with same server and same Agent(s) . I made a small change on this one and got new version , but still getting error .

17:06:30 INFO Agent1

Submitted for execution: PID: 3327309

17:06:31 ERROR execute flow-run

Failed to load and execute flow run: PermissionError(13, 'Permission denied')

Does anyone have any links to docs/code for setting up a Prefect Orion webserver for testing client interractions specifically?

Hi All, am trying to deploy a flow.. but instead of deploying the flow it is running it..am a little confused. could anyone help me? below is the statement:

prefect deployment build ./data_source_flows/master_flow.py:master_flow -n master_flow -q test --storage-block github/flows --apply

Hi, I've looked through a couple questions on the helm/k8s setup for Orion but havent found my answer. This is probably just me missing something about k8s itself. Im just trying to test self-hosting orion on my local machine using the helm chart found here and then opening the locally running dashboard. The command im using is

helm install prefect-orion-poc prefect/prefect-orion --values  helm-orion-values.yaml

and my values file is

namespaceOverride: helm-prefect
postgresql:
  auth:
    password: password
ingress:
  enabled: true

It outputs

Get the application URL by running these commands: <http://prefect.local/>

but that comes back as an unknown host. Im pretty new to K8S so any help is appreciated.

Can someone help me with this question? https://stackoverflow.com/questions/74757050/how-to-deploy-locally-multiple-flows-using-prefect-2-0

Hello, does anyone use Prefect Helm Chart? https://github.com/PrefectHQ/prefect-helm#the-database-deploys-correctly-but-other-services-fail-with-bad-password I think I have this problem but I am not sure I understood the solution

Hey 🙂 I have shared code that is used by both prefect flows + system where no prefect is installed. Thus I need a logger which is able to log everywhere. I created an own logger but have some problems with prefect to get it running. I only found documentation on how to set up a custom logger for v1: https://docs-v1.prefect.io/core/concepts/logging.html#extra-loggers But I think it should work the same for v2. I also had a look here: https://discourse.prefect.io/t/prefect-logging-faq/1476

Hey, I'm running prefect orion on Docker. The prefect UI launched fine but I'm keeping getting this error. What am I doing wrong?

Hi! I'm using Prefect 1.2.4 with docker storage and kubernetes agent I need to change the umask of the flows as they are writting to shared storage. I tried setting it in the .bashrc as I did for my users outside of flows, but apparently it is ready only for interractive sessions so not with flows! I see the recommended way is to set the umask in the entrypoint of the docker, example :

ENTRYPOINT ["./set_umask.sh"]

https://codeyarns.com/tech/2017-07-21-how-to-set-umask-for-docker-container.html#gsc.tab=0 Is there a way to override the entrypoint of the flow ? Or any other solution to set umask?

Hi all. I'm using run_deployment on existing deployments (Prefect 2.7.1) but I notice a 5 to 15 sec delay in the start of the flow run. I dont use scheduled runs so it should start right away. Should I look for parameters or config to remove the delay or is it something enviroment related?

Hi I was wondering if there are any guidelines/ideas on running subflows. As far I know we can do it either; by referencing/calling them from your code directly or using

run_deployment

. Assuming that all prefect flows are contained in the same repository, and are deployed to the same orion server. Are there any benefits/disadvantages over one or the other?

I’m attempting to set up Prefect 2.0 and I want to run flows in a docker container. My docker image is in an Azure Registry. I created a block referring to the image. I’m getting the following error: No module named prefect.engine.__main__; ‘prefect.engine’ is a package and cannot be directly executed. Any clue what I’m misssing?

I have got below error while running my code in Prefect cloud. The flow generated large number of tasks, so I am assuming there is a limit for number of tasks per flow?

12:48:12.659 | ERROR | prefect.infrastructure.process - Process 'optimal-toucanet' exited with status code: -9; This indicates that the process exited due to a SIGKILL signal. Typically, this is caused by high memory usage causing the operating system to terminate the process.

Hello, I’m reposting an issue i’m having as I have not been able to resolve. I have created a deployment using an ECStask block. The ECStask block is configured to run an existing task definition. When I try to run the deployment I get the following error: Submission failed. KeyError: "No class found for dispatch key 'ecs-task' in registry for type 'Block'." This error shows up under details of the flow run but there are no other logs. I already tried registering the block again. cc: @Taylor Curran

Hello, I have 4 tasks that run successfully when running locally using FlowRunner. When running in Prefect Cloud, I get the following results How can I troubleshoot this? • TypeError("'generator' object is not callable") • SystemError('unknown opcode') • SystemError('Objects/dictobject.c1527 bad argument to internal function') • The 4th task hangs indefinitely

I’d like to follow a workflow with Prefect 2.0 of separate

@task

functions called from a

@flow

- all of our work is done outside of Python called via make commands, which I currently use

shell_run_command

for. However, you can’t put

shell_run_command

within a function decorated with

@task

. I could just remove the

@task

decorator. Is there a workflow folks use to leverage

@task

decorator that run shell commands?

Hi...I need help printing task results using Prefect 1: seems like printing the output only prints the task name, not task result:

from prefect import task, Flow

@task
def get_value():
    return 10

with Flow("task-results") as flow:
    v = get_value()
    print(v)

state = flow.run()

I get

<Task: get_value>

instead of

10

. What am I doing wrong?

I haven’t been able to find info on this yet, but I’m guessing it exists in docs somewhere: file structure. AFAICT

prefect deployment build

only allows for pointing at one file. If you want to pull in imports from other files like utility functions (not pypi packages) how do you suggest doing that? I assume we don’t have to have all code in the one file the deployment build command points at?

Hello, I have created a deployment using an ECStask block. The ECStask block is configured to run an existing task definition. When I try to run the deployment I get the following error: "Submission failed. botocore.errorfactory.AccessDeniedException: An error occurred (AccessDeniedException) when calling the RegisterTaskDefinition operation: User <> is not authorized to perform: ecs:RegisterTaskDefinition on resource: * because no identity-based policy allows the ecs:RegisterTaskDefinition action." Why does the agent need the ability to register a task definition if it already exists?

Hello community, absolute prefect noob here, so far i like all i have seen! kudos for the team! one silly questions; i have a python script i use to fetch data from an api and save in a remote storage, the script has a lot of recurrency and takes about 1 hour to complete i took the same script added the necessary decorators and minimal adjustment to fit prefect pattern, and when i run the script in prefect it takes 11 hours to complete is there any red flag related to performance i should be aware of that i may have completely ignored? the script is too simply for that amount of time its literally request api -> convert json to pandas -> calculate one column — repeat until its over --> save to storage as csv

I don’t see any docs on what the process is for updating code for flows deployed to Prefect Cloud? Is there something like

prefect deployment update

? Do you just nuke your deployment.yaml file, and rerun

prefect deployment build

then

prefect deployment apply foo-deployment.yaml

? Maybe it depends on where storage is located? Im using GCS storage.

in the prefect 1 graphql api, is there any way to query all tenants using one api key? or is one api key only able to query one tenant? I'd like to make a table that logs all of our flow runs and want to make sure I structure things correctly

Are there any examples/recipes for ECS deployment implemented in python?

Hello all - I like the radar view of tasks from within a single flow. Is there any way to visualize the tasks from subflows? I find stepping into one or many sub-flows becomes a large mental exercise to remember where I’ve navigated from within the UI. How should I go about visualizing or querying upstream or downstream dependencies with Prefect 2?

I have a new issue with a deployment. I get the following error: prefect.exceptions.ParameterTypeError: Flow run received invalid parameters: - start_date: field required - end_date: field required - initial_url: field required - cursor_url: field required However, when I run the flow locally, there are no errors. For reference, this is the last line of the flow and as you can see the parameters are populated. Is there something different I need to do in the deployment?

if __name__ == "__main__":
    extract_load_events ('20221030','20221030','<https://us1.api.clevertap.com/1/events.json','https://us1.api.clevertap.com/1/events.json?cursor=>')

For running

prefect agent …

on a VM as a service, does anyone have a setup for injecting secrets into the

supervisord.conf

file for the prefect url and api key? A solution with

chamber

would be ideal, but curious about any solution that works

Hi, I am trying to write the flow in local first before deploying to cloud. I found that the local db can lock occasionally if the flow is terminated in the middle. I need to remove the ~/.prefect/orion.db to recover. I wonder if there are properly way to release the lock when I terminate the flow run execution.

hi! i was trying out notification on orion, i can see an option of tags there, how do i add tags to flow run ? do i add it while building the deployment or is there a parameter that can be added in the flow wrapper

After some troubleshooting, I found that its due to the change to running subflow which propose to use asyncio.gather on the subflow. When changing the flow to async, we can not use task.submit anymore or else it will face strange behaviour which some task won’t be run. It would be good that the document put emphasis on this part. Right now the closest message I got is “Note, if you are not using

asyncio.gather

, calling

submit

is required for asynchronous execution on the

ConcurrentTaskRunner

.”

Hey gang, I am trying to save a DbtCliProfile block that contains DatabaseCredentials for a postgres db, but am running into an error:

dbt_cli_profile = DbtCliProfile(
    name="dbt_dwh",
    target=DEFAULT_BLOCK,
    target_configs=PostgresTargetConfigs(
        credentials=DatabaseCredentials.load(DEFAULT_BLOCK), schema="public"
    ),
)
dbt_cli_profile.save(name="dbt_dwh")

TypeError: Type is not JSON serializable: URL

Has anyone else seen this?