Hi! We are writing out some telemetry and logs to our own postgres instance (stuff that sits outside of the purview of the prefect logs). We have a LogHandler for this, and it calls out to sqlalchemy. Is it recommended to use the PostgresExecute task as opposed to using our own bespoke DB handlers? In that case, we might handle the ORM call out with PostgresExecute.run(..)

When I run

prefect agent docker start

is there a way I can specify the image it's pulling from?

Hi Good afternoon, I am trying to use the Mapping Functionality with LocalDaskExecutor to run a set of Tasks parallel, in my case (16 of them at the same time) using a simple flow. I tried both Threads and Processes as the scheduler and it didn't run the tasks parallel. But when in check the mapped function via the Schematic, I only see jobs starting and running one at a time. Please see the images below: Could you please let me know how to get the parallel task execution using a LocalDaskExecutor. Thanks, Sen

I'm a bit lost / stuck regarding DockerRun on ACR, combined with Azure storage. So, my Docker agent requires the AZURE_STORAGE_CONNECTION_STRING to pull the flow from the store. But when I pass it as an env in the DockerRun() params, I keep getting the same error that it can't be found.. And no matter what I'm changing, I can't get rid of the error.

Failed to load and execute flow run: Exception('Azure connection string not provided. Set `AZURE_STORAGE_CONNECTION_STRING` environment variable or save connection string as Prefect secret.')

First I was working on the Local storage, but that gave me ModuleErrors and an old prefect thread explained that it should be solved through the use of Azure/Github/etc storages.

with Flow(
    "extract",
    storage=Azure(container="prefect-flows"),
    run_config=DockerRun(image="<http://xxx.azurecr.io/extract:latest|xxx.azurecr.io/extract:latest>", labels=["extract"], env=env_config.dict()),
) as flow:
    print(env_config.dict(), file=stderr)
    # Prints: {'AZURE_STORAGE_CONNECTION_STRING': 'DefaultEndpointsProtocol=https;AccountName=xxxxxx==;EndpointSuffix=<http://core.windows.net|core.windows.net>'}

Is there a way to allow archived flows to be run? Or to make it so that flows are not automatically archived when a new flow by the same name is registered? Running on ECS, often we will make some changes to the docker image, then run via prefect to test them out. But reverting to the previous flow_id gives the error

Flow xxx is archived

.

Greetings everyone, I am currently using prefer server deployed on kubernetes with gitlab storage. I want to push some modular code on git and import custom functions in my prefect script but seems like there are issues in the execution. Can anyone guide with an example or something as to how can i write a script that contains a prefect flow and import custom functions from other files in the script.

Hi. When i run the command prefect server start --expose, the graphql doesnt listen on all interface. It only listens for localhost. What should i do to fix it? Thanks

Are there limits on how many STDOUT lines will be captured in a given timeframe? When using

print()

statements in code and using the

@task(log_stdout=True)

decorator, running a particular flow manually emits 204 log lines, whereas only 103 lines end up in prefect.

Hi all. I have a task that returns a json. However, when I try to convert it into a python dict, I get this error. is there a way to do this conversion?

TypeError: the JSON object must be str, bytes or bytearray, not FunctionTask

Question. If a task were to fail due to insufficient memory/cpu requirements, would it be possible to restart the task or workflow and request more resources?

Hi all, im trying to map dbt commands to a task, run_dbt_command, however i keep getting this error, Could not infer an active Flow context while creating edge to <Task: dbt run tag:>. This often means you called a task outside a

with Flow(...)

block. If you're trying to run this task outside of a Flow context, you need to call `DbtShellTask(...).run(...). My code is below. Any ideas what's wrong? Thank you!

Hi! How do I persist the logs generated by the server? I would like the logs to stay even if I restart the server

Hi everyone I have a Prefect server and agent that is running on a virtual machine. When I add flow runs to a certain project, I add custom labels to make them easy to distinguish in the UI. What is the easiest way to update the already running Agent's labels so it can pick up new ones from the newly added flows? I have been trying to use the GraphQL to do this but I am not sure if there is built in functionality already for this? Thanks, Aniruddha

Hi! how to parallel task working in kubernetes agent? I have got flow and 10 tasks inside and I want them to work in 3 threads. LocalDaskExecutor/DaskExecutor not working

Good evening everyone. I am trying to install prefect server and then pip install "prefect[dev]". When I do the installation on prefect[dev] I am getting the error below. I have setup a virtual environment in andaconda and I downloaded the prefect server docs from git. I'm new to programming so any help is appreciated. Creating library build\temp.win-amd64-3.10\Release\ast3/Custom\_ast3.cp310-win_amd64.lib and object build\temp.win-amd64-3.10\Release\ast3/Custom\_ast3.cp310-win_amd64.exp ast.obj : error LNK2001: unresolved external symbol _PyUnicode_DecodeUnicodeEscape build\lib.win-amd64-3.10\typed_ast\_ast3.cp310-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120 [end of output]

hey everyone! thanks for having this community! i recently upgraded from prefect 2021.11.30 to 2022.01.25, and when i run

prefect get flow-runs -s

i get:

prefect.exceptions.ClientError: [{'message': 'unexpected null value for type "String"', 'locations': [{'line': 2, 'column': 5}], 'path': ['flow_run'], 'extensions': {'path': '$.selectionSet.flow_run.args.where._and[0].flow._and[0].project.name._eq', 'code': 'validation-failed', 'exception': {'message': 'unexpected null value for type "String"'}}}]

any tips on how to get around this? i do believe that

prefect-server database upgrade

was run post-upgrade

Hi All, I am trying to run a flow with some labels with

["aniruddhas", "parallel_example"]

. I have an Agent whose label's are dynamically updated every time a flow is registered so that any new labels on flows are added to the agent. So now my Agent contains these labels :

["aniruddhas", "prefect_spider", "simple_news_spider", "parallel_example"]

. I have now experimented with it and found that when my flow has just the label

["aniruddhas"]

, the Agent is able to pick them up and run them. But if the flow run has its original labels:

["aniruddhas", "parallel_example"]

, the Agent is not able to pick them up. Why is this the case? Thanks, Aniruddha

Hi All; Are there any thoughts out there on how to make a DbtShellTask dynamic? As in certain parameters would be provided to the flow and based on those the DbtTask would get created and run subsequently? It’d be great if I can see an example.

Hi, I'm doing some tests using the LocalDaskExecutor. Some flows are ending with errors with not much explanation in the pod log. Pod prefect-job-12d6087d-594qj failed. Container 'flow' state: terminated Exit Code:: 139 Reason: Error Do you know what 139 exit code means?

hi all, I am running

prefect docker agent start..

with some labels on my local machine, and am running into this error that I am unable to figure out. I have registered my flow, and it shows up in Prefect UI, but when I run that flow, it goes into Submitted State, and the logs of prefect docker agent show this error:

docker.errors.NotFound: 404 Client Error for <http+docker://localhost/v1.41/containers/4675972bf8cb3506f9edd11788ac666dc61cef7d896c2e5367f38e9a9f3fdea2/json>: Not Found ("No such container: 4675972bf8cb3506f9edd11788ac666dc61cef7d896c2e5367f38e9a9f3fdea2"

Hi. I have a flow that copies data from S3 to Snowflake. It works fine locally, but when I run it in ECS using Fargate, I'm getting a strange error.

Failed to load and execute flow run: ImportError("cannot import name 'List' from 'pip' (/usr/local/lib/python3.7/site-packages/pip/__init__.py)")

Is there a way to pass the value of a Parameter into a PrefectSecret as a string? Is something like this possible?

run_env = Parameter("run_env", default="DEV")
sf_user = PrefectSecret(SF_USER + "_" + run_env)

Hi - I'm using Prefect Server and I'm trying to register a flow into a project but I'm getting the following error:

prefect register --project "Tests" --path hello_world.py
... lots of output ...
prefect.exceptions.AuthorizationError: Malformed response received from Cloud - please ensure that you are authenticated. See `prefect auth login --help`.

hi! where can i find a list of all of the environment variables i.e.

PREFECT__*

that i can use to configure prefect?

How to get some kind of alert like slack notification when Agent last query was more than 10 minutes:

Hi all, Is it possible to cancel a flow after it ran for an X amount of time? Let's say I want to cancel flows that take longer than 1 hour? How would I approach this?

Is there any documentation/examples/recipes for how to deploy custom private modules in a github repo with a flow?

Hello all Hope that you are well. I have a flow which runs a series of shell commands but the first command is to make a Singularity image with the command

make_shell

. This command does not return any exit codes, so when I tried to run it as a standalone task, my server crashed. What would be the best way to ensure that this command is passed off first before executing other shell tasks? Thanks,

Hi folks, Is there a way to drop the default label when using

prefect build

to generate json for flow registration? There doesn't seem to be an option in the cli as there is for local agents, unless I've missed something.