prefect-community #prefect-community

prefect-community

Kyle McEntush

12/24/2022, 7:19 PM

I hope this is a relatively simple question: using prefect 2.0, how can I get a task to return two values? as a function it is something like

def f():

✅ 1

Kyle McEntush

12/24/2022, 7:20 PM

i know prefect 1.0 had

nout

as a task arg, but I can't find the equivalent in the docs...

✅ 1

Shubham Wani

12/24/2022, 7:39 PM

#needHelp Using Prefect 2.7, I'm trying to connect a MYSQL databse to write data through a prefect task. once I run

cursor.executemany(query, values),

I run the

connection.commit()

to to transfer all data from buffer to the table in dataset. Error faced in the following command:

connection.commit()

Error:

00:12:33.556 | ERROR   | Task run 'load_records-41eb504c-0' - Encountered exception during execution:
Traceback (most recent call last):
  File "/home/shubhamwani/nice/edilitics-server/edilitics-env/lib/python3.8/site-packages/prefect/engine.py", line 1346, in orchestrate_task_run
    result = await run_sync(task.fn, *args, **kwargs)
  File "/home/shubhamwani/nice/edilitics-server/edilitics-env/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 69, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(call, cancellable=True)
  File "/home/shubhamwani/nice/edilitics-server/edilitics-env/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/home/shubhamwani/nice/edilitics-server/edilitics-env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/home/shubhamwani/nice/edilitics-server/edilitics-env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/home/shubhamwani/nice/edilitics-server/prefect_flows/load/prefect2_tasks.py", line 166, in load_records
    (
TypeError: cannot unpack non-iterable NoneType object

I've tried using different sql connectors(mysqlclient, mysql-connector, pymysql), but the error persists. is this somehow related to prefect? anyone else facing this issue? Any help is welcomed, Thanks.

✅ 1

Shruti Hande

12/26/2022, 4:40 AM

prefect.exceptions.PrefectHTTPStatusError: Server error '500 Internal Server Error' for url '<https://api.prefect.cloud/api/accounts/2f48cb6f->
0049-4569-879d-7123d9113e31/workspaces/9da885d9-9d9f-4ee9-9b42-84bfb4a3d9aa/work_queues/name/queue_5'
Response: {'exception_message': 'Internal Server Error'}

✅ 1

Aditya Sharma

12/26/2022, 8:03 AM

Hi i am trying to restart the docker agent which is running on linux. i am not able to restart the docker agent

Aditya Sharma

12/26/2022, 8:03 AM

and the flows are failing as below

Aditya Sharma

12/26/2022, 8:04 AM

anybody who can add some views on this

Jelle Vegter

12/26/2022, 1:32 PM

Hi, it looks like the agent isn’t healthy for some reason. Are there logs from the agent?

Kelvin DeCosta

12/26/2022, 1:50 PM

Never used Azure, but my bet would be on the

prefect

blocks not being registered. The hosted agent must run something that looks like this:

prefect block register -m prefect_azure.container_instance

Ollie Sellers

12/26/2022, 9:33 PM

Hello everyone, I have a quick couple of questions around persistent input and result storage: 1) will restarted mapped tasks use results for the partial mapped result or does the task need to finish completely? 2) will the stored results be used even if there are changes to the code in the task if params are the same? Apologies if these are easily answerable through the docs, I couldn't locate the answer in my first search. I am also currently using v1. Thanks, Ollie

Rikimaru Yamaguchi

12/27/2022, 12:58 AM

Is there any way to check if my flow is a sub flow or not? thank you.

✅ 1

David Cheedie

12/27/2022, 2:54 PM

Good morning, we received this error on our remote machine which is running prefect agent as a systemd service

David Cheedie

12/27/2022, 3:03 PM

Good morning, we received this error over the weekend on our remote machine which is running prefect agent as a systemd service

Dec 24 22:04:58 linux prefect[279492]: 22:04:58.602 | ERROR   | prefect.agent - Server error '500 Internal Server Error' for url

The service failed, tried to restart a few times, was unsuccessful, and then our scheduled jobs on prefect cloud switched to state=Late I'd like to rule out that this was an error on the prefect cloud side, does anyone know anything about prefect cloud issues around 12/24 22:04 UTC?

✅ 1

Michael Levenson

12/27/2022, 4:41 PM

Hi all I am getting a weird bug in production for a flow with a kubernetes run config that hangs when I redeploy it

Prefect Core Version: 1.2.4

I trigger a flow run from a github actions step and it just hangs (happened yesterday and the run state was still in progress for 2 days) even though the job does not exist I know the flow did not finish because I see that my ml model tracking service did not register a new model I triggered the flow manually and it successfully completed. This flow is also scheduled and completes. What could be causing this issue?

PKay

12/27/2022, 5:14 PM

Hi all, anyone found some good documentation on how to build custom docker images. Really confused by the Prefect blocks UI and logging not providing any information. Usually the agent crashes and resulting in the flow run to enter the state of "PENDING" or "CRASHED", provides no information why the image (from registry) wasn't pulled and executed.

✅ 1

Chris Whatley

12/27/2022, 5:54 PM

Is prefect_slack deprecated? I can’t get it to be recognized as a block with cloud or with a local orion setup that I’m testing.

✅ 1

John-Craig Borman

12/27/2022, 7:35 PM

Hi all, I'm working on unit testing some flows in Prefect 2 - what are the best practices for populating Blocks via Pytest fixtures?

👀 1

✅ 1

Sebastian Steele

12/27/2022, 8:57 PM

Hey everyone is there a way to get a workers poll history through the CLI? i want to see if one failed

Matt Morris

12/27/2022, 9:06 PM

Hey everyone, just upgraded from prefect 1 to 2 and enjoying a lot of the new features, especially the

ConcurrentTaskRunner

! However, I am noticing that the speed of using

prefect_dask.DaskTaskRunner

is slower than using

prefect.executors.DaskExecutor

(prefect 1). In both cases I was using processes rather than threads. Speed depends on the task, but some tasks that read from s3 are taking a little over twice as long with the

DaskTaskRunner

. Has anyone faced something or similar, or any pointers as to why I may be experiencing this? Thanks!

👀 1

Rikimaru Yamaguchi

12/28/2022, 5:30 AM

hello. Is there any way to run flow triggered api?　Thanks!

✅ 1

Mohit Singhal

12/28/2022, 11:45 AM

Hi , is there any way to use .fn() and .submit() simultaneously while calling a task function. considering we have to pass parameters as well?

Abosede Oladipo

12/28/2022, 12:59 PM

Hello.

Abosede Oladipo

12/28/2022, 1:47 PM

Hello everyone Is there a way to get a list of all blocks in a workspace using Python code? I know that it can be done via the cli with

prefect block ls

, but I can't seem to find anything in the docks using python code.

✅ 1

Tadej Svetina

12/28/2022, 6:00 PM

Hi, is there a way to use S3 storage (for deployments/blocks) without explicitly providing the secret key/id? I would like to have the authentication handled automatically by AWS SDK, and will be provided using env vars or instance metadata

Dalin Stone

12/28/2022, 7:25 PM

Hi Team! 👋

Ken Nguyen

12/28/2022, 7:53 PM

Hi! I recently had a working config of my Prefect 1 flow by importing the following modules:

import prefect
from prefect import task, Flow
from prefect.storage import GitHub
from prefect.run_configs import ECSRun
from prefect.client import Secret

However, when I run my code now (Prefect 2.7.4), I get a

ModuleNotFoundError: No module named 'prefect.storage'

error (same goes for

prefect.run_configs

and

prefect.client

). Was there a recent update to the way we should import these modules?

CA Lee

12/29/2022, 8:39 AM

Hello, I have a question regarding scheduling (exceptions)

✅ 1

Paco Ibañez

12/29/2022, 3:03 PM

Hello! I noticed that flow function comments are shown as description in the flow's deployment, is there a way to format the comments? I have tried html and markdown with no luck.

@flow
def hello():
   """
   Is there a way to get the text in here to be formatted in the awesome prefect UI?
   """
   pass

✅ 1

Chris Gunderson

12/29/2022, 3:57 PM

Hi Prefect Team, We have our dev environment up and running for Prefect 2 now. When setting up the Prefect 2 production environment we encountered an issue where we cannot access our database. We are using an ECS Task to run our flows in dev and prod. The dev instance is able to read data and execute the program, but production is not able to. The ECS Task in production is using the same VPC, Subnet, routes and security group as development. Is there another possible reason for this failure?

✅ 1

Dalin Stone

12/29/2022, 4:20 PM

Hi Team! 👋 Is it possible to have a custom docker image in your local file system be referenced by the docker-container infrastructure block when building a deployment or am I required to have my custom docker image on Docker Hub/Docker Registry and access it through those systems?

✅ 1

Title

Dalin Stone

12/29/2022, 4:20 PM

✅ 1

Anna Geller

12/29/2022, 4:44 PM

yes it is, you would need to set the image pull policy to Never, then Prefect will only look for images available locally

🙌 2

Dalin Stone

12/29/2022, 5:27 PM

You're the best Anna! 🥂

😄 1

@Anna Geller How can I pass the '-p {host_port}:{container_port}' option to the docker infrastructure block to properly map ports? When I try to connect to a external/remote mysql server instance from my code, I receive error '2002, can't connect to mysql server {my_server}, 115' when running the deployment. I think this is because my container doesn't have the properly mapped ports to connect to an external/remote mysql server. None of the documentation mentions how to pass the -p port argument

Anna Geller

12/30/2022, 12:58 AM

what's your setup? are you on Prefect Cloud or self-hosting? if you're self-hosting or just exploring things locally with docker, check out this docker compose setup https://discourse.prefect.io/t/running-prefect-2-with-docker-compose/1516

✅ 1

Dalin Stone

12/30/2022, 3:12 PM

@Anna Geller We are using Prefect Cloud, is it possible to pass that -p in the docker container infrastructure block or is there another way we could map ports for the container?

This works great, thank you Anna! 🥂

🙌 1

View count: 3