Ryan Sattler
08/27/2021, 1:40 AMWilson Bilkovich
08/27/2021, 2:23 AMtornado.util.TimeoutError: Operation timed out after None seconds
before? I’m getting that in my Dask KubeCluster.Jacob Hayes
08/27/2021, 4:23 AMflow.schedule
)? I've set them up in the UI a few times, but occasionally renamed flows or changed projects and lose failure alerts. 🙈Konstantin
08/27/2021, 6:55 AM6 August 2021,10:31:49 agent01 INFO Submitted for execution: PID: 135
26 August 2021,10:31:49 agent03 INFO Submitted for execution: PID: 140
26 August 2021,10:31:49 agent02 INFO Submitted for execution: PID: 136
26 August 2021,10:31:49 execute flow-run ERROR Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/Users/user1/'")
26 August 2021,10:31:49 execute flow-run ERROR Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/Users/user1/'")
26 August 2021,10:31:49 execute flow-run ERROR Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/Users/user1/'")
tell me where to look for the errorBruno Murino
08/27/2021, 10:07 AMprefect.Client
to run a graphql query, but I’m not sure how to pass filters to it… I’m using a workaround where I pass the whole query as a string but the curly braces “{” everyone are making passing variables hard, so I’m wondering if there’s a better wayItalo Barros
08/27/2021, 12:02 PMConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))
If the Task or Flow tries to retry the execution the error pops again and only goes away if the Task/Flow is Cancelled and executed again. There's any way to bypass that?Wilson Bilkovich
08/27/2021, 1:19 PMapollo
pod logs, but I see this in the Agent logs:
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='prefect-server-initial-apollo.prefect', port=4200): Max retries exceeded with url: /graphql (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused'))
(prefect is my namespace, prefect-server-initial-apollo is the name of the service)Konstantin
08/27/2021, 2:37 PMhow to understand what he is missing
Konstantin
08/27/2021, 2:39 PMBastian Röhrig
08/27/2021, 2:46 PMwith Flow("filtering") as flow:
a_s = get_a_s() # ["a1", "a2", "a3"]
b = get_b() # "b2"
a_s_and_b = combine_a_s_and_b.map(a_s = a_s, b = unmapped(b)) # [("a1", "b2"), ("a2", "b2"), ("a3", "b2")]
FilterTask(filter_func=a_and_b_end_on_the_same_character)(a_s_and_b) # [("a2", "b2")]
Evan Curtin
08/27/2021, 2:58 PMExecuteNotebook
task and it’s spilling the cell content to stdout during normal execution, is there an option to supress? I’m using a target file to cache the output and that’s working fineJean Da Rolt
08/27/2021, 5:03 PMDanny Vilela
08/27/2021, 10:39 PMBlake List
08/27/2021, 11:53 PMFabian Brück
08/28/2021, 11:15 PMprefect run --name hello-world --execute
Looking up flow metadata... Done
Creating run for flow 'hello-world'... Done└── Name: malachite-lobster└── UUID: 381ad17b-e2d8-4e8b-84aa-ab880d2f0458└── Labels: ['agentless-run-149b4f62']
└── Parameters: {}
└── Context: {}
└── URL: <https://cloud.prefect.io/data-and-bi/flow-run/381ad17b-e2d8-4e8b-84aa-ab880d2f0458>
Executing flow run...
└── 01:11:26 | INFO | Creating subprocess to execute flow run...
└── 01:11:32 | INFO | Downloading flow from <s3://bucket/hello_world.py>
└── 01:11:32 | ERROR | Error downloading Flow from S3: An error occurred (ExpiredToken) when calling the GetObject operation: The provided token has expired.
An error occurred (ExpiredToken) when calling the GetObject operation: The provided token has expired.
Wilson Bilkovich
08/29/2021, 4:43 PMRuntimeError: Unable to find any timezone configuration
Zac Chien
08/29/2021, 6:01 PMWilson Bilkovich
08/29/2021, 7:17 PMFabian
08/30/2021, 5:36 AMCA Lee
08/30/2021, 7:08 AMBouke Krom
08/30/2021, 7:58 AMsubmittable
in the docs... Any pointers?Jelle Vegter
08/30/2021, 11:15 AMYD
08/30/2021, 2:52 PMTony Yun
08/30/2021, 3:46 PMcreate_flow_run
in code. What’s the simpliest thing to change by giving the flow run a customized run-id? Assume I only need to trigger the flow run using schedules or UI.bral
08/30/2021, 7:09 PMchicago-joe
08/30/2021, 7:49 PMFile "D:\venv\poetry\.venv\lib\site-packages\prefect\core\task.py", line 159, in init
old_init(self, *args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'state_handers'
python-BaseException
@task(log_stdout = True, nout = 2, max_retries = 3, retry_delay = timedelta(minutes = 15), state_handers = [handler])
Michael S
08/30/2021, 8:01 PMDockerAgent()
on my
local computer, and I am using S3
(private bucket) for my flow storage. I am having
the flow run with DockerConfig
, with a custom image. I am able to
register the flow, and I see it in S3.
Error: When trying to run the Flow, I get Error downloading Flow from S3: Unable to
locate credentials
What I tried:
1. At first, I thought, oh!, the agent doesn’t have the creds (at first I thought the server might need it as well, but I don’t think that’s true). I shut down the agent,
export AWS_ACCESS_KEY_ID=...
and export AWS_SECRET_ACCESS_ID=...
, in the shell, and restarted. This didn’t work.
2. I then figured maybe the container needs the creds. I (temporarily) hardcoded them into my docker image. This works! This could be a fine workaround. I could build the image with --build-arg
to remove the hardcoding, but I don’t really want to bake my secrets in the container environment. I rather pass them in at runtime (via the agent)-- i.e., something like docker run -e AWS_ACCESS_KEY_ID...
What I don’t understand: What is the best workflow for having docker containers pull flows from S3? I was looking in the prefect Secrets
, but this seems like it’s for accessing secret within tasks, but getting flows from somewhere. Am I misunderstand how Secrets
can be used?
What would be nice: An example of the best practice.
Sorry if this is such an obvious question! Still trying to get my bearings here.Vincent
08/30/2021, 9:16 PMElijah Moreau-Arnott
08/30/2021, 10:39 PMget_task_run_result
to retrieve task results from the child flow, however I was getting errors where it couldn't find a task with the provided slug (which I had set to 'output') - I thought it was an issue with my results location until I realized that despite annotating the child task like so: @task(slug="output",result=GCSResult(...
the slug name (as visible in the context) was output-copy
. I now got get_task_run_result
working by grabbing the task with slug output-copy
but was unsure of how the -copy
came about. Is this documented anywhere?John Marx
08/30/2021, 11:42 PMJohn Marx
08/30/2021, 11:42 PM