Gabriel Milan
11/18/2021, 1:47 PMregister
, all my new or modified flows get registered fine. On the other side, some flows I've deleted are still registered on my server. Is there a way of syncing these?Dekel R
11/18/2021, 2:18 PMSash Stasyk
11/18/2021, 3:34 PMAleksandr Liadov
11/18/2021, 5:37 PMlucas oliveira
11/18/2021, 6:44 PMSubmitted for execution: Task arn:aws:ecs:us-east-2:<my-account>:task/my-fargate-cluster/1eb7b9cbd17c448ab4fcdc76e4692297
Has anyone experienced a similar problem and can you help me?
Thanks!Aqib Fayyaz
11/18/2021, 7:13 PMVipul
11/18/2021, 7:15 PMVipul
11/18/2021, 8:22 PMkiran
11/18/2021, 8:39 PMnohup
instead of Supervisor? I’m thinking something simple like this nohup prefect agent local start 2>&1 > /tmp/prefect_local_agent.log &
John T
11/18/2021, 9:00 PMPrefectResult
? I’m currently encountering this error:
TypeError: PrefectResult only supports JSONSerializer or DateTimeSerializer
Kevin Kho
11/18/2021, 10:13 PMJohn Muehlhausen
11/18/2021, 11:08 PMHugo Shi
11/18/2021, 11:43 PMDominic Pham
11/19/2021, 12:15 AMJeremiah Lethoba
11/19/2021, 8:03 AMAJ
11/19/2021, 8:50 AMAdam Everington
11/19/2021, 10:19 AMflow.register(project_name='my-project', indempotency_key=flow.serialized_hash())
on prefect server will the version number still get bumped each time?Florian Kühnlenz
11/19/2021, 11:42 AMAqib Fayyaz
11/19/2021, 12:18 PMprefect agent kubernetes install -k API_KEY | kubectl apply --namespace=my-namespace -f -
are they both the same, like i have one deployed with prefect server on gke but i need to use prefect cloud so is one agent enough for both server and cloud or i have to deploy seperate one for cloud?ek
11/19/2021, 3:26 PMLeon Kozlowski
11/19/2021, 6:03 PMMax Kolasinski
11/19/2021, 9:31 PMall_failed
or any_failed
- if the ETL Task fails, it shouldn’t run. What I believe I would need is something like an on_x_task_failed
option- it seems like the available options are way too broad to be useful.
• I then looked into some of the ideas on the Conditional Logic page, but this seems clumsy for a few reasons. I need a Task specifically to check the State of the Validation Task, and then on our Schematic View we have additional Tasks showing up for each Case Task as well as the Merge Task. All combined, it makes our Schematic look like the image below which seems crazy for what is effectively if x do y
.
I feel like I have to be approaching this in completely the wrong way- if anyone had any ideas or suggestions I would be extremely grateful.Tom Shaffner
11/19/2021, 11:16 PMAndré Bonatto
11/20/2021, 12:50 PMfrom prefect import Flow, Parameter, task
from etl import prepare, load
from crawlers import crawler1, crawler2
from typing import Dict, Callable, String
prepare = task(prepare)
load = task(load)
def make_standard_pipeline(flow_name: String, func :Callable, func_params:Dict):
with Flow(flow_name) as flow:
params = {k: Parameter(k, default = v) for k,v in func_params.items()}
df = func(**params)
df = prepare(df)
df = load(df)
return flow
pipe1 = make_standard_pipeline('flow1', task(crawler1), {})
pipe2 = make_standard_pipeline('flow2', task(crawler2), {'type' : 'xxx'})
Locally this code runs fine and I can also register these on the prefect server. However, when I try to run the flows, only the first flow defined in the file runs successfully (I tested reordering the flows). For the other flows, I get Key Error saying it couldn't found task slug crawler2. Does anyone has hints on what could be causing this problem?
Thank you.Chen Di
11/20/2021, 3:30 PMManuel Gomes
11/21/2021, 5:40 PMValueError: Filename must be a string
So there is clearly some sort of.. unwrapping/unpacking that I'm missing?
my flow is likewise in the thread, as is the invocation.
So... would someone please tell me in which exact angle I should smack my forehead and the correct octave of the "duh!"... plus maybe what I should be doing instead?Wilhelm Su
11/21/2021, 8:54 PMAnh Nguyen
11/22/2021, 10:21 AMJean-Baptiste Six
11/22/2021, 2:05 PMslack_notifier
, without success.
• Then I tried with SlackTask
, this is my piece of code (didn't work) :
def post_to_slack_on_failure(task, old_state, new_state):
if new_state.is_failed():
if isinstance(new_state.result, Exception):
value = "```{}```".format(repr(new_state.result))
else:
value = str(new_state.message)
msg = (
f"The task `{prefect.context.task_name}` failed "
f"in a flow run {prefect.context.flow_run_id} "
f"with an exception {value}"
)
SlackTask(
message=msg,
webhook_secret="<https://hooks.slack.com/services/*******/*******/*******>"
).run()
return new_state
• Finnally, I tried with a custom solution (didn't work, again) :
def post_to_slack_on_failure(task, old_state, new_state):
if new_state.is_finished():
msg = "Task {0} finished in state {1}".format(task, new_state)
# replace with your Slack webhook URL secret name
secret_slack = cast(str, Secret("<https://hooks.slack.com/services/*******/*******/*******>").get())
<http://requests.post|requests.post>(secret_slack, json={"text": msg})
return new_state
This is my test flow :
@task
def task_error():
raise Exception("Test")
with Flow("Test Slack", state_handlers=[post_to_slack_on_failure]) as flow:
task_error()
flow.run()
(I also tried EmailTask with smtp_type="INSECURE", and once again it didn't work)
Need some help please 🙏Margaret Walter
11/22/2021, 4:18 PMMargaret Walter
11/22/2021, 4:18 PMAnna Geller
11/22/2021, 4:21 PM