Antonio Manuel BR
02/15/2022, 7:05 AMJean-Baptiste Six
02/15/2022, 9:28 AM@task
def subtask():
return 1
with Flow("subflow") as subflow:
subtask()
@task
def main_task():
subflow.run()
with Flow("main_flow") as main_flow:
main_task()
But I faced this Error :
_Unexpected error while running flow: KeyError('Task slug init_dirs-1 is not found in the current Flow. This is usually caused by a mismatch between the flow version stored in the Prefect backend and the flow that was loaded from storage.\n- Did you change the flow without re-registering it?\n- Did you register the flow without updating it in your storage location (if applicable)?')_
I precise that I had already register the _main_flow_ (without the subflow inside) and it worked, but then I updated it and I registered the subflow, and it failed, could you help me please ? 🙏
The task "_init_dirs_" is in the main_flow, finishes in a success state, but this error append when subflow.run() is called (and init_dirs is not in the subflow)Martin Teller
02/15/2022, 10:37 AMFlorian Kühnlenz
02/15/2022, 1:11 PM{
flow_group(
where: {flows: {name: {_ilike: "%name%"}, archived: {_eq:false}}} ) {
id
name
flows{
name
id
archived
}
}
}
Will not actually filter for not archived versions. What am I doing wrong?Simon Stusak
02/15/2022, 4:15 PMZach Schumacher
02/15/2022, 4:21 PMNatsume Kirito
02/15/2022, 4:37 PMGabriel Milan
02/15/2022, 4:41 PMvalues.yaml
file such as
prefectConfig:
services:
towel:
max_scheduled_runs_per_flow: 50
and when I try to upgrade my deployments with this, the towel pod doesn't get redeployed and the issue is still there. Is this way correct or should I do anything else?Adi Gandra
02/15/2022, 5:13 PMXavier Babu
02/15/2022, 5:47 PMJake
02/15/2022, 6:59 PMConnectionClosedError('Connection was closed before we received a valid response from endpoint URL: "<insert_s3_url_here>.prefect_result".',)
It is causing our flows to fail intermittently and I’m not sure how to debug this.David Wang
02/15/2022, 8:24 PMKevin Mullins
02/15/2022, 8:34 PMPrefect Core encourages the use of small "helper" tasks. Won't those these inflate my billable usage?
Tasks that run for less than one second do not count as billable usage, so you can continue to design workflows that follow best practice and include as many helper tasks as you need.
Now that I’m using Prefect more and watching my task usage. I’ve noticed that both Secrets (EnvVarSecret in this case) which behind the scenes are a task and very simple tasks that just wrap parameters/task results into another structure vary quite a bit in runtime and often exceed the <1 second rule and thus will get billed. This is especially worrisome to me for secrets these tasks are necessary for securely handling things and some of our flows have quite a few secrets.
I’m not sure what a solution would be, but wanted to see if anybody has noticed this and if there could be anything done to ensure these simple tasks don’t inflate actual bills.Chowdary
02/15/2022, 10:04 PMChowdary
02/15/2022, 10:05 PMbrian
02/15/2022, 10:56 PM--param
to set parameters locally. I’ve just started working on a flow that will depend on a prefect secret, and would love to be able to set a value for the secret locally. Is there a way to do this? My naive hope was that prefect run
would have a --secret
flag similar to --param but this doesn’t seem to be the caseWilliam Grim
02/16/2022, 12:26 AMNitin Bansal
02/16/2022, 6:29 AMMukamisha jocelyne
02/16/2022, 8:45 AMNoah Holm
02/16/2022, 10:56 AMtask1
-> task2
-> task3
Task3 is the last task of the flow and has other upstream dependencies which might cause task3 to be skipped in a flow run. Today a flow run got the following states: Failed
-> TriggerFailed
-> Skipped
I assume the flow is considered successful since task3 was skipped and skipped states are considered successful (that makes sense). But we don’t want to consider a flow run successful if any tasks in the flow has a failed state. Refreshing my skills in the docs I’m thinking of handling it with state handlers, but I assume that a state handler on the flow would still see the flow state as successful due to the last skipped task. Is the only way forward here to have a state handler on each of the tasks in the flow?Chris Arderne
02/16/2022, 11:49 AMDaniel Nilsen
02/16/2022, 12:08 PMJason Motley
02/16/2022, 1:00 PMRichard Hughes
02/16/2022, 2:55 PMGuillaume Latour
02/16/2022, 3:04 PMTypeError: cannot pickle 'lxml.etree.XMLSchema' object
Of course the result of one of my task is of this type and I would prefer not to change it.
I tried to launch that process again with config.flows.checkpointing = "false"
in the ~/.prefect/config.toml
file but I got the same error...
This is an error that I cannot reproduce locally with prefect run <flow>
. Do you guys have any leads on how I can solve this issue or at least reproduce the error locally?Emma Rizzi
02/16/2022, 3:47 PMSean Talia
02/16/2022, 4:12 PMSam Werbalowsky
02/16/2022, 6:21 PMChris Reuter
02/16/2022, 7:43 PMYoutube▾
Jason Motley
02/16/2022, 8:11 PMSET procedure2 := bla bla
etc..Jason Motley
02/16/2022, 8:11 PMSET procedure2 := bla bla
etc..Kevin Kho
02/16/2022, 8:12 PMpymysql
does. What is your error?Jason Motley
02/16/2022, 8:15 PMpymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SET @includeLostPipeline := FALSE' at line 2")
Kevin Kho
02/16/2022, 8:15 PMJason Motley
02/16/2022, 8:17 PMKevin Kho
02/16/2022, 8:20 PMJason Motley
02/16/2022, 8:22 PM