Mars
08/16/2022, 11:57 PMif __name__ == '__main__': myflow()
. This is without a running Orion server.
The code in my flow that creates the Secret block appears to work. However, when I try use my Secret block in my 2.0 flow the prefect client raises a PrefectHTTPStatusError, and says ‘Block document not found’.
In Prefect 1.0 I could use PREFECT__CONTEXT__SECRETS__
to set the values I need, but I don’t see any mention of similar functionality for 2.0.Jaime Raldua Veuthey
08/17/2022, 7:52 AMTim Helfensdörfer
08/17/2022, 9:37 AMTim Helfensdörfer
08/17/2022, 10:15 AMMohamed Ayoub Chettouh
08/17/2022, 10:29 AMprefect block type ls
does give the block types.
UPDATE2: Something is off, when hosting on localhost and doing an ssh tunnel on it it works like a charm. When hosting on 0.0.0.0
the page is empty.
UPDATE3: Now it works on the public IP as well ?!? Maybe it needs to start without --host
the first timeAndreas Nigg
08/17/2022, 11:27 AMprefect deployment build r_script_automation.py:r_script_automation --name retention_cohort_analysis_deployment -t k8s -sb gcs/gcs-prefect-stprage -ib kubernetes-job/generic-k8s-job -o cohort_deployment.yaml
However, as soon as I add infrastr. overrides, the flow stays in "pending" state forever
prefect deployment build r_script_automation.py:r_script_automation --name retention_cohort_analysis_deployment -t k8s -sb gcs/gcs-prefect-stprage -ib kubernetes-job/generic-k8s-job -o cohort_deployment.yaml --override image=europe-docker.pkg.dev/vol-at/rm-datateam-repository/r-script-automation:beta-23 --override env.GIT_PYTHON_REFRESH="quiet" --override env.GOOGLE_APPLICATION_CREDENTIALS="/google/.google-secret-key.json" --override env.CHROMIUM_FLAGS="--no-sandbox"
If I manually delete the infra_overrides-block from the deployment-yaml - the flow works again. So somehow this env.xyz - overrides are the problem. Can you point me to my mistake?
Edit: I tested overriding the "image" - this works. It really only fails when overriding the environment variables....Sachin Ranveer
08/17/2022, 11:53 AMJason Thomas
08/17/2022, 12:33 PM2.0.4
Hi all, I just upgraded from 2.0b8
to 2.0.4
.
In v2.0b8
errors within flows/tasks were caught by Prefect and the flow would continue, now in 2.0.4
they are being raised and crashing my run. How can I get back to the previous behavior?
I’ll post my code in the threadPedro Machado
08/17/2022, 12:39 PMHanno Küpers
08/17/2022, 12:59 PMstate.result()
only returns _Result(key='82ecff4d3a454e77b08b5ce93e1dd418', filesystem_document_id=UUID('af2119c0-c631-45ed-af6a-0cff1cf396a5'))
. I tried different return value types for the flow (str, dict). What am I missing here? How can I easily access the return value of a succesful flow run? Thank youMars
08/17/2022, 2:03 PMHamza Naanani
08/17/2022, 2:05 PMcloud login
command ?
I tried setting prefect config set PREFECT_API_URL
and PREFECT_API_KEY
as a setting but it doesn't seem to work.Alberto Fernández
08/17/2022, 2:12 PMTom Klein
08/17/2022, 3:04 PMMapped
- with no mapped tasks being created, and then to the flow itself just immediatley moving to the Failed
stated - even though nothing failed?Darren
08/17/2022, 3:08 PMssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)
my credentials look like this email_server_credentials = EmailServerCredentials(
username="demo@company.com",
password="mypasswordhere",
smtp_server="smtp.office365.com",
smtp_type="STARTTLS",
smtp_port=587
)Neil Natarajan
08/17/2022, 3:26 PMNot Ready
with the future.result()
resolving to None
. The issue is that two of the final tasks of my flow accept futures of previous tasks as input and are listed in the wait_for
section as well. These final two tasks instead are stuck in a Pending
state and this causes the flow to hang instead of running to completion. Any advice on how to properly implement the ideal behavior (tasks that depend on futures from previous tasks that fail or didn't run should also fail/not run and flow should run to completion) would be greatly appreciated, thank you!
In the logs, I see the flow encountered an UpstreamTaskError
.Ben Strange
08/17/2022, 3:29 PMSam Garvis
08/17/2022, 4:20 PMLucien Fregosi
08/17/2022, 4:38 PMKubernetesJob
is it possible to specify a node-selector
and tolerations
to be able to assign the worker pod to a specific node pool ?
I can’t see it in the doc
Thanks for your helpJeffrey Lam
08/17/2022, 4:48 PMDavid Elliott
08/17/2022, 5:05 PMJosh Paulin
08/17/2022, 5:48 PMKhuyen Tran
08/17/2022, 7:12 PMemo loic
08/17/2022, 7:47 PMTony Yun
08/17/2022, 8:07 PMIlya Galperin
08/17/2022, 8:31 PMkubernetes-job
infrastructure block in Prefect 2.0? In 1.0, we could accomplish this by using the KubernetesRun
module and passing it like so: KubernetesRun(job_template_path="my/local/path.yaml")
It seems like we can try to mimic the functionality in 2.0 by using some hacks to pipe the output from our job template .yaml file, represent it as a one-line JSON string then force that output into --override job=
but this doesn’t seem ideal.Ross Teach
08/17/2022, 8:56 PMThis work queue uses a deprecated tag-based approach to matching flow runs; it will continue to work but you can't modify itI noticed this warning on the Prefect Cloud UI. Is there any documentation on the recommended approach to use?
Kathryn Klarich
08/17/2022, 9:30 PMAaron Goebel
08/18/2022, 12:10 AMParameters
or something analogous as input to a KubernetesRun
to request a specific amount of resources on the fly? I'm working w/ Fargate and want to give users flexibility to run flows with differing amounts of computeMichael Levenson
08/18/2022, 2:47 AM