Christian Nuss
05/02/2022, 7:39 PMChris Reuter
05/02/2022, 8:10 PMZach Schumacher
05/02/2022, 9:18 PMMatt Alhonte
05/02/2022, 9:56 PMECSRun
would I put Dask security args? Like tls_ca_file
and stuff?Anthony L
05/02/2022, 10:02 PMrandom
generates the same values between runs.
I'm using Prefect Cloud 1.0 and running a local agent (with Prefect 1.2.1 library installed).
Here's my flow:
import prefect
from prefect import task, Flow
from random import random
@task
def task1():
logger = prefect.context.get("logger")
random_val = random()
<http://logger.info|logger.info>(f"{random_val=}")
if random_val < 0.5:
raise Exception()
with Flow("random-bug-flow") as flow:
task1()
flow.register(project_name="ant-flows")
I manually scheduled a run of this flow in Prefect Cloud and I see the random value in the logs. However, when I schedule another run (and any run thereafter), it generates the same "random" value. Is this expected?Gaurav
05/02/2022, 10:52 PMArtem Vysotsky
05/03/2022, 2:12 AMCole Murray
05/03/2022, 5:19 AMBernard Greyling
05/03/2022, 5:45 AMdavzucky
05/03/2022, 6:25 AMSuresh R
05/03/2022, 6:37 AMprefect.exceptions.ClientError: [{'path': ['secret_value'], 'message': 'An unknown error occurred.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]
Domenico Di Gangi
05/03/2022, 7:09 AMThomas Mignon
05/03/2022, 7:29 AMSang Young Noh
05/03/2022, 8:23 AMAndreas Nord
05/03/2022, 8:54 AMUserWarning: Task <Task: task_name> has retry settings but some upstream dependencies do not have result types. Cloud features such as input caching and resuming task runs from failure may not work properly.
The (automatic) retrying seems to be working fine, not sure what this warning refers to. I am aware that I cannot resume task run from failure, but not sure why that is related to retry settingsRon Levi
05/03/2022, 10:18 AMAmruth VVKP
05/03/2022, 10:45 AM@flow(name='primary_flow', version='1', task_runner=SequentialTaskRunner())
def primary_flow(**kwargs):
... # perform data processing tasks, all tasks are non-async
for cli_command in parsed.result():
shell_run_command(cli_command) # I've also tried using asyncio.run for running the shell task
Scenario 2 -
Primary Flow: Uses sequential task runner, a bunch of processing tasks. The final task to build and call a CLI is done through another async subflow
@flow(name='primary_flow', version='1', task_runner=SequentialTaskRunner())
def primary_flow(**kwargs):
... # perform data processing tasks, all tasks are non-async
for _ in parsed.result():
cli_flow(_) # I've also tried using asyncio.run for running the shell task
@flow(name='cli_flow', version='1', task_runner=SequentialTaskRunner())
async def cli_flow(parsed):
for _ in parsed:
... # Build CLI
await shell_run_command(cli_command)
I am getting this exception thrown (attached screenshot) and the CLI task is not getting started/executed. Can someone help me out on this one please (I am terrible with asnyc processes)?Mars
05/03/2022, 2:29 PMprefect k8s agent install
. I’ve uploaded my flow to a private GitHub repo and registered it with Prefect. And I’ve added a Cloud Secret called GITHUB_ACCESS_TOKEN
that holds a valid GitHub personal access token. When I run my flow the agent’s GitHub storage gives me an UnknownObjectException(404, 'Not Found')
error. If I change the flow to use a different Cloud Secret key for the PAT, such as access_token_secret='MYKEY'
, then the agent tells me ValueError('Local Secret "MYKEY" was not found.')
.
How can I introspect the kubernetes agent to verify that the GitHub PAT secret is being loaded from Prefect Cloud correctly?Jai P
05/03/2022, 3:09 PMJakub Cieslik
05/03/2022, 4:54 PMGowtham Manne
05/03/2022, 5:22 PMDavid Beck
05/03/2022, 5:29 PMprefect.
, e.g. prefect.flow_name
. Given the documentations cautioning of renaming context attributes, I'm trying to brainstorm the simplest and most effective way of doing this across all our flows. Do you have suggestions?Andrew Lawlor
05/03/2022, 6:25 PMPod prefect-job-5e3af599-tl2xs failed. No container statuses found for pod
where can i look for a more detailed message? any idea what actually caused it to fail?
also, of those jobs, all but one passed. the one that failed had the same message (no container statuses found).Artem Vysotsky
05/03/2022, 6:55 PMchicago-joe
05/03/2022, 7:34 PMBradley Hurley
05/03/2022, 8:34 PMExecuteNotebook
Task.Andrew Lawlor
05/03/2022, 8:41 PMMatthew Roeschke
05/03/2022, 9:07 PMprefect register -p project
to get the names of the current flows that would have been registered
2. Get the IDs of the flows through the GraphQL API
3. Delete flow IDs that that do not have names from step 1
Is there a more sensible way to get the “current flows” in a project beside the prefect register
trick?Matt Alhonte
05/03/2022, 9:35 PMKilled
) and the mapping stays in a mapped
stateCole Murray
05/04/2022, 5:45 AMCole Murray
05/04/2022, 5:45 AMAnna Geller
05/04/2022, 9:30 AM