paimoe
05/17/2022, 7:31 AMSander
05/17/2022, 7:38 AMValentin Baert
05/17/2022, 10:20 AMMoss Ebeling
05/17/2022, 11:28 AMkushagra kumar
05/17/2022, 12:12 PMNEWBIE Here:
Hi, Is there a possibility of non-linearity and cross- and intra-flow/ work flow interaction in prefect 2.0
? Need to do some POC.Valentin Baert
05/17/2022, 12:13 PMTyndyll
05/17/2022, 12:22 PMValentin Baert
05/17/2022, 12:38 PMFlorian Guily
05/17/2022, 12:56 PMFrank Hereford
05/17/2022, 2:58 PMSylvain Hazard
05/17/2022, 3:08 PMfrom loguru import logger
and uses that for logging. Unfortunately, those don't appear in Prefect logs which is not great.
I've tried adding PREFECT__LOGGING__EXTRA_LOGGERS="['loguru']"
in my environment but doesn't seem to work. Is there another way to do this ?Koby Kilimnik
05/17/2022, 3:32 PMKoby Kilimnik
05/17/2022, 6:57 PMJason
05/17/2022, 7:06 PMDanny Vilela
05/17/2022, 7:17 PMFlow
initialization? Or is the best way to unset prefect.config.flows.checkpointing
before building the Flow
?Jeffery Newburn
05/17/2022, 7:31 PMMike Vanbuskirk
05/17/2022, 8:32 PMSubmitted for execution: Task arn:<task-arn>
with no further logs generated. In cloudwatch, it only shows:
2022-05-17T16:18:52.575-04:00 [2022-05-17 20:18:52,575] INFO - agent | Deploying flow run f2f45aac-ccfb-4bb9-88db-fb1d00426989 to execution environment...
2022-05-17T16:18:53.805-04:00 [2022-05-17 20:18:53,805] INFO - agent | Completed deployment of flow run f2f45aac-ccfb-4bb9-88db-fb1d00426989
Jason Motley
05/17/2022, 8:55 PMenv={"EXTRA_PIP_PACKAGES": "mypackage--index-url=<https://my_url>"})
Leon Kozlowski
05/17/2022, 8:56 PMHoratiu Bota
05/17/2022, 9:09 PMCommand ["/bin/sh","-c","prefect execute flow-run"]
(however i need to run prefect in bash with a conda environment activated). Is there any way to configure the ECS task to remove the ["/bin/sh/", "-c",]
before the prefect command? My container is configured with the appropriate entrypoint, runs bash and activates conda at startup, but /bin/sh -c
undoes that setupJessica Smith
05/17/2022, 9:44 PMprefect.exceptions.ClientError: [{'path': ['set_task_run_states'], 'message': 'State update failed for task run ID d253a979-2ad4-4add-9c60-e569d75758e6: provided a running state but associated flow run 165e6a96-0e0d-464b-9a85-fc97454b9375 is not in a running state.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]
I have a flow that kicks off a few hundred flow runs, and it was working fine in a test environment, but I got this as soon as I tried to run it in a production environmentJeffery Newburn
05/17/2022, 10:01 PMdatetime.now()
inside of a flow, not a task will that update on each run?Daniel Saxton
05/17/2022, 10:10 PMDavid Beck
05/17/2022, 10:20 PMdavzucky
05/18/2022, 1:30 AMJan Bršťák
05/18/2022, 8:24 AMprefecthq/prefect:1.2.1-python3.9
image, and we are using Prefect Cloud. All seems to be fine with the Agent, we can see it in the UI, and also in Cloudwatch logs, whenever we trigger a flow run, it says Deploying flow run…
and Completed deployment of flow run
. But no Flows are getting started, it gets stuck on Submitted every time (it worked before, but we changed image for agent from prefecthq/prefect:0.14.13-python3.8
and also roles in AWS which have the same permissions as original had). There are no errors, and nothing in logs. I’m not really sure what to do? Thanks for any help. Run config looks like this: RUN_CONFIG = ECSRun(
_labels_=["prod"],
_task_role_arn_="arn:aws:iam::XXX:role/prefectTaskRole",
_execution_role_arn_="arn:aws:iam::XXX:role/prefectTaskExecutionRole",
_task_definition_arn_="prefect-task:4",
_run_task_kwargs_=dict(
_cluster_="XXX",
),
)
Florian Guily
05/18/2022, 10:16 AMDaniel Sääf
05/18/2022, 12:32 PM>>> result = leonardo_dicaprio("leo")
However, when i try to create the deployment i get a:
ImportError: cannot import name 'hello_world' from 'test' (/usr/local/Cellar/python@3.10/3.10.4/Frameworks/Python.framework/Versions/3.10/lib/python3.10/test/__init__.py)
So basically, my question is: how do i import functions from local python-files in a flow?
(Let me know if this isn’t the right place for this type of questions!)yair friedman
05/18/2022, 12:44 PMKrzysztof Ciesielski
05/18/2022, 1:10 PMmax_retries
+ retry_delay
task parameters. I wanted to wrap some of my tasks with retries, and I thought it should be enough to add these parameters to the @task
annotation. However, when a task raises an exception, it shows in the Prefect Cloud UI that it's state is "Retrying Task (after attempt 1 of 61)", but after a short while the entire flow gets cancelled.
Example:
@task(log_stdout=True, max_retries=60, retry_delay=timedelta(seconds=10))
def perform_something(param1: Optional[str],
param2: any,
year_month_str: str,
environment_label: str) -> datetime: