Jon Young
12/30/2022, 2:33 PMinit
or `exit`:
# Prefect has its own implementation of a context manager,
# which it calls a Resource Manager.
# pylint and mypy are unhappy with the implementation.
# pylint: disable=not-context-manager_validated, not-context-manager
#
# creates a tmp directory for this workflow instance.
# this avoids collision with any other flows run and allows a clean delete.
with resource_managers.TemporaryDirectory( # type: ignore[attr-defined]
consumer_code=consumer_code_validated, # type: ignore[arg-type]
provider_code=provider_code_validated, # type: ignore[arg-type]
resource_type=resource_type_validated, # type: ignore[arg-type]
) as tmp_dir:
Santhosh Solomon (Fluffy)
01/01/2023, 12:51 PMBernardo Galvao
01/06/2023, 11:34 AMprefect_orion.1.vvr5ju6d6s5x@SEMRI01 | 11:00:23.861 | ERROR | prefect.orion.services.telemetry - Failed to send telemetry:
prefect_orion.1.vvr5ju6d6s5x@SEMRI01 | Shutting down telemetry service...
is Orion taken down? is --analytics-off
the way to ensure this error does not happen?Anna Geller
01/06/2023, 2:53 PMBernardo Galvao
01/06/2023, 4:12 PMSander
01/09/2023, 10:20 PMBernardo Galvao
01/10/2023, 2:44 PMAll connection attempts failed
Is the Prefect agent trying to connect to PREFECT_API_URL
?Bernardo Galvao
01/12/2023, 9:35 AMBernardo Galvao
01/16/2023, 12:19 PMBernardo Galvao
01/18/2023, 12:33 PMJulian Brendel
01/18/2023, 12:48 PMBernardo Galvao
01/18/2023, 2:56 PMBernardo Galvao
01/19/2023, 10:10 AMprefect deployment build
?
(Same way you can pass an override to a docker-container)
Edit: corrected the command from git to prefect, my badBernardo Galvao
01/19/2023, 10:45 AMprefect-gitlab
on the client side for this error not to occur?
Or does it not match the slug name?wences
01/24/2023, 7:47 PMKelvin DeCosta
01/25/2023, 7:49 AM.map
for all the tasks in a batch.
While this works, it feels very hacky and isn't ideal for performance.
Any help is appreciatedJohn Kang
01/27/2023, 3:28 PMJ
01/28/2023, 6:34 AMHa Pham
01/30/2023, 8:22 AMwences
02/01/2023, 4:16 PMSlackbot
02/01/2023, 5:42 PMzlee
02/03/2023, 8:37 PM.apply()
them. (we use s3 storage, and run our flows via Kubernetes jobs, in images which have the flow repo installed as a python package.
This works fine except that when we build the Deployments in end-user repos, we run into the problem that the files containing the imported flows are not under the same working directory that we are building the Deployments from. The flow repo is installed as a python package and thus is under /opt/env/.../site-packages/...
. This causes the code that sets the Deployment's entrypoint to fail with an error like <path_to_flow_file> is not in the subpath of <cwd> OR one path is relative and the other is absolute.
.
We are working around this by importing the flows and modifying their .___module___
property to 'trick' prefect into thinking that the flows come from the end-user repo. Is there something I'm missing here that would allow us to do this in a less hacky way, or is this pattern just not something prefect supports? Thanks!Richard Alexander
02/08/2023, 4:05 PMfrom prefect import flow, get_run_logger
@flow()
async def my_flow():
logger = get_run_logger()
# Start async process and pass logger
Will the logger also be async?
As a bit of background, we have an async process that spawns multiple other async processes and will create a good deal of logging that we need to catch with the prefect logger. We want to make sure that the logging won't cause any blocking issues. Is passing a logger in this way a safe way to go about it for async code?Jon Young
02/10/2023, 5:02 PMOluremi Akinwale
02/13/2023, 10:02 AMAlireza
02/14/2023, 9:49 AMGreat Expectation
is no longer supported in v 2.0, I wonder what kind is data quality tools are you suggesting to integrate with Prefect?Stefan
02/17/2023, 9:41 AMdef main(date: DateModel = None) -> None:
How can I use the current date for flow_run_name? Unless doing data for another day, I don't call the flow with any arguments.Serina
02/17/2023, 4:46 PMKelvin DeCosta
02/21/2023, 12:05 PMasync
execution of tasks via .map
.
In the documentation for the .map
function, it mentions that if the task that is being mapped is async
, then the .map
call must be awaited.
However, I keep getting an error with the type checker stating that there are no overloads of .map
that can be awaited.
What should could be done to fix this?Stefan
02/23/2023, 8:19 PMStefan
02/23/2023, 8:19 PMDylan
02/23/2023, 8:21 PMStefan
02/23/2023, 8:21 PMDylan
02/23/2023, 8:22 PMStefan
02/23/2023, 8:27 PMDylan
02/23/2023, 8:27 PM