Sander
05/19/2022, 2:15 PMfrom logging import getLogger
logger = getLogger(__name__)
Class MyClass:
def __init__(self):
pass
def my_method(self):
<http://logger.info|logger.info>(‘hi’)
Does that make sense?Vadym Dytyniak
05/19/2022, 3:18 PMKyle Austin
05/19/2022, 3:34 PMJohn Muddle
05/19/2022, 3:47 PMDylan
05/19/2022, 5:51 PMjob_template_path
now. Same template loaded via S3 during flow init didn’t throw this error, but registration blows up with this error now.Jacqueline Riley Garrahan
05/19/2022, 7:07 PMprefect_test_harness
in 2.0 and am wondering if any such fixtures exist for 1.2? Are there any general guides for setting up tests for projects using prefect?Florian Guily
05/19/2022, 7:50 PMprefect auth list-keys
command but not in prefect cloud. Is it normal ?Madison Schott
05/19/2022, 8:02 PMNikhil Jain
05/19/2022, 9:19 PMstorage.add_flow()
method. Is there a way around it?Nikhil Jain
05/19/2022, 9:34 PMDylan
05/19/2022, 10:22 PMLuke Segars
05/20/2022, 12:38 AMNash Taylor
05/20/2022, 1:15 AMpip install prefect>=2.0b
) into an environment with mypy, and since there doesn’t appear to be a py.typed
file in prefect
, mypy is skipping it:
Skipping analyzing "prefect": module is installed, but missing library stubs or py.typed marker
I threw a py.typed in there (touch py.typed
), but obviously this is less than ideal. Am I missing something obvious?Luke Segars
05/20/2022, 2:30 AMDaniel Sääf
05/20/2022, 5:48 AMSylvain Hazard
05/20/2022, 8:57 AMloguru
logger in Prefect tasks as described here.
At first glance it works pretty well but it looks like DEBUG
logs aren't sinked into Prefect. Any idea what I could do ?Sander
05/20/2022, 9:29 AMMarwan Sarieddine
05/20/2022, 11:57 AMValentin Baert
05/20/2022, 11:59 AMflow
decorated function "MissingContextError: No settings context found"
What am I missing ?Florian Guily
05/20/2022, 1:28 PM--param bool_param=True
but True
is treated as a String during flow execution. How can i pass a boolean param in cli for flow run ?Suresh R
05/20/2022, 1:48 PMAndrew Lawlor
05/20/2022, 2:32 PMKubernetes Error: pods failed
and no container statuses found for pod
and in GKE i see SyncLoop DELETE
and audit_log, method: "io.k8s.core.v1.pods.delete", principal_email: "system:serviceaccount:kube-system:generic-garbage-collector"
and SyncLoop REMOVE
Bob De Schutter
05/20/2022, 2:58 PMfor flow in flows:
flow.register(
project_name=f"Data Platform - {os.environ.get('PREFECT_TARGET_ENV', 'Develop')}",
add_default_labels=False,
set_schedule_active=True if os.environ.get('PREFECT_TARGET_ENV', 'Develop') == 'Production' else False
)
This has worked for me before but since I upgraded prefect to 1.2.1, I'm getting an error message related to the Azure storage I'm using for the flows:
azure.core.exceptions.ResourceExistsError: The specified blob already exists
Any idea why this strategy is not working anymore?James Keegan
05/20/2022, 3:47 PMShellTask
task) but keep running into this error. Any suggestions?
The SSH key is set up correctly, but there doesn't appear to be a user associated with the ShellTask
commands.
This wasn't an issue previously, as older flow images can pull git projects just fine. Only came up recently after I rebuilt the image again (Prefect version & python version are still the same).
Cloning into './dbt_project'...
No user exists for uid 1000190000
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
James Keegan
05/20/2022, 5:00 PMcloudpickle
v2.1.0 was just released an hour ago and it has broken the healthcheck when we try to register a flow. We're using the prefecthq/prefect:0.15.6-python3.7
as a base image.
We now get this error when building the docker image.
Step 21/21 : RUN python /opt/prefect/healthcheck.py '["/opt/prefect/flows/prefect-dbt-run-modelling.prefect"]' '(3, 7)'
---> Running in 1d14c333ced9
Beginning health checks...
System Version check: OK
Traceback (most recent call last):
File "/opt/prefect/healthcheck.py", line 152, in <module>
flows = cloudpickle_deserialization_check(flow_file_paths)
File "/opt/prefect/healthcheck.py", line 44, in cloudpickle_deserialization_check
flows.append(cloudpickle.loads(flow_bytes))
AttributeError: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from '/usr/local/lib/python3.7/site-packages/cloudpickle/cloudpickle.py'>
Florian Guily
05/20/2022, 5:21 PMconfig.py
file in flow_utilities
folder like this from flow_utilities.config import funcA funcB
. When i do so, i get an error. So i tried to reproduced it with the dummy code you have on the image. I get the same error which is ModuleNotFoundError: No module named 'flow_utilities'
. This really seems stupid to me as it is just importing some func from local file but somehow i can't make it work with this file architecture. What am i doing wrong ?Florian Guily
05/20/2022, 5:46 PMSecret.get()
it apparently only search for local secrets. The doc says i have to change prefect.config.use_local_secrets
to false but i can't understand how... I suppose it is in the config.toml file but as the flow has to run on an eks cluster, how can i specify this for the cluster ?Mike Vanbuskirk
05/20/2022, 7:24 PMMalthe Karbo
05/20/2022, 7:54 PMAlvaro Durán Tovar
05/20/2022, 8:42 PMdef build(...):
with Flow(...) as Flow:
...
return flow
is it possible to use docker storage? thinking on possible issues trying to find the path of the flow inside the docker file, there won't any "flow" variable on the module level