Jake
05/10/2022, 6:05 PMRaviraja Ganta
05/10/2022, 6:08 PMBob Colner
05/10/2022, 6:47 PMalex
05/10/2022, 7:00 PMKathryn Klarich
05/10/2022, 7:03 PMDylan
05/10/2022, 7:21 PMJason
05/10/2022, 7:33 PMname
of a SnowflakeQuery
task similar to @task to give it a human meaning name? I didn't see it in https://docs.prefect.io/api/latest/tasks/snowflake.htmlSander
05/10/2022, 7:53 PMJoseph Vedam
05/10/2022, 8:05 PMFrederick Thomas
05/10/2022, 8:22 PMPaco Ibañez
05/10/2022, 8:31 PMScott McCallen
05/10/2022, 8:34 PMprefect register
command. There are a couple of scenarios where I want to change aspects of the flow run container based on Flow Parameters. For instance, I'd like the flow container to have additional Job annotations in some runs of the flow, but not all runs of the flow. I'm not sure how to do this dynamically when the flow is started. The only thing I've been able to figure out so far is that I need multiple registrations of the same flow - each registration having a different Kubernetes Job template. Is there a dynamic way to adjust/modify the Job template for a given registered flow?Michelle Brochmann
05/10/2022, 8:48 PM{"detail":"Not Found"}
instead of the view I wanted to look at. It would be very nice if I could take advantage of a url to quickly get to the place I wanted - like if I wanted to remove the “timeline” part to just get to the view for the specific run.
Thanks! 🙂Andrew Lawlor
05/10/2022, 10:43 PMFailed to retrieve task state with error: ClientError([{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}])
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/engine/cloud/task_runner.py", line 154, in initialize_run
task_run_info = self.client.get_task_run_info(
File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 1479, in get_task_run_info
result = self.graphql(mutation) # type: Any
File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 473, in graphql
raise ClientError(result["errors"])
prefect.exceptions.ClientError: [{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}]
before running any tasksDekel R
05/11/2022, 7:50 AMException raised while calling state handlers: ClientError([{'path': ['secret_value'], 'message': 'An unknown error occurred.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}])
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/client/secrets.py", line 137, in get
value = secrets[self.name]
KeyError: 'SLACK_WEBHOOK_URL'
I get this error randomly once in a while - anyone have the same issue?
I didn’t reregister or touched the 2 flows for at least 2 months now.
Thankstas
05/11/2022, 8:10 AMFlorian Guily
05/11/2022, 9:29 AMThomas Opsomer
05/11/2022, 10:52 AMstored_as_script=True
. Any idea how to overcome this ?Klemen Strojan
05/11/2022, 11:34 AMKubernetesRun
and Docker
storage, something people using Prefect in production might find interesting. If there are any other posts or documentation on this topic I would appreciate if you can point me there.Jeremy Savage
05/11/2022, 12:26 PM@task(max_retries=200)
def some_task():
if helpers.test_if_job_done() is False:
raise RETRY(
"Work not done yet, retrying in 20 seconds.",
start_time=pendulum.now().add(seconds=20),
)
TIAPatrick Koch
05/11/2022, 1:56 PMJessica Smith
05/11/2022, 3:57 PMJason
05/11/2022, 3:58 PMJessica Smith
05/11/2022, 4:18 PMBinoy Shah
05/11/2022, 4:19 PMMateo Merlo
05/11/2022, 4:31 PMArthur Jacquemart
05/11/2022, 4:36 PMKendal Burkhart
05/11/2022, 6:28 PMstorage = Docker(registry_url=os.getenv("REGISTRY_URL"),
image_name=os.getenv("IMAGE_NAME"),
base_image="<http://nnnnnnnnnnn.dkr.ecr.us-west-2.amazonaws.com/flow-base-image:latest|nnnnnnnnnnn.dkr.ecr.us-west-2.amazonaws.com/flow-base-image:latest>"
)
and built from a very simple Dockerfile:
FROM prefecthq/prefect:latest-python3.8
ENV PYTHONPATH=$PYTHONPATH:/
COPY ./utilities utilities
Currently, this setup fails. When the flow runs in ECS, it exits immediately with an error:
Exit Code 1
Command ["/bin/sh","-c","prefect execute flow-run"]
As part of my troubleshooting, I bypassed using the image in ECR and set
the base image in the Docker storage for the flow:
storage = Docker(registry_url=os.getenv("REGISTRY_URL"),
image_name=os.getenv("IMAGE_NAME"),
base_image="prefecthq/prefect:latest-python3.8"
)
This too failed with the same error.
I reviewed the logs for builds that did not set the base image, and saw this
image being used by default:
prefecthq/prefect:0.15.4-python3.8
I then used this image in my Docker storage:
storage = Docker(registry_url=os.getenv("REGISTRY_URL"),
image_name=os.getenv("IMAGE_NAME"),
base_image="prefecthq/prefect:0.15.4-python3.8"
)
This flow runs successfully.
Updating my Dockerfile and using that build from ECR also works.
So…does anyone have any idea as to why using prefecthq/prefect:latest-python3.8 fails?
I would prefer not to pin the version in my Dockerfile.Chris Reuter
05/11/2022, 6:55 PMJason
05/11/2022, 7:43 PMsave_s3 = Parameter(...)
if save_s3:
load_s3(dataset)
Jason
05/11/2022, 7:43 PMsave_s3 = Parameter(...)
if save_s3:
load_s3(dataset)
Anna Geller
05/11/2022, 7:46 PMcase()
Jason
05/11/2022, 7:47 PM