Tejas Shah
02/22/2024, 6:07 AMflow.deploy(
name="flow-deployment-docker",
work_pool_name="ecs-work-pool",
image=f"tejasanilshah/prefect-images:{commit_hash}",
parameters={"date": datetime.date.today()},
tags=["ecs", "using-deploy"],
schedule=CronSchedule(cron="0 8 * * *"),
is_schedule_active=True,
)
However, when the flow is deployed, the parameter is frozen to the date of deployment and not to the current date.Marvin
02/22/2024, 6:07 AMCronSchedule
. The typical behavior for parameters set at deployment is that they maintain the same value with which they were deployed.
A possible solution to ensure that the parameter is dynamic would be to set the parameter within the flow definition itself instead of setting it at deployment time. When the flow triggers according to the schedule, the datetime.date.today()
function would run on each execution, giving you the current date.
I would recommend looking deeper into the Prefect documentation or reaching out on the Prefect community Slack or GitHub for an example or further advice. As I'm currently tasked with utilizing chroma to find documentation and not searching external community platforms, I can't directly pull information from Slack or the forums for you. Would you like to continue searching the documentation or inspecting GitHub issues for similar cases?Tejas Shah
02/22/2024, 6:16 AMMarvin
02/22/2024, 6:16 AMNone
object as a parameter to a flow in Prefect due to JSON serialization issues.
However, I can suggest a general solution for this kind of issue. When facing serialization problems with the None
object, it typically indicates that you are trying to use Python's None
in a context that expects JSON serialization, like parameters in Prefect flows. JSON doesn't have an explicit None
value but uses null
instead.
Before passing None
as a parameter, make sure you are using it in a way that is compatible with JSON serialization. Prefect uses JSON serialization for flow parameters, so if you want to pass None
, you might have to either:
1. Pass "None"
as a string and handle it within your flow.
2. Use a custom serializer that handles the None
value if the default JSON serializer is unsuitable.
Would you like to look more into the Prefect documentation or GitHub issues to find a more precise answer?Tejas Shah
02/22/2024, 6:19 AMMarvin
02/22/2024, 6:19 AMdeployment.yaml
configuration file. If you want to have the ability to manually trigger flows with a specific date or have a date inserted at runtime when they're not manually triggered, this can be handled within the deployment configuration by setting up your parameters accordingly.
For more details on configuring parameters in deployments, you can refer to the Prefect documentation on deployments here.
It appears that life—the rest of it, I mean—is bound to be somewhat problematic.Marvin
02/22/2024, 6:54 AMdatetime.date.today()
, is frozen at the time of deployment rather than being dynamic for each run, then you may need to take an alternative approach.
Instead of setting the parameter directly in your flow.deploy
call, you could use a task within your flow that retrieves the current date each time the flow is run. This task would replace the direct parameter definition.
Here is a more detailed explanation on parameters that you can provide via the deployment.yaml
configuration and override them at runtime: Prefect Parameters.
If you're expecting me to say something along the lines of "Don't panic," you'll be sorely disappointed.