Hello everyone! I have a flow that i want to sched...
# prefect-server
Hello everyone! I have a flow that i want to schedule with cron string with different parameters for each cron. The problem is, i am building a UI for non programmer users to be able to schedule some reports, So the cron string is not predefined. I was thinking about creating a non scheduled masterflow that will a have as parameter a string containing the cron schedule and within that flow, i would create and register another scheduled childflow with the correct schedule. I would run the masterFlow with graphql query each time a user create a new report schedule. What do you think? Is that possible?
Hi @Mac Gréco Péralte Chéry - that is possible, though it could get a little messy. Essentially you are using Prefect as your application’s API - you hit a route to run the master flow, and it registers an appropriately-configured child flow. If it’s helpful, we are working on making flow schedules customizable from the Prefect UI, which should land either this month or next.
@Jeremiah Thanks for your quick answer! The second option would be to store some information about the schedule report in a database (report, cron_string, next_schedule_date ) and on the prefect server have a Masterflow that check regularly in this database if there is any reports which next_schedulate_date<= now . From there, for each scheduled report , this Masterflow will call another Childflow that will do the rest of the job, If the Child flow succeed, i go to the database again and update the next_schedule_date base on the cron string. And REPEAT. This is a little messy too...🙄
The perfect solution for my use case would be to have a graphql mutation that create scheduled flow with parameters defaults set to specified values from an unscheduled flow with parameters .
I think under the circumstances your original proposal is the best one - essentially using a custom flow to create your optimal API.
Hi @Jeremiah I tried to implement the first solution I have registered successusfully a very basic flow with one task and within that task i tried to register another flow but when i try to run the MasterFlow I get an error:
Copy code
uly 3rd 2020 at 10:56:25pm | prefect.CloudTaskRunner
ERROR lens
Unexpected error: SystemError('unknown opcode',)
Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.6/site-packages/prefect/engine/runner.py", line 48, in inner
    new_state = method(self, state, *args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/prefect/engine/task_runner.py", line 822, in get_task_run_state
    self.task.run, timeout=self.task.timeout, **raw_inputs
  File "/home/ubuntu/.local/lib/python3.6/site-packages/prefect/utilities/executors.py", line 186, in timeout_handler
    return fn(*args, **kwargs)
  File "C:/Users/Admin/PycharmProjects/project/app.py", line 12, in basic_task
  File "C:/Users/Admin/PycharmProjects/project/app.py", line 23, in base_task
SystemError: unknown opcode
In the past, whenever we’ve seen that error it’s because of a Python version mismatch - typically, the version of Python used to build and serialize the flow is different than the version being used to execute it. This can happen, for example, if you’re using a Docker container — it looks like you’re building with Python 3.6 based on the traceback; is it possible you’re trying to run a flow that was serialized with Python 3.7 (or vice versa?).
I don’t know that many related issues have been opened on the repo itself, other than this one, but I suggest looking at the Python versions being used to serialize / run.
Thanks for your answer. I decided to make the test locally before to be sure it can works. I get this error
Copy code
July 4th 2020 at 9:14:43pm | prefect.CloudTaskRunner
ERROR lens
Unexpected error: RuntimeError('Attempting to call `flow.register` during execution of flow file will lead to unexpected results.')
@Jeremiah Forget about my latest answer. I switched to using the client and this code seems to works (a test code) :
Copy code
from prefect import task, Flow
from prefect import Client

def basic_task():
    client = Client( "<http://localhost:4200/graphql>")
    f = Flow("childFlow")
    client.register(flow=f, build=False)

with Flow(name="MasterFlow") as masterFlow:
I will continue to update the code (S3 storage, parameters etc ...) and see if it still work.