https://prefect.io logo
Title
d

Denis Pokotylyuk(External)

03/24/2022, 9:46 PM
Hi there! I have a question about schedules. How can I fetch schedules for a flow group and specific parameters. Example: I have very many schedules. If I fetch with this query it will be a lot of data for the client. I need to fetch with a filter based on “targetProjectId” from “parameter_defaults”, see the image. It is possible? Thanks :)
query GetSchedules {
  flow_group(where: {id: {_eq: "eb595947-xxxxxx"}}) {
    schedule 
  }
}
k

Kevin Kho

03/24/2022, 9:50 PM
Hey @Denis Pokotylyuk(External), this is quite tricky. I’ll try it
🙏 1
I am unsure where you are querying from but for the first one you gave, you can do:
query GetSchedules {
  flow_group(where: {id: {_eq: "7fb20810-cd37-41cf-9b04-218c284f6db5"},
                    default_parameters: {_contains: {x: "something else "}}}) {
    schedule
    default_parameters
  }
}
👀 1
Are these parameters created on the Flow group (UI) or through code?
You really this might be easier to just bring in Python and manipulate there
d

Denis Pokotylyuk(External)

03/24/2022, 10:06 PM
I create schedules by using graphQL mutation:
set_flow_group_schedule
k

Kevin Kho

03/24/2022, 10:10 PM
I think my query might be enough then to point you in the right direction I hope
d

Denis Pokotylyuk(External)

03/24/2022, 10:10 PM
default_parameters for my flow group is empty:
query GetSchedules {
  flow_group(where: {id: {_eq: "eb595947-xxxxx"}}) {
		default_parameters
  }
}
{
  "data": {
    "flow_group": [
      {
        "default_parameters": {}
      }
    ]
  }
}
How can I query on that then?
default_parameters: {_contains: {x: "something else "}}}) {
What is “x: “something else “” in my case? 🙂
parameter_defaults and default_parameters are different things, right?
k

Kevin Kho

03/24/2022, 10:12 PM
Ah ok I was setting it here
I guess you set it from the clock on the Flow?
d

Denis Pokotylyuk(External)

03/24/2022, 10:13 PM
But these parameters will be the same for all schedules under the flow group, right?
k

Kevin Kho

03/24/2022, 10:14 PM
They will I was just thinking you set it here. There’s just a couple of places to set 😅
d

Denis Pokotylyuk(External)

03/24/2022, 10:16 PM
I need to filter by parameters which are not common. Some schedules will be set for a “targetProjectId” = “2", some schedules for “targetProjectId” = “3", and I need to fetch schedules for specific targetProjectId, since it will be very many schedules. If you understand. 🙂
k

Kevin Kho

03/24/2022, 10:17 PM
Are you filtering on the Flow run level or on the schedule level?
d

Denis Pokotylyuk(External)

03/24/2022, 10:20 PM
I need to filter on schedule level. When I create a schedule for a flow group (by set_flow_group_schedule or in UI) I register some parameters. Then I want to get all schedules with specific parameters.
k

Kevin Kho

03/24/2022, 10:20 PM
Ok
So I have this
And I want to try just pulling
value two
d

Denis Pokotylyuk(External)

03/24/2022, 10:23 PM
Yes, exactly 🙂
all with “p”: “value two”
k

Kevin Kho

03/24/2022, 10:26 PM
Yeah I’m not positive this can be done unfornately because the kost you can do it check the
clocks
. Not something as nester as this. You’d have to execute that in Python
d

Denis Pokotylyuk(External)

03/24/2022, 10:29 PM
Ok, understand. But can it be done in Python? How then? Do you have a link to the related docs?
k

Kevin Kho

03/24/2022, 10:34 PM
from prefect.client import Client

query = """
query {
  flow(where: {id: {_eq: "608f7f60-cc0d-4abc-a63a-bb653ff2c1e5"}}) {
    id
    schedule
  }
}
"""

client = Client()
res = client.graphql(query)
print(res["data"])
You can use the Prefect Client and then
res["data"]
will be a dictionary you can manipulate
d

Denis Pokotylyuk(External)

03/24/2022, 10:37 PM
But this is actually the same query. It gets all the schedules, but then it is just to filter out, right?
Thank you a lot for helping me, btw ❤️
k

Kevin Kho

03/24/2022, 10:38 PM
Yes exactly filter it in Python because we can’t do it at the query level
Of course!
d

Denis Pokotylyuk(External)

03/24/2022, 10:39 PM
Ok, the problem that I can have hundreds of schedules, and I wanted to filter it on query level. 🤔 I do not like the idea to fetch the whole list of schedules.
k

Kevin Kho

03/24/2022, 10:43 PM
I don’t think we have a choice here because the GraphQL can’t filter on that. But actually you might have another problem. It works like this. Prefect schedules the next 10 flow runs for a given flow. The scheduler runs every 90 seconds. So let’s say you have 20 scheduled runs in the next minute, only 10 will be schedules and the other 10 will not be because of the cap in the number of runs that will be scheduled. After 10/20 run, the scheduler will run again, and then you’ll get another 10/20.
So for a given Flow, if you are firing them concurrently, you can only have a max of 10 schedules. Does that make sense?
d

Denis Pokotylyuk(External)

03/24/2022, 10:52 PM
But if I run every schedules about once or twice a day, not so very often. Can I have more than 10?
k

Kevin Kho

03/24/2022, 10:55 PM
I think…as long as you don’t have more than 10 in a two minute time frame it should be fine.
But yes this is unrelated to the graphql lol. I don’t think you have a choice there because it can’t be filtered on the query level.
1
d

Denis Pokotylyuk(External)

03/24/2022, 10:56 PM
Ok, I think I have to spread them over time. 🙂
I see. Thank you very much for your time! Nice support through the Slack! Now it is a bed time 🥱
👍 1
k

Kevin Kho

03/24/2022, 11:00 PM
Of course! 🙂