https://prefect.io
Join Slack
Hi there! I have a question about schedules. How can I fetch schedules for a flow group and specific...
# ask-community
d
Hi there! I have a question about schedules. How can I fetch schedules for a flow group and specific parameters. Example: I have very many schedules. If I fetch with this query it will be a lot of data for the client. I need to fetch with a filter based on “targetProjectId” from “parameter_defaults”, see the image. It is possible? Thanks :)
Copy code
query GetSchedules {
  flow_group(where: {id: {_eq: "eb595947-xxxxxx"}}) {
    schedule 
  }
}
k
Hey @Denis Pokotylyuk(External), this is quite tricky. I’ll try it
🙏 1
I am unsure where you are querying from but for the first one you gave, you can do:
Copy code
query GetSchedules {
  flow_group(where: {id: {_eq: "7fb20810-cd37-41cf-9b04-218c284f6db5"},
                    default_parameters: {_contains: {x: "something else "}}}) {
    schedule
    default_parameters
  }
}
👀 1
Are these parameters created on the Flow group (UI) or through code?
You really this might be easier to just bring in Python and manipulate there
d
I create schedules by using graphQL mutation:
Copy code
set_flow_group_schedule
k
I think my query might be enough then to point you in the right direction I hope
d
default_parameters for my flow group is empty:
Copy code
query GetSchedules {
  flow_group(where: {id: {_eq: "eb595947-xxxxx"}}) {
		default_parameters
  }
}
Copy code
{
  "data": {
    "flow_group": [
      {
        "default_parameters": {}
      }
    ]
  }
}
How can I query on that then?
Copy code
default_parameters: {_contains: {x: "something else "}}}) {
What is “x: “something else “” in my case? 🙂
parameter_defaults and default_parameters are different things, right?
k
Ah ok I was setting it here
I guess you set it from the clock on the Flow?
d
But these parameters will be the same for all schedules under the flow group, right?
k
They will I was just thinking you set it here. There’s just a couple of places to set 😅
d
I need to filter by parameters which are not common. Some schedules will be set for a “targetProjectId” = “2", some schedules for “targetProjectId” = “3", and I need to fetch schedules for specific targetProjectId, since it will be very many schedules. If you understand. 🙂
k
Are you filtering on the Flow run level or on the schedule level?
d
I need to filter on schedule level. When I create a schedule for a flow group (by set_flow_group_schedule or in UI) I register some parameters. Then I want to get all schedules with specific parameters.
k
Ok
So I have this
And I want to try just pulling 
value two
d
Yes, exactly 🙂
all with “p”: “value two”
k
Yeah I’m not positive this can be done unfornately because the kost you can do it check the 
clocks
. Not something as nester as this. You’d have to execute that in Python
d
Ok, understand. But can it be done in Python? How then? Do you have a link to the related docs?
k
Copy code
from prefect.client import Client

query = """
query {
  flow(where: {id: {_eq: "608f7f60-cc0d-4abc-a63a-bb653ff2c1e5"}}) {
    id
    schedule
  }
}
"""

client = Client()
res = client.graphql(query)
print(res["data"])
You can use the Prefect Client and then 
res["data"]
will be a dictionary you can manipulate
d
But this is actually the same query. It gets all the schedules, but then it is just to filter out, right?
Thank you a lot for helping me, btw ❤️
k
Yes exactly filter it in Python because we can’t do it at the query level
Of course!
d
Ok, the problem that I can have hundreds of schedules, and I wanted to filter it on query level. 🤔 I do not like the idea to fetch the whole list of schedules.
k
I don’t think we have a choice here because the GraphQL can’t filter on that. But actually you might have another problem. It works like this. Prefect schedules the next 10 flow runs for a given flow. The scheduler runs every 90 seconds. So let’s say you have 20 scheduled runs in the next minute, only 10 will be schedules and the other 10 will not be because of the cap in the number of runs that will be scheduled. After 10/20 run, the scheduler will run again, and then you’ll get another 10/20.
So for a given Flow, if you are firing them concurrently, you can only have a max of 10 schedules. Does that make sense?
d
But if I run every schedules about once or twice a day, not so very often. Can I have more than 10?
k
I think…as long as you don’t have more than 10 in a two minute time frame it should be fine.
But yes this is unrelated to the graphql lol. I don’t think you have a choice there because it can’t be filtered on the query level.
✅ 1
d
Ok, I think I have to spread them over time. 🙂
I see. Thank you very much for your time! Nice support through the Slack! Now it is a bed time 🥱
👍 1
k
Of course! 🙂
7 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by