Hi, I was wondering if it is possible to grab/filter by a flow parameter’s value via a GraphQL query. I’ve been messing around with the interactive API and can see you can filter

flow_runs

by parameter by using something like:

flow_runs(where:{parameters: {_has_key: "process_date"}}

but what I would like to do is filter the query for any flows that ran on a specific process date, such as, 8/17/2021, rather than filtering for flows that have the

process_date

parameter. Thanks!

Hey @Ben Sack, I am not 100% sure this can be done with a query only. Am checking, otherwise you might have to do that filter on the response in Python

Sorry just seeing this now, but thanks for the input! I didn’t think it was possible but I wanted to check before proceeding with any workarounds

So I asked the team…and I don’t believe it’s possible yep

Thanks again Kevin! As a workaround, I am attempting to query all of the successful flow runs given a specific flow name, grab the parameters from the flow runs and then filter through the results using python. Strangely enough, many of the recent flow_runs that come back from my query have

"parameters": {}

, yet when I check in the parameters section for the flow_runs in the Perfect UI you can plainly see that there are parameters. Any ideas on why this would be? I can provide my query and results if that would help better understand the issue. Thanks

Yeah! I think there is another field for

default_parameters

.

parameters

are the ones explicitly supplied, but there is a

default_parameter

in the flow or flow run where you are probably passing the parameters through. Does that make sense for you?

Hey Kevin, so I tried using the

default_parameters

but that didn’t seem to do the trick either since that is a filter for the

flow_group

rather than

flow_run

. Here is the query I’m running and a snippet of the Prefect UI so you can better see what I’m struggling with. Query:

{
  flow(
    where: {_and: [{name: {_eq: "snowflake_cost_usage"}}, {project: {name: {_eq: "DSS-DE-Prod"}}}]}
  ) {
    id
    name
    flow_runs(where: {state: {_eq: "Success"}}) {
      parameters
      start_time
      id
      state
      name
    }
  }
}

Part of the response:

{
            "parameters": {},
            "start_time": "2021-09-12T04:00:09.013801+00:00",
            "id": "aa1c50c3-5aed-44e2-90bb-634b605152a6",
            "state": "Success",
            "name": "zippy-malamute"
          },

Snippet of specific flow_run’s parameters from Prefect UI:

I see your point. Are these supplied to the flow or on the default parameters on the schedule (flow group)? And no worries at all 🙂 Here to help.

I believe they are supplied to the flow here:

with Flow("snowflake_cost_usage") as flow:
    process_date = Parameter(name="process_date", default=date_minus_seven())
    spoke_accounts = Parameter(
        name="spoke_accounts", default=list(SPOKE_ACCT_DBS.keys())
    )
    spoke_dbs = Parameter(name="spoke_dbs", default=list(SPOKE_ACCT_DBS.values()))
    hub_account = Parameter(name="hub_account", default=HUB_ACCT)
    hub_db = Parameter(name="hub_db", default=HUB_DB)

but I do see them in the Default Parameters section in the settings in Prefect UI as well. The parameters are static except for the

process_date

Gotcha I’ll ask the UI team how that info is retrieved

Thanks a lot!

Did you try getting them at the flow level like this?

query {
  flow_run {
    parameters
    flow {
      parameters
    }
  }
}

Hi @Ben Sack - if you want to query your runs directly to see what they ran with, you can do this from the IAPI:

query($parameters: jsonb) {
  flow_run(where:{parameters: { _contains: $parameters }}) {
    id
    name
    parameters
  }
}

and in the

Query Variables

section, you would paste the parameters you’re looking for, like this:

{
  "parameters": {
    "lat": 37.2761
  }
}

This is great let me give it a shot now - thanks again for assist!

For sure! Let us know if that doesn’t work

@nicholas for the

query variables

section I am getting the error with that syntax saying

Expected value of type "jsonb"

You’ll want to use key/value there, so:

"parameters": {
  "process_date": "<<the process data you're looking for"
}

Yeah so that was what I originally had but it still yells at me regarding expecting ‘jsonb’ type

Do you mean like this?

query {
  flow_run (where: {flow: {name: {_eq: "test"}}}) {
    parameters
    flow  {
      name
      parameters
    }
  }
}

exactly! Sorry that was a very simply question just getting use to the graphQL syntax.

No worries. I chatted with nicholas about this and there really might be a bug on our end because the parameters should be populated and it shouldn’t be that hard. The UI has some code that collects the parameters in these different places and reduces them to produce the parameters in the card. This shouldn’t have to be this hard to query. Default parameters from the flow group are passed to the flow run, BUT default parameters from the flow aren’t

Sorry for the slowness over here, been in and out of meetings

Thanks! So both are working now but unfortunately the

default_parameters

for

process_date

is always the same (since it is the

process_date

that was initially calculated at flow build) rather than the

process_date

that is generated from the flow’s docker storage (stored as script) each time the flow runs

I think it is a bug for the population of

parameters

part yep. Will open a ticket for this today. Just wondering, does the

process_date

show up in the UI correctly?

Thanks! I’ll take a look now

Gotcha so it may have been a change introduced on our end? Thanks for all of the info. I haven’t gotten a chance to write the ticket yet but I’ll include these images.

Awesome, thanks for all the help!

I am so sorry, it’s been on my to do list but I get to writing the issue. Will do ASAP and ping you today, but on the development side, we are definitely aware of it but haven’t gotten to it yet.

Ok cool - no problem at all, thanks!

Hey @Ben Sack, I tried to make this as succinct as possible. Can I get feedback if it accurately describes your issue? https://github.com/PrefectHQ/prefect/issues/5009

Thank you, I will take a look today!

Not deployed yet though

No problem, appreciate the update! My coworker was working on a separate prefect project and discovered the use of KV Stores in Prefect. By adding logic to the flow registration processes, we can populate the Prefect KV store from the config file within the repo. We figured this would be a way we could store the last run date as a key value pair in the kv store for the flow and use that date to calculate the

process_date

. Any thoughts/feedback on this method?

That sounds good yep