https://prefect.io logo
Powered by
Title
r

Richard Alexander

09/29/2022, 1:58 PM
I'm trying to pass a list as a parameter in the Prefect 2.3 UI, and I can't seem to get the format right. I want to start the run via UI interface. I've used square and curly brackets... single and double quotes, but I continue to get the same error: 
ParameterTypeError('Flow run received invalid parameters:\n - part_list: value is not a valid list')
When starting a flow run via UI, what is the correct format that the UI expects when passing a list as a parameter?
✅ 1
j

Jenny

09/29/2022, 2:46 PM
Hi @Richard Alexander - are you able to upgrade to the latest version of Prefect? We made a lot of improvements to how the ui handles inputs in our recent releases.
r

Richard Alexander

09/29/2022, 2:59 PM
I can give that a shot @Jenny. Once I upgrade, what format should I use?
I upgraded to 2.4.2 @Jenny, but I'm still having the same problem... I can't get the UI to accept a list. Not sure if it matters, but I'm specifically passing a list of strings. Any other ideas?
j

Jenny

09/29/2022, 3:21 PM
Are you using type hinting? You can see more in the docs - worth also checking the link in there to pydantic. https://docs.prefect.io/concepts/flows/
z

Zanie

09/29/2022, 3:21 PM
Can you share the output of 
prefect flow-run inspect <your-flow-run>
?
r

Richard Alexander

09/29/2022, 3:26 PM
Yes, I have specified 
part_list: string
.
Here's the output of my last attempt @Zanie with curly brackets and double quotes, though I have tried square brackets too: 
FlowRun(
    id='91e20552-b28b-4f40-a7f3-10c49ff3d6be',
    created=DateTime(2022, 9, 29, 15, 13, 47, 577427, tzinfo=Timezone('+00:00')),
    updated=DateTime(2022, 9, 29, 15, 13, 51, 979492, tzinfo=Timezone('+00:00')),
    name='omicron4-minhiriath',
    flow_id='254e5f71-eeb0-45c0-a3e1-6239cea4946f',
    state_id='c2ae0c28-c3b7-4c25-a928-e685e1f6baa6',
    deployment_id='9f138df5-710c-4940-a52d-08ee52349cf2',
    work_queue_name='general',
    flow_version='d290e9f5a2cfbe5193231cb5645881c5',
    parameters={
        'part_list': '{"170-215", "66205-85", "3059258-01"}'
    },
    empirical_policy=FlowRunPolicy(retries=0, retry_delay=0),
    state_type=StateType.FAILED,
    state_name='Failed',
    expected_start_time=DateTime(2022, 9, 29, 15, 13, 47, 577235, tzinfo=Timezone('+00:00')),
    infrastructure_document_id='fa337e2c-92f3-4c42-b48e-2aed81a67be5',
    state=State(
        id='c2ae0c28-c3b7-4c25-a928-e685e1f6baa6',
        type=StateType.FAILED,
        name='Failed',
        timestamp=DateTime(2022, 9, 29, 15, 13, 51, 976706, tzinfo=Timezone('+00:00')),
        message="Validation of flow parameters failed with error: ParameterTypeError('Flow run received invalid parameters:\\n - part_list: value is not a valid list')",
        data=DataDocument(
            encoding='cloudpickle',
            blob=b'gAWVgQAAAAAAAACMEnByZWZlY3QuZXhjZXB0aW9uc5SMElBhcmFtZXRlclR5cGVFcnJvcpSTlIxN\nRmxvdyBydW4gcmVjZWl2ZWQgaW52YWxpZCBwYXJhbWV0ZXJzOgogLSBwYXJ0X2xpc3Q6IHZhbHVl\nIGlzIG5vdCBhIHZhbGlkIGxpc3SUhZRSlC4=\n'
        ),
        state_details=StateDetails(flow_run_id='91e20552-b28b-4f40-a7f3-10c49ff3d6be')
    )
)
z

Zanie

09/29/2022, 3:28 PM
Ah thanks! You should have the type be 
List[str]
in your flow definition then in the UI you’ll want to pass it as 
["foo", "bar"]
If you can share what you enter in the UI then the result of the inspection we should be able to get to the bottom of this 🙂
r

Richard Alexander

09/29/2022, 3:29 PM
Great, I'll give that a go first. Thanks for the help so far! ;-)
Now it's treating the entire thing as a string @Zanie: 
flow_version='3eab00f5dd40a198fd2e4f61c712950b',
    parameters={
        'part_list': [
            '"170-215", "66205-85", "3059258-01"'
        ]
    },
I changed my method as you suggested: 
@flow()
def my_flow(part_list: list[str]):
	...
And entered this in the UI: 
"170-215", "66205-85", "3059258-01"
Whoops... I had a lowercase "l" in "list". That might be the problem...
z

Zanie

09/29/2022, 3:50 PM
Did you re-register the deployment after updating the flow?
It may need to be re-registered for the UI to know what the type should be
r

Richard Alexander

09/29/2022, 3:52 PM
I did, but I got a strange error: 
UserWarning: Block document has schema checksum sha256:d28332d78ac90882bfeeef79f98cd26c99cda6a7c6d3a48d56a6bd67d7360648 which does not match the schema checksum for class 'Process'. This indicates the schema has changed and this block may not load.
  self.infrastructure = Block._from_block_document(
Not sure if that has a bearing on the issue.
@Zanie I have tried a couple more times after fixing the "List" type hint, but the input is being treated as a single long string everytime: 
FlowRun(
    id='ada166b4-5279-449a-adc1-b0922c2d6b65',
    created=DateTime(2022, 9, 29, 15, 50, 0, 80785, tzinfo=Timezone('+00:00')),
    updated=DateTime(2022, 9, 29, 15, 50, 7, 965278, tzinfo=Timezone('+00:00')),
    name='theta5-magus-manifold',
    flow_id='254e5f71-eeb0-45c0-a3e1-6239cea4946f',
    state_id='386e6aa3-53b1-414b-a314-036061a56d32',
    deployment_id='9f138df5-710c-4940-a52d-08ee52349cf2',
    work_queue_name='general',
    flow_version='2db1960d59edb08a61304c30ce80861f',
    parameters={'part_list': ['["170-215", "66205-85", "3059258-01"]']},
    empirical_policy=FlowRunPolicy(retries=0, retry_delay=0),
    state_type=StateType.COMPLETED,
    state_name='Completed',
    run_count=1,
    expected_start_time=DateTime(2022, 9, 29, 15, 50, 0, 80471, tzinfo=Timezone('+00:00')),
    start_time=DateTime(2022, 9, 29, 15, 50, 6, 424794, tzinfo=Timezone('+00:00')),
    end_time=DateTime(2022, 9, 29, 15, 50, 7, 963240, tzinfo=Timezone('+00:00')),
    total_run_time=datetime.timedelta(seconds=1, microseconds=538446),
    estimated_run_time=datetime.timedelta(seconds=1, microseconds=538446),
    estimated_start_time_delta=datetime.timedelta(seconds=6, microseconds=344323),
    infrastructure_document_id='fa337e2c-92f3-4c42-b48e-2aed81a67be5',
    state=State(
        id='386e6aa3-53b1-414b-a314-036061a56d32',
        type=StateType.COMPLETED,
        name='Completed',
        timestamp=DateTime(2022, 9, 29, 15, 50, 7, 963240, tzinfo=Timezone('+00:00')),
        message='All states completed.',
        data=DataDocument(encoding='result', blob=b'{"key": "5eaebc2484eb4f2bb910f1091d98f5a5", "filesystem_document_id": "7fc03c9d-acbb-4246-9ef4-079a74f9d80f"}'),
        state_details=StateDetails(flow_run_id='ada166b4-5279-449a-adc1-b0922c2d6b65')
    )
)
z

Zanie

09/29/2022, 4:07 PM
Weird. Can you share a screenshot of how you’re entering it in the UI?
r

Richard Alexander

09/29/2022, 4:39 PM
z

Zanie

09/29/2022, 4:40 PM
Oh it looks like it knows its a list and it wants you to just input each value separately, cc @Jenny
r

Richard Alexander

09/29/2022, 4:40 PM
@Then when I press enter, I just noticed that it's already indicating that it is a single string:
z

Zanie

09/29/2022, 4:41 PM
You should be able to do just 1 -> Enter 2 -> Enter 3 -> Enter
r

Richard Alexander

09/29/2022, 4:42 PM
Oh man... I have a list size of 1K... I'd rather not do that! 😉
But I will test that
z

Zanie

09/29/2022, 4:42 PM
Yeah sounds like we ought to provide a way to enter data in a raw format 😄
r

Richard Alexander

09/29/2022, 4:47 PM
Yes, looks like you're right. The only way to make it work currently is to enter the list one at a time. I would VERY much appreciate a raw format method. Nearly all of my flow runs have list parameters in the 1-10K range.
Thanks for the help today Michael and Jenny! 😄
z

Zanie

09/29/2022, 4:48 PM
You can create runs outside of the UI with requests directly to the API
If that’d help in the meantime 🙂
r

Richard Alexander

09/29/2022, 4:48 PM
Oh... via command line? Or some other method?
j

Jenny

09/29/2022, 4:49 PM
If you can, I believe that if you don't specify it as a list of strings it should give you a json input.
r

Richard Alexander

09/29/2022, 4:50 PM
Right. Isn't that what I was dealing with in the beginning though? Before adding the type hint? I'll revert that and double check.
z

Zanie

09/29/2022, 4:51 PM
If it’s not specified as a list of strings the client won’t parse it from JSON into a list of strings at runtime
We’ll need the ability to do raw inputs in the UI to support this kind of use-case
r

Richard Alexander

09/29/2022, 4:52 PM
Right, I think that was my initial problem.
z

Zanie

09/29/2022, 4:54 PM
Yeah we rely on those type annotations to cast the parameters passed by the API to a run into Python datatypes
import asyncio
from prefect import get_client

async def create_run():
    async with get_client() as client:
        flow_run = await client.create_flow_run_from_deployment(
            "<deployment-id>", parameters={"your-param": ["your", "big", "list"]}
        )
        print(flow_run.id)
        

asyncio.run(create_run)
We’re still working on parameter specification for the 
prefect deployment run
command. Parsing JSON from the CLI is kind of a pain.
j

Jenny

09/29/2022, 6:33 PM
https://github.com/PrefectHQ/prefect/issues/7021 Feel free to add any extra info/context.
:thank-you: 1
#prefect-uiJoin Slack
Powered by