https://prefect.io logo
Join SlackCommunities
Powered by
Hi, I have os.path.join(path, folder) in my task, ...
# ask-community
e
Hi, I have os.path.join(path, folder) in my task, and I can run flow with it from python script but not on prefect UI. The error message is: Unexpected error: TypeError('expected str, bytes or os.PathLike object, not NoneType',). Anyone has any thoughts? Thanks in advance.
k
Hey! Welcome to the Prefect Slack channel. Could you share the flow script?
I think you passing a task to a non-task or something like that
e
with Flow(“fv_grid_flow”) as flow:     path = Parameter(“inputs”, required=True)     read_file(path)
parameter is saved in a json file
k
Oh I see. In this case, are you passing the value when you trigger from the UI since there is no default value here?
e
so you are saying UI won't read json but instead i need to type the parameters on UI?
k
No no. I’m saying that if you have
Copy code
test = Parameter("test", required=True)
if you go to the UI, you have to supply a value to 
test
because there is no default value set. But if you do:
Copy code
test = Parameter("test", default=1)
then you don’t need to supply in the UI because it can use the default. So it looks like the error is saying that the parameter is 
None
? This is what making me wonder if there is a value supplied Also, can I see the definition of 
read_file
? Is that a task?
e
yes, it's a task. so i can supply a default value on UI, but it will still read parameters from json file?
k
If you supply the path to that Parameter, yes this should work because the 
read_file
task will read in the json file
e
you are right about this, it solved the problem. thanks a lot!
k
Of course!
e
hi another error also only on UI, [WinError 2] The system cannot find the file specified, any thoughts?
k
Are you using Local agent?
e
yes
k
So the agent starts the Flow as a Python process in the same working directory. Maybe you need to specify the right working directory so that it can load the things it needs? There is an example of configuring that here .
Are you running the agent in the same machine you registered the Flow from?
e
yes, i am running in the same machine
k
Ok I think you just need to start the agent in the right location or use the 
LocalRun
to specify the working directory
e
With set_temporary_config(get_server_config(server, port)):    LocalAgent(import_paths=python_path).start()
this is how i start agent, you mean i can use hostname_label to make it start in the right machine>
?
k
Wait, no. The labels are correct in that they are the way to specify which machine to start on. But the 
working_dir
should be defined on the Flow side. In you flow,
Copy code
from prefect.run_configs import LocalRun

tasks....

with Flow(...) as flow:
    tasks....

flow.run_config = LocalRun(working_dir="/path/to/working-directory")
and then register this. When the agent picks up this Flow, it will run the Flow in that working directory. I think this should help you locate the files on the machine.
Is that error from a task for did the Flow fail to start? The 
WinError
?
e
i did what you suggested about run_config, i think the run stuck somewhere, i see the status "scheduled", but not "submitted"
to your question, it fails to deploy
k
Oh I see if it’s scheduled but not submitted. That is a labelling issue most of the time. What are the labels for your flow and agent?
The flow labels need to be a subset of the agent labels
e
hostname=socket.gethostname() labels=[hostname] flow.environment = LocalEnvironment(executor=executor, labels=labels)
i don't pass a label to agent, my flow label is my machine name
k
Environments are deprecated and they were replaced the RunConfigs so you shouldn’t use LocalEnvironment. You want RunConfiguration instead, specifically the LocalRun to pair with Local Agent. The agent would need the flow label with the machine name to be able to pick up the flow. If you start the agent with the CLI command:
Copy code
prefect agent local start
The agent will have the hostname by default also. You can also remove the label from the Flow but for LocalAgent, it would be good to have labels on both Flow and agent
e
i see, let me take a look
Hi, thanks to your help, i was able to get my flow running on UI. But it returns error when i add map to my task: _grid__client.map(batchName, xmlPath, unmapped(config_file), unmapped(grid_config)). the error was GRAPHQL_VALIDATION_FAILED: Cannot query field "get_or_create_task_run_info" on\n type "Mutation".
k
can you show be the flow code?
Copy code
with Flow(...) as flow:
    ....
and what version of Prefect are you on?
e
sure, bear with me for a sec, my code is in another remote machine, i have to type them in this chat, the v is 0.13.19
k
no worries
e
With Flow(“fv_grid_flow”) as flow:   Inputs = Parameter(“inputs”, required = True)   batchName, xmlPath, config_file, grid_config = _prepare_requests(inputs)   mapped_results = _grid_client.map(batchName, xmlPath, unmapped(config_file), unmapped(grid_config))
so batchName and xmlPath are lists with only 1 element
config_file is a string and grid_config is a tuple
k
can i see the __grid__client definition?
e
it's a long piece of code, but essentially, it's calling an in-house function of gridclientscenario analysis
if i don't use map, and just pass in the element in the first two arguments, it runs well
The GraphQL query was:\n\n mutation {\n get_or_create_task_run_info(input: { flow_run_id: "c87a2e12-3add-4247-a42d-410398b4395b", task_id: "da84f441-cb7c-4292-81d9-db87353cc461", map_index: 0 }) {\n id\n version\n serialized_state\n }\n }\n\nThe passed variables were:\n\n null\n',)
the error is passing a null value to the GraphQL
@task(name=“gridClient”) Def _grid_client(batch_name, xml_path, xml_config_file, grid_config):   Logger = prefect.context.get(“logger”)   Result = grid_client([batch_name, xml_path, xml_config_file, grid_config])   logger.info(“PV generated for %s”, result) Return result
k
Is your code calling the query? What are you trying to do with the query?
Or is that Prefect calling the query?
e
that's prefect calling the query
k
Can you try not mapping but passing all 4 
batch_name
, 
xml_path
, 
xml_config_file
, 
grid_config
?
I am wondering if it’s one of the configs and not the mapping that causes this
e
yeah, i tried with this, mapped_results = _grid_client(batchName[0], xmlPath[0], config_file, grid_config), and it worked
a
@E Li can you share what Prefect version do you use in the environment from which you registered this flow, and which Prefect version runs on this agent? This may be a version mismatch according to this: https://github.com/PrefectHQ/prefect/issues/4640
e
the are all 0.13.19, i actually had to downgrade my prefect to match what's running on the agent
k
Are you using local agent?
e
yes
a
Thanks for confirming that. But this error really looks the same as in the Github issue 🤔 and there it was the version mismatch. Could you perhaps test this flow with a local agent based on the latest version 0.15.7, and use also 0.15.7 during registration and check if the error happens again? You could create a new agent in your terminal using:
Copy code
prefect agent local start
e
yep, let me do that
🙌 1
k
Are you on server or Cloud?
e
in the meanwhile, attached log for additional info
log.rtf
i'm on server
k
ohh it looks like it’s your server version that needs to be upgraded i think
upvote 1
e
ok, how should i do that upgrade?
k
You can find the instructions here .
e
yeah, i don't think i can register with 0.15.7 to a server with lower version
k
you can’t but i think when you were on 0.13.19, I think your server was an even lower version
e
right
hi, is there a way to check the version running on server?
k
Yeah using the graphql api. Let me look
Copy code
query Api {
  api {
    backend
    mode
    version
    core_version
    release_timestamp
  }
}
e
it works, thank you
6 Views
Open in Slack
PreviousNext
Powered by