https://prefect.io logo
Powered by
Title
s

Scarlett King

10/05/2021, 2:53 PM
Hi, is there a way for my local agent to run flows that are stored on Azure Blob storage? I can see the flows are registered locally and stored on the blob storage but when I tried to run them, they are not started
k

Kevin Kho

10/05/2021, 2:54 PM
Hey @Scarlett King, what error do you get?
s

Scarlett King

10/05/2021, 2:55 PM
Just no error, when I kicked off the quick run on the UI, the flow just doesn’t start
I can see that it is stored on the blob storage container that I specified and I can see it on the UI on localhost:8080
The local agent just doesn’t seem to pick up this flow
k

Kevin Kho

10/05/2021, 3:03 PM
Is the Flow stuck in Scheduled? or does it get Submitted?
s

Scarlett King

10/05/2021, 3:06 PM
It’s stuck in scheduled
k

Kevin Kho

10/05/2021, 3:07 PM
What are the labels for your flow and agent?
s

Scarlett King

10/05/2021, 3:13 PM
The flow is None, the agent is my machine name
k

Kevin Kho

10/05/2021, 3:15 PM
I think None itself is a label which is why it’s not getting picked up. Can you try matching them or removing the agent one with 
prefect agent … start --no-hostname-label
?
s

Scarlett King

10/05/2021, 3:18 PM
Great, thank you for that. I now have an error on the agent terminal: ‘run_config’ of type ‘KubernetesRun’, only ‘LocalRun’ is supported. I have set up the flow with KubernetesRun for flow run on our AKS cluster. Do you recommend to use LocalRun when develop locally and change to KubernetesRun when pushing to the cluster or is there a way to do KubernetesRun locally so we don’t have to change the codes before pushing?
@Kevin Kho I removed the KubernetesRun part and now the flow is stopped at execute flow-run step with the error: Failed to load and execute Flow’s environment: AttributeError(‘NoneType’ object has no attribute ‘rstrip’)
I think it’s just to do with version between flow built and execution env
Is there a way to specify which python version it is for the prefect image pulling from Docker? I need to register the flow on python 3.8.10 to make it the same as my local machine since I can’t easily update my python version to 3.8.12
k

Kevin Kho

10/05/2021, 4:50 PM
Ohh one sec I know that error
See this , there was a bug with that connection string not being serialized. Fixed in 0.15.6 i think
within the same Python major version should be fine
s

Scarlett King

10/05/2021, 5:17 PM
I changed to the docker image for prefect 0.14.22 as the python image it bases on is 3.8.10 and reinstalled prefect on my machine to match this version. Now it’s back to show the AttributeError that I mentioned above
Ah, I didn’t see your message above. So what does this mean? Should I upgrade everything back to latest?
k

Kevin Kho

10/05/2021, 5:22 PM
I think there might be a workaround here ?
s

Scarlett King

10/05/2021, 6:04 PM
So I added the env var AZURE_STORAGE_CONNECTION_STRING in both config.toml and as —env when starting the agent
it seems to move to a new error. Now it’s saying it can’t instantiate PosixPath due to python version differences between the flow built (3.8.12) and the execution environment 3.8.10
k

Kevin Kho

10/05/2021, 8:35 PM
Could you show me the error message? Haven’t seen this before
If you make a local environment with 3.8.10, does the same error appear when you do 
flow.run()
?
s

Scarlett King

10/06/2021, 7:56 AM
Hello, sorry I can’t copy the error message out as VPN blocks that. However, I did a few tests. 
flow.run()
works fine. On this flow, I’m using Path from pathlib to handle file path and such. I would have a task that take filepath as an input and would pass 
filepath= ROOT_DIR / ‘filename.csv’
where 
ROOT_DIR = Path(__file__).resolve().parent
. So I removed these and set the file path as 1 string, I believe this only occurs when I pass the value to a task function. When I run this inside a task it works fine with no error. 
def save_file(df):
  file_path = ROOT_DIR / ‘file_name.csv’
  df.to_csv(file_path)
It throws the PosixPath error if I do 
def save_file(df, file_path):
  df.to_csv(file_path)

with Flow(‘abc’, storage=azure_store, run_configs=LocalRun()) as flow:
  df = query_db(sql_query)
  save = save_file(df, file_name=ROOT_DIR / ‘file_name.csv’)
It also throw error when I instantiate KubernetesRun like below 
kubernetes_run = KubernetesRun(image=os.environ.get(“IMAGE_URL”), job_template_path=os.environ.get(“JOB_TEMPLATE_PATH”, ROOT_DIR / “job_template.yaml”))
And for KubernetesRun, the error is thrown at the flow registration step
It works if I just replace 
ROOT_DIR / “job_template.yaml”
with 
”./job_template.yaml”
however, if I have it like this, it will throw error when I do 
flow.run()
as it can’t find the job_template.yaml file
So I got this resolved by converting the Path object to string. It works perfectly with 
str(ROOT_DIR / “job_template.yaml”)
k

Kevin Kho

10/06/2021, 2:32 PM
Thanks for the detailed solution. I am surprised that there was an inconsistency in the Python minor version
s

Scarlett King

10/06/2021, 3:32 PM
Hey, np. Thank you very much for your help too
#prefect-serverJoin Slack
Powered by