Hi guys, I am trying to understand a specific behavior when flows are being built and sent to the server. In one of my tasks I am defining the URL for Livy as follows

livy_url = f"https://{host}:{port}/gateway/default/livy/v1"

host and port are referring to variables that have the following code:

os.environ.get('LIVY_HOST')

and

os.environ.get('LIVY_PORT')

the livy_url is getting the variables from my local machine when the flow is being built. and thus the values for host and port are incorrect.

If you place the assignments of host and port inside a task, those variables will be evaluated on the machine running your flow instead of the machine where you build&register the flow

Hi Julian, I think i got where the problem is. You are right. The assignments were being done in my code and then being passed to the task while setting up the task. Moved them inside the task and all is good. Thanks for that