https://prefect.io logo
Title
r

Robert Kowalski

03/23/2023, 10:32 AM
Hi,
Deployment.buld_from_flow()
use client from get_client() function , this function use defaults for api url and api key base on environment variables. Is there any way to inject PrefectClient instance to Deployment or override defaults after import any from prefect ?
# HERE i can set variables with api url and api key , and get_client() function works correctly 
# os.environ['PREFECT_API_KEY'] = api_key
# os.environ['PREFECT_API_URL'] = api_url


from prefect import flow
from prefect_gitlab import GitLabRepository
from prefect.deployments import Deployment


api_url = ''
api_key = ''

prefect_client = PrefectClient(
        api=api_url,
        api_key=api_key,
        httpx_settings=None,
)


# here its too late to setup env variables and get_client() have they defaults

# if i want load storage i can specified which client should be used 
storage = GitLabRepository.load(name=gitlab_store_name, client=prefect_client)


@flow(log_prints=True)
def tmp():
    print('test')

# But when i doing deployment, i can not choose which prefect client should be use and get_client() is used to getting PrefectClient instance 
d = Deployment.build_from_flow(
    flow=tmp,
    name='test',
    storage=storage,
    apply=True,
)
The imports have to be placed after setup env variables for the client to works. Its any possibility to inject Prefect client? ( all import should be at the top of file )
r

redsquare

03/23/2023, 10:43 AM
Changing it on deploy makes no sense to me - what problem are you trying to solve?
r

Robert Kowalski

03/23/2023, 11:11 AM
Its very annoying to setup env variables before each of my script to auto deploying, all my script have to ugly structure like
1. setup_variables()

2. import prefect stuff or import other my modules, which import some stuff from prefect

3. do rest of my code
I store prefect_api_key, workspace_id and account_id in consul. I used my own env variable ( eg, MY_ENV) to getting prefect configs according to value ( production , test, dev) I want create deploy script that is simple as possible to users, and i was very disappointment that i can't pass client argument 🙂 If i understand prefect code correctly it is impossible to do it and always Deployment use get_client(), correct ?
r

redsquare

03/23/2023, 11:26 AM
yeah since you have specified storage blocks etc that would need to exist so switching at the deploy stage would not make sense
r

Robert Kowalski

03/23/2023, 11:37 AM
storage and deployment have the same client config, but client is provide in different way ( by get_client() or directly create PrefectClient by myself) , so storage exists
storage is created in another script and here is only loaded
r

redsquare

03/23/2023, 1:56 PM
do you use any CI/CD as this takes the pain away
e g girhub actions per env