Chris Gunderson
01/10/2023, 2:11 PMprefect deployment build -n fidelity-allocations-deployment -q ecs-worker -a -sb s3/sra-s3 -ib ecs-task/default src/main/prefect/flows/allocations/prefect_fidelity_allocations.py:FidelityAllocationsFlow --cron "13 15 * * 1-5"
^^This is my current deployment command using the CLIMiguel Moncada
01/10/2023, 2:27 PMdataflows/deployments
that is imported and called from the root of the repo through a python script deployments.py
.
āāā README.md
āāā config
ā āāā agent
ā ā āāā Dockerfile
ā āāā runner
ā āāā Dockerfile
āāā dataflows
ā āāā __init__.py
ā āāā _version.py
ā āāā deployments
ā ā āāā __init__.py
ā ā āāā _constants.py
ā ā āāā hello_deployment.py
ā āāā flows
ā āāā __init__.py
ā āāā hello_flow.py
ā āāā utils
āāā deployments.py
āāā docker-compose.yml
āāā docs
āāā requirements.txt
āāā setup.py
āāā tests
ā āāā README.md
ā āāā conftest.py
ā āāā unit
ā āāā __init__.py
ā āāā flows
ā āāā __init__.py
ā āāā test_hello_flow.py
āāā tox.ini
Chris Gunderson
01/10/2023, 3:13 PMDanilo Drobac
01/10/2023, 4:08 PMdeployments.py
?Miguel Moncada
01/10/2023, 4:09 PMimport os
import logging
import argparse
from dataflows.deployments._constants import (
FLOW_DEPLOYMENT_DICT,
DEPLOYMENT_FILE_FUNC_DICT,
)
def get_file_name_file_extension(file_path: str) -> tuple:
"""Function to extract the file name and extension from a file URI
Args:
file_uri (str): URL pointing to the file
Returns:
tuple: file name and extension
"""
file_name_with_ext = os.path.basename(file_path)
file_name, file_ext = os.path.splitext(file_name_with_ext)
return (file_name, file_ext)
def get_deploy_function(file_name: str) -> callable:
"""Function to get the deployment function for a flow
Args:
flow (str): flow file name without extension
Returns:
callable: deployment function
"""
deploy_function = FLOW_DEPLOYMENT_DICT.get(file_name)
if not deploy_function:
deploy_function = DEPLOYMENT_FILE_FUNC_DICT.get(file_name)
return deploy_function
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument(
"-f",
"--file_path",
help="File path",
required=True,
)
args = parser.parse_args()
logger = logging.getLogger()
logger.setLevel(<http://logging.INFO|logging.INFO>)
file_name, _ = get_file_name_file_extension(args.file_path)
deploy_function = get_deploy_function(file_name)
if deploy_function:
<http://logger.info|logger.info>(f"Deploying {file_name} using {deploy_function.__name__}")
deploy_function()
<http://logger.info|logger.info>(f"{file_name} has been deployed!")
else:
<http://logger.info|logger.info>(
f"No deployment function found for {file_name},"
"skipping..."
)
Chris Gunderson
01/10/2023, 4:10 PMprefect deployment build -n fidelity-allocations-deployment -q ecs-worker -a -sb s3/sra-s3 -ib ecs-task/default src/main/prefect/flows/allocations/prefect_fidelity_allocations.py:FidelityAllocationsFlow --cron "13 15 * * 1-5"
Danilo Drobac
01/10/2023, 4:10 PMChris Gunderson
01/10/2023, 4:15 PMMiguel Moncada
01/10/2023, 4:15 PMdataflows/deployments
or dataflows/flows
Chris Gunderson
01/10/2023, 4:17 PM