Thomas Furmston
with Flow( 'common_flow', storage=Docker( base_image='my-docker-image:latest', local_image=True, )) as flow: num_days_parameter = Parameter('num_days', required=True) num_back_fill_days_parameter = Parameter('num_back_fill_days', required=True) end_date_parameter = Parameter('end_date', required=True) served_ads_command = ShellTask( name='my_first_task', command=construct_etl_command( app_mode=settings.app_mode, table_name='task1', num_days_cli_arg='num_back_fill_days', num_days=num_back_fill_days_parameter, end_date=end_date_parameter, database=settings.database, ), stream_output=True, )
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.