Hi there! I have some (not all) tasks that need ne...
# ask-community
j
Hi there! I have some (not all) tasks that need need PandasSerializer, and also need to use different result types depending on if dev or production. I have a solution for this (below), but wondered if anyone had a cleaner solution?
Copy code
condition = True
resultType = LocalResult if condition else S3Result

@task()
def create_list():
    return [1,2,3]

@task(result=resultType(serializer=PandasSerializer(file_type='parquet')))
def to_df(a_list):
    return pd.DataFrame({'col1': a_list})

with Flow(
        name="my-flow",
        result=resultType(serializer=PickleSerializer())
) as my_flow:
    my_list = create_list()
    my_df = to_df(my_list)
d
Hi John If you need to support different support types across environments, this pattern is appropriate! I can't offer too much more in the way of suggestions here other than maybe you want to select your task options using some sort of environment variable that's easy to pick up. One way is:
Copy code
environment = 'DEV'
task_config = {
    'DEV': LocalResult(serializer=PandasSerializer(file_type='parquet')),
	'PROD': S3Result(serializer=PandasSerializer(file_type='parquet')),
} 
  
@task()  
def create_list():  
    return [1,2,3]  
  
@task(result=task_config[environment])  
def to_df(a_list):  
    return pd.DataFrame({'col1': a_list})  
  
with Flow(  
        name="my-flow",  
        result=resultType(serializer=PickleSerializer())  
) as my_flow:  
    my_list = create_list()  
    my_df = to_df(my_list)
Of course, you risk issues arising in your flows that can happen outside of Prefect by handling multiple result types, so if it's possible in the future to unify them and put as much of the logic within flows and tasks as possible, you might have a better experience!
👍 1
j
Thanks. That is more clear that way