alexandre kempf
01/04/2020, 1:33 PMfrom prefect import task, Parameter, Flow
import numpy as np
@task
def augment_data(a, b=10):
return np.sqrt(a)+np.sqrt(b)
with Flow("augment") as flowAug:
a = Parameter("a")
aug = augment_data(a, 10)
@task
def load_data(c):
return c
@task
def train_model(model, d):
print("Training, {} with {}!".format(model, d))
return model
with Flow("training") as flowModel:
init = Parameter("c")
model = Parameter("model")
data = load_data(init)
state = flowAug.run(a=data)
aug_data = state.result[aug].result
result = train_model(model, aug_data)
state_model = flowModel.run(c=5, model="SVM")
josh
01/06/2020, 3:11 PMaugment
flow inside of a task? Something like:
@task
def run_flowAug(data):
return flowAug.run(a=data)
And then add that task to your training
flowDylan
01/06/2020, 6:15 PMalexandre kempf
01/07/2020, 10:52 AM@task
def sumsqrt(a, b):
return np.sqrt(a)+np.sqrt(b)
(note that I'm using sqrt in order to avoir the add operation from prefect that can hide a bug if a and b are task and not float or int)
The values I return is not a task in my function definition, but in the flow it is.
Therefor I tried something like
@task
def run_flowAug(data):
state = flowAug.run(a=data)
return state.result["aug"].result
so I return something that is not a task.
And in the main flow it is NOT considered a task.
As you see I'm a bit lost 😛 But I don't want to take too much of your time, so I'll try to figure out what is wrong on my side and then ask more specific questions 🙂