https://prefect.io
Join Slack
If I may point out a design inconsistency: that parameters and secrets sometimes are "values as decl...
# ask-community
h
If I may point out a design inconsistency: that parameters and secrets sometimes are "values as declaration time" and sometimes "values as output by tasks" — this looks like an anti-pattern to me:
Another inconsistency: there's no default for secrets.
k
On the first one, yes your are right. This is a known issue and is being tackled here: https://github.com/PrefectHQ/prefect/issues/4377
Do you mean grabbing a default secret if one is not supplied?
h
Yes
Useful for testing environments
k
I see maybe you should an issue for that one.
h
Why not unify the API?
Remove 
Secret
if it's not supposed to be used
Make everything that can be a task, possible to give as a task
k
I have some errands I need to run today, but I'll definitely respond to you on Monday about this 😅
h
Ensure all API surface areas are available an an immutable fashion; you don't gain anything by 'currying' the state of your tasks with a two-layered 1) constructor, 2) run method; only complexity
Document the 
@task
surface area; it's not always working and 
kwargs
is not very discoverable when you want to override "statically" configured properties
And why reverse the source array and applicable function in 
map
? Wouldn't it be easier to have an overload? results.map(opts, task(lambda (opts, res): res + 1)) where 
opts
is an object with the "unmapped" parameters?
Also, there type annotations are lacking for 
map
having lots of 
Any
values; I'd be happy to review fixes for that — I know my way around typed functional DSLs
I'll quickly throw in here also that secrets should probably 1) on a flow level, 2) on a group/team level, or at least be possible to set on a flow level, since they are "similar" to parameters, except that they are secret
k
Hey @haf, we have the issue open to remove the two layered constructor and run and push them to run time. One of the issues was that using parameters in construction was not possible. You can PR the refactored tasks.
Will bring up to team about 1, default secrets and 2, flow level/group level secrets.
You can open a issue for the map
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by