Hi
New to Prefect, wondering if anyone is using Pr...
# prefect-communitym
Mike Grabenstein
11/19/2020, 9:01 PMHi
New to Prefect, wondering if anyone is using Prefect to manage data flows with tasks written in Node, Java or Clojure.
And how would you do it? Shell task that kicks the Node or Java/Clojure code? Or Container autorun like task?
Thanks.
d
Dylan
11/19/2020, 9:18 PM
Hi @Mike Grabenstein! Welcome 😄
I’d probably recommend the ShellTask as it will handle creating a subprocess & recording logs in the Flow nicely for you
Dylan
11/19/2020, 9:19 PM
We do also have Tasks in the task library that run containers or Kubernetes jobs, if you would prefer to use those
Dylan
11/19/2020, 9:19 PM
Let me know if you have any feedback or questions!
m
Mike Grabenstein
11/19/2020, 9:24 PMLike to see a prefect.tasks.core.function.FunctionTask* of some sort be able to call over to Java via Jython. That would be slick.
Mike Grabenstein
11/19/2020, 9:32 PMAny support for SQS, or a broker like RabbitMQ? @Dylan
d
Dylan
11/19/2020, 9:37 PM
Would you mind opening an issue for the Jython task?
Dylan
11/19/2020, 9:39 PM
We support event-driven flows through API calls, but we don’t have a steaming solution at this time. We do have a PIN (Prefect Improvement Notice) that discusses it and we’d love to hear your thoughts
Dylan
11/19/2020, 9:39 PM
m
Mike Grabenstein
11/20/2020, 2:48 AMPlanning on using the AWS lambda.
Jython, sure where do I create an issue at?
Mike Grabenstein
11/20/2020, 2:49 AMI expected to use lambda originally something else, but the node apps are accessible via Lambda as well.
Mike Grabenstein
11/20/2020, 5:55 PM@Dylan https://github.com/PrefectHQ/prefect/issues/3698
That work well?
5 Views