m

    Mike Grabenstein

    1 year ago
    Hi New to Prefect, wondering if anyone is using Prefect to manage data flows with tasks written in Node, Java or Clojure. And how would you do it? Shell task that kicks the Node or Java/Clojure code? Or Container autorun like task? Thanks.
    Dylan

    Dylan

    1 year ago
    Hi @Mike Grabenstein! Welcome 😄 I’d probably recommend the ShellTask as it will handle creating a subprocess & recording logs in the Flow nicely for you
    We do also have Tasks in the task library that run containers or Kubernetes jobs, if you would prefer to use those
    Let me know if you have any feedback or questions!
    m

    Mike Grabenstein

    1 year ago
    Like to see a prefect.tasks.core.function.FunctionTask* of some sort be able to call over to Java via Jython. That would be slick.
    Any support for SQS, or a broker like RabbitMQ? @Dylan
    Dylan

    Dylan

    1 year ago
    Would you mind opening an issue for the Jython task?
    We support event-driven flows through API calls, but we don’t have a steaming solution at this time. We do have a PIN (Prefect Improvement Notice) that discusses it and we’d love to hear your thoughts
    m

    Mike Grabenstein

    1 year ago
    Planning on using the AWS lambda. Jython, sure where do I create an issue at?
    I expected to use lambda originally something else, but the node apps are accessible via Lambda as well.