• William Jamir

    William Jamir

    2 months ago
    Hi, I have one flow where I have two steps that I use a map,
    Task A
    to do an upload and
    Task B
    to monitor if everything went fine after the upload from Task A. I noticed that using LocalDaskExecutor, only 10 tasks are processed at a time It’s possible to run more than 10 mapped tasks in parallel? Using LocalDaskExecutor with Prefect 1.2.2
    William Jamir
    1 replies
    Copy to Clipboard
  • i

    Isara Ovin

    2 months ago
    Hi can someone please share a some content on handling multiple work flows.
    i
    Anna Geller
    4 replies
    Copy to Clipboard
  • d

    Denis Sh

    2 months ago
    Good day everyone, did I miss something in UI or Orion self-hosted backend is not allowing to edit parameters of deployment/flow in UI? Thanks
    d
    Anna Geller
    3 replies
    Copy to Clipboard
  • d

    Denis Sh

    2 months ago
    And another one - if flow-decorated main function is async def'ined and consists of await asyncio.gather() then FlowRunner will properly asyncio.run() it?
    d
    Anna Geller
    3 replies
    Copy to Clipboard
  • Rajvir Jhawar

    Rajvir Jhawar

    2 months ago
    I think prefect 2.0 docs are off, there is line the about work-queues and agents section that states this: “You must start an agent within the environment in which it will execute flow runs.” This statement is a little off and should come with note stating the agent doesn't necessarily have to be running in the same execution environment as the flow. For instance you can have an agent running in a VM connected to a k8 cluster using kubectl and run k8 jobs that way.
    Rajvir Jhawar
    Anna Geller
    +1
    3 replies
    Copy to Clipboard
  • Faheem Khan

    Faheem Khan

    2 months ago
    Prefect 2.0 Hi, my code runs fine when I run it with python abc.py but it doesn't do anything when I run it from UI i.e prefect deployment create abc.py. it spins up a new container when I run it from UI but then shutdowns I don't see any logs for debugging, the flow status shows completed. I am using dockerflow runner with DaskTaskRunner.
    Faheem Khan
    Anna Geller
    5 replies
    Copy to Clipboard
  • Shivam Bhatia

    Shivam Bhatia

    2 months ago
    Hi, I am trying to host a prefect 2.0 instance on google cloud platform compute engine. Could someone please share some documentation?
    Shivam Bhatia
    Anna Geller
    2 replies
    Copy to Clipboard
  • komal azram

    komal azram

    2 months ago
    Hi I am trying to run a sync connection task in airbyte using prefect. but getting this error sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table block_schema_reference already exists. when I sync from airbyte it works.
    komal azram
    Anna Geller
    9 replies
    Copy to Clipboard
  • Black Spy

    Black Spy

    2 months ago
    Hi, Can you help me to do create dynamic DAG flow based on configuration file(Yaml) and also how to do parallel scheduling at the sametime @Anna Geller
    Black Spy
    1 replies
    Copy to Clipboard
  • Tom Matthews

    Tom Matthews

    2 months ago
    Context: I have a number of tasks that are essentially API calls which can be run in parallel and take ~ 1 second each but every second of latency saved is very important. I’d like to use Prefect in production and take advantage of the native async support, but async support is only available in prefect 2.0 and that is not recommended in production. Is there any recommendations here, would it be safe to use prefect 2 in Production, or is there anyway to use asyncio in prefect 1.0 (I couldn’t find any documentation for this), or would it be a good idea to just use the DaskExecutor in prefect 1.0 with the
    threads
    scheduler? 🙏
    Tom Matthews
    Ron Levi
    +1
    12 replies
    Copy to Clipboard