Hey, does anyone in the community did incremental etl with Prefect ? We are considering doing it and maybe there are mistakes that can be avoided ^^ Thanks !

Hi All , I am having issue with prefect server. Then server is starting on server ip and config.toml is set to apollo_url="http://10.1.x.x:4200/graphql" everything starts normally and no errors during start , the ui shows green connection to API, however 1-2 minutes later it turns yellow and stops working , i turn debug=true but I am not able to find logs are written or how to troubleshoot this problem. Any help would be appreciated (this is local install on ubuntu)

Have you seen this issue when launching jobs from the ECS Agent? I'm troubleshooting and not sure where to start with it. botocore doesn't like something about the parameters. Prefect version is 1.0

How do I create and run flows on a schedule in Prefect Orion? Let’s say I have this:

schedule = IntervalSchedule(interval=datetime.timedelta(seconds=10))
deployment_spec = DeploymentSpec(name="hola", flow=greetings_flow, schedule=schedule)

Do I now use OrionClient and some methods to deploy? We currently use Prefect core in a pod and simply do flow.run() with a schedule attached, and I’m looking for a working example to do the equivalent in Orion (even if I gather behind the scenes it will do something else via a server and DB). Basically I’m looking for this: https://orion-docs.prefect.io/concepts/deployments/#running-deployments-with-the-api 🙂

Hello: I am seeing a message > Work queue does not have any actively polling workers ready to execute work. But can find no documentation on this. Can someone assist?

Hi there! Back again- I've been building a fun little data ecosystem with prefect as our primary gardener. We're approaching a tipping point of turning some of these tools into user-facing products- The biggest bugbear though is that the prefect workers I run on our machines will occasionally (although reliably) crash once every couple of weeks, requiring a manual reboot. I'm only running them manually in tmux windows, as opposed to as systemd services- just while I've been in dev phase- but I wanted to get some insight into what might be going on here. I'll copy the error in the thread:

hey @Marvin why my flow don't fail if it have timeout of 30 sec here goes the code """ from prefect import flow, task import time @flow(name='my-flow-check', timeout_seconds=30, log_prints=True) def greetings(): print("comes into greetings") time.sleep(40) print("greeting from prefect") greetings() """

Hello, I'm creating my first flow run on local for the first time, and i want to set it up with how (i think) my prod set up will interact with prefect. I have a work pool created and a local agent listening to it, and I deployed my Flow to prefect cloud like so:

if __name__ == "__main__":
    
    deployment: Deployment = Deployment.build_from_flow(
        flow=test_flow,
        name="example",
        version="1",
        tags=["demo"],
        work_pool_name='local_agent_work_pool'
    )

    deployment.apply()

I confirmed that the deployment worked in the Prefect Cloud UI. In my flask endpoint, I am trying to submit the flow run to Prefect cloud like so, but it feels wrong:

@flask_route.route("/prefect", methods=['POST'])
def start_prefect_flow():
    # prefect_client = get_client(
    #     # api_key='redacted'
    # )
    prefect_client = PrefectClient(
        api='redacted',
        api_key='redacted'
    )

    res = prefect_client.create_flow_run(test_flow)
    print(res)

First, I don't think I should be creating the client this way, but unsure how else I should be passing my api/api key information to it. This is also returning a coroutine object instead of what I thought would be information about my flow run, but maybe that makes sense since I don't see a flow run being kicked off. Some assistance here would be super helpful!

Hello everyone! I use aws for my work. On my MacOS laptop .aws contains configuration files that are responsible for accessing my work cloud. I want to test prefect with another aws account but I'm afraid something might go wrong. As far as I understand, Prefect uses boto3, and boto3 will take configs from the default path. How to distinguish between a working aws profile and a home one, and what is the most important thing how to say about this to Prefect?

Is it possible in Prefect to invoke R script with-in a task. I have a code in R script and that can only run in the R-studio, I don't want to convert it into python. How can I invoke that R code with prefect. Is there an interface between R and Prefect