Hello. I'm trying to use ShellTask locally on Windows using Prefect core, but I'm having issues. I found this old Issue on Github saying that ShellTask isn't windows-compatible. Is there a fix or any workaround?

Hi! I recently added a schedule to one of my flows and pushed it to cloud. I now have the latest version and -1 version both scheduled and am unable to find where to disable the -1 version.

Hi everyone, I am having a hard time filtering graphQL flows by agent id or label name. any ideas on how to accomplish this?

Hi.. quick question can I have a task that returns a list of strings then from that list I'll loop over it and do some logic. the issue I have is

object is not iterable

is there anyway to return the result of a task as a normal python object list, dict and not a task (please note i don't know the length of list and its very dynamic)

I’m running a Prefect Flow on a Docker Agent with a DaskExecutor. The flow runs fine for awhile, then I get an error from the CloudFlowRunner

Unexpected error: KilledWorker('{{TaskRunName}}', <WorkerState '<tcp://127.0.0.1:44295>', name: 1, status: closed, memory: 0, processing: 72>)

Any idea what might be causing this?

is there a trigger like “at least one successful, and the rest skipped”. I have some logic like so

@task
def required():
    pass


@task
def optional():
    pass


@task
def final():
    pass


with Flow('example'):
    res = required()
    with case(some_case, True):
        optional_res = optional()

    final(res, upstream_tasks=[optional_res])

If optional is skipped, its okay to run final. If required fails, or optional fails, I dont want to run final

the example is simple, but final would have additional upstream dependencies, which would be treated like res

Hey there! I’m trying to figure out the easiest way to trigger a prefect cloud (1.0) job via the API - I know I can use the GraphQL API but I don’t have a ton of experience with GraphQL and I want to trigger a flow run based on knowing the flow name — not the id. I also need to trigger a flow on the latest version of the flow — and there will be several versions. This all needs to take place within the context of an automated build process (GCP cloud build) so I don’t want to have to install the cli or anything - I just want to drop in an API key via a secret and run a simple script to trigger the flow at the end of a build. Ideally it would be just a bash/curl script but I could also write a basic python script to do it.

Hey - I have a flow that has a few clocks eg

au_harness_win_place = CronClock(
    cron="33 * * * *",
    parameter_defaults={
        "race_types": ["Harness"],
        "market_type_codes": ["WIN", "PLACE"],
        "event_type_ids": None,
        "market_countries": ["AU"],
        "table_name": "harness_market_catalogue_au"
    },
    start_date=datetime.now(tz=timezone("Australia/Brisbane")),
)

that is called like so

with Flow(
    name="betfair_market_catalogues",
    storage=Storage.in_a_s3_bucket(),
    run_config=RunConfig().fargate_on_ecs(),
    schedule=Schedule(clocks=[au_nz_greys_win_place, au_harness_win_place]),
    executor=LocalDaskExecutor(scheduler="threads"),
) as flow:

If I want to call this via the cli it doesnt like the None values and the python objects, do I have to accept stringified versions of this or does prefect handle this on its own? eg, how would I run this locally

prefect run -p flows/market_catalogues/betfair_market_catalogues.py --param table_name="harness_market_catalogue_au" --param race_types=["Harness"] --param market_type_codes=["WIN", "PLACE"] --param event_type_ids=[] --param market_countries=["AU"]

doesnt do the job

We get a pickle thread lock error in a task when using paramiko. This is probably a problem with pickling ssh stuff. What is the alternative? How are people using paramiko with Prefect?

hi , i am executing my flow not from main flow. when i run it on the default executor everything is running file, when i use DaskExecutor it crashs with the following message 2022-06-22 140640,649 prefect.TaskRunner INFO Task ‘train_model[0]’: Starting task run... 2022-06-22 140640,650 prefect.TaskRunner INFO Task ‘train_model[1]’: Starting task run... The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. 2022-06-22 140655.255 python[95140:4859256] * Terminating app due to uncaught exception ‘NSInternalInconsistencyException’, reason: ‘NSWindow drag regions should only be invalidated on the Main Thread!’ please advise.

Hello, community! I’m trying to build a flow with docker tasks. So below is some information: • prefect server • prefect docker and local agents are started • task with pulling image from hub.docker.com (it’s pulling ok) • task that create container • task that run container But when I run flow locally there is an error (screen). Could you please help with it? P.S. code in thread

We are using Prefect cloud and We want to enable task level SLA. Whenever the task in the flow is delayed, We want to get notification. is that possible?

I have a question about: restarted flows with trigger:any_successful tasks.

What is the best practice for implementing fan-out-to-fan-in patterns in Prefect 2.0? As a test scenario, I tried to create a flow that 1. Gets all AWS S3 bucket names 2. Spawns a task per bucket and counts the objects 3. Collects all count outputs and calculates the sum (Code is in the first comment) I get some errors (also in the comment) but I have the feeling that I'm using Prefect 2.0 in a way that might be wrong. How should I structure my flow to do it in the most Prefect-y way? I loosely based the idea on this amazing Prefect blog post.

Hey!I am facing issue while running prefect on VM instance of GCP.I am unable to open UI with external_IP:portNumber. Could anyone suggest solution to it?

Is there an updated timeline for the release of prefect 2.0? I see here

The Prefect 2.0 beta period is expected to last for at least the second quarter of 2022.

And that 2.0 is not recommended for production at this time. Is this still accurate? For context, we are looking to build an internal tool using prefect as the orchestrator and are debating whether to start with 2.0 or use 1.0 and migrate. We would be hoping to start using the tool Q3 this year.

I am trying to run a flow through an ECS Agent on FARGATE. I have successfully deployed the prefect agent on cluster. But I'm unable to run a registered flow. I'm using prefect 1.2.2. Logs only say

Submitted for execution: Task arn:

Can anybody suggest a solution?

Hi maybe this should be more specific, not everyone knows how container works. here should be clear that it’s a Docker container. instead of just “container”

Hello again! Is there any way to do Bitbucket storage through SSH? There seems to be a generic Git storage module which supports it, but it's a bit unclear whether this is compatible with a Bitbucket remote

Hi I’m getting Storage permission while creating deployments with gcs Traceback (most recent call last): File “/usr/local/lib/python3.9/site-packages/google/cloud/storage/blob.py”, line 2567, in upload_from_file created_json = self._do_upload( File “/usr/local/lib/python3.9/site-packages/google/cloud/storage/blob.py”, line 2368, in _do_upload response = self._do_multipart_upload( File “/usr/local/lib/python3.9/site-packages/google/cloud/storage/blob.py”, line 1903, in _do_multipart_upload response = upload.transmit( File “/usr/local/lib/python3.9/site-packages/google/resumable_media/requests/upload.py”, line 153, in transmit return _request_helpers.wait_and_retry( File “/usr/local/lib/python3.9/site-packages/google/resumable_media/requests/_request_helpers.py”, line 148, in wait_and_retry response = func() File “/usr/local/lib/python3.9/site-packages/google/resumable_media/requests/upload.py”, line 149, in retriable_request self._process_response(result) File “/usr/local/lib/python3.9/site-packages/google/resumable_media/_upload.py”, line 114, in _process_response _helpers.require_status_code(response, (http.client.OK,), self._get_status_code) File “/usr/local/lib/python3.9/site-packages/google/resumable_media/_helpers.py”, line 105, in require_status_code raise common.InvalidResponse( google.resumable_media.common.InvalidResponse: (‘Request failed with status code’, 403, ‘Expected one of’, <HTTPStatus.OK: 200>)

Is there a way to lock down which `Storage`s an agent is allowed to run? Our concern is if an API key is stolen, and someone submits a flow that e.g. uses a github storage pointing to their own repo and so runs arbitrary code on our ECS agent

hi team, 1.0 question, are task serializers stored in the Prefect database as part of the task state/result? I’m trying to use

get_task_run_result

to fetch task results from a different flow. The result location is correct but the serializer defaults to Pickle while that task uses PandasSerializer, so it errors out with Pickle

For Enterprise users of Prefect Cloud — is there any way we can determine when a licensed user was last active / logged in from the graphql API? Users are across multiple tenants, fwiw Querying

audit_log

returns this information in the

message

field, but looks to only be for the current tenant only. Wondering if there is a way to get this across all tenants/users associated with a license

how do I enable MFA for my account? Or is that not available in the Free plan?

In 2.0 how could I manually run a flow / replay a flow run to be able to use pdb for debugging purposes? I have tried

python -m prefect.engine <flow_run_id>

but I get

This run has already terminated

Hey folks Quick one from me. In Prefect 1.0, there are pre-configured tasks available, for example the AirbyteConnectionTask. I'm trying to trigger an Airbyte connection using Prefect 2.0, but from what I can tell, there's no out-of-the-box tasks available? Am I missing something, or do we need to build everything from scratch with Prefect 2.0?

since the update to prefect 2.0b7, I've been having issues with calling result() related to what I think is prefect storage to s3 broken.

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

Hello everybody, I have two quick questions 1. Does a mapped task (of size n) count for 1 task or n tasks in terms of billing? 2. I have a workflow that generates a list of size n and then all subsequent tasks use

map

for each item. On the UI it says "No heartbeat detected from the remote task; marking the run as failed.", but the result of the workflow is marked as "Success". I checked my database and everything looks like the task succeeded. Any idea what is going on and how can I avoid it? Have a nice day!