Same here, also can't login into the cloud with a new API key

The status page is still green: https://prefect.status.io

@Chris White Is anyone looking at this??

All of our jobs are in the Scheduled State

we are also experiencing this, jobs have not been running for about an hour

Oh, thank goodness. I was going crazy. I’m having the issue as well.

Fair to say this looks like a general outage.

Any approximate ETA @Chris White?

How can we pause a task until a specified time and how to resume the same task before the timeout with some external triggers @Chris White

Just saw my flows start

Yea it seems our fix has taken some time to have an effect but the good news is that I am seeing late runs begin to get submitted and are now successfully running; we are still investigating though

Same here

I’m getting this error when using the FivetranSyncTask

Traceback (most recent call last):
  File "/Users/slloyd/projects/dwbi-orchestration/.venv/lib/python3.8/site-packages/prefect/engine/task_runner.py", line 880, in get_task_run_state
    value = prefect.utilities.executors.run_task_with_timeout(
  File "/Users/slloyd/projects/dwbi-orchestration/.venv/lib/python3.8/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "workable/src/flow.py", line 64, in fivetran_sync
    status = FivetranSyncTask.run(
TypeError: method() missing 1 required positional argument: 'self'

creds is passed in from a PrefectSecret task.

@task
def fivetran_sync(connector: str, creds: dict) -> dict:
    status = FivetranSyncTask.run(
        api_key=creds['api_key'],
        api_secret=creds['api_secret'],
        connector_id=connector
        )
    return status

Hello everyone! The Cloud API issue should now be resolved, and we took a few extra steps that should prevent it from recurring; I don't have a full post mortem yet but in short this appears to have been caused by an incredibly large burst of logs entering the system that caused cascading effects. We'll be sure to take our learnings here to prevent this from occurring again and my sincere apologies for your disruption in service!! I hope everyone continues to have a good morning, afternoon or evening depending on where you are

Hi, I just wanted to highlight this issue https://github.com/PrefectHQ/server/issues/357, we've also seen this, so we're sticking to 2022.01.25 for now. Any ideas when this might be addressed?

Hi all, anyone know if there is a way to retrieve the run config parameters, specifically the labels, from within a flow programmatically at flow runtime? Haven't been able to find it in the docs.

Hello all. Does anyone have any experience with or resources for using Prefect with GCP serverless? (cloud functions/cloud run) We are currently running a lot of our workflows using GCP serverless (via cloud functions or cloud run); and we have been using pub/sub queues as an orchestration tool to communicate between tasks. The pub/sub 10 minute message acknowledgement limit is starting to become a problem as some our tasks are taking longer than that to complete, so we wish to look into alternative tools.

Hello Prefect team, How a prefect cloud enterprise customer can get a technical support with a bug with prefect cloud itself?

Hi, I saw that there were some issues earlier today with Prefect Cloud. We’re currently still seeing a lot of our ECSRun tasks fail to begin execution correctly- are we potentially seeing delayed after effects from the earlier service disruption?

Howdy - I've attempting to push a hello world job to ECR using the Docker storage for use with ECS. It looks like it fails on the last part, after passing health checks, where it's attempting to push to the registry. It's triggering this exception:

<https://github.com/PrefectHQ/prefect/blob/master/src/prefect/storage/docker.py#L610-L613>

Has anyone run into a similar exception? I'm successfully authed to ECR from my local docker --login as well.

I have an ECSRun for a hello world that appears to be hanging. I've successfully pushed the image to ECR via the Docker() storage, and the task shows up as scheduled in Prefect Cloud - which can also see the ECS Agent I added following the guide (https://docs.prefect.io/orchestration/agents/ecs.html#running-ecs-agent-in-production). I'm a bit puzzled on how to proceed to debug why the agent isn't popping the task from the queue in the cloud.

Hi I have a question regarding scheduling prefect flows with parameters, I think I'm missing something simple : how do you pass db creds, if not through parameters (without hard-coding passwords, paths, etc)? I've posted my questions on SO: https://stackoverflow.com/questions/71875658/running-registering-prefect-flows-with-parameters-on-schedule, please let me know what you think. thanks!

Is it possible to return a Socket from a task?

Hi, I'm looking into running Prefect on Kubernetes clusters in different regions, and had a question about handling GCSResults. If we use the

@task(result=GCSResult(bucket=<bucket>))

method of configuring a task result, is the bucket path fixed at flow registration time? If so, is there a way it can be dynamically set at flow run time? What I'm hoping to do is have flows that can be registered to run on different clusters (using agent labels), and have their GCSResult bucket path be configured via an env var on the cluster. That way we can re-use the same flow code across different clusters but have different results buckets depending on the cluster.

Do you need to set the image on ECSRun for the task to use a a specific image or will it associate the Docker storage pushed to ECR in the same flow as the base image to use?

Cryptic errors in ShellTask - Hi, I'm running a flow with a number of shellTasks, and encountering errors like:

2022-04-14 09:42:06-0600] ERROR - prefect.TaskRunner | Task 'ShellTask[0]': Exception encountered during task execution!
Traceback (most recent call last):
  File "/blah/python3.9/site-packages/prefect/engine/task_runner.py", line 880, in get_task_run_state
    value = prefect.utilities.executors.run_task_with_timeout(
  File "/blah/python3.9/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "/blah/python3.9/site-packages/prefect/utilities/tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "/blah/python3.9/site-packages/prefect/tasks/shell.py", line 131, in run
    tmp.write(command.encode())
AttributeError: 'list' object has no attribute 'encode'

I have a couple of questions: • Is there a way to tag shell_tasks such that you can see some clue as to which one failed? • Can I get a better description of the failure At the moment I have

shell_task = ShellTask(log_stderr=True, return_all=True, stream_output=True)

Is it a valid use-case to use Prefect for the sake of having a managed way of running tasks rather than "data pipelines"? I just want the workflow aspect of it.

Can I run Prefect fully on-premise without cloud features whatsoever?

So I have a Fargate ECS cluster, but when I try to scale up the cpu count with ECSRun() I get:

[14 April 2022 2:14pm]: An error occurred (InvalidParameterException) when calling the RunTask operation: No Fargate configuration exists for given values.

. The weird thing is that 4096 appears to be a valid entry: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html. Is it possible I screwed up the definition of my fargate cluster?

What are the best practices for managing Docker image names?

docker_storage = Docker(
    registry_url=environ["REGISTRY_URL"],
    dockerfile="./Dockerfile",
    image_name="{edited}-prod-platform-prefect-{project}",
    image_tag="latest",
)

It seems like each flow could demand its own image in order to separate dependencies, which would mean creating an ECR repo for each workflow? I suppose this wouldn't be that difficult to script with Github Actions and aws-cli globbing a directory for workflow names?