prefect-community #prefect-cloud

I’ve canceled a flow run, but it’s stuck in state “Cancelling” for at least 10 minutes. I see this issue https://github.com/PrefectHQ/prefect/issues/4490 not sure if it’s related. Any ideas?

Tibs

01/05/2023, 5:29 AM

Hi, I am trying to optimize deployments to prefect cloud. I am creating from a github action 6 deployments and 8 blocks (aws credentials, s3 bucket, 6 ecs task blocks), using Python API. Currently this takes around 12 mins to deploy, which is too slow for us, as we will be adding a lot more. The storage block is an S3 bucket, infrastructure block for every flow is an ECSTask. Does anyone has an idea why it's slow and how can we optimize this?

brett

01/05/2023, 5:57 PM

On v1, is it possible to cancel all scheduled runs for a Flow?

Ben Muller

01/05/2023, 6:22 PM

Is there anyway to configure a longer token expiry when logged in to the cloud UI? I'm finding myself having to send a password about 5 times a day.

upvote 1

Óscar Reyes

01/05/2023, 8:29 PM

Hi, I’m the Data Scientist Leader in a private company that has almost all the services running on AWS. We are planning to create a new data pipeline, but not sure at this point if it is better for us to install prefect in our AWS cloud and then manage it, or go for a Prefect Cloud account. Which are the benefits of Prefect Cloud? What are the computational resources of Prefect Cloud for highly demanding computational intensive data pipelines? The numbers of flow executions are limited by the kind of Prefect Cloud account?

Matt Racki

01/05/2023, 8:27 PM

Hi community! I'm trying to set up an organization on prefect cloud, but just get a blank screen. Is anyone else running into this issue? https://app.prefect.cloud/organizations/create

✅ 1

01/05/2023, 10:39 PM

Hey. We’re having problems running Prefect using the tutorials provided on the website. Tried to upgrade python and install prefect latest version, but we still face this error with the basic example:

Copy code

python3 test.py
Traceback (most recent call last):
  File "/Users/ax/prefect/test.py", line 9, in <module>
    flow = Flow("hello-flow", tasks=[hello_task])
  File "/Users/ax/prefect/env/lib/python3.10/site-packages/prefect/context.py", line 176, in __register_init__
    __init__(__self__, *args, **kwargs)
TypeError: Flow.__init__() got an unexpected keyword argument 'tasks'

✅ 1

01/05/2023, 10:39 PM

What’s the reason for that?

01/05/2023, 11:04 PM

Are late “flow runs” normal for the prefect.cloud ?

01/05/2023, 11:20 PM

How to run a flow on Prefect cloud with curl?

Andy Yeung

01/06/2023, 10:06 AM

Hi, wanna ask what is the best way to call the subflow. right now I am using something like below:

Copy code

parallel_subflows = [
    subflow1(ref_id),
    subflow2(ref_id)
]
await asyncio.gather(*parallel_subflows)

but I can't control the random generated flow run name in this way. How can I configure the flow run parameter of the subflow?

Jacob Gora

01/06/2023, 7:35 PM

Hi, anyone having problems getting the prefect cloud 2.0 password reset e-mail?

Siva Balusu

01/07/2023, 3:10 AM

Hi I am on prefect 2.0 cloud and my deployment suddenly stopped working with the below error. Deployment pushed the flows to s3 bucket but agent is having issues to get those flows?

Copy code

Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 260, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/deployments.py", line 167, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 463, in get_directory
    return await self.filesystem.get_directory(
  File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 308, in get_directory
    return self.filesystem.get(from_path, local_path, recursive=True)
  File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 113, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 98, in sync
    raise return_result
  File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 53, in _runner
    result[0] = await coro
  File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 551, in _get
    rpaths = await self._expand_path(rpath, recursive=recursive)
  File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 751, in _expand_path
    out = await self._expand_path([path], recursive, maxdepth)
  File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 775, in _expand_path
    raise FileNotFoundError(path)
FileNotFoundError: ['prefect-bucket/prefect_test2']

Siva Balusu

01/07/2023, 3:10 AM

@here any help is much appreciated

Siva Balusu

01/07/2023, 3:10 AM

agent is in kubernetes

Siva Balusu

01/07/2023, 3:10 AM

Copy code

I have no name!@prefect-agent-656bfcdc7d-pqjzx:~$ pip list | grep "prefect"
prefect                    2.7.1
prefect-aws                0.2.3
prefect-dask               0.2.2
prefect-kv                 0.1.0
prefect-snowflake          0.25

Michael Viescas

01/07/2023, 4:59 AM

Hi. We're running @Anna Geller’s dataflow-ops project and the ecr-ecs-block github actions flow stopped working with a 409 error.

run: |    cat <<EOF > ecs_block.py    from prefect_aws.ecs import ECSTask

Michael Viescas

01/07/2023, 5:10 AM

Hi. We're running @Anna Geller’s dataflow-ops project: https://github.com/anna-geller/dataflow-ops/blob/main/.github/workflows/main.yaml The ecr-ecs-block github actions flow stopped working with a 409 error: "Client error '409 Conflict' for url '***/block_documents/'" It's happening during the "ecs.image=" line.

Copy code

- name: Prefect ECS block
        run: |
          cat <<EOF > ecs_block.py
          from prefect_aws.ecs import ECSTask
          
          block_ = "$BLOCK"
          ecs = ECSTask.load(block_)
          ecs.image = "${{ steps.build-image.outputs.image }}"
          ecs.save(block_, overwrite=True)
          
          EOF
          python ecs_block.py

Does the ECSTask block still support changing the image after loading it from an existing block? I'm having trouble finding it when looking at the documentation here: https://prefecthq.github.io/prefect-aws/ecs/#prefect_aws.ecs.ECSTask.image Prefect Version: 2.7.7

✅ 1

Elliott Wilson

01/07/2023, 5:49 PM

Hello team! I am getting this, this morning when I try and apply a deployments I have refreshed my api keys the url is returning the below:

✅ 1

Christopher Boyd

01/09/2023, 2:52 PM

Hi Tadej, Would you be able to open a github issue for this behavior? Thank you!

Ben Muller

01/09/2023, 6:49 PM

My flow is stored in s3 and I'm getting these logs FYI

Downloading flow code from storage at ''

🤷 1

Florian Giroud

01/10/2023, 1:09 PM

Hi, I have a quick question about Prefect Cloud and the “7 days history”. After 7 days, what is deleted exactly ? is it only the logs ? or is it also the entry in the list ? I was wondering if if would still see the landing times and statuses after 7 days, or if it’s everything that gets deleted Tx

Stéphan Taljaard

01/10/2023, 2:02 PM

Hi. How long should a web session to the UI https://app.prefect.cloud/ be valid for? I used to be able to log in maybe once per day. Now my session expires much more often, maybe requiring a login 3-4 times per day.

plus one 3

✅ 1

Tadej Svetina

01/11/2023, 6:57 PM

Oh nvm - getting them now, seems it took prefect some time to start sending notifications

Shayne Hodge

01/12/2023, 8:57 PM

Hello, We're trying to migrate from Prefect 1 to 2, but we want to run both concurrently on the same data to verify the output is the same. Both lib versions can't be installed at once, so I'm using the REST API to trigger jobs. We have a flow set up that works and that we've tested launching through the UI. I've followed this guide, and my code looks something like this:

Copy code

data = {
        'parameters': params, 
        'flow_id': 'cfd9e6db...',
        'deployment_id': '24583f30...4',
        'infrastructure_document_id': '0181dc52...',
        # tried without the next key entirely, and in "PENDING"
        "state": {"type": "RUNNING"}
    }
    r = <http://requests.post|requests.post>(PREFECT2_URL, json=data, headers={'Authorization': f'Bearer {api_key}'})

The flow is created, but it doesn't pick up the work queue properly, and so it just stays in the state but nothing happens. See below, the green run was triggered in the UI, the other two were sent in as "SCHEDULED" and "RUNNING", respectively. Any suggestions on getting this to run would be much appreciated. Thanks!

Luis Pinto

01/13/2023, 6:54 PM

Hello, One question, if I deploy a Flow in prefect cloud and I set up a schedule time, Where the computing executing power comes from if I turn off the laptop?

✅ 1

Chris Whatley

01/15/2023, 1:13 PM

Are notifications from the cloud meant to be mutually exclusive? I have two going to different slack rooms and only one fires.

✅ 1

Ben Muller

01/19/2023, 2:44 AM

prefect cloud issues? I just got some super strange notifications to our slack:

Copy code

Rendering the template exceeded the CPU, memory, or time limit.
Template source:
Flow run {{ flow.name }}/{{ flow_run.name }} entered state {{ flow_run.state.name }} at {{ flow_run.state.timestamp }}.
Flow ID: {{ flow_run.flow_id }}
Flow run ID: {{ flow_run.id }}
Flow run URL: {{ flow_run|ui_url }}
State message: {{ flow_run.state.message }}
Show less

👍 1

hard same 1

✅ 1

Giuliano Mega

01/20/2023, 11:44 AM

Hello there, since upgrading my agents to 2.7.8 (Prefect Cloud + GKE autopilot), my agent logs are full of those:

Copy code

--- Orion logging error ---
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/logging/handlers.py", line 151, in send_logs
    await client.create_logs(self._pending_logs)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/orion.py", line 1830, in create_logs
    await <http://self._client.post|self._client.post>(f"/logs/", json=serialized_logs)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post
    return await self.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/base.py", line 253, in send
    response.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/httpx/_models.py", line 745, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '429 Too Many Requests' for url '<https://api.prefect.cloud/api/accounts/[account]/workspaces/[workspace]/logs/>'
For more information check: <https://httpstatuses.com/429>
Worker information:
    Approximate queue length: 0
    Pending log batch length: 458
    Pending log batch size: 164880
The log worker has tried to send these logs 4 times and will now drop them.

which I guess isn't my fault as the Prefect agent is in control of rate limiting calls to the logging API. Looking into Prefect Cloud's UI it indeed seems to be dropping logs, which I don't think was happening before. Is this a known issue? Anything I can do to mitigate?