Hello All! I am using prefect to run a Python script and I ran into an issue around a specific task that is querying a database and saving the results into the dataframe. Please see the error below. The query takes some tiem to execute ~40 seconds but is not particularly large only around 200MB. Any guidance would be helpful. Thanks!

No heartbeat detected from the remote task; marking the run as failed.

PrefectLive is starting in 10 minutes with @Anna Geller! https://prefect-community.slack.com/archives/C036FRC4KMW/p1654090047453119

Hi, I registered flow to S3 storage using AWS SSO credentials.

with Flow("LiveLots-ETL-Parent", storage=S3(bucket=config_bucket,
                                            client_options={
                                                 "aws_access_key_id":aws_access_key_id,
                                                 "aws_secret_access_key":aws_secret_access_key,
                                                 "aws_session_token":aws_session_token},)) as f:

Local agent is running on container as ECS task. The ECS task is attached with permission to access all S3 bucket.

{
  "Statement": [
    {
      "Action": [
        "s3:*"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::*"
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::wp-livelots-pt/*",
        "arn:aws:s3:::wp-live-lots/*"
      ]
    }
  ],
  "Version": "2012-10-17"
}

Initially flow is running without issue. After one hour, I am getting

Failed to load and execute flow run: ClientError('An error occurred (ExpiredToken) when calling the GetObject operation: The provided token has expired.')

Looks like ECS task is using the credentials when I registered flow. Please help

Hey guys i'm run my stack on ECS, but when i have a bunch of flows running in parallel sometimes i start get timeout for apollo service/hasura that lead the flow to be considered a zombie task and get killed/fail, what i can do to reduce this ocurrence?

Hey why would registering a flow cause the code in the flow to run?

I have an issue with the .prefect/results folder taking up a large amount of space and it's causing issues on our server. Can these results be safely removed? Is there a way to automatically clean up these files? I am using local storage. Thanks for the help!

Good morning 🙂 is there an issue with the GitHub storage? All the flows are failing now since the repo can't be found but it is still in the same location. The token shouldn't have expired.

Hi everyone! We use the EmailTask in prefect.tasks.notifications to send mail notifications from our Prefect flows via smtp-mail.outlook.com. Unfortunately, this class utilizes basic authetication implementation. Due to deprecation of basic authentication in Exchange Online, we now need to switch to modern authentication for all of our services. Does anyone know if Prefect has implemented this or has plans to do so? Otherwise, has anyone else implemented this using other libraries eg. from Microsoft? Thank you in advance. Can this be fixed by using smtp_type = SSL? Do we have to deal with certificates in order to switch to SSL? Extract of the code we use:

mail_settings = {
    "mail_title": "Title",
    "mail_recipient": "<mailto:recipient.1@enova.no|recipient.1@enova.no>,recipient.2@enova.no",
    "mail_sender": "<mailto:sender@enova.no|sender@enova.no>",
    "mail_smtp_server": "<http://smtp-mail.outlook.com|smtp-mail.outlook.com>",
    "mail_smtp_port": 587,
    "mail_smtp_type": "STARTTLS",
}

with case(new_file, True):
        email_sent = email(
            subject=mail_settings["mail_title"],
            email_to=mail_settings["mail_recipient"],
            email_from=mail_settings["mail_sender"],
            smtp_server=mail_settings["mail_smtp_server"],
            smtp_port=mail_settings["mail_smtp_port"],
            smtp_type=mail_settings["mail_smtp_type"],
            msg_plain=email_body,
            attachments=[most_recent_file["local_path"]],
        )

Hey, any idea if the map function will be out during beta stage ?

Hello - I am running into an issue where docker containers built on M1 Mac’s vs Intel Mac’s perform differently; the M1 Mac containers will get stuck in a “scheduled” state. Has this problem cropped up before?

Is anyone else experiencing weird css on prefect cloud? When I looked this morning I got this... It looks blank but the text is there, it's just the same color as the background

Hi Prefect Team, I am using Linux System and Prefect v2. I want to confirm a few things regarding deployment. I see that when I run the following command - 'prefect deployment create XYZ.py' I noticed that it is also running the flow and task. Is this expected? I assumed that deployment create would only create the deployment or update it. This happens even though my cron schedule setting is in the future.

Hi all, using the

S3Upload

module in prefect is it possible to specify the content-type?

One of my flows has the following error in prefect cloud. Any ideas what it might be from?

Failed to retrieve task state with error: ClientError([{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}])
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prefect/engine/cloud/task_runner.py", line 154, in initialize_run
    task_run_info = self.client.get_task_run_info(
  File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 1479, in get_task_run_info
    result = self.graphql(mutation)  # type: Any
  File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 473, in graphql
    raise ClientError(result["errors"])
prefect.exceptions.ClientError: [{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}]

Hi, maybe someone knows why it does not work for me. I'm trying to run simple flow (one flow with print) locally in manually built docker container and I get:

root@3b776c4684ea:/prefect/jobs# python3 simple.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/asgi_lifespan/_concurrency/asyncio.py", line 17, in wait
    await self._event.wait()
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Is there any known issue with removing parameters from a Flow and Prefect Cloud still showing the removed Parameter?

Hi, i’m developping flows in local mode, with docker containers I would like to know if there is any way to persist project and flows. My problem is the following: I create a Project, then a register my flows. Then when I restart docker all is gone, no project neither flows. I would like to know how the project persistence works in local mode. I found that flows are stored in filesystem by default at the following path ~/.prefect so why my project keep removed ? Thx for reading

I'm getting a weird issue when querying flow runs in the GraphQL api.

{
  "errors": [
    {
      "path": [
        "flow_run",
        0,
        "id"
      ],
      "message": "Cannot return null for non-nullable field flow_run.id.",
      "extensions": {
        "code": "INTERNAL_SERVER_ERROR"
      }
    }
  ],
  "data": null
}

I don't know how these values can be null if they...can't be null.

Hi, Prefect stores all these information about flow run and shows nicely with UI like below, is the a way to download run data so we can perform analysis. Eg, average duration of task..etc

Is there a way to have Flows run as a different Linux user (as in, not

root

?)

Hi.. I got this error `ValueError: A

path

must be provided to show where flow

.py

file is stored.` My flow code like this!

with Flow('flow_step1', storage=Local()) as flow:
    engine = create_engine('<mysql+pymysql://wonsun.jeong>:~~~~~~~~~~~~~~~', echo=True)
    connection = engine.connect()
    
    before = check_db()
    after = check_storage()
    
    answer = coin_inspection(after, before)
    
    with case(answer, False):
        new_datas = action_if_false()
        raw_extract = each_file_info.map(new_datas)
        final = each_ecg_meta.map(raw_extract)
        
    with case(answer, True):
        action_if_true()
           

flow.storage = Local(stored_as_script=True)
flow.register(project_name='try')
flow.run() 
# flow.visualize()

This python file name is 'flow_step1.py'. What's the problem?? Also, most weird thing is window(local) already run 'flow_step1' and create result files but prefect web UI can't detect this flow! I'm so confused because i don't understand exactly Flow, Register, Agent, Server, Local, Path.. in Prefect. Help me please.. 🤯

hi, i am experimenting with samples files with prefect, i have to questions, how to solve "*No module named \'lib\'"),* happens when the agent start executing the task

EU and North America support If we get the enterprise Prefect addition, will we be able to use the same service for both EU and North America? • performance wise, is there any impact? • Legally, does having a single instance for both region is accepted by EU countries?

Hello, I am using Jupyter task (https://docs.prefect.io/api/latest/tasks/jupyter.html#executenotebook) to execute notebook to generate HTML outputs in my flow, was wondering if there's a way to pass in

allow-errors

equivalent of jupyter-nbconvert (https://nbconvert.readthedocs.io/en/latest/config_options.html)

Hi, I'm working with Prefect and having a flow_id in my database. But I don't find anywhere to list the flow_run_id(s) of my flow_id. I would like to have something like get_flow_info for flows such as get_flow_run_info could return for flow runs : https://github.com/PrefectHQ/prefect/blob/9f415155a661614386a7041329d221cb001ccbe8/src/prefect/client/client.py#L1195 Do you have any solution? Thank you

Hi, we're having an issue with prefect cloud and the heartbeat mechanism marking a task as failed, but not marking the flow as failed. We are running flows using an EKS cluster that uses spot instances, so we rely on the heartbeat mechanism to know to retry flows that die because the underlying node is reclaimed by AWS. For flows where we're encountering the issue(see flow

9a3a44b8-efeb-48bf-b07c-22a90ca9b63e

), the log messages state:

No heartbeat detected from the remote task; marking the run as failed.

for some task, but the flow is still in a running state. The issue is intermittent,

1f570a06-56c8-46e4-acf5-3e0cf6d12b75

is a flow where the heartbeat marked the same task as failed and it also successfully marked the flow as failed. Any insight into what may be going on here would be much appreciated!

hi community! i'm following this to add extra loggers for my runs, but the log messages are not showing up on prefect server, anyone have similar issues?

Hi all, is a simple way to view a distribution of all scheduled prefect Flows? For example, under the calendar tab currently we can see each Flow that is scheduled that day but not all of them overlapping. Is there a way to get this view?

Hi guys, I'm new to prefect, tried to install prefect 2.0 on Ubuntu WSL and after tunning "prefect orion start" getting an message "INFO: 127.0.0.1:37140 - "POST /graphql HTTP/1.1" 405 Method Not Allowed"

can someone point me how to fix this?