Hello all, I've noticed the following error intermittently using Prefect Cloud for the last few weeks. I'm running my agent on an EC2 instance. My deployments use docker containers. I updated recently from 2.1.0 to 2.3.2. The same errors have occurred with both versions. Any insight into why this may be happening?

prefect.exceptions.PrefectHTTPStatusError: Server error '500 Internal Server Error' for url '<https://api.prefect.cloud/api/accounts/043b2649-9d07-4c5e-8225-521ba2275e68/workspaces/689b139b-a725-4c2b-b167-86a705b8789d/task_runs/7d0c4bc0-a212-49b3-94e2-f1b9bdee765f/set_state>'
Response: {'exception_message': 'Internal Server Error'}
For more information check: <https://httpstatuses.com/500>

Hello All, When re-installing my helm chart to AKS via the Helm Chart, I’m running into this issue:

Error: failed to fetch <https://prefecthq.github.io/prefect-helm/charts/orion/prefect-orion-0.5.1.tgz> : 404 Not Found

The target version is: 2.3-python3.9. Has something changed to where this file no longer exists? That assumption doesn’t seem quite right.

Hi All, I am trying to implement a date parameter in prefect UI like so

start_date = Parameter("start_date", default="2022-09-11")

What I see on the UI after registering the flow is in the first attached image. start_date shows up as string. What I’d like to see by default is in the second image, I’d like start_date to show up as date type instead of as string with the date time picker functionality. How can I achieve this?

Hey all, does anyone know how can I retrieve an Infrastructure object from Orion? I haven’t seen it covered in the docs

Has anyone figured out how to overwrite parameters in prefect 2 for a flow on a per schedule basis, the way you could in prefect 1? A lot of overhead in Prefect 2 right now if you just want to send a simple variable change for a flow depending on the time the flow runs. Currently, the only solution I've found is to have 2 separate deployments registered with different schedules and variables.

Do flows and tasks need to be in the same Python script in Prefect 2.0?

Hello, I have been testing out prefect 2.0, we have been using 1.0 for over a year and love it. I have a GKE autopilot cluster set up on GCP with an agent running in a workload. I have been testing out running a very simple flow, and everything is working well. However, when I look at the logs from the agent workload, it picks up the flow run, says the pod is starting, and then says the pod never started. The pods do start, the flow runs, and I see its successful in the UI. I am just wondering why the agent says the pod was never started. This is my first time using kubernetes btw so bare with me.

We have some flows that could run locally for small datasets. What is the recommended way to invoke a flow and dynamically change the associated TaskRunner?

This message was deleted.

Hey folks! I’m seeing this rather cryptic pickle error when running Prefect locally (Prefect 2.4.0, Python 3.10) - my code doesn’t use Pydantic’s

AnyUrl

or subclasses within the code, do you have any idea what might be causing the pickle error?

Hi, any suggestions on how to pass a dataframe as a parameter to a flow? This breaks in latest Prefect:

from prefect import flow
import pandas as pd
import numpy as np


@flow
def do_nothing(df):
    return None


if __name__ == "__main__":
    df = pd.DataFrame(np.random.random((10_000, 20)))
    do_nothing(df)

Traceback in thread

Hello! I have a question regarding how resources are being used in Prefect. I am using the DaskExecutor with LocalCluster(2 workers, and 4 threads per worker) and am testing processing 800 files which I am splitting evenly into 8 batches. Originally I had the prefect task running each of the batches passed to them and processing those in a loop, but when I change this to explicitly run threading within the tasks (still mapping to them) using the existing client like how Kevin suggested in this POST, I am seeing significant time savings (25 - 50% faster). My question is are there drawbacks to performing batch processing this way/is this the right way to do this or were we not utilizing Prefect in the correct way previously?

For those interested, I'm creating a Prefect Collection to interact with Jinja. I'm still working on it but I've already posted the basics on GitHub: https://github.com/hallenmaia/prefect-jinja

Does anyone know if it’s possible for one’s own Prefect Cloud account to kick off a flow that’s in another Prefect Cloud account? My org delivers data to another org that also uses Prefect and it’d be cool if once my data delivery finishes, it could automatically kick off their pipeline that handles said data. Maybe if not cross-cloud, there could be an email trigger (if the other org gave me their account and flow IDs)?

Hello Team, I am getting intermittent crashes on prefect without any reason being shown Is there any bug in the system causing this?

Hey all, is there a sync OrionClient? I’m developing a CLI with custom features and the sync client would be very helpful

Been trying to get the prefect helm chart working and getting a little confused

In Prefect 1.0, is it possible to schedule a flow, but not allow (disable certain roles) users to run it?

Hi. At last I'm properly getting to 2.0! success kid Seems I'm running into the same issue as Keith here Getting

ModuleNotFoundError: No module named 'gcsfs'

It's strange because the environment that my agent is running in has

gsfs

installed and I have

"EXTRA_PIP_PACKAGES": "gcsfs"

in my Docker Block... Any tips?

Hi All, I’ve deployed Prefect Orion to AKS. We’ve connected the UI via ingress with an alias. API calls seem to be working with the alias. However, the UI via the alias seems to be giving this error. I’ve been changing the Orion API URL to see if that resolves it to no avail. Does anyone have any insight to what might be happening here?

I couldn't find a hello@prefect.com email to direct this at so apologies for dropping it here. I've been looped in to help with some last minute promotion/sponsorship for PyData Miami which is taking place next Thursday (9/22/2022): https://pydata.org/miami2022/ There is sponsorship opportunities if you think Prefect would be interested: https://pydata.org/miami2022/wp-content/uploads/2022/08/PyData-Miami-2022-Sponsorship-Prospectus-1.pdf You could also encourage attendance to your colleagues and network. Let me know if you have any questions.

After installed Prefect Orion 2.4, getting the following error while starting Prefect Orion. Please help me to fix this issue. sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class 'asyncpg.exceptions.UndefinedObjectError'>: operator class "gin_trgm_ops" does not exist for access method "gin" [SQL: CREATE INDEX CONCURRENTLY trgm_ix_flow_name ON flow USING gin (name gin_trgm_ops); ]

Do you know if it would be possible to convert the snowflake_utils to work with the SnowflakeConnector block? I’ve had a try but I get invalid username/password although I’ve used the same block in snowflake_query without an issue

Hello. I’m starting to see a low of flows runs crashing when trying to update task state (500 error from https://api.prefect.cloud/api/accounts/<uuid>/workspaces/<uuid>/task_runs/<uuid>/set_state) Full stack trace in the thead. Is there any way to diagnose further?

We are getting a flow failure with this message:

Submitted for execution: Task arn:aws:ecs:us-west-2:<>:task/prod-prefect-cluster/c01a3ca13b3741a89e58b6c33dda0671
INFO prefect-server.Lazarus.FlowRun Rescheduled by a Lazarus process. This is attempt 1.

Any tips on how to debug this? I can’t find the logs for these tasks in cloudwatch.

Question regarding trigger.any_failed:

Hi Team - When running my docker image in the terminal. I wanted to get a list of block types: prefect block type ls There was an aws credential block type, but I am not able to create it, nor see that type in blocks. Is this a type that was abandoned?

Is it expected that this graphql query would return a flow group id rather than an actual flow id? Does

client.create_flow_run

work off of flow group ids?

{
  flow(
    where: {name: {_eq: "FLOW_NAME"}, archived: {_eq:false}},
    order_by: {created: desc}
  ) {
    id
    name
    project {
      name
    }
  }
}

The concept of Agent and Work Queues is really useful. But as far as I understand, Work Queue's hold only flows, not tasks. What queue do tasks go into? Is there a way for a user to tap into that queue?

Hi I have

main

flow, which calls to a subflow

go_eat

, each time with different parameters. something of the sort:

@flow
def go_eat():
    ...

@flow
def main():
    animals = ['dog', 'cat', 'monkey']
    for animal in animals:
        go_eat(animal)

I want to make those

go_eat

flows to run in parallel, and I wonder how to implement this. it would be best if I could make up to 2 animals to

go_eat

in parallel, but no more than that. I'm not sure if I somehow better use the prefect queue concurrency mechanism, or if I better implement this in pure python. any suggestions?