Hi guys, I configured a RemoteFilesystem Block in the UI. I´ve copied my credentials as a dictionary into the settings box. After saving it is formatted in the following way, and when starting a flow, prefect isn´t able to read the settings correctly.

"{\n  \"key\": \"my_access_key\",\n  \"secret\": \"my_super_secret_secret\",\n  \"client_kwargs\": {\n    \"endpoint_url\": \"<https://123456abcdefg.r2.cloudflarestorage.com>\"\n  },\n  \"s3_additional_kwargs\": {\n    \"ACL\":\"private\"\n },\n}"

Is this a bug in the UI, or is there any “trick” to avoid this?

I take it that you need a remote filesystem block to deploy flows remotely? When I deploy with filestorage, the remote prefect server is trying to load a file path that looks like a path in my local system. I have github, but haven't been able to figure out why the deploy does not upload files properly. I get the following warning:

Deployment storage GitHub(repository='<'redacted'>, reference=None, 
access_token=SecretStr('**********'), include_git_objects=True) does not have upload capabilities; no files uploaded.  Pass --skip-upload to suppress 
this warning.

This is after I execute the following deploy command:

prefect deployment build -n sync_repository_tables_flow -q default flows/sync_repository_tables_flow.py:sync_repository_tables_flow -a -sb github/prod-gbi-github-dataflowdeployments

Anyone know what I'm missing?

Also, is there a way to import block settings from one server to another?

Hi Guys, I wrote this piece on utilizing prefect and google cloud run for serverless deployment of your data pipelines. Please check it out and drop comments/feedbacks. https://medium.com/@nwosupaul141/serverless-deployment-of-a-prefect-data-pipeline-on-google-cloud-run-8c48765f2480

[Kedro Pipeline, deployment with Prefect] Hey everyone, has anyone been able to successfully create a Prefect 2 deployment from a Kedro project? If so, could you share some insights or tips on how to go about it? I'm currently working on a project and would love to integrate Prefect into my Kedro pipeline, but I'm not sure where to start. Any help would be greatly appreciated! I am trying to convert the

register_prefect_flow.py

script given by Kedro documentation. Unfortunately, I don't seem to succeed because Prefect 1.0 API is different from Prefect 2.0. Thanks in advance !

if someone can help me about how to submit spark jobs or use prefect to orchestrate spark jobs, please share with me some links

Hi team! Is there currently a way to create notifications programatically? Either using cli, or the python interface? We’re trying to have everything as infrastructure-as-code as much as possible.

Hi everyone, I try to disable the telemetry service in Prefect by using the config.toml and currently trying to verify if the telemetry is already disable or not. I try to output the logs in .prefect folder but it seems like no logs appeared in .prefect folder. May I know if you have any idea on how to verify it?

Hi team! Im having issues accessing all of my deployments. If I click on a deployment in the deployment tab, nothing happens, and my web console gets a Javascript error (see screenshots). Has only happened so far since the Prefect 2.8.4 upgrade (self hosted). Have cleared cache and refreshed, rolling back to 2.8.3 to see if that fixes things now. Curious if anyone else has run into issues. Confirming that rolling back to 2.8.3 has all the UI working properly again, but the DB roll back failed and so now I’m just waiting on my team to delete and recreate the pg db.

Can someone help me about to how to deploy pyspark jobs or use prefect to orchestrate pyspark jobs on dataproc(GCP). please share with me some links

Hello everyone. I was experiencing an issue when creating a Deployment, which I documented here: https://discourse.prefect.io/t/error-when-creating-a-deployment-with-the-cli-modulenotfound/2426 Unfortunately it has not garnered very much attention from what I can tell. What it bols down to, is that creating a Deployment using the Prefect CLI yields:

Script at 'data_exports/extract_data.py' encountered an exception: ModuleNotFoundError("No module named 'pandas'")

which kind of confuses me, since the host where I am creating the deployment is not where it will run. My questions boil down to: 1. Why does the CLI need to know

pandas

(or any other library that I’m using)? 2. Does this mean that I should set up a different venv for every one of my deployments?

Hello everyone, Just wondering if prefect supports adding additional logs to the flow logs. I am running a cached task with the following parameters.

cache_key_fn=task_input_hash,
    cache_expiration=datetime.timedelta(hours=2),
    persist_result=True,
    result_storage=LocalFileSystem(), # Change to remote in future
    result_serializer=JSONSerializer()

For every task call , I want to log the

cache_key

, if the task returned the cached result OR if the result were cached. Is this supported by prefect by default? / What are the best practices for this type of logging to be implemented?

Hi all! Yesterday I had all my flows running properly, but all my flows today have had failed with the same error. I can no longer even get a simple toy model working, every flow responds with

No such file or directory

If anyone has any ideas as to what might be causing issues with S3 block file downloading or a way to debug it, please let me know!

Hi all, I am trying to create an automation, but I don't know if it is even possible. The automation I have in mind should

cancel

all sub-flow-runs of a meta-flow-run iff the meta-flow-run is

cancelled/cancelling

. Could I achieve this with a custom-trigger?

Hello folks, say there's a pipeline running in a docker container on some EC2 instance. If Prefect is added to said pipeline, is it then possible to schedule/trigger the executions through Prefect Cloud?

Hello, I need to build a prefect image using another distro. Although version 2.x includes a complete Dockerfile I am not able to find a similar one in v1.4.1 Where I can get a full Dockerfile for v1.4.1 ?, how the args are populated in Prefect CI process?, after reviewing https://docs-v1.prefect.io/core/development/overview.html I can't figure it out.

Hi all, slightly confused when looking this up in the documentation (I have asked this question before, but came up with no luck) Is there any roadmap/intention for prefect 2 to allow the the manual naming of cache file names similar to older prefect versions "target" argument? The current solution of using hashed file names doesn't seem that useful when it comes to archiving data

Hi guys, when trying to run a deployment via https://XXX/api/deployments/12345/create_flow_run it does not run the deplyoment and send back an error: via browser

{
  "detail": "Method Not Allowed"
}

via API docs on prefect server

"detail": "Data integrity conflict. This usually means a unique or foreign key constraint was violated. See server logs for details."

Goal is to trigger a deplyoment/ flow from another when data was loaded by another tool. Thanks in advance. 🙂

Any idea how to use the FlowFilter to get runs withs created_at between two dates?

client.read_flow_runs(flow_filter=FlowFilter(created_at>xxx, created_at<xxx, name={"any_": ["flow1"]}), limit=5)

Hello Everyone, I am using the latest version of prefect 2.8.4 in a local machine. I am using the UI to manage the server. but I am having an issue with the deployments webpage. I am able to see al of them as in the first image, But whenever I click on any of these deployments I get an empty webpage. But from the page where all the deployments are listed I am able to do a custom or quick run and itt works perfectly. but I am not able to see the deployment page. Thanks

I've been seeing this error pretty routinely (3-4 times per day), and there are not any task errors not sure if this is specific to prefect

2.7.10

and that I would benefit from upgrading, but the traceback below:

prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
06:30:38 AM

Hi, I trying extend flow decorator to add some defaults parameters to

flow

decorator and it is possible, but after this I have in UI ( in deployment section -> parameters ) only args and kwargs. I want see here

param

. Is there a magical way to achieve this? code:

from prefect import flow
from prefect.deployments import Deployment


def flow_wrapper(func):
    default_retries = 3

    @flow(name=func.__name__, retries=default_retries)
    def wrapper(*args, **kwarg):
        return func(*args, **kwarg)

    return wrapper


@flow_wrapper
def tmp_flow(param: int = 1):
    print(param)


if __name__ == '__main__':
    # tmp_flow(param=1)
    environment = 'prod'
    Deployment.build_from_flow(
        flow=tmp_flow,
        name=environment,
        path=f'.',
        infra_overrides={"env": {
            "PREFECT_LOGGING_LEVEL": "DEBUG",
        }},
        work_queue_name=environment,
        apply=True,
        work_pool_name=environment
    )

Is it possible to disable the cancel button for certain flows?

Any chance I can grab account and workspace_id out of a python object easily? I'm doing error flows and I want to dynamically build the cloud url in every workspace without adding new configuration

With a self-hosted prefect orion, how does one generate an API key?

Hello Everyone, I am using this module.When I run

kubectl cluster-info

it is giving timeout error. In AWS, I see that cluster is created but it shows, there is a new version of Kubernetes.Do you think that updating cluster will resolve the issue? Because It also says there might be RBAC permission issues in auth config map.

Over the weekend I tried the Cloud Storage example locally and it worked fine. Now trying it again (nothing changed), I'm getting the following error:

ValueError: Unable to find block document named my-uniq-bucket for block type gcs-bucket

• Prefect server and agent are running • Deployment has been applied via :

python gcs_example.py

Any one got this error in Prefect 2.8.3 while starting the server? If so, please share the root cause. 195640.777 | ERROR | prefect.server.services.flowrunnotifications - Unexpected error in: CompileError('The ORM FromStatement construct only supports being invoked as the topmost statement, as it is only intended to define how result rows should be returned.')

What's a best practice for registering blocks in a

blocks/

folder for e.g.? Via the CI/CD as with

deployments

?

Anyone familiar with this error?

too many values to unpack

on

prefect deployment build

Details in thread