Hi guys, can I use S3 block to download files from S3 bucket? or am I just using the block as secret to the AWS S3Download task? any examples?

Hi, I try to build a deployment and upload to minio storage(remote-file-system), the package s3fs already install and i put this lines to head of flow.py

from prefect.filesystems import RemoteFileSystem
remote_file_system_block = RemoteFileSystem.load("minio-s3")

I get the error:

root@fd9f23f7b06c:~/flows# prefect deployment build rest_flow_3.py:websites2 -n eli -q check_url_status -sb remote-file-system/minio-s3
Found flow 'test url check flow'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 230, in coroutine_wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 181, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/usr/local/lib/python3.8/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/usr/local/lib/python3.8/site-packages/prefect/cli/deployment.py", line 988, in build
    deployment = await Deployment.build_from_flow(
  File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 755, in build_from_flow
    await deployment.upload_to_storage(ignore_file=ignore_file)
  File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 600, in upload_to_storage
    file_count = await self.storage.put_directory(
  File "/usr/local/lib/python3.8/site-packages/prefect/filesystems.py", line 368, in put_directory
    self.filesystem.put_file(f, fpath, overwrite=True)
  File "/usr/local/lib/python3.8/site-packages/fsspec/asyn.py", line 114, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/fsspec/asyn.py", line 99, in sync
    raise return_result
  File "/usr/local/lib/python3.8/site-packages/fsspec/asyn.py", line 54, in _runner
    result[0] = await coro
  File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 1101, in _put_file
    await self._call_s3(
  File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 332, in _call_s3
    await self.set_session()
  File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 492, in set_session
    self.session = aiobotocore.session.AioSession(**self.kwargs)
TypeError: __init__() got an unexpected keyword argument 'url'
An exception occurred.

my setting is:

{  "url": "<https://172.23.0.5:900>",  "accessKey": "!!!!!!!!!!!!!!!!!",  "secretKey": "===============",  "api": "s3v4",  "path": "auto"}

Hi all, prefect 2.0. I am getting the following error while building a deployment.

ValueError: Unable to find block document named mini1 for block type RemoteFileSystem

I am getting this error while executing a flow via Cloud Prefect 2.0 "RuntimeError: This portal is not running". Any clue what is going wrong. The same flow was working fine before I use Cloud Prefect 2.0. Actually it finishes all my flows and tasks successfully and at the end it crashes and it happens to all my existing flows. line 1512, in propose_state raise prefect.exceptions.Abort(response.details.reason) prefect.exceptions.Abort: This run has already terminated.

Hi, I'm new to Prefect and installed a local server. Last week everything seemed to work and I started to write my first flows. They were executed as expected, also the UI worked great. However today morning, I tried to continue working on my flows, but for some reason the UI was frozen and I just restarted the containers through docker-compose (down and up again). After that I experienced a curious / weird output in the Postgres logfile (I will explain below). Even the UI was working fine again, the logfile of Postgres was quite confusing. So I thought, that something has messed up with my agent or other communication issues between Prefect server and the database. I decided to make a new clean setup in a new clean virtual environment. I also setup a path to a new database, and deleted the .prefect folder in the home directory. But even with the clean setup the weird behavior of Postgres started right away: ==> More details in thread

Hello, I’m new to Prefect and I need some help getting started. I’m currently struggling with two issues: 1. I’m setting up Prefect server on EKS using Helm. After

helm upgrade

ran successfully, I get commands such as

1 Run the following command to get the UI URL:

  UI_HOST=$( \
    kubectl get svc \
    --namespace prefect \
    --template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}" \
    prefect-server-ui \
  ) \
  && echo "UI available at: http://$UI_HOST:8080"...

when I run the above command, I get an error saying

error: error executing template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}": template: output:1:10: executing "output" at <index .status.loadBalancer.ingress 0>: error calling index: index of untyped nil

Then I have to run

kubectl port-forward <name of the service> <port>:<port>

to get ui and apollo servers running. Is there a way to run this without port forwarding? 2. I’m trying to run a hello world example flow on prefect server on my EKS cluster. I tried to trigger it from my command line by creating a project and running

prefect register --project test-project -p hello/

. The registering seem to run successfully but I don’t see the flow on the ui. I’m also not able to manually create flow project using ui. Am I interacting with two different prefect engine here? How do I register prefect flow so that it is running on my prefect server on EKS cluster?

Hi, how can I pass environment variables to Docker agent without restarting? is there any options to set it in flow, for example pass the path for *.env file or make visible variables from current terminal session to Local agent?

Hey guys, I am getting this error after helm install prefect/prefect-worker

Hello, I have successfully deployed a Kubernetes Prefect Agent on Azure Kubernetes Cluster. I am trying to run a simple flow that utilizes a LocalDaskExecutor on the AKS Virtual Nodes. For this, I am using a custom job template for the pod, because it needs some customized node selectors and tolerations that Azure publishes. the following is snippet of my job_template:

job_template={
            "apiVersion": "batch/v1",   
            "kind": "Job",
            "spec": {
                "template": {
                    "metadata": {
                        "labels": {
                            "execution-model": "serverless"
                        }
                    },
                    "spec": {
                        "containers": [
                            {
                                "name": "flow"
                            }
                        ],
                        "nodeSelector": {
                            "execution-model": "serverless"
                        },
                        "tolerations": [
                            {
                                "key": "<http://virtual-kubelet.io/provider|virtual-kubelet.io/provider>",
                                "operator": "Exists"
                            }
                        ]
                    }
                }
            }

However the flow fails. When i ran kubectl get events. I notice the following output:

Warning   ProviderCreateFailed   pod/prefect-job-XXXXX-XXXXX   ACI does not support providing args without specifying the command. Please supply both command and args to the pod spec.

Just some more information - I also ran the same flow successfully on a alternate deployment on AWS EKS Fargate, using an AWS Kubernetes Agent. Any guidance is really appreciated :)

Did anyone have this problem where the python client hangs on trying to set up deployment?

deployment = Deployment.build_from_flow(
    flow=load_data_from_minio,
    name="load_data_from_minio",
    work_queue_name="default",
    schedule=schedule,
    storage=storage_block,
    infrastructure=infra_block,
)

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    deployment.apply()

Running this code just seems to hang forever. No debug logs either. Full code in thread