Hi all! I was wondering how we can change the UI port from 8080 to something else. I have changed the

PREFECT__SERVER__UI__PORT

and

PREFECT__SERVER__UI__HOST_PORT

to try and follow the config.toml, but the UI service is still launching and saying “👾👾👾 UI running at localhost:8080 👾👾👾” and I dont know what else to change in the UI service 😞 As a follow up, what is the difference between the UI port and the UI host port? EDIT: I see echo

UI running at localhost:8080

is hardcoded in ./start_server.sh in the UI service, is this meant to be the case?

@Marvin how to setup basic auth for oss prefect

Hey guys, I'm kinda struggling to set up a Prefect on our company server. So it's running and all that, but I had to setup a reverse proxy (Nginx) as well, so that people can access the UI via standard https at 443 and at a company subdomain (internal that is). The UI itself is accessible, but it's trying to make REST API requests to http://127.0.0.1/api/something which is obviously not working. Is there any env setting I can use to make it understand it should use https://prefect.internal.company.com/api instead? I couldn't find anything on that topic anywhere 😞 Thanks!

Hello friends! I am running into some issues using an ECSCluster with the DaskExecutor. A brief overview of my setup: using Prefect cloud tied in with AWS infrastructure (e.g. running prefect-agent as a service on an ECSCluster) A flow might look something like this:

file_list = ['./file_1.txt', './file_2.txt', './file_3.txt', './file_4.txt', './file_5.txt', './file_6.txt', './file_7.txt', ..., './file_100.txt']

@task
def test_task(input_file):
    result = some_function(input_file)
    return result

def ecs_cluster(n_workers=4):
    """Start a ecs cluster using the same image as the flow run"""
    return ECSCluster(
        n_workers=n_workers, image=prefect.context.image, region_name="us-east-1"
    )


with Flow(
    "test flow",
    storage=S3(bucket="storage"),
    run_config=ECSRun(task_definition=task_definition),
) as flow:
    test_task.map(input_file=file_list)

flow.executor = DaskExecutor(
    cluster_class=ecs_cluster, cluster_kwargs={"n_workers": 10}
)

How should web applications (Node.js) listen/subscribe to Prefect workflows completion? I have a task that trains a deep learning model through a Web Application. The “*Start training*” button on the web app sends a

POST /model

REST API call to the Node.js server which then in turn calls Prefect 2.0 using the REST API to start a new run (of an existing deployment). The workflow will train the model, run inference on some data and then persist the output to a database. This is obviously an asynchronous operation that may take a few minutes (or more) to complete. Assuming the workflow succeeded, I would like to notify the users (those who have Chrome opened to my web app) that something has happened, i.e. training completed. How should the Node.js be notified when the flow has finished (either successfully / failed)? Is there like a middleware / RabbitMQ / other message queue that the Node.js app can subscribe to, onto which Prefect publishes event? If not, does Prefect expose other broadcast events? And if not, should I poll periodically from my app and maintain state diff? Thanks!

Hello, I am new to prefect and investigating several use-cases to assess if Prefect would be a good fit in the toolbox to build a team of engineers dealing with data integration between our company's API and our customers systems. For some use-cases we have to periodically pull files from a remote SFTP server, do some manipulations with the file and some APIs then finally upload a sort of "response" file over the SFTP server. I couldn't find something related to SFTP in the built-in tasks list https://docs.prefect.io/api/latest/tasks/function.html What would be the best way to deal with SFTP file exchange with prefect ? Should I just use a custom function task and a python sftp library or is there a more suitable task ?

Can I return a python list from a Task implementation? I'm getting `TypeError: Task is not iterable. If your task returns multiple results, pass

nout

to the task decorator/constructor, or provide a

Tuple

return-type annotation to your task.` but I can't use

nout

because I don't know the result size and it is not a good fit for Tuple (since it's a list of tuple)

I’m starting to use

create_flow_run

,

wait_for_flow_run

, and

map

to fan-out via sub-flows. Is there any way to get more friendly task names using these tasks so I can more easily identify what child task is still running? Right now I have to go into the parent flow, look under logs for the URL for the created child flow run and then navigate separately. Ideally (maybe in Orion some day) it would be nice if the lineage between parent and child flows could be visualized; however, I realize this may be difficult thing to do. I’m just trying to find ways to make it as user friendly for an engineer to go in and understand which sub-flows failed or are running/etc.

Hey guys, I’m running a flow with a DaskTaskRunner that spawns a task that takes around 20seconds and apparently it’s running sequentially?! My machine has 16 threads and I see in the UI each task only being run after another one finishes. Any inputs?

23:04:54.057 | INFO    | prefect.engine - Created flow run 'optimal-snail' for flow 'test1'
23:04:54.057 | INFO    | prefect.task_runner.dask - Creating a new Dask cluster with `distributed.deploy.local.LocalCluster`
23:04:55.980 | INFO    | prefect.task_runner.dask - The Dask dashboard is available at <http://127.0.0.1:8787/status>
23:05:00.371 | INFO    | Flow run 'optimal-snail' - Created task run 'Execute values of the query-8165e3c8-0' for task 'Execute values of the query'
23:05:00.372 | INFO    | Flow run 'optimal-snail' - Executing 'Execute values of the query-8165e3c8-0' immediately...
23:05:24.612 | INFO    | Task run 'Execute values of the query-8165e3c8-0' - Finished in state Completed()
23:05:26.370 | INFO    | Flow run 'optimal-snail' - Created task run 'Execute values of the query-8165e3c8-1' for task 'Execute values of the query'
23:05:26.370 | INFO    | Flow run 'optimal-snail' - Executing 'Execute values of the query-8165e3c8-1' immediately...
23:05:42.533 | INFO    | Task run 'Execute values of the query-8165e3c8-1' - Finished in state Completed()
23:05:44.295 | INFO    | Flow run 'optimal-snail' - Created task run 'Execute values of the query-8165e3c8-2' for task 'Execute values of the query'
23:05:44.296 | INFO    | Flow run 'optimal-snail' - Executing 'Execute values of the query-8165e3c8-2' immediately...
23:06:09.538 | INFO    | Task run 'Execute values of the query-8165e3c8-2' - Finished in state Completed()

As you see in these logs it’s not really spawning more tasks. I was under the impression that the call to a function with the

@task

decorator would be non-blocking if made within a flow

@flow

which uses DaskTaskRunner

@Marvin How do I pass a kubernetes secret as an environment variable during deployment when using

.deploy()