I've got prefect server and all its components deployed and talking to each other, now trying to interact w/ it from the cli and running into 400 errors trying to do the first few steps (create tenant)

this issue to override the default apollo url was closed a few days ago but I am still unable to get this to default to anything other than localhost: https://github.com/PrefectHQ/ui/issues/99

Hello everyone! I have a question about having an option of running the same workflow both with the prefect server and without it. For example, when we just want to evaluate the correctness of the workflow and write some unit tests for it, we run it in a console without the server, using

LocalResult

. But then we'd like to register the same workflow on the server and run it there on a large scale, using

PrefectResult

and cache to the database. We're struggling to get the

LocalResult

and

PrefectResult

behave in the same way, so we could just switch between one and the other. Does this make sense at all - to have a switch between offline and server run? Or should one result class handle all cases, perhaps a custom one?

Hey, I want to contribute to an open pull Request https://github.com/PrefectHQ/prefect/pull/3330 The production Code seems fine to me and the core maintainers as well. However, due to a failed unittest this branch cannto be merged into master yet, so I wanted to fix this unittest. How should I push the Unittest-changes into this branch, since I want this feature as soon as possible. Do I have to create a new Branch (with identical changes + testfix) or can I somehow correct the Test and it will be merged soon?

Hi Prefect Community, just started going over prefect docs, blogs and videos. We are a big on-premise data company (in a highly regulated environment) and trying to see if prefect is a good solution for us. We are a big pyspark shop on CDH (Cloudera hadoop distro) and using ML (h2o). Looking at prefect to help us with mainly 2 things: 1. document data and state dependencies across our pyspark code, orchestrate pyspark based flows. Reading up on prefect, it seems like it can do these 2 things but a little lost on architecture of how this can be deployed. Basic questions - would prefect agent be running on the edge (hadoop gateway servers) and prefect backend server on K8s cluster? How would flow orchestration work? Would agent be kerberos aware? Sorry for lot of these basic questions - could not find info on prefect deployment in an on-prem setup with spark being the execution engine.

hi guys i have an empty flow

with Flow('flow', storage=docker_storage) as flow:
    passs

using docker storage

docker_storage=Docker(registry_url="<http://gcr.io/repo|gcr.io/repo>", dockerfile='Dockerfile')

where the dockerfile is just

FROM prefecthq/prefect:all_extras-0.13.7

and the agent logs show

[2020-09-23 09:01:30,367] INFO - agent | Pulling image <http://gcr.io/repo/flow:2020-09-23t08-59-44-707304-00-00|gcr.io/repo/flow:2020-09-23t08-59-44-707304-00-00>...
ERROR: gcloud failed to load: No module named '_sqlite3'
    gcloud_main = _import_gcloud_main()
    import googlecloudsdk.gcloud_main
    from googlecloudsdk.api_lib.iamcredentials import util as iamcred_util
    from googlecloudsdk.core.credentials import transports
    from googlecloudsdk.core.credentials import http
    from googlecloudsdk.core.credentials import creds as core_creds
    import sqlite3
    from sqlite3.dbapi2 import *
    from _sqlite3 import *

This usually indicates corruption in your gcloud installation or problems with your Python interpreter.

Please verify that the following is the path to a working Python 2.7 or 3.5+ executable:
    /opt/prefect/.local/share/virtualenvs/prefect-Ta6MxEs8/bin/python3

If it is not, please set the CLOUDSDK_PYTHON environment variable to point to a working Python 2.7 or 3.5+ executable.

If you are still experiencing problems, please reinstall the Cloud SDK using the instructions here:
    <https://cloud.google.com/sdk/>

so this seems like it is the agent not having proper GCR auth? how can i configure the docker agent for this? is there a way to configure the image of the agent like it is possible to configure the image for the flow?

Hi ! I've updated server, now all tasks running for more than 15min are killed by the ZombieKiller ! How can I disable it ?

I reverted back to 0.13.4 ...

Hi, I would like some help understanding the utility of the Docker image (https://hub.docker.com/r/prefecthq/prefect) I thought I could use this and run it to get a prefect environment running locally.. but trying to start the prefect server I get

No such file or directory: 'docker-compose': 'docker-compose'

Should docker-compose not be installed inside the image and this be a read-to go dev environment?

Hi. I just updated to prefect 0.13.8 and I'm seeing logs for graphql foreign key violations on starting the server. I cleaned up the existing docker images and volumes, and everything in ~/.prefect except config.toml and backend.toml, just to see if that changes anything, but I still see it.

Hi, I was trying to add a new cloud hook on Prefect Core for Web with URL as http://127.0.0.1:5000/alert to test this feature but it keeps complaining that it is invalid URL. Is it possible to test this feature in development environment on local host? Thanks

Hi all, we’re trying to figure out how to build the image of a Prefect Flow inside a Jenkins pipeline. We would want to avoid using Prefect directly to build the image, mainly because we don’t want to have Jenkins to depend on Prefect for building docker containers. Have you find yourself in such situation before? How did you manage to create Prefect docker images without relying on storage.build() and flow.register()? Do you think it is something doable?

Design question. Why did you choose to front Hasura with Apollo instead of A) the other way around or B) no Apollo at all, and just use Hasura and APIs? I've got a decision to make about a platform, and I'm amazed by Hasura. Can I bother you for a few moments of your time as to how Apollo ended up in the mix? Clearly a low priority, but I'm awfully curious. Thanks!

Hi, I have a master flow which triggers several child flows (using FlowRunTask). When looking at the logs of a specific run of the master flow I can see it triggered the child flows, but I can’t get from there to the child logs. Is there a way to enable it? In other words, I want to know if a flow was triggered by another flow or by itself. Is there a way to know that using the UI? Thanks!

Hello guys, I'm very happy to be heard. I have a simple question: does Prefect run on Spark Cluster? Because I didn't find that in a clear explanation and all the documentation is oriented Dask oriented. Can someone explain this to me? Thanks

Is there somewhere I can find more info about task health checks? I've got some flows failing because of them and it's not clear what might actually cause that. Are these healthchecks run in a thread from the flow process or something like that?

Hey Team! I see that the nomad-agent was deprecated; is there a particular reason this was the case?

Hey everyone, my teammate @Zaid Naji and I are trying to run flows using the Fargate agent. We're getting this error when we try to run a flow:

An error occurred (InvalidParameterException) when calling the RegisterTaskDefinition operation: Invalid 'cpu' setting for task.

The entrypoint we're using for the agent is:

["prefect","agent","start","fargate","cpu=256","memory=1024","networkConfiguration=$NETWORK_CONFIGURATION"],

Does anyone know what we should we be setting for

cpu

to get it working?

Hey, a basic stuff but wanted to confirm.. to be able to see and run the flow in my own infrastructure I need to register the flow in the cloud account? No way to use it without the account? I ran prefect server but cannot force to register flow to local instance

Hey Team! If i were running prefect server locally (not on cloud), how would i swap out the docker image of postgres with a postgres i already have running? Would it be a change in the following for docker-compose.yml: PREFECT_SERVER_DB_CMD: ${PREFECT_SERVER_DB_CMD:-"echo 'DATABASE MIGRATIONS SKIPPED'"} PREFECT_SERVER__DATABASE__CONNECTION_URL: ${DB_CONNECTION_URL} would this be recommended? or you think prefect should have its own dedicated postgres instance?

Hi, apologies for a super basic question but I can't seem to get prefect server to recognize the value of "server.ui.apollo_url" ... the line exists in the config.toml file, but the dashboard still shows it as "localhost" and suggests that the value isn't set. It seems to be the same issues as https://stackoverflow.com/questions/63719945/how-to-configure-graphql-url-in-prefect-server-0-13-5 but I just installed this today, and that issue is marked as resolved

hey, I want to deploy my flow using docker. I'm using the code from prefect docs : Flow with docker (first image attached). I do not push it to any registry as i want to run the docker image on my local machine. After starting the docker agent using

prefect agent start docker

i am able to register the flow to the UI but it always remains in pending state and never gets executed(refer images attached). Can someone explain where i am going wrong?

Hi, has anyone example code of a flow to run a docker image on a schedule ?

Hi 👋 If anyone has experience starting

prefect server start

via a running docker image, can you please help me fix an error I keep getting : / I summarized it here: https://github.com/PrefectHQ/prefect/discussions/3449

Hi! I have Prefect server & UI running locally (

prefect server start

) and my flows are executing via the agent (

prefect agent start

). I was wondering how I could get them to run on Dask? I have

dask-scheduler

and a few

dask-worker

processes running. When I run my flow using

flow.run(executor=DaskExecutor(address="<tcp://xxx:8786>"))

everything works well, but when I run the flow via prefect server’s UI it doesn’t run on Dask. What am I missing? I tried running the server and agent with

export PREFECT__ENGINE__EXECUTOR__DEFAULT_CLASS="prefect.engine.executors.DaskExecutor"

and

export PREFECT__ENGINE__EXECUTOR__DASK__ADDRESS="<tcp://xxx:8786>"

set but this didn’t help.

Other than running scheudling for the flows and acting as the api backbone for the prefect UI; the major roles that prefect server performs is to: 1. checkpoint tasks?

attempting to understand the scalability differences for single node prefect server vs prefect cloud

is the major scaling problem for concurrent flows, that the prefect server needs to be available to recieve checkpointing data and store it?

Hi, I am having a dependency issue and I am unsure whether anyone has encountered this before