https://prefect.io logo
#prefect-community
Title
# prefect-community
l

Lewis Bails

06/18/2020, 5:11 PM
Hello! I'm trying to get Prefect going on an on-prem server and I'm having some trouble getting the API to connect (it never connects, just keeps spinning) I assume I'm doing something wrong in my config.toml?
n

nicholas

06/18/2020, 5:13 PM
Hi @Lewis Bails - you'll want to add:
Copy code
[server.ui]
graphql_url = "<http://10.10.0.40:4200/graphql>"
so that the UI knows where the API is 🙂
l

Lewis Bails

06/18/2020, 5:23 PM
Thanks for the swift reply. I've changed my config.toml to this? Does this look better? Still having an issue connecting. I removed the docker images and ran "prefect server start". The flow was registered at localhost still, is something wrong there?
n

nicholas

06/18/2020, 5:28 PM
Hm interesting; is the
config.toml
you're showing on the server? Or on your local machine?
l

Lewis Bails

06/18/2020, 5:29 PM
on the server
n

nicholas

06/18/2020, 5:37 PM
Got it, can you share your local config as well? I'm not sure why the server config wouldn't be applying, that looks correct.
l

Lewis Bails

06/18/2020, 5:48 PM
message has been deleted
Copy code
# debug mode
debug = false

# base configuration directory (typically you won't change this!)
home_dir = "~/.prefect"

backend = "cloud"

[server]
host = "<http://localhost>"
port = "4200"
host_port = "4200"
endpoint = "${server.host}:${server.port}"

    [server.database]
    host = "localhost"
    port = "5432"
    host_port = "5432"
    name = "prefect_server"
    username = "prefect"
    # set to "" to generate a random password each time the database starts
    password = "test-password"
    connection_url = "postgresql://${server.database.username}:${server.database.password}@${server.database.host}:${server.database.port}/${server.database.name}"
    volume_path = "${home_dir}/pg_data"

    [server.graphql]
    host = "0.0.0.0"
    port = "4201"
    host_port = "4201"
    debug = false
    path = "/graphql/"

    [server.hasura]
    host = "localhost"
    port = "3000"
    host_port = "3000"
    admin_secret = "" # a string. One will be automatically generated if not provided.
    claims_namespace = "hasura-claims"
    graphql_url = "http://${server.hasura.host}:${server.hasura.port}/v1alpha1/graphql"
    ws_url = "ws://${server.hasura.host}:${server.hasura.port}/v1alpha1/graphql"
    execute_retry_seconds = 10

    [server.ui]
    host = "<http://localhost>"
    port = "8080"
    host_port = "8080"
    endpoint = "${server.ui.host}:${server.ui.port}"
    graphql_url = "<http://localhost:4200/graphql>"

    [server.telemetry]
    enabled = true

[cloud]
api = "${${backend}.endpoint}"
endpoint = "<https://api.prefect.io>"
graphql = "${cloud.api}/graphql/alpha"
use_local_secrets = true
heartbeat_interval = 30.0
diagnostics = false

# rate at which to batch upload logs
logging_heartbeat = 5

queue_interval = 30.0

    [cloud.agent]
    name = "agent"
    labels = "[]"

    # Set to `DEBUG` for verbose logging
    level = "INFO"

    # Agents require different API tokens
    auth_token = ""

    # Internal address for agent health checks, etc...
    agent_address = ""

        [cloud.agent.resource_manager]
        # Separate loop interval for resource managers
        loop_interval = 60


[logging]
# The logging level: NOTSET, DEBUG, INFO, WARNING, ERROR, or CRITICAL
level = "INFO"

# The log format
format = "[%(asctime)s] %(levelname)s - %(name)s | %(message)s"

# additional log attributes to extract from context
# e.g., log_attributes = "['context_var']"
log_attributes = "[]"

# the timestamp format
datefmt = "%Y-%m-%d %H:%M:%S"

# Send logs to Prefect Cloud
log_to_cloud = false

# Extra loggers for Prefect log configuration
extra_loggers = "[]"

[flows]
# If true, edges are checked for cycles as soon as they are added to the flow. If false,
# cycles are only checked when tasks are sorted (for example, when running or
# serializing the flow). Defaults to false because it can affect the performance of
# large flows.
eager_edge_validation = false
# If true, `flow.run` will run on schedule by default.
# If false, only a single execution will occur (no retries, etc.)
run_on_schedule = true
# If true, tasks which set `checkpoint=True` will have their result handlers called
checkpointing = false

    [flows.defaults]
        [flows.defaults.storage]

        # the default storage class, specified using a full path
        default_class = "prefect.environments.storage.Local"

[tasks]

    [tasks.defaults]
    # the number of times tasks retry before they fail.
    # false indicates that tasks should never retry (equivalent to max_retries = 0)
    max_retries = false

    # the amount of time tasks should wait before retrying, in seconds.
    # false indicates that tasks have no default value (users must specify one to set it)
    retry_delay = false


[engine]

    [engine.executor]

    # the default executor, specified using a full path
    default_class = "prefect.engine.executors.LocalExecutor"

        [engine.executor.dask]
        # the default scheduler address for the DaskExecutor.
        address = ""

        # the default Cluster class to use to create a temporary dask cluster
        cluster_class = "distributed.deploy.local.LocalCluster"

    [engine.flow_runner]
    # the default flow runner, specified using a full path
    default_class = "prefect.engine.flow_runner.FlowRunner"

    [engine.task_runner]
    # the default task runner, specified using a full path
    default_class = "prefect.engine.task_runner.TaskRunner"
Hmm, I may have found the problem. I dont think ~ isnt pointing to the dir with the right home. I'm running within a conda environment
I'll let you know if that fixes it
n

nicholas

06/18/2020, 5:51 PM
Ah that would explain it.
l

Lewis Bails

06/18/2020, 6:10 PM
j

justin

06/18/2020, 6:25 PM
I have the same issue. Where would we find the config.toml?
Is this the same as the
backend.toml
in
~/.prefect
@Lewis Bails Should you backend say cloud? even though you are on-prem?
l

Lewis Bails

06/18/2020, 6:36 PM
I call "prefect backend server" before "prefect server start", so I don't think that's my problem. I don't think backend.toml is the same thing. I had to create config.toml at ~/.prefect, but then, in desperation, changed the config.toml in my conda environment site-packages/prefect dir. I haven't been able to solve my problem.
j

justin

06/18/2020, 6:37 PM
There is a similar discussion between Rafal and nicholas above
n

nicholas

06/18/2020, 6:44 PM
It's not the same as
backend.toml
, and @Lewis Bails is correct, this is one you'll need to create yourself.
l

Lewis Bails

06/18/2020, 6:47 PM
@nicholas am I right in saying the config at ~ will override fields in the base config I pasted above?
n

nicholas

06/18/2020, 6:54 PM
Let me walk through some steps and see if these match up with what you've done: 1. On your server, Install Prefect
pip install prefect
or
conda install -c conda-forge prefect
2. Also on the server, create and modify
~/.prefect/config.toml
, which should include the following:
Copy code
[server]
  [server.ui]
  graphql_url = "<http://your_server:4200/graphql>"
3. Still on the server, run
prefect server start
4. Back to your local machine, also install prefect using the steps above, creating a new
config.toml
, which should include the following:
Copy code
backend = "server" # this would also be set by running `prefect backend server`

[server]
host = "<http://your_server> # tells prefect where your flows should be available after registration
Now when you register a flow from your local machine, it should be registering with your server deployment; it'll display a URL as such. When in the UI, if you've followed the steps above, the API Healthcheck tile should show your server url as the endpoint it's trying to hit.
Yes that's correct @Lewis Bails
l

Lewis Bails

06/18/2020, 6:58 PM
In that case I'm pretty lost haha. I can ping http://10.10.0.40:4200 and get the "GET query missing" message, but I cant get the UI to look at anything but http://localhost:4200/graphql for the api. blurg.
n

nicholas

06/18/2020, 6:59 PM
Does my message above match with what your deployment looked like?
l

Lewis Bails

06/18/2020, 7:02 PM
oh sorry didnt see that message @nicholas. i was registering flows from within the server. i.e. develop flow scripts on my machine, move them to the server, and register them there. I'll do it this way and see how I go.
n

nicholas

06/18/2020, 7:03 PM
Ah in which case, the output was correct, right? The flows are available at localhost when you're on the server.
j

justin

06/18/2020, 7:03 PM
I noticed this in the docker-compose on github. It is sending graphql to 4201
n

nicholas

06/18/2020, 7:04 PM
But either way it looks like something's up with your server config, because the UI should display whatever you've set the server config
graphql_url
to (after you've restarted the server)
@justin the
graphql_url
in this case is a bit of a misnomer; the client is really what we're after from an API-standpoint, which would be the Apollo GraphQL client (at 4200)
j

justin

06/18/2020, 7:05 PM
ah ok
l

Lewis Bails

06/18/2020, 7:05 PM
Yeah I think the message from the flow registration is okay.. maybe, im not sure
So here's my ~/.prefect/config.toml I removed the docker ui image, too
Copy code
[server]
    [server.ui]
    graphql_url = "<http://10.10.0.40:4200/graphql>"
n

nicholas

06/18/2020, 7:19 PM
Nice, that's the config on your server, right? Not local?
l

Lewis Bails

06/18/2020, 7:19 PM
Correct
aaaand its connected.
n

nicholas

06/18/2020, 7:20 PM
Great, so when you run
prefect server start
, you should see a message in the startup output something like "Replacing Graphql references with 10.10.0.40:4200/graphql"
oh great!
l

Lewis Bails

06/18/2020, 7:22 PM
..and he never shut the server down again I joke. Thanks for your patience.
n

nicholas

06/18/2020, 7:23 PM
No worries! Networking is a pain in the ass in any scenario
Prefect has a lot of moving parts!
a

Anish Chhaparwal

09/10/2020, 7:02 AM
@Lewis Bails  @nicholas i'm facing the exact issue to the one you guys have discussed. i have the server up and running from a ec2 instance (not at the default port but from 8888, and graphql port 4200). i have follow all the step but nothing works out. There is one slight difference though, when i look at the API status from UI it says attempting to connect httphttpMy_IP_Addess_:4200/graphql. i am attach an image for clarity. secondly i did not delete the UI image. can you elaborate if you delete only the UI image or the entire list of prefect images on docker. Also post deleting it do i need to reinstall anything?
n

nicholas

09/10/2020, 3:53 PM
Hi @Anish Chhaparwal - take a look at the URL displayed in your API tile: it's not a valid URL (it has 2 http protocols)
a

Anish Chhaparwal

09/10/2020, 4:16 PM
@nicholas I did notice that but am not able to set it right. I am attaching an image of my config.toml. Another attempt was to remove the prefix http:// and just put my graphql_url as my_ip_address:4200/graphql but that doesnt work either. Do you have any further suggestions on how i can get rid of the http:// appearing twice?
n

nicholas

09/10/2020, 4:36 PM
@Anish Chhaparwal - this thread is out of date and is dealing with an older version of Prefect, so that way of setting the endpoint is out of date. If you're on Prefect 0.13.0+ you'll be able to set your endpoint (and test it) from the
Home
page of the UI
a

Anish Chhaparwal

09/10/2020, 5:14 PM
@nicholas I am using prefect 0.13.5 and do get the option of setting graphql endpoint but doesnt work from their either. (ref attached image)
n

nicholas

09/10/2020, 5:17 PM
Can you visit that endpoint (http://15.207.55.236:4200/graphql) in your browser? If not, Apollo isn't exposed correctly on whatever machine you're running Prefect Server. This could be something you need to configure with your Cloud provider by exposing that port on your machine.
a

Anish Chhaparwal

09/10/2020, 6:15 PM
No i cannot open it in my browser. I can't find the default port for apollo. Do the --server-port tags in prefect server start refer to apollo? Could you list out all the ports that are suppose to be open by default so i can configure my cloud provider. Alternately you can also suggest how I could set up apollo endpoint using the cli. I'm new to the community and would like to let you know you patience and speedy reply is grately appreciated! :)
n

nicholas

09/10/2020, 6:35 PM
@Anish Chhaparwal - would you mind starting a new thread? I don't want to keep bothering others in this thread.