Timo
08/26/2022, 1:59 PMRyan Peden
08/26/2022, 2:20 PMprefect orion start
only runs the Prefect API and UI.
If you want flows that run remotely, you will need to set up remote storage for your flow, create a deployment, and then start an agent that connects to your deployment's work queue.
You can start an agent on the same machine as the API and UI if you want to, or you can run the agent on a different machine. Just make sure you run prefect config set PREFECT_API_URL=<http://your-prefect-url>
so the agent knows how to reach the API server.Timo
08/26/2022, 2:24 PMprefect orion start
on the remote VM. If I call it on the browser I see the exact same things (flow runs, blocks...) like if I run it on my local machine. I expected an empty environment when I access the remote instance.Timo
08/26/2022, 2:27 PMRyan Peden
08/26/2022, 2:44 PMprefect config view
on your local machine, does it show the address of the remote Orion server for PREFECT_API_URL?
The reason I ask is that if you run flows locally with PREFECT_API_URL set to the remote Orion server's address, your local flow run results will be sent to the remote Orion server and will show up there.Timo
08/26/2022, 2:56 PMTimo
08/26/2022, 2:57 PMTimo
08/26/2022, 3:01 PMprefect orion start --port 4444 --host 0.0.0.0
If I look at the settings page on the REMOTE server it prints out this:
Orion Settings
PREFECT_HOME: ~/.prefect
PREFECT_DEBUG_MODE: false
PREFECT_CLI_COLORS: true
PREFECT_CLI_WRAP_LINES: true
PREFECT_TEST_MODE: false
PREFECT_TEST_SETTING:
PREFECT_API_URL:
PREFECT_API_KEY:
PREFECT_CLOUD_URL: <https://api.prefect.cloud/api>
PREFECT_API_REQUEST_TIMEOUT: 30
PREFECT_PROFILES_PATH: ${PREFECT_HOME}/profiles.toml
PREFECT_LOCAL_STORAGE_PATH: ${PREFECT_HOME}/storage
PREFECT_LOGGING_LEVEL: INFO
PREFECT_LOGGING_SERVER_LEVEL: WARNING
PREFECT_LOGGING_SETTINGS_PATH: ${PREFECT_HOME}/logging.yml
PREFECT_LOGGING_EXTRA_LOGGERS:
PREFECT_LOGGING_ORION_ENABLED: true
PREFECT_LOGGING_ORION_BATCH_INTERVAL: 2
PREFECT_LOGGING_ORION_BATCH_SIZE: 4000000
PREFECT_LOGGING_ORION_MAX_LOG_SIZE: 1000000
PREFECT_AGENT_QUERY_INTERVAL: 5
PREFECT_AGENT_PREFETCH_SECONDS: 10
PREFECT_ORION_DATABASE_PASSWORD:
PREFECT_ORION_DATABASE_CONNECTION_URL: sqlite+aiosqlite:///${PREFECT_HOME}/orion.db
PREFECT_ORION_DATABASE_ECHO: false
PREFECT_ORION_DATABASE_MIGRATE_ON_START: true
PREFECT_ORION_DATABASE_TIMEOUT: 1
PREFECT_ORION_DATABASE_CONNECTION_TIMEOUT: 5
PREFECT_ORION_SERVICES_SCHEDULER_LOOP_SECONDS: 60
PREFECT_ORION_SERVICES_SCHEDULER_DEPLOYMENT_BATCH_SIZE: 100
PREFECT_ORION_SERVICES_SCHEDULER_MAX_RUNS: 100
PREFECT_ORION_SERVICES_SCHEDULER_MAX_SCHEDULED_TIME: 8640000
PREFECT_ORION_SERVICES_SCHEDULER_INSERT_BATCH_SIZE: 500
PREFECT_ORION_SERVICES_LATE_RUNS_LOOP_SECONDS: 5
PREFECT_ORION_SERVICES_LATE_RUNS_AFTER_SECONDS: 5
PREFECT_ORION_API_DEFAULT_LIMIT: 200
PREFECT_ORION_API_HOST: 127.0.0.1
PREFECT_ORION_API_PORT: 4200
PREFECT_ORION_UI_ENABLED: true
PREFECT_ORION_UI_API_URL:
PREFECT_ORION_ANALYTICS_ENABLED: true
PREFECT_ORION_SERVICES_SCHEDULER_ENABLED: true
PREFECT_ORION_SERVICES_LATE_RUNS_ENABLED: true
PREFECT_ORION_SERVICES_FLOW_RUN_NOTIFICATIONS_ENABLED: true
Timo
08/26/2022, 3:06 PMRyan Peden
08/26/2022, 3:54 PMprefect orion database reset -y
on the remote Orion instance?Timo
08/27/2022, 7:42 AMprefect block type ls
I can see all the default blocks in the terminal but they are not visible in the UI. I tried different browsers and even tried to access the instance from a remote terminal server (same behavior). I noticed in the browser console that it tries to connect to the api of localhost. This doesn't seem to be right (see image in the post below).
2. After starting the local instance I see all the flow_runs, blocks etc. of my local instance even if I access the URL of the remote
Again I started prefect orion start --host 0.0.0.0 --port 8445
on the remote server. I also resetted the remote database with no success.Timo
08/27/2022, 7:53 AMTimo
08/27/2022, 8:06 AMTimo
08/29/2022, 5:22 AMprefect config set PREFECT_API_URL="http://<<server ip/url>>:<<server port>>/api"
on the remote machine it works now. 🎉