Darragh
05/14/2020, 10:20 AMDylan
05/14/2020, 1:19 PMflow.register
2. Are you building a docker container locally? If so, where are you pushing the image?
3. Make sure that apollo’s port is exposed on your Prefect Server instance
4. Would you mind sharing your .prefect/config.toml
configuration that points to your remote Prefect Server instance on your machine?[server]
[server.ui]
graphql_url = "INSERT_ENDPOINT_HERE:4200/graphql"
Darragh
05/14/2020, 1:28 PMDylan
05/14/2020, 1:28 PMSimon Basin
05/14/2020, 2:27 PMDylan
05/14/2020, 2:43 PMSimon Basin
05/14/2020, 2:47 PMDylan
05/14/2020, 2:58 PMSimon Basin
05/14/2020, 3:04 PMDarragh
05/14/2020, 3:17 PMDylan
05/14/2020, 3:18 PMDarragh
05/14/2020, 3:20 PMpip install prefect
right?Dylan
05/14/2020, 3:21 PM[server.ui]
that interact with prefect server, you’ll need to update the config.toml
on the machine where you’re running Prefect ServerDarragh
05/14/2020, 3:26 PM~./prefect/config.toml
[presumably this overrides the one instaled with Prefect?] with it’s own IP in it for the graphql endpoint?
• Local machine - flow.register() - what do I need to alter in my local Prefect installation to get that flow registered with http://EC2_BOX ??nicholas
05/14/2020, 3:38 PMconfig.toml
installed with the package, you'll only need to modify the one at `~/.prefect/config.toml`; this may not be created automatically so you may have to mkdir ~/.prefect && touch ~/.prefect/config.toml
2. You're correct that your EC2 Prefect installation will need its own ~/.prefect/config.toml
(the one you just created), in which you'll override the graphql_url
in the way @Dylan posted above. As you said, this will be the IP address of the EC2 you've installed the stack on (basically it'll direct the UI to itself)
3. On your local machine, you'll run prefect backend server
and point prefect to your remote installation. You can do this by running export PREFECT___CLOUD__API=<http://EC2_BOX>_
Darragh
05/14/2020, 3:39 PMnicholas
05/14/2020, 3:40 PM<http://123.456.789.1111:4200/graphql>
instead of <tel:1234567891111|123.456.789.1111>:4200/graphql
• if you're spinning up the Prefect Server docker network anywhere but locally, including in a VM or a remote machine with AWS, GCP, Azure etc, you'll need to let the Prefect Server installation know where it's installed so that when you access the UI in your browser (the client), it knows where to look for its APISimon Basin
05/14/2020, 3:49 PMPREFECT__CLOUD__API=_
nicholas
05/14/2020, 3:50 PMPREFECT___CLOUD___API
, Slack likes to interpret my underscores as italics <.<Simon Basin
05/14/2020, 4:06 PMnicholas
05/14/2020, 4:10 PMSimon Basin
05/14/2020, 4:11 PMnicholas
05/14/2020, 4:12 PMprefect agent start --name "Super Awesome Local Agent" --label Nicholas-Little-Mac.local
Simon Basin
05/14/2020, 4:16 PMnicholas
05/14/2020, 4:17 PMprefect backend server
and it'll know to poll localhost:4200/graphql
Simon Basin
05/14/2020, 4:19 PMnicholas
05/14/2020, 4:22 PM--verbose
flag? That way we can see the outputSimon Basin
05/14/2020, 4:39 PMnicholas
05/14/2020, 4:42 PMWaiting flow flow runs...
line?Simon Basin
05/14/2020, 4:45 PMDylan
05/14/2020, 4:50 PMnicholas
05/14/2020, 4:50 PM[2020-05-14 16:40:55,812] DEBUG - Version Group Tests | Verbose logs enabled
[2020-05-14 16:40:55,812] DEBUG - Version Group Tests | Environment variables: []
[2020-05-14 16:40:55,812] DEBUG - Version Group Tests | Max polls: None
[2020-05-14 16:40:55,812] DEBUG - Version Group Tests | Agent address:
[2020-05-14 16:40:55,812] DEBUG - Version Group Tests | Prefect backend: server
[2020-05-14 16:40:56,155] DEBUG - Version Group Tests | Import paths: []
[2020-05-14 16:40:56,155] DEBUG - Version Group Tests | Show flow logs: False
____ __ _ _ _
| _ \ _ __ ___ / _| ___ ___| |_ / \ __ _ ___ _ __ | |_
| |_) | '__/ _ \ |_ / _ \/ __| __| / _ \ / _` |/ _ \ '_ \| __|
| __/| | | __/ _| __/ (__| |_ / ___ \ (_| | __/ | | | |_
|_| |_| \___|_| \___|\___|\__| /_/ \_\__, |\___|_| |_|\__|
|___/
[2020-05-14 16:40:56,155] INFO - Version Group Tests | Starting LocalAgent with labels ['Nicholas-Little-Mac.local', 'NicholaittleMac.fios-router.home', 'azure-flow-storage', 'gcs-flow-storage', 's3-flow-storage']
[2020-05-14 16:40:56,156] INFO - Version Group Tests | Agent documentation can be found at <https://docs.prefect.io/orchestration/>
[2020-05-14 16:40:56,156] INFO - Version Group Tests | Agent connecting to the Prefect API at <https://localhost:4200/graphql>
Simon Basin
05/14/2020, 4:56 PMDylan
05/14/2020, 5:10 PMSimon Basin
05/14/2020, 5:32 PMDylan
05/14/2020, 6:05 PMSimon Basin
05/14/2020, 6:55 PMDylan
05/14/2020, 7:07 PMDarragh
05/14/2020, 7:43 PMnicholas
05/14/2020, 7:50 PM:4200/graphql
; that port (4200) will need to be opened to outside traffic.
As for logging, perfect server start
has a lot of output, including network traffic to the APi, which should prove useful. Unfortunately I don't believe the .register
command has any additional loggingDarragh
05/14/2020, 7:51 PMnicholas
05/14/2020, 7:52 PMDarragh
05/14/2020, 7:55 PMnicholas
05/14/2020, 7:55 PMDylan
05/14/2020, 7:56 PMDarragh
05/14/2020, 8:10 PMDylan
05/18/2020, 3:40 PM378 Flow: <http://localhost:8080/flow/87adb8df-c3d1-4745-af0b-31752f1091b3>
has localhost in the URLhttp://{IP dotted quad}/flow/{uuid}
nicholas
05/18/2020, 3:57 PMDarragh
05/18/2020, 4:05 PMnicholas
05/18/2020, 4:37 PMPREFECT_CLOUD_API
. 2) Since this is using server, the output of flow.regsiter
is based on the prefect.config.server.ui.endpoint
var, which defaults to localhost
. This isn't a true reflection of where the UI is deployed though, since Prefect has no way of knowing unless that variable is explicitly set.Darragh
05/18/2020, 4:40 PMnicholas
05/18/2020, 4:41 PMgraphql_url
variable before starting it), you should be able to see the flow thereDarragh
05/18/2020, 4:45 PMnicholas
05/18/2020, 6:02 PMDarragh
05/18/2020, 6:05 PMnicholas
05/18/2020, 6:06 PM