• s

    Saoirse Amarteifio

    2 years ago
    hi there. ive just started playing around with prefect and having fun. i have set up various working examples e.g. connecting to static and dynamics dask execution environments. I am a little confused about how i would combine dask/prefect-server + ui on kubernetes and submit flows from a client (e.g. connecting to a dask scheduler running on kubernetes and being able to monitor them in a UI). The UI could be running anywhere i guess im just not sure how this fits together. prefect-server needs docker compose etc so i guess we dont run this on kubernetes? anyway, a little lost on this monitoring part and not even sure what is the right question. would love to hear folks thoughts on this.
    s
    Chris White
    2 replies
    Copy to Clipboard
  • Vikram Iyer

    Vikram Iyer

    2 years ago
    Hey Guys, I have recently started evaluating prefect for my ML work at my current company. I have a few questions around "how to" use prefect for my local deployment. Setup:- I have a backend service that runs inside a docker container on a host machine. Prefect server (prefect server start) and agent (prefect agent start) run on the same host. Question:- How do I register flows from within my container to the server on host and make them run from UI? So, when the service is running on the host I can simple do the below steps
    1. flow.register() # this will register the flow and hence I can see it on the UI
    2. From the UI, I can go to the particular flow and run it. # easy
    How do I do something similar, with the only exception of the service and code residing inside a docker container? End Notes:- I might be looking at it from a completely wrong perspective, please correct my understanding if necessary. Cheers! Vikram
    Vikram Iyer
    Kyle Moon-Wright
    2 replies
    Copy to Clipboard
  • r

    Rob Harrop

    2 years ago
    Hello all. I'm trying to connect the CLI to our Prefect Cloud. I have version 0.12.1 for the CLI and I'm getting this odd error:
    Traceback (most recent call last):
      File "flow.py", line 38, in <module>
        main()
      File "flow.py", line 36, in main
        flow.register(**register_params)
      File "/Users/rdh/Library/Caches/pypoetry/virtualenvs/pipelines-3RvCBXzu-py3.8/lib/python3.8/site-packages/prefect/core/flow.py", line 1479, in register
        registered_flow = client.register(
      File "/Users/rdh/Library/Caches/pypoetry/virtualenvs/pipelines-3RvCBXzu-py3.8/lib/python3.8/site-packages/prefect/client/client.py", line 687, in register
        res = self.graphql(
      File "/Users/rdh/Library/Caches/pypoetry/virtualenvs/pipelines-3RvCBXzu-py3.8/lib/python3.8/site-packages/prefect/client/client.py", line 226, in graphql
        raise ClientError(result["errors"])
    prefect.utilities.exceptions.ClientError: [{'path': ['create_flow_from_compressed_string'], 'message': '[{\'extensions\': {\'path\': \'$.variableValues\', \'code\': \'validation-failed\'}, \'message\': "no such type exists in the schema: \'flow_group_insert_input\'"}]', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]
    r
    Zachary Hughes
    4 replies
    Copy to Clipboard
  • b

    bruno.corucho

    2 years ago
    Hello everyone! TL;DR (and probably newbish question) I'm having some issues using Docker Storage. I did a mockup flow, and it runs fine locally, but after setting the flow.storage = Docker() and try to register it, it cannot find the modules I'm trying to import in my script. I presumed it was something regarding the current sys.path not being set to the project's root directory? (as opposed to running it locally). What I'm doing:
    1. prefect backend server #switch to local server
    2. prefect server start
    3. python register_flow.py (imports * + flow.storage = Docker(...) `flow.register(name)) #in the future would like to set up registry_url = my_aws_ECR
    4. prefect agent start docker #Start a docker agent
    Currently following this documentation.
    b
    1 replies
    Copy to Clipboard