• Levi Leal

    Levi Leal

    1 year ago
    Trying to add DaskExecutor to a flow I run on K8s. I'm currently using KubernetesRun and Docker for storage. When I add DaskExecutor I get and error while building the flow image. On the thread the code I'm using
    Levi Leal
    5 replies
    Copy to Clipboard
  • Josh Greenhalgh

    Josh Greenhalgh

    1 year ago
    Hey all - I am registering my flows with the following (to a server deployment on k8s);
    def register_deploy():
        # pylint: disable=missing-function-docstring
        with prefect.context(
            config={
                "cloud": {"graphql": os.environ["GRPAHQL_ENDPOINT"]},
                "server": {"ui": {"endpoint": os.environ["UI_ENDPOINT"]}},
            }
        ):
    
            for flow_name, flow in all_flows.items():
                <http://logging.info|logging.info>(f"Registering flow: {flow_name}")
                flow.register(project_name=PROD_PROJECT, labels=[SHARED_LABEL])
    This works (in terms of registering) but the stdout is the following;
    Result check: OK
    Flow URL: <http://localhost:8080/default/flow/a6013647-a31e-4d3c-9e7e-82e69c02f71e>
     └── ID: d78a0fc5-3990-461d-91d7-593a4278b6b2
     └── Project: prod
     └── Labels: ['prefect-agent']
    I thought setting
    config.server.ui.endpoint
    would lead to that endpoint being printed but it seems to not (
    localhost:8080
    as opposed to
    UI_ENDPOINT
    ) - any ideas?
    Josh Greenhalgh
    m
    12 replies
    Copy to Clipboard
  • Josh Greenhalgh

    Josh Greenhalgh

    1 year ago
    Hi Again; I am running into the following error when registering flows on CI (works perfectly from my local machine);
    Josh Greenhalgh
    nicholas
    13 replies
    Copy to Clipboard
  • Josh Greenhalgh

    Josh Greenhalgh

    1 year ago
    Haha Me again! So I am having some trouble with attaching
    Result
    classes to my flows. If I specify the result class in the same file that the flow is defined via
    flow.result = ...
    everything works as expected. However if I import the flow def and then attach the result class (same way just outside the flow def) I do not get any stored results. I have put together a minimal repo here -> https://github.com/josh-gree2/minimal-repo/compare/result-in-cli...master (the two branches show what I mean)
    Josh Greenhalgh
    Michael Adkins
    +1
    21 replies
    Copy to Clipboard
  • g

    Gustavo Silva

    1 year ago
    Hi everyone
  • g

    Gustavo Silva

    1 year ago
    I would like to ask how do I get more information to find the issue with a failed flow. In my logs I juts got
    Process PID 266 returned non-zero exit code
    g
    Kyle Moon-Wright
    +1
    6 replies
    Copy to Clipboard
  • d

    Dana Merrick

    1 year ago
    is there a way to edit the command on k8s jobs? it's
    prefect execute flow-run
    but I want to add a command before that
    d
    Kyle Moon-Wright
    12 replies
    Copy to Clipboard
  • c

    Chohang Ng

    1 year ago
    Could anyone help me understand updating flows? Do I always need to re-register all my flows and flow of flows? I built a script that register all my flows and flow of flow. However everytime I make an update to my flows, it takes a long time as if it is running a whole cycle on the backend. Am I doing it right?
    c
    Michael Adkins
    2 replies
    Copy to Clipboard
  • c

    Charles Leung

    1 year ago
    Hey Team! i have some general questions about secrets and script storage. When i use Gitlab Storage to register flows, it seems that they still look for secrets i plan to add onto the agent during execution; yet an error is thrown when i'm just registering the flow:
    ValueError: Local Secret "VAULT_TOKEN" was not found.
    Is this the expected behavior? should all secrets be registered wherever flows are created/registered?
    c
    Michael Adkins
    30 replies
    Copy to Clipboard
  • Josh Greenhalgh

    Josh Greenhalgh

    1 year ago
    Hello again! So I have run into the following issue - seems to have appeared out of nowhere since don't think it was happening yesterday... I am registering my flows on CI - get the following output;
    INFO:root:Registering flow: weather_data_etl
    Result check: OK
    Flow URL: <http://localhost:8080/default/flow/b366cae3-8200-42d9-87b5-f621711e897a>
     └── ID: 2570985c-efd0-4b29-88f1-96b86862d34a
     └── Project: prod
     └── Labels: ['prefect-agent']
    INFO:root:Registering flow: test_flow
    Result check: OK
    Flow URL: <http://localhost:8080/default/flow/96b48708-c731-4dd7-9ba0-f354b8ef7028>
     └── ID: 1307c478-cc0e-49ee-b4b8-b00a8834eed6
     └── Project: prod
     └── Labels: ['prefect-agent']
    So all seems good... However if I go to the UI there are no flows visible at all - that being said if I look into the postgres DB then the flows with the above IDs both exist...
    Josh Greenhalgh
    nicholas
    5 replies
    Copy to Clipboard