• a

    Adam Roderick

    2 years ago
    Does Docker storage attempt to authenticate with the docker registry? I'm getting a "no basic auth credentials" error from
    flow.register()
    a
    nicholas
    2 replies
    Copy to Clipboard
  • a

    Adam Roderick

    2 years ago
    Hi, I'm trying to work through flow registration with cloud using Docker storage
    a
    Chris White
    +1
    17 replies
    Copy to Clipboard
  • i

    Ivan Shumilin

    2 years ago
    Hello! Does Prefect has notion of priority? Something like priority_weight in Airflow. So I could raise some flow runs.
    i
    Jeremiah
    3 replies
    Copy to Clipboard
  • Arsenii

    Arsenii

    2 years ago
    Where can I submit small Cloud UI-related bugs like this? Clicking on
    Cloud Hooks
    here opens https://docs.prefect.io/orchestration/concepts/cloud-hooks.html while the correct URL is https://docs.prefect.io/orchestration/concepts/cloud_hooks.html (notice the
    _
    instead of
    -
    )
    Arsenii
    Jeremiah
    +1
    4 replies
    Copy to Clipboard
  • r

    Rafal

    2 years ago
    Do you know where this error comes from?
    r
    j
    +2
    26 replies
    Copy to Clipboard
  • m

    Marwan Sarieddine

    2 years ago
    Hello everyone - I think I found two issues in
    prefect install agent kubernetes
    --namespace
    only applies to the rbac in the generated manifest but not to the deployment - [i.e agents always gets deployed to default namespace] • Can’t specify a
    --name
    to the cli call - so the agent’s name always gets registered as
    agent
    - apparently this is done in
    prefect agent start
    - I stand corrected
    m
    Dylan
    9 replies
    Copy to Clipboard
  • m

    Michael Reeves

    2 years ago
    Hi all, I've started using prefect and I think its a great api for managing workflows (I see it as a human friendly wrapper around dask orchestration). Thanks for excellent documentation and a super useful tool! I'm trying to determine how best to use prefect within an orchestration workflow design. The workflow is essentially three generic steps:1. spinning up various services (various web servers and processes) 2. manage these services: state, variables/properties, tasks (prefect seems to fit this, although I may need to find a good way to store state/data) 3. detect failure/errors in services and trigger redundancy/recovery tasks I am torn on how to use prefect with 1 and 3. I don't think using prefect alone is the best solution, but I also think prefect(dask scheduler/executor) is a good solution for starting services (1) and handling triggers (3). However I'm not sure if prefect is ideal for active monitoring. Is there a prefect usecase for monitoring log files/services, or if there is a better tool for this job? I didn't see any mention of monitoring within the prefect core. I'm not talking about monitoring specific prefect tasks which is obvious that prefect should monitor its own tasks, more so on monitoring errors/issues in services. If prefect isn't ideal for active monitoring of services, I envision using something like prometheus to watch data/logs from the services and trigger redundancy/recovery tasks in prefect if prometheus alerts or detects failures/issues. Are there any ideas for a better integration of prefect into this workflow?
    m
    Dylan
    +1
    6 replies
    Copy to Clipboard
  • g

    Geoffrey Gross

    2 years ago
    Hey everyone got a question. I am currently working on building a flow via the
    Docker
    storage class. Right now my flow imports env vars from a
    config.py
    , found in the same directory with the file that holds my flow, which uses
    os.environ
    . I would like to set those environment variables at run time but when, the healtchecks run and deserialize my flow in the docker build step, it is trying to evaluate those environment variables. This causes an error since I don't have defaults set. Does anyone have any ideas of have to handle this without ignoring healthchecks and putting environment variables into the Dockerfile that gets generated?
    g
    Dylan
    15 replies
    Copy to Clipboard
  • Scott Zelenka

    Scott Zelenka

    2 years ago
    When using a KubernetesExecutor, is it possible to have dynamic resource allocation? We can currently use the
    job_spec
    file to specify the K8 resources. But we have a use case where, in most situations the volume of work expected from our Flow fits within the specified resources, occasionally an input Parameter to the Flow will be such that it requires more resources. We're getting by this today by setting the resources to the maximum value, but (in theory) that restricts the number of concurrent Flows we can execute in the same environment. Curious if there's an ability to adjust the memory resources required in the
    job_spec
    based on the value of an input Parameter to the FlowRun?
    Scott Zelenka
    1 replies
    Copy to Clipboard
  • Darragh

    Darragh

    2 years ago
    Morning all! Curiosity question, anyone tried cross cloud platform usage? I.e. host Prefect server and agent on AWS and run a flow on Digital Ocean?
    Darragh
    nicholas
    5 replies
    Copy to Clipboard