• m

    Marko Jamedzija

    1 year ago
    Hello 🙂 I recently deployed Prefect Server 0.14.15 on a test k8s cluster. After playing around with it I noticed that k8s agent only runs the flows for the first tenant I created. For the projects I created from other tenants, all their flows are just stuck in
    scheduled
    state (becoming late runs). Is this a bug, or there’s a way of configuring k8s agent to work with more than one tenant? If so, how? Thanks 🙂 (I deployed the services using the prefecthq helm chart)
    m
    Kevin Kho
    +1
    6 replies
    Copy to Clipboard
  • Brent Bateman

    Brent Bateman

    1 year ago
    Who's the admin for this workspace? I want to leave it/ not just sign out of it?
    Brent Bateman
    Dylan
    4 replies
    Copy to Clipboard
  • Joe McDonald

    Joe McDonald

    1 year ago
    So what is the chance of getting official PrefectHQ docker images hosted on https://public.ecr.aws/ so that ECS services can avoid the docker hub rate limits? Would be nice to convince Hasura to do the same thing…
    Joe McDonald
    Kevin Kho
    5 replies
    Copy to Clipboard
  • t

    Trevor Kramer

    1 year ago
    I'm confused about how to use @resource_manager effectively. I am using it to create an RDS instance in AWS. I can create it fine in setup and destroy it in cleanup but I don't understand how to get the endpoint out of it. It seems to wrap it in a ResourceManager so I can't access any attributes of my class from the flow. How can I use the client as instantiated here
    with OracleRDS() as client
    ?
    t
    Kevin Kho
    13 replies
    Copy to Clipboard
  • Oussama Louati

    Oussama Louati

    1 year ago
    Hello 🙂, I deployed Prefect server 0.14.14. Everything works fine locally. I am a bit confused so if anyone can help me: • I have a workload of Machine learning tasks That i would like to execute inside a docker container • I have everything packaged inside a container • I have an agent running on my machine "prefect agent docker start --show-flow-logs -l agent-name --no-pull" • When i do
    docker run my_image
    , i get this error each time:
    Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/root/'")
    Should i run the agent inside the container ? Thank you
    Oussama Louati
    Kevin Kho
    87 replies
    Copy to Clipboard
  • Paul Prescod

    Paul Prescod

    1 year ago
    This may be outside of the design criteria for Prefect. The use-case consists of two tasks. Task A is CPU-bound. Task B is network bound. Task A generates data that Task B uploads to a slow server. I want a pool of Task A workers to feed a pool of Task B workers. For now, they are all subprocesses of a single parent process. The number of Task A processes should be the same as the number of Cores (not including "hyperthreads"). I'm more flexible about the number of Task B processes.
    Paul Prescod
    Kevin Kho
    8 replies
    Copy to Clipboard
  • Hygor Knust

    Hygor Knust

    1 year ago
    Is there a way to deploy custom Kubernetes CRDs using the K8s Task Library? I'm trying to deploy a SparkApplication CRD but it seems that there is no task that accomplishes this. Should I use the kubernetes-python inside a Python Task instead? Thanks
    Hygor Knust
    Kevin Kho
    8 replies
    Copy to Clipboard
  • j

    Jonathan Buys

    1 year ago
    Is there a way to setup SSO with Office365 in Prefect Cloud?
    j
    Dylan
    4 replies
    Copy to Clipboard
  • x

    xyzy

    1 year ago
    I tried writing a SQLiteConnection ResourceManager as a demo, but when executing with DaskExecutor it leads to problems with having different threads:
    ProgrammingError('SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 8032 and this is thread id 2308.')
    Is there a way to work around this other than using a LocalExecutor? e.g. a parameter for the ResourceManager to tell it to keep everything that uses it in the same thread?
    x
    Dylan
    12 replies
    Copy to Clipboard
  • Josh

    Josh

    1 year ago
    Not sure if this should be considered a feature request or discussion idea, but I would hope for updates to patch versions not to introduce breaking changes. I understand we’re still in initial development (0.y.z), it is disheartening to login to monitor flows to see that everything has failed in the last 24 hours because I haven’t updated updated the flows to the newest release. Specifically 0.14.15 introduced things like •
    terminal_state_handlers
    on Flow objects that will cause a flow to fail if they aren’t present even though the 0.14.15 version defaults to None. •
    log_output
    on the
    ExecuteNotebook
    class that defaults to False on the 0.14.15 version, but the execution fails if a flow on a previous version is being run with the attribute not specified
    Josh
    Chris White
    +1
    28 replies
    Copy to Clipboard