• Tom Forbes

    Tom Forbes

    1 year ago
    When using
    flow.run()
    is it possible to override the result class/type? Doing:
    flow.run(result=LocalResult(...))
    fails with:
    TypeError: run() got an unexpected keyword argument 'result'
    Tom Forbes
    Kevin Kho
    5 replies
    Copy to Clipboard
  • Raúl Mansilla

    Raúl Mansilla

    1 year ago
    Hello all, I´m trying to get a flow running using a docker agent that it´s already running in docker…I get the agent connected to prefect server and the flow gets to the agent when I run it, but It never finish…
    Raúl Mansilla
    m
    31 replies
    Copy to Clipboard
  • Paulo Benatto

    Paulo Benatto

    1 year ago
    Hi guys, how r u doing? I'm new to prefect and i was reading: https://docs.prefect.io/core/idioms/parallel.html and i think im missing something 😏. I would like to build a list of tasks to be executed in parallel. See this dummy example:
    @task
    def create_task_list(ids):
        l = []
        for i in ids:
            l.append(ReferralUID())
        return l
    
    @task
    def generate_ids():
        return ["id1", "id2", "id3"]
    
    
    
    with Flow("parallel-execution") as flow:
    
        ids = generate_ids()
        
        list_of_tasks = create_task_list(ids)
    
        request_referral = RequestReferralDetail()
    
        # list_of_tasks should be a list, but i'm returning a task. how to inject a list on bind?
        request_referral.bind(request_details=list_of_tasks, flow=flow)
    
        flow.visualize()
        flow.run(executor=LocalDaskExecutor())
    Thanks
    Paulo Benatto
    Rob Fowler
    7 replies
    Copy to Clipboard
  • c

    Chris Bowen

    1 year ago
    Hello, I'm getting a Prefect server set up on a RHEL server for the first time. I have a question related to the local agent- is there any way to run the agent "detached" like the server command? With the server, I can run
    prefect server start --detach
    so it doesn't run interactively. Is there a comparable solution for agents? When my ssh session with the server ends, so does my agent. I might just be missing something obvious. Appreciate any help.
    c
    Kevin Kho
    2 replies
    Copy to Clipboard
  • YD

    YD

    1 year ago
    Try to set rootless server on CentOs 7, followinghttps://docs.docker.com/engine/security/rootless/ not so clear to me how to install
    newuidmap
     and 
    newgidmap
    also not so clear what exactly I should add to
    /etc/subuid
     and 
    /etc/subgid
    if someone can help with a step by step example, it will be very helpful thanks
    YD
    Kevin Kho
    +1
    5 replies
    Copy to Clipboard
  • YD

    YD

    1 year ago
    another question on installation following https://docs.prefect.io/core/getting_started/installation.html If I want to change the port from 8080 to some other port, how to do this? this is for a case that something else is installed on this server and uses the 8080 port
    YD
    Kevin Kho
    2 replies
    Copy to Clipboard
  • Shaoyi Zhang

    Shaoyi Zhang

    1 year ago
    Hi everyone, what’s the best way to report a potential security bug with Prefect?
    Shaoyi Zhang
    1 replies
    Copy to Clipboard
  • YD

    YD

    1 year ago
    How to keep the UI running on the server? If I run
    prefect server start
    , it will stop when exit the terminal I tried
    sudo systemctl prefect server start
    , but this does not look like to proper way
    YD
    1 replies
    Copy to Clipboard
  • YD

    YD

    1 year ago
    General question... I had some sample workflows running on my local machine (on my laptop), http://localhost:8080/default I've installed Prefect server on a VM. when I go to
    <VM ip address>:8080
    , without running anything, it is somehow shows the workflows and projects I had on my local machine. How can this be ?
    YD
    Kevin Kho
    6 replies
    Copy to Clipboard
  • ash

    ash

    1 year ago
    Hello everyone , I am a little confused on the below mentioned part
    import pymongo
    import pandas
    from reports.config import mongo_config
    import sklearn
    from prefect import Flow
    from prefect.storage import Docker
    
    @task
    	def say_hello():
    		print("hello world !!!")
    
    with Flow("hello world") as flow:
    	say_hello()
    
    
    flow.storage = Docker(registry_url = , image name = "hello world flow")
    flow.register("demo")
    In above code I am importing three external libraries i.e pandas, sklearn, pymongo and
    mongo_config
    which here contains configuration related information for connecting with mongo
    When I register a flow, lets say for code above ,
    
    A.) step(1) A docker image containing everything including external libraries, mongo config and flow code will be built and saved to container registry.
    	step(2) Its Metadata including a schedule if any , its path to dependencies from container registry etc will be saved on postgres.
    	step(3) When the kubernetes agent polls and have to run the above flow, it will create a pod and dependencies will be installed and after task completion pod is terminated.
    
    	Thats my understanding of how things are working, please correct me if am wrong on any of above.
    	Now one thing here is what if mongo config changes, whenever we built pipelines for reporting all we want to do is just change config at one place and  changes   are incorporated for every other report but going on with above approach , i might need to re-register every flow to let it engulf the updated config, Thats what i thinking over here, can you suggest someways of how can i change config at one place so that all the flows knows it and i don't have to re-register all my flows.
    
    B.) One way that i think will be able to solve this is when I use github as storage as the code is read from github so the updated config will also be taken into consideration possibly but there is one issue in this approach ,
    when the pod is created to run the script how the dependencies will be installed on the pod  since we don't have docker image this time?
    ash
    m
    +1
    22 replies
    Copy to Clipboard