Thread
#prefect-community
    Valantis Hatzimagkas

    Valantis Hatzimagkas

    6 months ago
    Hello! I have noticed the following behavior and I cannot figure out what is going on. (I am performing the following action on my local machine) I have done he following:1. I have created a flow which has a task that gets some data 2. I am using Docker storage 3. I have done the required steps(building base image, passing dependencies, running server with --expose, adding a docker agent that communicates in the same network with my server) My Issues: I have added a for loop in my task, this is causing the task to be stuck in the running state I have attempted a dictionary comprehension inside my task, it caused a ι SystemError('Objects/dictobject.c:1555: bad argument to internal function') I tried the same piece of code in local storage and it works fine, so I think there is something with Docker storage that I am missing.
    Abhishek

    Abhishek

    6 months ago
    @Valantis Hatzimagkas can you share the code or a gist?
    Valantis Hatzimagkas

    Valantis Hatzimagkas

    6 months ago
    Yes of course!
    from prefect import task, Flow, prefect
    from prefect.storage import Docker
    import requests
    
    lat = 40
    lon = 45
    API_key = 'my_key'
    url = f'<https://api.openweathermap.org/data/2.5/weather?lat={lat}&lon={lon}&appid={API_key}>'
    
    @task
    def get_weather():
        return requests.get(url).json()
    
    
    @task
    def logger(res):
        for k, v in res['main'].items():
            print(f'{k}: {v}')
    
    
    with Flow('spray_service') as flow:
        data = get_weather()
        logger(data)
    
    if __name__ == "__main__":
        flow.storage = Docker(image_name="my_image", image_tag="latest", local_image=True,
                              dockerfile='Dockerfile')
        #flow.register('test')
        flow.run()
    Anna Geller

    Anna Geller

    6 months ago
    Running docker agent with Server deployed in docker compose is a bit hard because you may end up running docker in docker. I think you would need to modify your docker compose setup so that the docker agent and your Server components are in the same Docker network. This thread seems similar and I added a couple of comments there and some links you may try, LMK if this is not helpful and you need more info https://prefect-community.slack.com/archives/C014Z8DPDSR/p1646249456379019?thread_ts=1646239884.507719&amp;cid=C014Z8DPDSR
    Your flow is perfectly fine and I'm pretty sure if you switch to local storage and local agent this should work, but docker agent is a bit more involved to set up with Server
    Valantis Hatzimagkas

    Valantis Hatzimagkas

    6 months ago
    Thanks Anna, yes I tried it also on local storage and it worked fine, both in the ui and executing it via flow.run() in my code. I will give it a shot with the thread you provided me. Thanks a lot!
    Thanks again Anna, I found my actual problem. I am going to leave this here just in case: My Docker file had:
    FROM prefecthq/prefect:0.15.13-python3.9
    But! my flow was serialized in *python 3.10(During registration prefect complained about this,* but silly me I did not notice it*)* This caused this strange issue in my case
    Anna Geller

    Anna Geller

    6 months ago
    Thanks for sharing and nice work figuring this out! 🙌