Thread
#prefect-community
    p

    Philip MacMenamin

    10 months ago
    prefect agent docker start
    results in
    requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff5ba1a23a0>: Failed to establish a new connection: [Errno 111] Connection refused'))
    Kevin Kho

    Kevin Kho

    10 months ago
    This is on server right? Did you spin up your server and can you access the UI?
    p

    Philip MacMenamin

    10 months ago
    $ uname -a
    Linux L02147210 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
    $ prefect diagnostics
    {
      "config_overrides": {
        "PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS": true,
        "PREFECT__FLOWS__CHECKPOINTING": true
      },
      "env_vars": [],
      "system_information": {
        "platform": "Linux-5.4.0-89-generic-x86_64-with-glibc2.29",
        "prefect_backend": "server",
        "prefect_version": "0.15.7",
        "python_version": "3.8.10"
      }
    }
    @Kevin Kho - this is on localhost
    Kevin Kho

    Kevin Kho

    10 months ago
    Are all of your images healthy? Does accessing the UI of server work (localhost:8080)0?
    p

    Philip MacMenamin

    10 months ago
    so the server has to run to start the agent?
    Kevin Kho

    Kevin Kho

    10 months ago
    Yes you need the server up and running because an agent polls a backend (Cloud or Server) every 10 seconds to pick up flows. So the agent is establishing a connection to server.
    I know you are on Server for security reasons right? But Cloud would be easier to test with non-sensitive data as you get the 20000 task runs for free every month
    p

    Philip MacMenamin

    10 months ago
    how are you defining Server and Cloud?
    as in, run on your infra, which you call Cloud?
    or my infra which is called Server?
    Kevin Kho

    Kevin Kho

    10 months ago
    Yeah exactly so Cloud is our managed offering of Server. Both are called backends.
    p

    Philip MacMenamin

    10 months ago
    I guess... the issue is I'd like to be able to just run flows locally without all that much hassle. ATM I just want to see if I can get a containerized flow up
    Anna Geller

    Anna Geller

    10 months ago
    In that case you should definitely go for Cloud. It’s hassle free - you sign up, create an API key, authenticate your terminal with this key (
    prefect auth login --key YOUR_KEY
    ) and you’re good to go. With Server you need to first configure it all. So I’d definitely recommend getting started with Cloud. You can always switch to Server later if you wish to do so.
    p

    Philip MacMenamin

    10 months ago
    basically I'd like an example to run https://docs.prefect.io/api/latest/tasks/docker.html
    as in, a flow that has say one task that's a containerized task.
    for example, i have a docker image A that does something I want, how do I roll that into a flow?
    I could presumably use a shelltask, and call into it, but I assumed seeing as there were dockertasks this would be more helpful.
    unless this sounds wrong headed.
    Kevin Kho

    Kevin Kho

    10 months ago
    Yeah you can try it out with Cloud and Cloud will orchestrate the spinning up of the container on your local and running the Flow there. I think there are two things here. One is you want to run a container that contains tasks, which you can do just by supplying the image to run on. This will be preferable if the code is Python because you get more granular observability. The second is the CreateContainer task you can use. This doc kinda shows what that looks like (ignore the Kubernetes portion)
    The Docker agent will launch your Flow as a container. No need to create a container if it’s just Python code.
    p

    Philip MacMenamin

    10 months ago
    ok - thanks guys. I'll give this a look.