• Daniel Black

    Daniel Black

    1 year ago
    Hello! I am trying to test a few flows in our prod environment by running a local agent in a docker container that simulates the environment. For some reason the agent is not picking up the registered flows even though they share labels. Does anyone know why this would be happening? For reference, it runs successfully if I run a local agent in my dev environment
    Daniel Black
    j
    10 replies
    Copy to Clipboard
  • Marco Petrazzoli

    Marco Petrazzoli

    1 year ago
    Hello, it's the right place to ask something about prefect errors?
    Marco Petrazzoli
    1 replies
    Copy to Clipboard
  • m

    Matthew Blau

    1 year ago
    Hello all, I have made some progress with using docker storage in my flow, however, I am now getting this error:
    ValueError: Your docker image failed to build!  Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.
    The full traceback is
    Step 12/22 : WORKDIR /app
     ---> Running in b820b907ef61
    Removing intermediate container b820b907ef61
     ---> fee83981b109
    Step 13/22 : RUN groupadd --gid ${GID} ${USERNAME}     && useradd --uid ${UID} --gid ${GID} --shell /bin/bash -M ${USERNAME}     && chown -R ${USERNAME} /app     && chgrp -R ${USERNAME} /app
     ---> Running in 7f6c8b47c9e3
    groupadd: invalid group ID 'rainbow'
    
    Removing intermediate container 7f6c8b47c9e3
    The command '/bin/sh -c groupadd --gid ${GID} ${USERNAME}     && useradd --uid ${UID} --gid ${GID} --shell /bin/bash -M ${USERNAME}     && chown -R ${USERNAME} /app     && chgrp -R ${USERNAME} /app' returned a non-zero code: 3
    Traceback (most recent call last):
      File "integration.py", line 11, in <module>
        from change_password import *
      File "/home/mblau/projects/experian-integration/change_password.py", line 5, in <module>
        from integration import modify_config
      File "/home/mblau/projects/experian-integration/integration.py", line 338, in <module>
        flow.register(project_name="test")
      File "/home/mblau/.local/lib/python3.8/site-packages/prefect/core/flow.py", line 1665, in register
        registered_flow = client.register(
      File "/home/mblau/.local/lib/python3.8/site-packages/prefect/client/client.py", line 782, in register
        serialized_flow = flow.serialize(build=build)  # type: Any
      File "/home/mblau/.local/lib/python3.8/site-packages/prefect/core/flow.py", line 1451, in serialize
        storage = self.storage.build()  # type: Optional[Storage]
      File "/home/mblau/.local/lib/python3.8/site-packages/prefect/storage/docker.py", line 360, in build
        self._build_image(push=push)
      File "/home/mblau/.local/lib/python3.8/site-packages/prefect/storage/docker.py", line 427, in _build_image
        raise ValueError(
    ValueError: Your docker image failed to build!  Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.
    what am I doing wrong?
    m
    j
    25 replies
    Copy to Clipboard
  • m

    matta

    1 year ago
    Is there a particular "Dask-ic" way to split a big CSV and make sure the compressed chunks are a specific size?
    m
    james.lamb
    4 replies
    Copy to Clipboard
  • p

    Peyton Runyan

    1 year ago
    Hi all! I'm having a bit of an issue with an empty UI when running
    prefect server start
    . I'll post the logs in the thread.
    p
    Michael Adkins
    +1
    29 replies
    Copy to Clipboard
  • Marco Petrazzoli

    Marco Petrazzoli

    1 year ago
    I'm missing something. It's possibile to launch a flow of docker images tasks?
  • Marco Petrazzoli

    Marco Petrazzoli

    1 year ago
    I'm using something like:
    def say_hello():
        container = CreateContainer(
            image_name="nonexistentname",
        )
        StartContainer(container_id=container, docker_server_url="arandomtext")
        WaitOnContainer(container_id=container)
    and it's always successful. Even with correct data i don't see my image executed
    Marco Petrazzoli
    m
    +1
    4 replies
    Copy to Clipboard
  • a

    Arnulfo Soriano

    1 year ago
    Hi, We would like to do flow registers from Github Actions when a new flow gets merged into master. To accomplish this, we have our graphql end point protected through SSL using a certificate and we define it as an environment variable PREFECT__CLOUD__GRAPHQL=https://protected.grpahql.endpoint.com so prefect from Github Action runner linux image has access to this. So this endpoint is only accessible though https and using ca certificates. I've tested that this end point works on Chrome with the give certificate and it works fine. However, when we try to call it through Github Actions runners i keep getting: prefect.TaskRunner | FAIL signal raised: FAIL('Failed to register flow_test, error: 400 Client Error: Bad Request for url: https://protected.grpahql.endpoint.com/\nThis is likely caused by a poorly formatted GraphQL query or mutation but the response could not be parsed for more details') Here is some of the things i've tried1) configured the certificate by adding it to the Github actions runner image at /usr/local/share/ca-certificates/graphCert.crt and then 'dpkg-reconfigure ca-certificates' 2) Declared python env variable to tell it where too look for the ca certs REQUESTS_CA_BUNDLE: /etc/ssl/certs/ca-certificates.crt Does any one of you have a similar setup where your graphql endpoint is protected in this way? When it's not protected our setup works just fine... Thank you in advance...
    a
    1 replies
    Copy to Clipboard
  • Fina Silva-Santisteban

    Fina Silva-Santisteban

    1 year ago
    Hi Prefect team! Can someone pls confirm or clarify the following understanding that I have about how Prefect works with Docker: • I can run an app inside a docker container, any dependencies needed, e.g. pandas library, will be installed inside that container. Our repo already has a Dockerfile and a docker-compose.yml which build an image and run the app inside a container successfully. Necessary commands to do that are
    docker-compose build app
    , then
    docker-compose run app
    .  • Prefect Agent runs flows. The Local Agent runs flows with the local setup. This is not feasible for production since we’d need to manually make sure that whichever machine is hosting the flow has all dependencies installed. Better: run flows within a docker container. This way we don’t have to worry about dependencies, since every machine will simply be running the container. To run flows in containers we need to use the Docker Agent. • The Docker Agent DOES NOT REQUIRE DOCKER STORAGE, but can work with Docker Storage if desired. The Docker Agent can run with a locally available image (image="example/image-name:with-tag"). If Docker Storage is used, you can provide a url (
    <http://registry_url|registry_url>
    )to where your image is hosted, e.g. Docker Hub, and that url is used to get the image. • I should be able to run the Docker Agent with the image I had previously created with docker-compose. (It doesn’t seem to be able to find the image, even though I specify the absolute path. )
    Fina Silva-Santisteban
    Michael Adkins
    18 replies
    Copy to Clipboard
  • m

    Martin Shin

    1 year ago
    Hi, we’re using a paid plan on Prefect Cloud but we are now seeing this error when trying to register new flows: 
    This tenant already has the maximum number of flows
    . Don't' see there's any mention of upper limit on flows? Any help will be appreciated!
    m
    1 replies
    Copy to Clipboard