https://prefect.io
Join Slack
Did anyone have this problem where the python client hangs on trying to set up deployment? ``` deplo...
# ask-community
b
Did anyone have this problem where the python client hangs on trying to set up deployment?
Copy code
deployment = Deployment.build_from_flow(
    flow=load_data_from_minio,
    name="load_data_from_minio",
    work_queue_name="default",
    schedule=schedule,
    storage=storage_block,
    infrastructure=infra_block,
)

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    deployment.apply()
Running this code just seems to hang forever. No debug logs either. Full code in thread
Copy code
import logging
import datetime
from prefect.deployments import Deployment
from prefect.orion.schemas.schedules import IntervalSchedule
from dataclasses import asdict

from precog_api.prefect.flows.load_data_from_minio_flow import load_data_from_minio
from precog_api.config import Config
from prefect.filesystems import RemoteFileSystem
from prefect.infrastructure import DockerContainer
from prefect.infrastructure.docker import DockerRegistry

registry_block = DockerRegistry.load("gitlab-docker-registry")

storage_block = RemoteFileSystem(
    basepath="<s3://flow-storage/load_data_from_minio>",
    settings={
        "use_ssl": False,
        "key": Config.MINIO_USER,
        "secret": Config.MINIO_PASSWORD,
        "client_kwargs": {"endpoint_url": f"http://{Config.MINIO_ENDPOINT}"},
    },
)

storage_block.save("prefect-load-data-from-minio", overwrite=True)

infra_block = DockerContainer(
    registry=registry_block,
    image="<http://registry.gitlab.com/periplo-innovation/project-precog/ml_db|registry.gitlab.com/periplo-innovation/project-precog/ml_db>",
    image_pull_policy="IF_NOT_PRESENT",
    networks=[Config.PREFECT_CONTAINER_NETWORK]
    if Config.PREFECT_CONTAINER_NETWORK
    else None,
    env=asdict(Config()),
)
infra_block.save("precog-api-docker", overwrite=True)

today = datetime.datetime.today()

schedule = IntervalSchedule(
    interval=datetime.timedelta(days=1),
    anchor_date=datetime.datetime(
        year=today.year,
        month=today.month,
        day=today.day,
        hour=1,
    ),  # Run every day at 1 AM
)


deployment = Deployment.build_from_flow(
    flow=load_data_from_minio,
    name="load_data_from_minio",
    work_queue_name="default",
    schedule=schedule,
    storage=storage_block,
    infrastructure=infra_block,
)

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    deployment.apply()
Returns this after a long wait:
Copy code
httpx.HTTPStatusError: Client error '422 Unprocessable Entity' for url '<http://prefect-server:4200/api/deployments/>'
For more information check: <https://httpstatuses.com/422>
I see no way to check what the error was exactly (code runs in container, modifying code inside to print the actual json response is a huge pain)
Weirdly I dont see this request in prefect server logs
Firgured out part of it: the long wait was because prefect was very slowly uploading flow code to minio. Fixing .prefectignore to not include .git folder made it faster
still same error
z
Hm, we should be auto-expanding those error messages
cc @Peyton Runyan looks like 
PrefectHTTPStatusError
may not be working as intended?
👀 1
@Boris Tseytlin I believe you’ll get more information server-side if you change the log level there?
b
@Zanie I set the log-level to DEBUG, but I don’t see any logs from the HTTP server, and certianly no errors ;(
All the blocks get created successfully, flow data gets uploaded to MINIO right. Something is wrong about the API call being sent to prefect API, but I can’t figure out what
z
Did you set 
PREFECT_LOGGING_SERVER_LEVEL
?
b
Yes, to DEBUG
message has been deleted
j
Do you have a lot of files in the current directory? My guess is that it tries to upload everything and it takes a lot of time
b
I think the problem is that it takes a lot of time to upload code to stroage. Even if LocalFileSystem is used. It timeouts before it happens
Which is weird because I only have code, its all just some kbs
@Zanie is there a way to modify the timeout for uploading flow code to storage?
Or maybe some more efficient way to do it?
Also I see that it tries to overwrite everything even if files exist. Would be good to have some kind of checksum, so files only get replaced if they dont exist
Gah, I made uploading a separate step, but it still errors on 
apply
Copy code
if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    print("Upload to stroage")
    deployment.upload_to_storage()
    print("Apply")
    deployment.apply(upload=False)
Ah I think I found the issue. My requirements had a non-fixed version of prefect package, so it got updated to 2.7.5, but my docker-compose had prefect 2.6.9. Now it works
👍 1
j
Good to hear, Boris. If you are using SQLite for the backing db, you'll want to upgrade to 2.7.6.
353 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by