Hi all I m having trouble configuring a deployment pipeline Prefect Community #ask-community

Hi all, I'm having trouble configuring a deploymen...

Leon Kozlowski

02/01/2023, 4:23 PM

Hi all, I'm having trouble configuring a deployment pipeline in Jenkins. The steps for our flow deployment process include: 1. Building and Pushing docker image for flow to ECR 2. Logging in via

prefect cloud login -w <WORKSPACE_HERE> -k <API_KEY_HERE>

3. Registering a block via

prefect block register --file src/block.py

4. Building a deployment via

prefect deployment build src/flow.py:<NAME>

(with many other options) 5. Applying a deployment via

prefect deployment apply <NAME>-deployment.yaml

This pipeline was working as expected in

prefect==2.4.0

however since upgrading to

prefect==2.7.5

we can only get to step 2 of this pipeline. After a successful login we see a warning log:

Copy code

/usr/local/lib/python3.10/site-packages/prefect/context.py:496: UserWarning: Failed to create the Prefect home directory at /.prefect
  with SettingsContext(profile=profile, settings=new_settings) as ctx:

This is being run inside of the docker image prefecthq/prefect:2.7.5-python3.10 Not sure of a path forward, I am also unsure why a

.prefect

would be needed for a cloud deployment inside of an ephemeral deployment pipeline

Leon Kozlowski

02/01/2023, 4:54 PM

The traceback concludes with

Copy code

File "/usr/local/lib/python3.10/site-packages/aiosqlite/core.py", line 405, in
connector
    return sqlite3.connect(loc, **kwargs)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open 
database file
(Background on this error at: <https://sqlalche.me/e/14/e3q8>)

Christopher Boyd

02/02/2023, 2:10 PM

When you login with prefect cloud login, it builds a profile in .prefect

Christopher Boyd

02/02/2023, 2:11 PM

My guess is maybe write permissions on / in the docker container

Leon Kozlowski

02/02/2023, 5:05 PM

I got the permissions working, but ran into another issue relating to a bug in

prefect==2.7.5

relating to sqlite migrations

Leon Kozlowski

02/02/2023, 5:06 PM

I have a separate question though, is it an issue that my deployment pipeline uses ephemeral containers? What is this sqlite database for, does this store state that shouldn't be wiped per deployment?

5 Views

Open in Slack

Previous Next