https://prefect.io logo
t

Tim Ricablanca

11/07/2022, 10:09 PM
hi folks! long time listener, first time caller 😄 — i have a question about upgrading and downgrading the database (we’re using a postgres container) along side prefect versions. i ran into an issue when trying to rollback from 2.6.6 to 2.6.4 and got this error on orion startup:
Copy code
alembic.util.exc.CommandError: Can't locate revision identified by '41e5ed9e1034'
i’ll post some details in a thread about what i tried before nuking the database and starting over, but 1) is there guidance on how i should be using
prefect orion database stamp
or
… revision
before upgrading orion from 2.6.4 -> 2.6.6 (or any other minor version) ? and 2) should i be running
… downgrade
from the 2.6.6 package or 2.6.4 package if i wanted to go from 2.6.6->2.6.4?
1
So I upgraded to 2.6.6, and saw some issues that we hadn’t seen in 2.6.4. We’re running Postgres and Prefect on separate containers talking over a Docker network. In order to downgrade, I left the database as-is and re-launched Prefect with the 2.6.4 container, that’s where I saw the above
alembic
error.
I had been able to roll-forward to 2.6.6 and everything started back up. However, we still wanted to try to revert to 2.6.4 and keep the run history.
I tried
prefect orion database downgrade
and then rolled the container back to 2.6.4 and then saw the following on startup:
Copy code
prefect-orion-1  | sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class 'asyncpg.exceptions.DuplicateTableError'>: relation "flow" already exists
prefect-orion-1  | [SQL:
prefect-orion-1  | CREATE TABLE flow (
prefect-orion-1  |     id UUID DEFAULT (GEN_RANDOM_UUID()) NOT NULL,
prefect-orion-1  |     created TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
prefect-orion-1  |     updated TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
prefect-orion-1  |     name VARCHAR NOT NULL,
prefect-orion-1  |     tags JSONB DEFAULT '[]' NOT NULL,
prefect-orion-1  |     CONSTRAINT pk_flow PRIMARY KEY (id),
prefect-orion-1  |     CONSTRAINT uq_flow__name UNIQUE (name)
prefect-orion-1  | )
prefect-orion-1  |
prefect-orion-1  | ]
prefect-orion-1  | (Background on this error at: <https://sqlalche.me/e/14/f405>)
prefect-orion-1  |
prefect-orion-1  | ERROR:    Application startup failed. Exiting.
…wait, sorry my steps are out of order. I did
downgrade
and saw the same
alembic
error and then in desperation removed the
alembic
version and made it worse and saw the following.
z

Zanie

11/07/2022, 10:14 PM
While on 2.6.6, you’ll need to downgrade to the specific database commit that 2.6.4 uses.
👀 1
t

Tim Ricablanca

11/07/2022, 10:14 PM
That that point I just dropped the PG database and started over. So really the question is — what’s the best way to roll back to avoid this going forward
And that is what
stamp
does? I couldn’t find a lot of docs except for the changelog on it
Or rather, stamp creates that commit version?
z

Zanie

11/07/2022, 10:15 PM
Ah no it’d be
prefect orion database downgrade -r <commit>
I don’t think we really expose a clean way to get the commit for a given Prefect version though.
Since our library versions are tracked as git tags there’s not a great way for us to expose a downgrade command that takes a prefect version right now. I’ll see if anyone has any great ideas for how we can expose that in the future so you don’t need to do so much.
t

Tim Ricablanca

11/07/2022, 10:34 PM
Okay, thanks! For the time being I'll see how far I can wire this up in Ansible but something more automatic or built into to the CLI would be nice
m

Mason Menges

11/11/2022, 10:34 PM
Hey @Tim Ricablanca we're tracking these issues here https://github.com/PrefectHQ/prefect/issues/7512 if you have some time to run through reporting the behavior you were seeing.
6 Views