https://prefect.io logo
d

davzucky

04/28/2022, 11:58 AM
Did active try to use Yugabyte instead of postgres with Prefect Server 1.x? Looking to get more resiliency and hot hot availability.
a

Anna Geller

04/28/2022, 12:08 PM
The easiest would be to give it a try. I'm always a bit skeptical when I see "Postgres-compatible" - it's still not Postgres? Where are you running this? I can totally understand the problem of ensuring resiliency, but have you looked at the Postgres managed services from cloud providers? they seem to provide features for more resiliency, not sure about hot-hot availability (multi-AZ RDS?)
d

davzucky

04/28/2022, 12:12 PM
I would love to have access to cloud providers and DBaaS, however, the journey is only starting and the movement is slow. At the moment we have everything on Kubernetes across multiple cluster, however postgres is not simple to setup for full high availability.
I will give it a try and let you know how it goes. I never tried it, however as this is certified as Postgres compatible by Hasura 2.x I want to give it a try. I will let you know the feedback. Do you know if Prefect server queries directly the Db or everything is handled using Hasura
a

Anna Geller

04/28/2022, 12:21 PM
I understand, migrating from on-prem to ā˜ļø is not easy Good question, let me ask the team
d

davzucky

04/28/2022, 12:25 PM
Thank you @Anna Geller for asking
šŸ‘ 1
a

Anna Geller

04/28/2022, 12:51 PM
btw do you plan to actually move to Prefect 2.0? šŸ˜‰ with Prefect 2.0, I checked with Bill and it works based on ORM and SQLAlchemy, so doing this should work in theory (I haven't tested that):
Copy code
export PREFECT_ORION_DATABASE_CONNECTION_URL="<postgresql+asyncpg://yb_db_user:yb_pwd@yb_host:5433/db_name>"
this would work because YougabyteDB supports SQLAlchemy
d

davzucky

04/28/2022, 12:57 PM
Yes I'm planning to migrate to prefect 2 however I need to be out of the beta stage. Looking forward about that
šŸ‘ 1
a

Anna Geller

04/29/2022, 1:36 AM
@davzucky I asked the team about YugabyteDB with Prefect Server and here is the answer: • in 1.0, almost everything goes through Hasura, so there is some chance that this might work and it would be best if you give it a try and report back - I'd appreciate that! • there are some places where we query the DB directly using
asyncpg
, and we also run migrations via Alembic, so those use cases don't go through Hasura and you would need to be more careful in those use cases So the bottom line is: there is no dedicated support for YugabyteDB but with some work on your end it could work, but no guarantee. But from a long-term perspective, you may be better off if you start using it with Prefect 2.0 - due to the ORM layer rather than Hasura, this is much easier
19 Views