Gabor Hosszu
07/22/2025, 10:59 AMprefect server start
command, nothing fancy. All good here, the only problem is, it's using its own SQLite database within the container, which means if this goes down, all data is lost. It's not easy to connect remote storage to Code Engine, plus it doesn't work well with prefect. So probably the best would be to connect it to an external postgres so the data remains persistent and I don't lose all my deployments every time the server has a hiccup. How would one go about this?Marvin
07/22/2025, 10:59 AMMarvin
07/22/2025, 11:00 AMPREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://username:password@host:5432/database_name>
Replace the following:
- `username`: Your PostgreSQL username
- `password`: Your PostgreSQL password
- `host`: Your PostgreSQL host address
- `database_name`: The name of the database you want Prefect to use
3. In your Code Engine project settings, add this environment variable. The exact steps would be:
- Go to your Code Engine project
- Select your Prefect Server application
- Add the environment variable under the configuration section
4. Important considerations:
- Ensure your PostgreSQL database is accessible from your Code Engine environment (check network/firewall rules)
- The database user should have all necessary permissions to create and modify tables
- It's recommended to use connection pooling if available in your PostgreSQL setup
- Store sensitive information like passwords using Code Engine's secrets management
5. After setting this up, when you restart your Prefect Server, it will automatically:
- Connect to the PostgreSQL database
- Create all necessary tables if they don't exist
- Start using PostgreSQL instead of SQLite
This way, even if your Code Engine container restarts or goes down, all your Prefect data (flows, deployments, flow runs, etc.) will persist in the PostgreSQL database.
A few tips:
- Make sure to backup your PostgreSQL database regularly
- Monitor your database connections and performance
- Consider using connection pooling if you expect high concurrency
- Use SSL connections if your database is hosted outside IBM Cloud
Would you like me to provide more specific details about any of these steps or help with troubleshooting the setup?