Gaël Grosch
01/27/2025, 3:56 PMDatabaseCredentials
? It seems like the github project for prefect_sqlalchemy
is deprecated but the integration inside Prefect (prefect[sqlalchemy]
) does not seems to support it.
What is the proper way to read from a Database Credentials
block? It it not marked as deprecated on the managed prefect (prefect.cloud)Marvin
01/27/2025, 3:56 PMDatabaseCredentials
block is still supported in Prefect, despite the deprecation of the prefect_sqlalchemy
GitHub project. The integration for working with databases using Prefect is now handled through the prefect-sqlalchemy
package, which you can install and use with Prefect.
Here's how you can read from a DatabaseCredentials
block:
1. Installation: Ensure you have the prefect-sqlalchemy
package installed. You can do this by running:
bash
pip install prefect-sqlalchemy
2. Registering Blocks: After installation, register the block types in the prefect-sqlalchemy
module to make them available for use.
3. Using the Block: You can save credentials to a block and then load the saved block that holds your credentials. Here's a basic example:
python
from prefect_sqlalchemy import DatabaseCredentials
# Load your credentials block
credentials_block = DatabaseCredentials.load("my-database-credentials")
# Use credentials block in your flow
For more information, you can refer to the Prefect SQLAlchemy Integration Documentation.
Additionally, the DatabaseCredentials
type is used in the Prefect flows as shown in this GitHub issue example: Delayed flow/task start.
If you're using Prefect Cloud, the DatabaseCredentials
block is not marked as deprecated, and you should be able to use it as described.Gaël Grosch
01/27/2025, 4:01 PMDatabaseCredentials
inside https://github.com/PrefectHQ/prefect (and in particular in <https://github.com/PrefectHQ/prefect/tree/main/src/integrations/prefect-sqlalchemy>
)
The project https://github.com/PrefectHQ/prefect-sqlalchemy is archived since april 2024Marvin
01/27/2025, 4:05 PMDatabaseCredentials
block was indeed part of the prefect-sqlalchemy
package, which has been archived since April 2024. This means that while the block was available for use in Prefect flows, its future support and updates are uncertain due to the archiving of the repository.
Given this situation, here's what you can do:
1. Existing Usage: If you have existing workflows using DatabaseCredentials
, they may continue to work as expected, but be aware that there might be no future updates or support for this block.
2. Alternative Solutions: Consider using alternative methods for managing database connections. You might directly use SQLAlchemy or other database connection libraries in your Prefect flows. Prefect's flexibility allows you to integrate any Python library, so you can manage connections and credentials programmatically.
3. Monitor Updates: Keep an eye on Prefect's official announcements or GitHub repositories for any new integrations or changes regarding database connection management.
For more details on the archived repository, you can view the discussions and issues on the prefect-sqlalchemy GitHub page.
If you need further guidance on setting up database connections manually in your flows, I can help with that as well.Marvin
01/27/2025, 4:07 PMDatabaseCredentials
block was part of the prefect-sqlalchemy
package, which has been archived. However, the documentation still references its usage for connecting to databases in Prefect flows.
Here's a summary of the current situation and recommendations:
1. Archived Repository: The prefect-sqlalchemy
repository is archived, which means no further updates or official support will be provided.
2. Documentation Reference: The Prefect documentation still references the prefect-sqlalchemy
package for database connections, suggesting that it was a valid method prior to the archiving. You can find more about this in the Prefect SQLAlchemy Integration Documentation.
3. Alternative Approaches:
- Direct SQLAlchemy Usage: You can directly use SQLAlchemy or another database library in your Prefect flows. This involves managing connections and credentials programmatically.
- Custom Blocks: Consider creating custom Prefect blocks to handle database credentials if needed.
Since the prefect-sqlalchemy
integration is no longer actively maintained, using direct database connection methods or custom solutions might be the best path forward for new projects or updates. If you have specific needs or further questions, feel free to ask!Nate
01/27/2025, 4:21 PMActive development of this project has moved within PrefectHQ/prefect. The code can be found here and documentation here. Please open issues and PRs against PrefectHQ/prefect instead of this repository.
prefect-sqlalchemy
just lives in prefecthq/prefect
now
what are you trying to do? this is usually the block that's most useful directlyGaël Grosch
01/28/2025, 3:57 PMDatabaseCredentials