Kingsley Blatter
01/07/2021, 8:09 PMHiro Rodriguez
01/20/2021, 11:05 PMKingsley Blatter
01/22/2021, 8:35 PMJeremiah
01/25/2021, 9:58 PMJeremiah
02/09/2021, 3:38 PMJim Crist-Harif
02/10/2021, 4:57 PMModule
storage class, for referencing flows importable from a Python module
• New tasks for working with Exasol
Big thanks to all our community members who contributed this week 👏👏:
• Alex P.
• Marwan S.
• Peter Roelants
• Timo S.
The full release notes can be found here: https://github.com/PrefectHQ/prefect/releases/tag/0.14.7Chris White
02/11/2021, 10:52 PMKingsley Blatter
02/12/2021, 9:10 PMJim Crist-Harif
02/17/2021, 3:35 PMCompressedSerializer
for compressing task results
• Added configurable timeouts to the Prefect Client
• Improved performance for all Prefect AWS integrations
And last but not least, Fivetran introduced their first official orchestrator integration with a new Prefect task (the FivetranSyncTask
); we’re excited to collaborate more with the Fivetran folks so give it a spin and let us know your feedback!
Thanks to all our community contributors this week 👏:
• Amanda Wee
• Andrew Hannigan
• Craig Wright
• Nick Acosta
• Timo S.
The full changelog can be found here: https://github.com/PrefectHQ/prefect/releases/tag/0.14.9Jim Crist-Harif
03/04/2021, 11:22 PMprefect server config
command that outputs a standalone docker-compose.yaml
file that can be used without the prefect CLI.
• New task library tasks for working with files/filesystems in prefect.tasks.files
• Added project_name
/`project_id` to flow run execution context in orchestrated flow runs
Thanks to all our community contributors this week 👏
• David Zucker
• Jacob Hayes
• Joël Luijmes
• Timo S.
• Yogi Patel
The full changelog can be found here: https://github.com/PrefectHQ/prefect/releases/tag/0.14.11.Jim Crist-Harif
03/10/2021, 11:16 PMJim Crist-Harif
03/10/2021, 11:55 PMHiro Rodriguez
03/16/2021, 1:25 PMJim Crist-Harif
03/24/2021, 10:46 PMprefect register
for easier registration of flows (docs).
• prefect build
for building flows without registration (docs).
Combined, these expose a lot of new & interesting deployment options. One is automatic flow reloading. The prefect register
CLI supports a --watch
flag which will watch your source files, and auto-register them on save. This can be useful for development, but also exposes an airflow-like pattern where new flows can be deployed by adding them to a directory.
We have several additional CLI improvements in the works, so stay tuned.
As always, big thanks to all our community contributors this release 👏 :
- Abid Ahmad
- Alexandr N. Zamaraev
- Jacob Hayes
- Timo S.
The full changelog can be viewed here: https://github.com/PrefectHQ/prefect/releases/tag/0.14.13
Happy Engineering!Kevin Kho
04/20/2021, 6:13 PMMichael Adkins
04/27/2021, 11:31 PMGit
storage allows you to use any git
based service for flow storage (this includes deploy token/key authentication!)
- Error messages have been improved in more than five places to help guide users to a resolution
- ShellTask
now works on Windows and supports more configurable logging
- Bitbucket
storage now supports cloud servers
- A custom terminal state handler can now be placed on a flow, allowing users to customize the final state of their flow with access to the states of their flow's reference tasks
- An architecture diagram has been added to the docs (https://docs.prefect.io/orchestration/#architecture-overview)
- The required docker-compose version to run prefect server start
has been lowered and documented
- Environment variables can be passed to the UniversalRun
config
- PostgresFetch
can return data column names
- prefect server start
can now connect to an separately managed postgres instance
There's always more, see the full changelog at https://github.com/PrefectHQ/prefect/blob/master/CHANGELOG.md
Thanks to our community contributors!
- Alessandro Lollo
- David Zucker
- Greg Lu
- James Lamb
- Sean Talia
- Jonathan Wright
- Ben Fogelson
- Gabriel Montañola
Remember this list isn't exhaustive and for every person listed in it, there are many more who have helped report issues and written PRs that help move us forward even if they aren't merged. Thanks to everyone here in our Slack community!Zach Angell
06/03/2021, 7:36 PMJeremiah
06/10/2021, 9:57 PMChris White
06/17/2021, 7:37 PMjustabill
07/02/2021, 4:29 PMJeremiah
07/15/2021, 4:13 PMjustabill
07/30/2021, 4:41 PMVaultSecret
task
• Expanded configuration to expose deeper settings for Docker flow runs
• Enabled loading of additional repository files with Git
storage
• Added documentation for querying account role based access control data
• Added --no-schedule
flag to prefect register
to disable flow schedules when registered
• Enabled recursive flow registration and builds through globs
• Enabled calculation of evaluation parameters on the fly through new evaluation_parameters
parameter for the RunGreatExpectationsValidation
task
• Increased robustness of flow runs to connection errors through improved exception handling
Special thanks to @Gabriel Montañola, @Nelson Griffiths, Pawel Janowski, @Yueh-Han, and @Chris Ottinger for contributing to these enhancements!justabill
09/09/2021, 9:32 PM0.15.5-python3.9
.
• While registering very large flows, it was possible to see timeouts or incomplete registration. Now, flow edges and tasks are registered in batches allowing for much larger flows.
• Agentless flow runs now receive the environment variables of the process that starts them; this makes agentless runs easier to use in production.
• --idempotency-key
support has been added to prefect run
• prefect run
will now ignore schedules on local flow runs by default and only a single run will be performed. A --schedule
flag was added if you’d like the command to go forever and run the flow on a schedule.
• Several task library improvements have been made, including a new SnowflakeQueryFromFile
task, improved logging in the AWS BatchSubmit
task, improved runtime handling for MySQL Tasks, and updated authentication for the DremioFetch
task.
• Log limits have been updated to reduce errors when many large logs are sent.
Note that this release includes several breaking changes:
• In a secure-by-default effort, services run by the prefect server
cli now listen to localhost by default and are not accessible from other machines; use --expose
to allow connections from a remote location.
• Prefect Server will need to be upgraded to version 2021.09.02
before flows can be registered from this version of Core.
Special thanks to Deepyaman Datta, Henri Hannetel, Johnny Tirado, @Kathryn Klarich, @Tenzin Choedak, and Vincent Xue for contributing to this release :thank-you:!Chris Reuter
09/27/2021, 2:56 PMYoutube▾
Chris White
10/05/2021, 4:42 PMKevin Kho
10/06/2021, 2:35 PMChris Reuter
10/08/2021, 1:05 PMChris Reuter
10/12/2021, 4:14 PMChris Reuter
10/20/2021, 2:17 PMChris Reuter
10/26/2021, 1:00 PMChris Reuter
10/26/2021, 1:00 PM