Does anyone have any examples of Microsoft Teams integrations? I know there is a Python wrapper library (https://pypi.org/project/pymsteams/) and really would like some type of exception reporting or notifications as a result of a Prefect flow.

Hello everyone, I’m trying to have an oriented approach to Flows and Tasks. The idea is to inherit specific flow types that already have some methods as tasks. Only overriding them if necessary. I realize that the decorator 

task

  doesn’t seem to work for class properties. Here is the error:

File "/Users/nuno/Developer/Data-Framework/data-prefect/data_prefect/utils/flows.py", line 67, in factory_flow
    flow.fetch()
  File "/Users/nuno/Developer/Data-Framework/data-prefect/.venv/lib/python3.8/site-packages/prefect/core/task.py", line 470, in __call__
    new.bind(
  File "/Users/nuno/Developer/Data-Framework/data-prefect/.venv/lib/python3.8/site-packages/prefect/core/task.py", line 511, in bind
    callargs = dict(signature.bind(*args, **kwargs).arguments)  # type: Dict
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/inspect.py", line 3025, in bind
    return self._bind(args, kwargs)
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/inspect.py", line 2940, in _bind
    raise TypeError(msg) from None
TypeError: missing a required argument: 'self'

It seems that I cannot pass the method “self” argument. Do you guys have any suggestion? Thank you in advance.

Hi all, I finally got around to writing up a fun personal project that I did with Prefect early this Summer. My family and I belong to a local outdoor pool that started requiring online reservations to limit the number of people at one time due to COVID-19. Reservations opened up exactly 48 hours in advance and were tough to get so I made a simple automated Bot to book our pool reservations. It turned out to be a really fun and practical use of Prefect! https://medium.com/@joe.schmid/how-prefect-kept-my-family-cool-in-the-pool-this-summer-21b1a3118a1f?sk=01fdb7427eb9fc3bf6a19b091e83a7c0

If you are using Python and need Technical Analysis. Check out Pandas TA. It is easy to use, has more than 120 indicators and utility methods. Multiprocessing. Easy to build Custom TA Strategies. Tightly correlated with TA-Lib and TradingView indicators. Example Notebooks to get started. And if you like it, I would appreciate a ⭐.

It is something I have been building as a core component in my future Prefect flows.

Hi everyone! Does anyone have an example project structure that includes tests? I’m struggling to get my imports etc working

Hello you fine folks! I loved using Prefect so much these last few weeks I created the simple guide I had to get started: https://lejimmy.com/how-to-automate-python-workflows-in-prefect-step-by-step-guide/ I would appreciate any feedback you fellas have to help improve it for beginners 🙂

hi, I'm new to the task engines. I'm trying to figure out how and if these tools are appropriate for my project. I would like to get data from an API and schedule ( in an appropriate time ) tasks to be performed on different APIs. how would this work with Perfect ? Would I have a task (not sure on the defienition here) that gets the data from the API the first and puts them in a database and then another task that schedules them or .. ?

Hi @tsar! Take a look at examples you can find in Prefect github repo. https://github.com/PrefectHQ/prefect/tree/master/examples/tutorial You can find some good starting points regarding your use case

Hello all! For those attending ODSC West virtually next week, I'm giving a talk about Prefect + Dask: https://odsc.com/speakers/next-generation-big-data-pipelines-with-prefect-and-dask/ Hope to see you there!

Hey all 👋 I’m Jacob. I work as a Data Science Engineer at Paidy, a fintech startup in Tokyo. We (@Arsenii and I) just released an article on how we used Prefect to build an in-house feature store on top of PostgreSQL that makes our lives much easier when building models. https://paidy.com/media_center/work_life/engineering/article/2XwaOiBrjiYXFpRPZWsv6B We are working on another post about how we deploy our flows that I think will be useful for a lot of people. Have a nice day! 🚀 (Sorry for the formatting issues, the blog was just released this week so there’s still some kinks to work out. 😅)

Hey all! My name is Ashton and I'm an ML Engineer at Sobeys, one of the main grocery retailers in Canada. I'm also one of the contributors that helped implement the Databricks tasks into Prefect. I wrote a blog post about how you can use these tasks in your Prefect flows to manage your Databricks jobs. Medium: https://towardsdatascience.com/tutorial-integrating-prefect-databricks-af426d8edf5c Personal Blog: https://ashton-sidhu.github.io/blog/prefect/databricks/etl/tutorial/guide/spark/2020/11/02/prefect-databricks.html If you have any questions or any feedback, feel free to shoot me a message! If you have any feedback or thoughts on the Databricks tasks (such as moving the connection string initialization inside the Databricks task 😉 ), I would love to hear them!

Hello friends, thought I’d share something fun. We’re a US-based digital bank and we’ve been using Prefect for our internal analytics jobs for a few months. But today, we went live with our new “wire processing system” that uses Prefect to generate and send wire processing instructions to the US Federal Reserve. Thought some of you might find such a use case interesting ;)

Hey guys I’m wandering if a lib to shedule notebook interest you https://github.com/jupyter-naas/naas It’s allow you to do naas.scheduler.add("* 9 * * *") in your notebook :)

Hey all 👋 , I’m a software engineer at Nvidia working on Rapids. I wrote a blogpost on how to how to use Prefect with GPU accelerated Rapids workflows: https://medium.com/rapids-ai/scheduling-optimizing-rapids-workflows-with-dask-and-prefect-6fc26d011bf. I also wanted to thank the Prefect slack community as I often found myself referring to questions already asked and answered. This alongside the great Prefect docs made the process really easy. 🚀

Hey everybody 🙂 I am working with a tech company in Europe who are looking to hire a Data Engineer on a 6 month contract, they are looking for somebody that has had exposure to Prefect. Tech is, Python, Airflow and Spark This will be a fully remote opportunity Please get in touch with me to discuss further - jack.clement@signifytechnology.com

I'm not too sure if this is the correct channel so let me know if there is a better place to post 🙂

Hi data lovers! If ensuring the reliability of your data pipelines is top of mind for you, I invite you to Data Quality Meetup on Thursday, November 19th at 9AM PST. It's a virtual event for the most active and hands-on data practitioners to share and discuss tools, ideas & success stories on managing data quality. 📈 We have an amazing list of speakers for the upcoming event including: • Scott Breitenother (co-founder of Locally Optimistic blog) • Dmitry Petrov (co-author of DVC – Data Version Control framework) • Mars Lan (co-author of DataHub) RSVP here If you are curious to learn more about Data Quality Meetup, you can check out the digests and recordings of the previous event.

Hi everyone, I just published a blogpost about a personal project 🤓 where I used Prefect to build a fully automated Medium stats pipeline 📊 to track and improve my writing performance on Medium. A big shoutout to @Dylan and @Jimmy Le for helping me with the Selenium issue and also to @Chris White for resetting my Cloud account after a stupid mistake! I hope you enjoy it: https://felix-vemmer.medium.com/building-a-fully-automated-medium-stats-pipeline-to-track-my-writing-performance-e76f791cdd59

Working at Endeavor (Ari from Entourage's company which is now a huge media holding company), building a central data warehouse for all its properties. Prefect's the star for getting all those acquisitions "onboarded". Don't have a blog post yet, but I'm hoping to maybe write one soon!

It's so great to be able to just write normal Python code with my normal tools and turn them into Prefect tasks and not have to worry about it lol. And I can comfortably say that the work we're doing would be basically impossible without the parameterized flows.

Yet another blog post mentioning awesome Prefect 🙂 https://towardsdatascience.com/15-essential-steps-to-build-reliable-data-pipelines-58847cb5d92f

We've been working for almost two months with prefect and we are loving it. Checkout our latest article written with the help of @Gareth Dwyer https://www.datarevenue.com/en-blog/what-we-are-loving-about-prefect

Here’s a great post from Kat Hempstalk at Smooth Sensors about building a data lake: https://medium.com/swlh/building-a-basic-data-lake-with-prefect-s3-glue-and-athena-676410fbbdc4

My brain hurts. Shout out to @Tyler Wanner, @Kyle Moon-Wright, @Kyle DeMerritt, @Amanda Wee for their support. Special shout out to @Anna Geller (old account) and @Laura Lorenz for their content on EKS and the new 0.14 updates. I've successfully deployed my flows using Prefect Cloud, AWS ECR, ECS, and Fargate. I'm a self taught dev and would love any feedback to clarify or include any concepts I may have luckily bypassed. https://lejimmy.com/distributed-data-pipelines-with-aws-ecs-fargate-and-prefect-cloud/

Prefect community is just amazing. Two people posting about the same thing on the same day! 😄 @Jimmy Le thanks for mentioning me. Here is my take on how to set up Prefect with an ECS agent: https://towardsdatascience.com/serverless-data-pipelines-made-easy-with-prefect-and-aws-ecs-fargate-7e25bacb450c

These are excellent, thanks for sharing y'all!

Awesome, thanks for sharing @Jimmy Le and @Anna Geller (old account)!!

Hello, as some of our projects in Infinite Lambda are using Prefect, we decided to share a bit of our great experience with this amazing tool. We needed a way to have a full CI/CD pipeline where we just push our code and it gets automatically deployed and then executed. Here is the article: https://infinitelambda.com/post/prefect-workflow-automation-azure-devops-aks/ I am sure it will be helpful to a lot of people. 😛refect:🚀