https://prefect.io logo
Title
t

Tony

12/14/2022, 5:43 PM
I have another issue in 1.0 that I am not sure how to classify. I have a download zip file task at the beginning of my flow. I have used BeautifulSoup to grab the dynamic zip file urls from the landing page and convert them into class called Page(). Then these links are exposed as a list of links. The first line of my Flow() is page = Page(). The problem is it seems this "page" object is cached as the newly posted links are not processed at all. It appears this is default behavior but How do I turn it off or bypass it? I need a new page = Page() object every time the flow runs... The flow is created with the following: result=PrefectResult(),storage=Local() Should I remove those?
m

Mason Menges

12/14/2022, 9:33 PM
Hey @Tony you'd likely need to use script storage to do this, by default the anything in the flow context is determined at registration time unless it's running within a task, (another option you could do here but might not be particularly efficient for your use case). That said this is workaround is no longer necessary in prefect 2 if you have the ability to create this flow Using that.