Using Scrapy and CrawlProcess in Prefect 2 Today I learned Scrapy CrawlProcess complains about needing to be the main process if used in tasks or subflows. I had to move the Scrapy process under the main flow for it to work.