https://prefect.io logo
j

Joseph Haaga

10/30/2020, 6:43 PM
I have a list of
newspaper.Article
objects (from
newspaper3k
) that I would like to analyze w/ a Spacy model. Is
unmapped
an appropriate way to pass in the
spacy
model to the task without initializing it each time? e.g.
Copy code
@task
def get_articles() -> List[Article]:
   ...
   return articles

@task
def load_spacy():
   return spacy.load("en_core_web_md")  # this is a slow operation

@task
def extract_organizations(article: Article, nlp) -> Set:
   return nlp(article.text).ents

with Flow("Extract Orgs from News Articles"):
   articles = get_articles()
   nlp = load_spacy()
   extract_organizations.map(articles, nlp=unmapped(nlp))
d

Dylan

10/30/2020, 6:43 PM
Hi @Joseph Haaga Welcome! Yes, you’ve nailed it 💯
🙏 1