Joseph Haaga
10/30/2020, 6:43 PMnewspaper.Article
objects (from newspaper3k
) that I would like to analyze w/ a Spacy model. Is unmapped
an appropriate way to pass in the spacy
model to the task without initializing it each time?
e.g.
@task
def get_articles() -> List[Article]:
...
return articles
@task
def load_spacy():
return spacy.load("en_core_web_md") # this is a slow operation
@task
def extract_organizations(article: Article, nlp) -> Set:
return nlp(article.text).ents
with Flow("Extract Orgs from News Articles"):
articles = get_articles()
nlp = load_spacy()
extract_organizations.map(articles, nlp=unmapped(nlp))
Dylan