Daniel Kornhauser
01/31/2022, 10:26 PMKevin Kho
Daniel Kornhauser
02/01/2022, 3:00 PMpipe-line-source-name
logic
tasks.py
flow.py
pubmed/
logic/
pubmed-bulk-download.py
pubmed-api-pull.py
pubmed-to-jsonl.py
pubmed-enrichment.py
pubmed-indexer.py
pubmed-storage.py
pubmed-tasks.py
pubmed-flow.py
We will have many other sources like rxiv, twitter, etc …
Each as a directory with the structure above
Wonder what is your opinion about the pubmed-
name redundancy, so far we haven’t needed to create multiple files containing tasks or flows, we actually place everything in a single flow.Kevin Kho
Daniel Kornhauser
02/01/2022, 3:13 PMKevin Kho