Varun Joshi
02/21/2021, 2:30 PMschedule = IntervalSchedule(start_date=datetime.datetime.now() + datetime.timedelta(seconds=1), interval=datetime.timedelta(minutes=1),)
as described on the website. Then why don't I see the flow running every minute.
2. If I make any changes to my flow code, will just running it again reflect changes in the UI?nicholas
Varun Joshi
02/22/2021, 7:29 AMschedule = IntervalSchedule(
start_date=datetime.datetime.now() + datetime.timedelta(seconds=1),
interval=datetime.timedelta(minutes=1),
)
with Flow("flowname", schedule= schedule) as flow:
source_system = Parameter("source_system",default="default")
source_database_id = Parameter("source_database_id",default=1)
result = function(source_system,source_database_id)
flow = Flow("flowname", storage=GCS(bucket="bucket-name"))
flow.storage.build()
flow.run()
flow.register(project_name="test_project")
Do I also have to go to the UI and setup the schedule?nicholas
flow.run()
is only used to run your flow locally. There's not need to include it if you're registering a flow with Prefect Cloud or Prefect Server
2: There's no need to call flow.storage.build
except perhaps in advanced scenarios - Prefect will do that for you when you call flow.register()
.Varun Joshi
02/22/2021, 8:00 AMnicholas
~/.prefect/config.toml
?Varun Joshi
02/22/2021, 8:28 AMnicholas
Varun Joshi
02/22/2021, 8:37 AM@task(log_stdout=True)
def metadata(source_system,source_database_id):
host=''
username=''
password = ''
database = ''
port = ''
charset = ''
metadata_cnxn = pymysql.connect(host=host, user=username, passwd=password,
db=database, port=int(port), charset=charset,
autocommit=True, cursorclass=pymysql.cursors.DictCursor)
params = [source_database_id,source_system]
database_cur = pvr_metadata_cnxn.cursor()
database_cur.execute(metadata_query, params)
database_result = database_cur.fetchall()
print(database_result)
nicholas
with Flow("flowname", schedule= schedule) as flow:
# ... other tasks
metadata()
Varun Joshi
02/23/2021, 5:03 AMnicholas