Channels
#prefect-community
Title
v
Varun Joshi
02/21/2021, 2:30 PMHey Prefect experts, I have a couple of questions:
1. I have entered a schedule (
Copy code
schedule = IntervalSchedule(start_date=datetime.datetime.now() + datetime.timedelta(seconds=1), interval=datetime.timedelta(minutes=1),)n
nicholas
02/21/2021, 5:29 PM
Hi @Varun Joshi - can you provide some more context to what you’ve tried? You’ll need to attach that schedule to your flow, and call flow.register(). Also when creating an interval schedule, there’s no need to add start_date unless there’s a specific point in the future you want to start the schedule on. The way you’ve constructed it would tell Prefect to start the schedule immediately, which is the default anyway.
v
Varun Joshi
02/22/2021, 7:29 AMHi Nicholas, here is what I've tried.
Copy code
schedule = IntervalSchedule(
start_date=datetime.datetime.now() + datetime.timedelta(seconds=1),
interval=datetime.timedelta(minutes=1),
)
with Flow("flowname", schedule= schedule) as flow:
source_system = Parameter("source_system",default="default")
source_database_id = Parameter("source_database_id",default=1)
result = function(source_system,source_database_id)
flow = Flow("flowname", storage=GCS(bucket="bucket-name"))
flow.storage.build()
flow.run()
flow.register(project_name="test_project")Also if I make any changes to my flow code, will the changes be updated if I run the code again?
n
nicholas
02/22/2021, 7:52 AM
Hi @Varun Joshi - a few comments on your code:
1:
flow.run()flow.storage.buildflow.register()Since you've defined a schedule on your flow, you don't need to also define one in the UI
Can you navigate to your flow's page in the UI and show me what you're seeing?
v
Varun Joshi
02/22/2021, 8:00 AMSure
🙌 1
I'm also using @task(log_stdout=True) but I'm not able to see my print statements
The schedule seems to be working now 🙂
However, I'm giving a couple of print statements in my task and I'm not able to see them in logs. Any clue why?
n
nicholas
02/22/2021, 8:16 AM
Hm, can you show me your task and your config at
~/.prefect/config.tomlv
Varun Joshi
02/22/2021, 8:28 AMIt's empty
n
nicholas
02/22/2021, 8:31 AM
Can you show me your task?
v
Varun Joshi
02/22/2021, 8:37 AM Copy code
@task(log_stdout=True)
def metadata(source_system,source_database_id):
host=''
username=''
password = ''
database = ''
port = ''
charset = ''
metadata_cnxn = pymysql.connect(host=host, user=username, passwd=password,
db=database, port=int(port), charset=charset,
autocommit=True, cursorclass=pymysql.cursors.DictCursor)
params = [source_database_id,source_system]
database_cur = pvr_metadata_cnxn.cursor()
database_cur.execute(metadata_query, params)
database_result = database_cur.fetchall()
print(database_result)Something on these lines
I tried using logger.info also. Still not able to see logs
So I just realized that I'm not able to see anything in tasks or in the schematic either
Sorry to be bothering you so much
n
nicholas
02/22/2021, 6:54 PM
Hi @Varun Joshi - please refrain from calling Prefect employees unless they've agreed to this beforehand; we answer support questions as quickly as possible in addition to our work of building and maintaining the various tools we're using!
To answer your question, it looks like you're not explicitly calling that task in your flow's context. It would look something like this:
Copy code
with Flow("flowname", schedule= schedule) as flow:
# ... other tasks
metadata()v
Varun Joshi
02/23/2021, 5:03 AMHi @nicholas, I apologize. That happened by accident
n
nicholas
02/23/2021, 8:38 AM
No worries, let me know if that helps answer your question
3 Views