https://prefect.io logo
v

Varun Joshi

02/21/2021, 2:30 PM
Hey Prefect experts, I have a couple of questions: 1. I have entered a schedule (
Copy code
schedule = IntervalSchedule(start_date=datetime.datetime.now() + datetime.timedelta(seconds=1), interval=datetime.timedelta(minutes=1),)
as described on the website. Then why don't I see the flow running every minute. 2. If I make any changes to my flow code, will just running it again reflect changes in the UI?
n

nicholas

02/21/2021, 5:29 PM
Hi @Varun Joshi - can you provide some more context to what you’ve tried? You’ll need to attach that schedule to your flow, and call flow.register(). Also when creating an interval schedule, there’s no need to add start_date unless there’s a specific point in the future you want to start the schedule on. The way you’ve constructed it would tell Prefect to start the schedule immediately, which is the default anyway.
v

Varun Joshi

02/22/2021, 7:29 AM
Hi Nicholas, here is what I've tried.
Copy code
schedule = IntervalSchedule(
    start_date=datetime.datetime.now() + datetime.timedelta(seconds=1),
    interval=datetime.timedelta(minutes=1),
)



with Flow("flowname", schedule= schedule) as flow:
    source_system = Parameter("source_system",default="default")
    source_database_id = Parameter("source_database_id",default=1)
    result = function(source_system,source_database_id)


flow = Flow("flowname", storage=GCS(bucket="bucket-name"))
flow.storage.build()
flow.run()
flow.register(project_name="test_project")
Do I also have to go to the UI and setup the schedule?
Also if I make any changes to my flow code, will the changes be updated if I run the code again?
n

nicholas

02/22/2021, 7:52 AM
Hi @Varun Joshi - a few comments on your code: 1:
flow.run()
is only used to run your flow locally. There's not need to include it if you're registering a flow with Prefect Cloud or Prefect Server 2: There's no need to call
flow.storage.build
except perhaps in advanced scenarios - Prefect will do that for you when you call
flow.register()
.
Since you've defined a schedule on your flow, you don't need to also define one in the UI
Can you navigate to your flow's page in the UI and show me what you're seeing?
v

Varun Joshi

02/22/2021, 8:00 AM
Sure
🙌 1
I'm also using @task(log_stdout=True) but I'm not able to see my print statements
The schedule seems to be working now 🙂
However, I'm giving a couple of print statements in my task and I'm not able to see them in logs. Any clue why?
n

nicholas

02/22/2021, 8:16 AM
Hm, can you show me your task and your config at
~/.prefect/config.toml
?
v

Varun Joshi

02/22/2021, 8:28 AM
It's empty
n

nicholas

02/22/2021, 8:31 AM
Can you show me your task?
v

Varun Joshi

02/22/2021, 8:37 AM
Copy code
@task(log_stdout=True)
def metadata(source_system,source_database_id):
    host=''
    username=''
    password = ''
    database = ''
    port = ''
    charset = ''
    metadata_cnxn = pymysql.connect(host=host, user=username, passwd=password,
                                   db=database, port=int(port), charset=charset,
                                   autocommit=True, cursorclass=pymysql.cursors.DictCursor)
    params = [source_database_id,source_system]


    database_cur = pvr_metadata_cnxn.cursor()
    database_cur.execute(metadata_query, params)
    database_result = database_cur.fetchall()
    print(database_result)
Something on these lines
I tried using logger.info also. Still not able to see logs
So I just realized that I'm not able to see anything in tasks or in the schematic either
Sorry to be bothering you so much
n

nicholas

02/22/2021, 6:54 PM
Hi @Varun Joshi - please refrain from calling Prefect employees unless they've agreed to this beforehand; we answer support questions as quickly as possible in addition to our work of building and maintaining the various tools we're using!
To answer your question, it looks like you're not explicitly calling that task in your flow's context. It would look something like this:
Copy code
with Flow("flowname", schedule= schedule) as flow:
    # ... other tasks
    metadata()
v

Varun Joshi

02/23/2021, 5:03 AM
Hi @nicholas, I apologize. That happened by accident
n

nicholas

02/23/2021, 8:38 AM
No worries, let me know if that helps answer your question
3 Views