https://prefect.io
Join Slack
For a private db in AWS can I connect to it using MysqlFetch in a function without `@task` decorato...
# ask-community
a
For a private db in AWS can I connect to it using MysqlFetch in a function without 
@task
decorator ? I get 
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on '<http://xyz-mysql8-cluster.cluster-fa43asdfd.us-west-2.rds.amazonaws.com|xyz-mysql8-cluster.cluster-fa43asdfd.us-west-2.rds.amazonaws.com>' (timed out)")
every time. Any suggestions on how to approach in solving this?
✅ 1
I am using 
DatesClock
as schedule and fetching the dates from a private db, which throws the 'can't connect to MySQL server' error.
a
it depends, which Prefect product are you using, and which version? what's your flow code?
a
Prefect cloud, version 1.2.2. Its a MySQLFetch call that is failing, and its outside the Flow. We need to call the MySQLFetch before to fetch dates from the db which will then be passed into schedule.
If i turn off the schedule then it works fine. Does MySQLFetch work outside a Flow?
a
no, all tasks must be called from the flow, although if you just want to test something out, you could manually trigger it outside of the flow by invoking the task's .run() method
Copy code
MySQLFetch(init_args).run(runtime_args)
a
no but its not actually a task. Its a simple function which involves calling the db to fetch dates.
Though the db is private and we use jumphost or vpn to connect to it. So my question is does MySQLFetch work as a non-task method and still connect to the db?
a
how would you do it in Python? once you figure it out, you can run it in Prefect
a
No but there are other db calls also which work fine when run as a task. But this is for setting the schedule so it can't run as a task.
I think the jumphost part is solved as prefect is deployed in the same vpc as the db. And we have a working flow of this same code registered in cloud and is picked up by an agent running in the vm.
a
this is not the intended use case fir using schedules, I would encourage you to rethink the process and define the schedule either in your code or via UI rather than retrieving it from MySQL DB
32 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by