https://prefect.io logo
Title
t

Tim Wisniewski

03/17/2023, 12:39 PM
Hi folks, I've built my first Prefect flow—it interacts with two banking APIs—and I'm ready to deploy it to production, but I'm struggling a bit to wrap my head around all the pieces I now seem to need.. I figured I'd deploy it to Prefect Cloud for simplicity's sake, but (a) my custom block doesn't seem to be working (it says it 'failed validation'), and (b) the docs seem to suggest that even though I'm running it in Prefect Cloud, I still need to bring my own storage (e.g. S3 bucket) and infrastructure (e.g. k8s cluster)? Am I understanding that correctly, or is there a simpler way to get a basic flow running in production? (Regarding (a), I created a custom block in a local package that just stores client_id/client_secret and has a refresh_token function. I import it via
from prefect_monzo import MonzoCredentials
)
r

redsquare

03/17/2023, 12:43 PM
Hey Tim - yes you do need to bring your own tin in one form or another - until prefect have some sort of managed execution - not sure if/when
Did you get the validation error when attempting to save your custom block?
t

Tim Wisniewski

03/17/2023, 12:50 PM
@redsquare By 'save' do you mean 'register'? If so, no—it said "Successfully registered 1 block", and then I configured it in the UI on Prefect Cloud.
I'd ran
prefect block register --file prefect_monzo/credentials.py
- the same way I did when I had it working locally
Okay, seems like there's a big gap between getting it running locally and getting it running in production. I'll look around for some tutorials for getting it running in production.
r

redsquare

03/17/2023, 12:58 PM
You can load the block with validate=False
t

Tim Wisniewski

03/17/2023, 1:07 PM
@redsquare er, I suppose I could, yeah, but doesn’t the error message imply I’m doing something wrong?
a versioning thing
t

Tim Wisniewski

03/17/2023, 1:14 PM
@redsquare I'm sorry, I just took another look at the error, and it's not related to my custom block; it's related to my SqlAlchemyConnector block.
from prefect_sqlalchemy import SqlAlchemyConnector
database_block = SqlAlchemyConnector.load("db")
prefect.exceptions.ScriptError: Script at 'sync.py' encountered an exception: RuntimeError("Unable to load 'db' of block type None due to failed validation. To load without validation, try loading again with `validate=False`.")
r

redsquare

03/17/2023, 1:16 PM
does that open in the UI?
t

Tim Wisniewski

03/17/2023, 1:17 PM
I added the SqlAlchemyConnector block from the UI, and named 'db', if that's what you mean?
r

redsquare

03/17/2023, 1:19 PM
yeah that form looks buggy in the ui depending on what you select
t

Tim Wisniewski

03/17/2023, 1:23 PM
Oh, I see what you mean—looks like it was missing a value but let me save anyway, and when I edited the value, it dropped the password
r

redsquare

03/17/2023, 1:23 PM
I've raised it with them
t

Tim Wisniewski

03/17/2023, 1:24 PM
Seems like it's running now. I've still got to run the agent from my laptop, but I got an error message that suggests it's at least running my code properly, so that's progress! Thanks for your help
👍 1
Just need to work out how to run the agent from prefect cloud rather than my laptop, but will dig back into the docs
r

redsquare

03/17/2023, 1:34 PM
yeah depends what infra you have available to you