I am trying to run a stitch job using the code exactly how it shows here. https://alessandrolollo.github.io/prefect-stitch/. I'm getting an error that says "TypeError: missing a required argument: 'credentials'".

Hi @Ashley Felber - the docs for that collection appear to be out of date from this, it looks like you'll need to create a

StitchCredentials

block and pass that into

start_replication_job

, where you could either register the block with your workspace and then create and instance of it in the UI (recommended) in the CLI

prefect block register -m prefect_stitch.credentials

(you only have to do this once) in your flow

from prefect_stitch.credentials import StitchCredentials

@flow
def your_flow():
   stitch_creds = StitchCredentials.load("my-new-creds")

   start_replication_job(credentials=stitch_creds)

or you could do everything programmatically (and keep your access token in a Secret block) in your flow

from prefect_stitch.credentials import StitchCredentials

@flow
def your_flow():
   stitch_creds = StitchCredentials(access_token=Secret.load("my-stitch-access-token"))

   start_replication_job(credentials=stitch_creds)

Great I will try that

Ahh I see, the task is written by the maintainer (which is not Prefect in this case) to just hit the endpoint and then return. We usually like to implement a

verb_and_wait_for_completion

which involves polling the status of the job that's kicked off, but I'm noticing here that the implementor may not have done so because there's no endpoint to check a replication job status that I can find, only start and stop. This looks like a limitation of stitch, but if there's an endpoint to check job status that I'm missing, I can open an issue to add support for that

Got it thanks. After digging in, there isn’t an endpoint to check the job status. The job is broken up into extraction and load. And it is possible to check the status of both of those through the API. It’s a bit tricky though because there isn’t a job ID linking each part. And a load only occurs if an extraction results in incremental changes. There also can be multiple loads that need to finish if there are multiple tables with incremental changes. However, the API doesn’t indicate if there are loads that need to done and how many/which ones after the extraction is complete. Anyway I just put together a hacky solution using the extraction and load status through the API that isn’t perfect but hopefully it will work!

if your solution ends up working for you, feel free to open an issue on prefect-stitch and then we can work on getting that functionality into the collection one way or another!