https://prefect.io logo
Title
b

BBPP

01/11/2023, 9:35 AM
Howdy. Is it possible to invoke Glue Job using Prefect?
1
p

Pim Claessens

01/11/2023, 9:56 AM
You can use the boto3 library for that: Glue — Boto3 Docs 1.26.46 documentation (amazonaws.com)
b

BBPP

01/11/2023, 9:58 AM
Would you mind sharing an example on how to do that?
p

Pim Claessens

01/11/2023, 10:02 AM
glue = boto3.client('glue')
job_run_id = glue.start_job_run(JobName=f'{job_name}')
And you can fetch the status of the job with:
glue.get_job_run(JobName=job_name, RunId=job_run_id.get("JobRunId")).get("JobRun").get(
        "JobRunState")
b

BBPP

01/11/2023, 10:03 AM
Thanks!