Vince Bob

    Vince Bob

    9 months ago
    hello, I am struggling on a great_expectations integration problem. I obviously use RunGreatExpectationsValidation task on a checkpoint I created in GE with:
    validation_task(
    context_root_dir=root_dir,
    checkpoint_name=expectation_checkpoint_name
    )
    When I run the command on GE (great_expectations --V3-api checkpoint run my_checkpoint), it works, but on prefect task, I have an exception: With GE V3 api:
    .....
    for batch in ge_checkpoint["batches"]:
    TypeError: 'Checkpoint' object is not subscriptable
    The same with GE V2 api
    ...
    for batch in ge_checkpoint["batches"]:
    TypeError: 'LegacyCheckpoint' object is not subscriptable
    Great_expectations=0.13.43 (also tried with 0.12.10 version) prefect=0.15.9 Anyone experienced this pb?
    Anna Geller

    Anna Geller

    9 months ago
    @Vince Bob it looks like this task has been written based on the V2 API and Great Expectations v3 introduces breaking changes, unfortunately. I’d like to reproduce the issue using this V2 example: https://github.com/superconductive/ge_tutorials/tree/main/getting_started_tutorial_final_v2_api Could you share your flow definition so that I can follow the same steps?
    @Vince Bob it could be that you would need to temporarily downgrade your Prefect version to 0.12 to make it work with the GE task: https://github.com/PrefectHQ/prefect/issues/4411
    Vince Bob

    Vince Bob

    9 months ago
    Do you mean only V2 generated expectations are supported?
    Thanks for the replies! I will check with V2 api and GE 0.12.0 version. I will also try with tutorials you linked
    Anna Geller

    Anna Geller

    9 months ago
    I believe so, but not 100% sure. Just opened an issue here: https://github.com/PrefectHQ/prefect/issues/5172