https://prefect.io logo
Powered by
Title
m

Miguel Moncada

01/04/2023, 11:36 AM
👋 Hi, I'm encountering the same error as described above, incognito did not help unfortunately. 
Submission failed. KeyError: "No class found for dispatch key 'cloud-run-job' in registry for type 'Block'."
Probably it's due to a misconfiguration on the deployment python file (or maybe the infrastructure block itself?), adding more info to the thread 🤔
✅ 1
Deployment file: 
from prefect.deployments import Deployment
from prefect_gcp.cloud_run import CloudRunJob
from prefect.filesystems import GCS
from prefect.orion.schemas.schedules import CronSchedule
from dataflows.flows.hello_flow import hello

storage = GCS.load("default")
infrastructure = CloudRunJob("default")


def deploy_hello_flow():
    deployment = Deployment.build_from_flow(
        flow=hello,
        name="hello_flow_deployment",
        work_queue_name="default",
        storage=storage,
        path="hello",
        infrastructure=infrastructure,
        schedule=CronSchedule(cron="* * * * *", timezone="UTC"),
    )
    deployment.apply()


if __name__ == "__main__":
    deploy_hello_flow()
I've tried recreating the CloudRun block without success
Also registering the 
prefect_gcp
blocks: 
❯ prefect block register -m prefect_gcp
Successfully registered 6 blocks

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Registered Blocks             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ BigQuery Warehouse            │
│ GCP Cloud Run Job             │
│ GCP Credentials               │
│ GcpSecret                     │
│ GCS Bucket                    │
│ Vertex AI Custom Training Job │
└───────────────────────────────┘

 To configure the newly registered blocks, go to the Blocks page in the Prefect UI.
And I think I've correctly checked that 
prefect-gcp
is installed in the docker container where the agent is running: 
root@prefect-cloud-vm:/opt/prefect# pip freeze
aiosqlite==0.18.0
alembic==1.9.1
anyio==3.6.2
apprise==1.2.1
asgi-lifespan==2.0.0
asyncpg==0.27.0
cachetools==5.2.0
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
cloudpickle==2.2.0
colorama==0.4.6
commonmark==0.9.1
coolname==2.1.0
croniter==1.3.8
cryptography==38.0.4
dask==2022.12.1
dataflows @ file:///opt/prefect
dateparser==1.1.5
distributed==2022.12.1
docker==6.0.1
fastapi==0.88.0
fsspec==2022.11.0
google-api-core==2.11.0
google-api-python-client==2.70.0
google-auth==2.15.0
google-auth-httplib2==0.1.0
googleapis-common-protos==1.57.0
greenlet==2.0.1
griffe==0.25.2
h11==0.14.0
h2==4.1.0
HeapDict==1.0.1
hpack==4.0.0
httpcore==0.16.3
httplib2==0.21.0
httpx==0.23.1
hyperframe==6.0.1
idna==3.4
Jinja2==3.1.2
jsonpatch==1.32
jsonpointer==2.3
kubernetes==25.3.0
locket==1.0.0
Mako==1.2.4
Markdown==3.4.1
MarkupSafe==2.1.1
msgpack==1.0.4
oauthlib==3.2.2
orjson==3.8.3
packaging==22.0
partd==1.3.0
pathspec==0.10.3
pendulum==2.1.2
prefect @ file:///opt/prefect/dist/prefect.tar.gz
prefect-dask==0.2.2
prefect-gcp==0.2.2
protobuf==4.21.12
psutil==5.9.4
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pydantic==1.10.3
Pygments==2.13.0
pyparsing==3.0.9
python-dateutil==2.8.2
python-slugify==7.0.0
pytz==2022.7
pytz-deprecation-shim==0.1.0.post0
pytzdata==2020.1
PyYAML==6.0
readchar==4.0.3
regex==2022.10.31
requests==2.28.1
requests-oauthlib==1.3.1
rfc3986==1.5.0
rich==12.6.0
rsa==4.9
six==1.16.0
sniffio==1.3.0
sortedcontainers==2.4.0
SQLAlchemy==1.4.45
starlette==0.22.0
tblib==1.7.0
text-unidecode==1.3
toml==0.10.2
toolz==0.12.0
tornado==6.2
typer==0.7.0
typing_extensions==4.4.0
tzdata==2022.7
tzlocal==4.2
uritemplate==4.1.1
urllib3==1.26.13
uvicorn==0.20.0
websocket-client==1.4.2
zict==2.2.0
Ok, testing something out, will update later
👍 2
I was checking, because importing 
prefect_gcp
from the VM's container was failing (I'm using this Dockerfile & requirements.txt for my agent and the container was apparently missing 
prefect_gcp
submodules 
bigquery
, 
cloud_storage
, 
cloud_run
)
but the same error prompts after installing, verifying I can now import 
prefect_gcp
and registering the blocks 
root@prefect-cloud-vm:/opt/prefect# prefect block register -m prefect_gcp
Successfully registered 6 blocks

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Registered Blocks             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ BigQuery Warehouse            │
│ GCP Cloud Run Job             │
│ GCP Credentials               │
│ GcpSecret                     │
│ GCS Bucket                    │
│ Vertex AI Custom Training Job │
└───────────────────────────────┘

 To configure the newly registered blocks, go to the Blocks page in the Prefect UI.

root@prefect-cloud-vm:/opt/prefect# python
Python 3.10.9 (main, Dec 21 2022, 08:51:48) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import prefect_gcp
>>>
Not sure what it was... it just works fine after re-creating the infrastructure
j

Joseph Desjardins

01/04/2023, 4:36 PM
Hi Miguel! My name is Joey, and I’m with our Product Advocate team here at Prefect. We’d love to hear how your experience with Prefect has been so far, and offer you time to meet with one of our engineers to answer technical questions. DM me if you’re interested, and we can set up a time to meet. Thanks, and looking forward to chatting soon!
🙇 1
n

Nelson Griffiths

01/18/2023, 10:25 PM
@Miguel Moncada did you ever figure out what was causing this? I am running into it again and can't seem to get it to work this time.
@Jeff Hale not sure if you have any insights on this?
m

Miguel Moncada

01/19/2023, 8:12 AM
@Nelson Griffiths mmm I don't think I did, to be honest. Have you tried registering the block from within the agent's environment and see if the error message is any different? I had adapted my template using Anna's as reference and have not encountered this error again (at least for now!)
n

Nelson Griffiths

01/19/2023, 2:25 PM
Thanks. If I go ahead and reset everything it works again. Not sure why I occasionally need to re-register blocks.
🤔 1
m

Miguel Moncada

01/19/2023, 2:28 PM
Where are you running your agent? in a VM?
n

Nelson Griffiths

01/19/2023, 2:59 PM
I was rubbing it in a vm until I was tired of debugging remotely and then just started running it locally on my laptop. Still hit similar issues though where every once in a while it stops working and I have to register blocks again and recreate deployments
27 Views
#prefect-gcpJoin Slack
Powered by