Channels
#prefect-gcp
Title
# prefect-gcp
m
Miguel Moncada
01/04/2023, 11:36 AM👋 Hi, I'm encountering the same error as described above, incognito did not help unfortunately.
Copy code
Submission failed. KeyError: "No class found for dispatch key 'cloud-run-job' in registry for type 'Block'."✅ 1
Deployment file:
Copy code
from prefect.deployments import Deployment
from prefect_gcp.cloud_run import CloudRunJob
from prefect.filesystems import GCS
from prefect.orion.schemas.schedules import CronSchedule
from dataflows.flows.hello_flow import hello
storage = GCS.load("default")
infrastructure = CloudRunJob("default")
def deploy_hello_flow():
deployment = Deployment.build_from_flow(
flow=hello,
name="hello_flow_deployment",
work_queue_name="default",
storage=storage,
path="hello",
infrastructure=infrastructure,
schedule=CronSchedule(cron="* * * * *", timezone="UTC"),
)
deployment.apply()
if __name__ == "__main__":
deploy_hello_flow()I've tried recreating the CloudRun block without success
Also registering the
prefect_gcp Copy code
❯ prefect block register -m prefect_gcp
Successfully registered 6 blocks
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Registered Blocks ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ BigQuery Warehouse │
│ GCP Cloud Run Job │
│ GCP Credentials │
│ GcpSecret │
│ GCS Bucket │
│ Vertex AI Custom Training Job │
└───────────────────────────────┘
To configure the newly registered blocks, go to the Blocks page in the Prefect UI.And I think I've correctly checked that
prefect-gcp Copy code
root@prefect-cloud-vm:/opt/prefect# pip freeze
aiosqlite==0.18.0
alembic==1.9.1
anyio==3.6.2
apprise==1.2.1
asgi-lifespan==2.0.0
asyncpg==0.27.0
cachetools==5.2.0
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
cloudpickle==2.2.0
colorama==0.4.6
commonmark==0.9.1
coolname==2.1.0
croniter==1.3.8
cryptography==38.0.4
dask==2022.12.1
dataflows @ file:///opt/prefect
dateparser==1.1.5
distributed==2022.12.1
docker==6.0.1
fastapi==0.88.0
fsspec==2022.11.0
google-api-core==2.11.0
google-api-python-client==2.70.0
google-auth==2.15.0
google-auth-httplib2==0.1.0
googleapis-common-protos==1.57.0
greenlet==2.0.1
griffe==0.25.2
h11==0.14.0
h2==4.1.0
HeapDict==1.0.1
hpack==4.0.0
httpcore==0.16.3
httplib2==0.21.0
httpx==0.23.1
hyperframe==6.0.1
idna==3.4
Jinja2==3.1.2
jsonpatch==1.32
jsonpointer==2.3
kubernetes==25.3.0
locket==1.0.0
Mako==1.2.4
Markdown==3.4.1
MarkupSafe==2.1.1
msgpack==1.0.4
oauthlib==3.2.2
orjson==3.8.3
packaging==22.0
partd==1.3.0
pathspec==0.10.3
pendulum==2.1.2
prefect @ file:///opt/prefect/dist/prefect.tar.gz
prefect-dask==0.2.2
prefect-gcp==0.2.2
protobuf==4.21.12
psutil==5.9.4
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pydantic==1.10.3
Pygments==2.13.0
pyparsing==3.0.9
python-dateutil==2.8.2
python-slugify==7.0.0
pytz==2022.7
pytz-deprecation-shim==0.1.0.post0
pytzdata==2020.1
PyYAML==6.0
readchar==4.0.3
regex==2022.10.31
requests==2.28.1
requests-oauthlib==1.3.1
rfc3986==1.5.0
rich==12.6.0
rsa==4.9
six==1.16.0
sniffio==1.3.0
sortedcontainers==2.4.0
SQLAlchemy==1.4.45
starlette==0.22.0
tblib==1.7.0
text-unidecode==1.3
toml==0.10.2
toolz==0.12.0
tornado==6.2
typer==0.7.0
typing_extensions==4.4.0
tzdata==2022.7
tzlocal==4.2
uritemplate==4.1.1
urllib3==1.26.13
uvicorn==0.20.0
websocket-client==1.4.2
zict==2.2.0Ok, testing something out, will update later
👍 2
I was checking, because importing
prefect_gcpprefect_gcpbigquerycloud_storagecloud_runbut the same error prompts after installing, verifying I can now import
prefect_gcp Copy code
root@prefect-cloud-vm:/opt/prefect# prefect block register -m prefect_gcp
Successfully registered 6 blocks
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Registered Blocks ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ BigQuery Warehouse │
│ GCP Cloud Run Job │
│ GCP Credentials │
│ GcpSecret │
│ GCS Bucket │
│ Vertex AI Custom Training Job │
└───────────────────────────────┘
To configure the newly registered blocks, go to the Blocks page in the Prefect UI.
root@prefect-cloud-vm:/opt/prefect# python
Python 3.10.9 (main, Dec 21 2022, 08:51:48) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import prefect_gcp
>>>Not sure what it was... it just works fine after re-creating the infrastructure
j
Joseph Desjardins
01/04/2023, 4:36 PMHi Miguel! My name is Joey, and I’m with our Product Advocate team here at Prefect. We’d love to hear how your experience with Prefect has been so far, and offer you time to meet with one of our engineers to answer technical questions. DM me if you’re interested, and we can set up a time to meet. Thanks, and looking forward to chatting soon!
🙇 1
n
Nelson Griffiths
01/18/2023, 10:25 PM@Miguel Moncada did you ever figure out what was causing this? I am running into it again and can't seem to get it to work this time.
@Jeff Hale not sure if you have any insights on this?
m
Miguel Moncada
01/19/2023, 8:12 AM@Nelson Griffiths mmm I don't think I did, to be honest. Have you tried registering the block from within the agent's environment and see if the error message is any different?
I had adapted my template using Anna's as reference and have not encountered this error again (at least for now!)
n
Nelson Griffiths
01/19/2023, 2:25 PMThanks. If I go ahead and reset everything it works again. Not sure why I occasionally need to re-register blocks.
🤔 1
m
Miguel Moncada
01/19/2023, 2:28 PMWhere are you running your agent? in a VM?
n
Nelson Griffiths
01/19/2023, 2:59 PMI was rubbing it in a vm until I was tired of debugging remotely and then just started running it locally on my laptop. Still hit similar issues though where every once in a while it stops working and I have to register blocks again and recreate deployments
27 Views