Danilo Drobac
06/10/2022, 11:39 AMprefect cloud login --key {key} --workspace {workspace}
related to timeouts?
I'm trying to run the command in a CI/CD pipeline through Google Cloud Build but it fails due to a timeout and I have very little information in the error.Anna Geller
06/10/2022, 11:41 AMprefect cloud login -k API_KEY -w "annaprefect/prod"
Danilo Drobac
06/10/2022, 11:42 AMAnna Geller
06/10/2022, 11:42 AMDanilo Drobac
06/10/2022, 11:43 AMAnna Geller
06/10/2022, 11:44 AMDanilo Drobac
06/10/2022, 11:47 AMStep #2 - "Prefect Version": Version: 2.0b6
Step #2 - "Prefect Version": API version: 0.5.0
Step #2 - "Prefect Version": Python version: 3.9.13
Step #2 - "Prefect Version": Git commit: 502ee008
Step #2 - "Prefect Version": Built: Tue, Jun 7, 2022 4:58 PM
Step #2 - "Prefect Version": OS/Arch: linux/x86_64
Step #2 - "Prefect Version": Profile: default
Step #2 - "Prefect Version": Server type: ephemeral
Step #2 - "Prefect Version": Server:
Step #2 - "Prefect Version": Database: sqlite
Step #2 - "Prefect Version": SQLite version: 3.34.1
Anna Geller
06/10/2022, 11:48 AMDanilo Drobac
06/10/2022, 11:48 AMAnna Geller
06/10/2022, 11:49 AMDanilo Drobac
06/10/2022, 11:50 AMStarting Step #2 - "Login to Prefect Cloud 2.0"
Step #2 - "Login to Prefect Cloud 2.0": Already have image (with digest): <http://gcr.io/nzyte-modern-data-stack/prefect-builder|gcr.io/nzyte-modern-data-stack/prefect-builder>
Step #2 - "Login to Prefect Cloud 2.0": Traceback (most recent call last):
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/anyio/_core/_sockets.py", line 186, in connect_tcp
Step #2 - "Login to Prefect Cloud 2.0": addr_obj = ip_address(remote_host)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/ipaddress.py", line 53, in ip_address
Step #2 - "Login to Prefect Cloud 2.0": raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
Step #2 - "Login to Prefect Cloud 2.0": ValueError: '<http://api-beta.prefect.io|api-beta.prefect.io>' does not appear to be an IPv4 or IPv6 address
Step #2 - "Login to Prefect Cloud 2.0":
Step #2 - "Login to Prefect Cloud 2.0": During handling of the above exception, another exception occurred:
Step #2 - "Login to Prefect Cloud 2.0":
Step #2 - "Login to Prefect Cloud 2.0": Traceback (most recent call last):
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
Step #2 - "Login to Prefect Cloud 2.0": stream: anyio.abc.ByteStream = await anyio.connect_tcp(
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/anyio/_core/_sockets.py", line 189, in connect_tcp
Step #2 - "Login to Prefect Cloud 2.0": gai_res = await getaddrinfo(
Step #2 - "Login to Prefect Cloud 2.0": asyncio.exceptions.CancelledError
Step #2 - "Login to Prefect Cloud 2.0":
Step #2 - "Login to Prefect Cloud 2.0": During handling of the above exception, another exception occurred:
Step #2 - "Login to Prefect Cloud 2.0":
Step #2 - "Login to Prefect Cloud 2.0": Traceback (most recent call last):
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/_exceptions.py", line 8, in map_exceptions
Step #2 - "Login to Prefect Cloud 2.0": yield
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
Step #2 - "Login to Prefect Cloud 2.0": stream: anyio.abc.ByteStream = await anyio.connect_tcp(
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
Step #2 - "Login to Prefect Cloud 2.0": raise TimeoutError
Step #2 - "Login to Prefect Cloud 2.0": TimeoutError
Step #2 - "Login to Prefect Cloud 2.0":
Step #2 - "Login to Prefect Cloud 2.0": During handling of the above exception, another exception occurred:
Step #2 - "Login to Prefect Cloud 2.0":
Step #2 - "Login to Prefect Cloud 2.0": Traceback (most recent call last):
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
Step #2 - "Login to Prefect Cloud 2.0": yield
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
Step #2 - "Login to Prefect Cloud 2.0": resp = await self._pool.handle_async_request(req)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
Step #2 - "Login to Prefect Cloud 2.0": raise exc
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
Step #2 - "Login to Prefect Cloud 2.0": response = await connection.handle_async_request(request)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/_async/connection.py", line 86, in handle_async_request
Step #2 - "Login to Prefect Cloud 2.0": raise exc
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/_async/connection.py", line 63, in handle_async_request
Step #2 - "Login to Prefect Cloud 2.0": stream = await self._connect(request)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/_async/connection.py", line 111, in _connect
Step #2 - "Login to Prefect Cloud 2.0": stream = await self._network_backend.connect_tcp(**kwargs)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/backends/auto.py", line 29, in connect_tcp
Step #2 - "Login to Prefect Cloud 2.0": return await self._backend.connect_tcp(
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
Step #2 - "Login to Prefect Cloud 2.0": stream: anyio.abc.ByteStream = await anyio.connect_tcp(
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
Step #2 - "Login to Prefect Cloud 2.0": self.gen.throw(typ, value, traceback)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
Step #2 - "Login to Prefect Cloud 2.0": raise to_exc(exc)
Step #2 - "Login to Prefect Cloud 2.0": httpcore.ConnectTimeout
Step #2 - "Login to Prefect Cloud 2.0":
Step #2 - "Login to Prefect Cloud 2.0": The above exception was the direct cause of the following exception:
Step #2 - "Login to Prefect Cloud 2.0":
Step #2 - "Login to Prefect Cloud 2.0": Traceback (most recent call last):
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/prefect/cli/_utilities.py", line 44, in wrapper
Step #2 - "Login to Prefect Cloud 2.0": return fn(*args, **kwargs)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 122, in wrapper
Step #2 - "Login to Prefect Cloud 2.0": return run_async_in_new_loop(async_fn, *args, **kwargs)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 69, in run_async_in_new_loop
Step #2 - "Login to Prefect Cloud 2.0": return anyio.run(partial(__fn, *args, **kwargs))
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 70, in run
Step #2 - "Login to Prefect Cloud 2.0": return asynclib.run(func, *args, **backend_options)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 292, in run
Step #2 - "Login to Prefect Cloud 2.0": return native_run(wrapper(), debug=debug)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
Step #2 - "Login to Prefect Cloud 2.0": return loop.run_until_complete(main)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
Step #2 - "Login to Prefect Cloud 2.0": return future.result()
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
Step #2 - "Login to Prefect Cloud 2.0": return await func(*args)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/prefect/cli/cloud.py", line 204, in login
Step #2 - "Login to Prefect Cloud 2.0": workspaces = await client.read_workspaces()
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/prefect/cli/cloud.py", line 82, in read_workspaces
Step #2 - "Login to Prefect Cloud 2.0": return await self.get("/me/workspaces")
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/prefect/cli/cloud.py", line 102, in get
Step #2 - "Login to Prefect Cloud 2.0": res = await self._client.get(route, **kwargs)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1751, in get
Step #2 - "Login to Prefect Cloud 2.0": return await self.request(
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1527, in request
Step #2 - "Login to Prefect Cloud 2.0": return await self.send(request, auth=auth, follow_redirects=follow_redirects)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1614, in send
Step #2 - "Login to Prefect Cloud 2.0": response = await self._send_handling_auth(
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1642, in _send_handling_auth
Step #2 - "Login to Prefect Cloud 2.0": response = await self._send_handling_redirects(
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1679, in _send_handling_redirects
Step #2 - "Login to Prefect Cloud 2.0": response = await self._send_single_request(request)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1716, in _send_single_request
Step #2 - "Login to Prefect Cloud 2.0": response = await transport.handle_async_request(request)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
Step #2 - "Login to Prefect Cloud 2.0": resp = await self._pool.handle_async_request(req)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
Step #2 - "Login to Prefect Cloud 2.0": self.gen.throw(typ, value, traceback)
Step #2 - "Login to Prefect Cloud 2.0": File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
Step #2 - "Login to Prefect Cloud 2.0": raise mapped_exc(message) from exc
Step #2 - "Login to Prefect Cloud 2.0": httpx.ConnectTimeout
Step #2 - "Login to Prefect Cloud 2.0": An exception occurred.
Anna Geller
06/10/2022, 12:14 PMpip install gcsfs
pip install -U "prefect>=2.0b"
all other dependencies from your Docker image can be used at runtime in your DockerFlowRunner or KubernetesFlowRunner, but they don't need to be available during deployment creation processDanilo Drobac
06/10/2022, 12:39 PMAnna Geller
06/10/2022, 1:29 PMFROM prefecthq/prefect:2.0b6-python3.9
Danilo Drobac
06/10/2022, 2:25 PMAnna Geller
06/10/2022, 2:31 PMDanilo Drobac
06/10/2022, 3:35 PMFROM prefecthq/prefect:2.0b6-python3.9
And it works correctly, because prefect version
is returning the correct values.cloudbuild.yaml
file is:
steps:
- id: 'Generate full list of flows/DeploymentSpecs'
name: 'ubuntu'
entrypoint: 'printf'
args:
- "'%s\n' workspace/flows/* > flow_list.txt"
- id: 'Login to Prefect Cloud 2.0'
name: '<http://gcr.io/{project-name}/{image_name}|gcr.io/{project-name}/{image_name}>'
args: ['prefect', 'cloud', 'login', '--key', '{insert_key}', '--workspace', '{insert_workspace}']
prefect cloud login
command to the Dockerfile, it successfully logs in.
I'm going to see whether the deploy command works as expected when the authorisation is handled in the image.
It feels like it's not the correct way of doing it, but if it works then at least I've found a workaround.Step #2 - "Create deployment": Loading deployment specifications from python script at
Step #2 - "Create deployment": 'flows/second_flow.py'...
Step #2 - "Create deployment": Creating deployment 'second-deployment' for flow 'main-flow'...
Step #2 - "Create deployment": Deploying flow script from '/workspace/flows/second_flow.py' using File
Step #2 - "Create deployment": Storage...
Step #2 - "Create deployment": Created deployment 'main-flow/second-deployment'.
Step #2 - "Create deployment": View your new deployment with:
Step #2 - "Create deployment":
Step #2 - "Create deployment": prefect deployment inspect 'main-flow/second-deployment'
Step #2 - "Create deployment": Created 1 deployments!
Finished Step #2 - "Create deployment"
2022/06/10 17:58:10 Step Step #2 - "Create deployment" finished
2022/06/10 17:58:11 status changed to "DONE"
Anna Geller
06/10/2022, 7:55 PMsteps:
- id: 'Generate full list of flows/DeploymentSpecs'
name: 'ubuntu'
entrypoint: 'printf'
args:
- "'%s\n' workspace/flows/* > flow_list.txt"
- id: 'Login to Prefect Cloud 2.0'
name: 'prefecthq/prefect:2.0b6-python3.9'
args: ['prefect', 'cloud', 'login', '--key', '{insert_key}', '--workspace', '{insert_workspace}']
Danilo Drobac
06/13/2022, 10:23 AMAnna Geller
06/13/2022, 11:09 AMDanilo Drobac
06/13/2022, 11:13 AMAnna Geller
06/13/2022, 11:15 AMDanilo Drobac
06/16/2022, 11:38 AMname: CI
on:
push:
branches: [ main ]
jobs:
list-flows:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- uses: actions/checkout@v3
- id: set-matrix
run: echo "::set-output name=matrix::$(ls flows/*.py | jq -R -s -c 'split("\n")[:-1]')"
deploy:
needs: list-flows
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
strategy:
matrix:
flows: ${{ fromJson(needs.list-flows.outputs.matrix) }}
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
uses: actions/setup-python@v3
with:
python-version: 3.9
- name: Dev dependencies
run: |
pip install gcsfs
pip install -U "prefect>=2.0b"
- name: Login to Prefect Cloud 2.0
run: prefect cloud login --key ${{ secrets.PREFECT_API_KEY }} --workspace 'danilodrobacnzytecouk/nzyte-prefect'
- name: Validate Prefect version
run: prefect version
- name: Authenticate to GCP
uses: 'google-github-actions/auth@v0.7.1'
with:
workload_identity_provider: 'projects/170961541935/locations/global/workloadIdentityPools/github-actions-pool/providers/github-actions-provider'
service_account: '<mailto:prefect-service-account@nzyte-modern-data-stack.iam.gserviceaccount.com|prefect-service-account@nzyte-modern-data-stack.iam.gserviceaccount.com>'
- name: Create deployment
run: prefect deployment create ${{ matrix.flows }}
Anna Geller
06/16/2022, 11:43 AMDanilo Drobac
06/16/2022, 11:58 AMThe bq and gsutil tools do not currently support Workload Identity Federation! You will need to use traditional service account key authentication for now.
So looks like you still need to use a service account for authentication when interfacing with GCS or BQ.
Typical, but still one to look out for when they update it!Anna Geller
06/16/2022, 12:04 PMDanilo Drobac
06/16/2022, 12:06 PMbq
and gsutil
are legacy packages, I guess they just haven't gotten round to updating the codebase, although you would think that should be part of the rollout of the new product itself so people don't run in to issues like this!Anna Geller
06/16/2022, 1:19 PM