https://prefect.io logo
Powered by
Title
v

Vadym Dytyniak

10/13/2022, 2:42 PM
Hi. I am getting weird error when trying to load String block. It works fine locally, but in Github action fails with the error in thread.
✅ 1
File "/usr/local/lib/python3.10/dist-packages/chronos/schedule/deployment.py", line 66, in deploy
    environment = String.load('environment').value
  File "/usr/local/lib/python3.10/dist-packages/prefect/utilities/asyncutils.py", line 212, in wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/prefect/utilities/asyncutils.py", line 141, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/usr/local/lib/python3.10/dist-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/usr/local/lib/python3.10/dist-packages/prefect/client/orion.py", line 82, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/prefect/blocks/core.py", line 648, in load
    block_document = await client.read_block_document_by_name(
  File "/usr/local/lib/python3.10/dist-packages/prefect/client/orion.py", line 1092, in read_block_document_by_name
    return BlockDocument.parse_obj(response.json())
  File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 743, in json
    return jsonlib.loads(self.text, **kwargs)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
j

James Sopkin

10/13/2022, 2:44 PM
Hi @Vadym Dytyniak are you loading your string blocks within a task or flow?
v

Vadym Dytyniak

10/13/2022, 2:45 PM
no, outside
to create deployment 
environment = String.load('environment').value
storage = S3.load(DEFAULT_S3_BLOCK)

infrastructure = _get_k8s_infra(
        namespace=NAMESPACE.format(env=environment),
        service_account_name=flow_to_service_account_name(flow.name),
        image=docker_image,
        cpu=cpu,
        memory=memory,
        disk_size=disk_size,
        use_public_subnets=use_public_subnets,
        env=env | {'EXTRA_PIP_PACKAGES': ' '.join(dependencies)},
    )

Deployment.build_from_flow(
        flow=flow,
        schedule=schedule,
        name=name,
        work_queue_name=work_queue_name,
        storage=storage,
        path=name,
        infrastructure=infrastructure,
        tags=tags,
    ).apply()
j

James Sopkin

10/13/2022, 2:55 PM
Could you try putting the blocks you load to within a task or flow(or an async function)?
I've done alot of testing with github action deployments and I typically run into issues when attempting to load blocks outside of a task or flow context
v

Vadym Dytyniak

10/13/2022, 2:58 PM
I found that issue not related to GHA, but I use different key
locally I used my personal key and there I use service account key
j

James Sopkin

10/13/2022, 2:58 PM
ahhh
v

Vadym Dytyniak

10/13/2022, 2:58 PM
looks like service account can't read string block
j

James Sopkin

10/13/2022, 2:58 PM
that would do it
v

Vadym Dytyniak

10/13/2022, 2:59 PM
but role is collaborator
or service account is not allowed to read blocks at all?
j

James Sopkin

10/13/2022, 2:59 PM
The service account may not have permission to access the specific workspace
https://docs.prefect.io/ui/organizations/?h=service+account
You can add a service account to the workspace though
v

Vadym Dytyniak

10/13/2022, 3:00 PM
It is already added
I use it for agent
wanted to test GHA using the same account
j

James Sopkin

10/13/2022, 3:05 PM
hmmm... I'll follow back on this. I am not sure why you are having issues loading blocks if the service account is already a collaborator on the workspace
v

Vadym Dytyniak

10/13/2022, 3:09 PM
give me some time to better test it
created different account and it works
but permissions are the same
j

James Sopkin

10/13/2022, 3:10 PM
Awesome! I am not sure why that is the case then
I assumed it was a permissions issue, but when you told me it was a collaborator already, I was confused, because I deployed a flow with GHA yesterday using a service account
v

Vadym Dytyniak

10/13/2022, 3:11 PM
How are you setting creds?
my code:
j

James Sopkin

10/13/2022, 3:15 PM
That is one way to do it. You can either use 
prefect config set
or you can just set the creds as env variables- for example 
on:
  push:
    branches:
      - "main"
      - "dev"
env:
  PREFECT_API_KEY: ${{ secrets.prefect_cs_service_account }}
  PREFECT_API_URL: ${{ secrets.PREFECT_API_URL }}
#prefect-communityJoin Slack
Powered by