https://prefect.io logo
Powered by
Title
r

Riccardo Tesselli

08/01/2022, 11:51 AM
hello, I’m struggling with loading a custom block from Prefect 2.0 cloud. I’ve created on cloud a block called 
test
of a custom block type 
Config
, and when trying to load its values within a flow with 
Config.load('test')
I get this error: 
except prefect.exceptions.ObjectNotFound as e:
>               raise ValueError(
                    f"Unable to find block document named {block_document_name} for block type {block_type_slug}"
                ) from e
E               ValueError: Unable to find block document named test for block type config
the weird thing is that if I load the block outside of a flow it works perfectly, the problem arises only when loading within a flow
ah, maybe I got it, the problem is not the flow, it’s pytest, because if I don’t run pytest it works fine
so, I’ve compared the two runs, the one within 
pytest
and the normal one, they differ in 
prefect.client.get_client
in line 110 
ctx = prefect.context.get_settings_context()
they differ because in the 
pytest
run in this entry for 
ctx
: 
PREFECT_API_URL=None
, while the other has the field set
the weird thing is that 
PREFECT_API_KEY
is set correctly in both runs
I can confirm this, so I’ve put in my flow this line 
<http://logger.info|logger.info>(f"PREFECT_API_URL: {PREFECT_API_URL.value()}")
when running normally (within 
if __name__ == '__main__':
) I get the correct URL, but if I run pytest (which then calls the flow) the result is 
PREFECT_API_URL: None
any idea of what’s going on?
a

Anna Geller

08/01/2022, 1:33 PM
you probably didn't register that block
check this https://docs.prefect.io/concepts/blocks/#registering-blocks
for help with unit testing https://discourse.prefect.io/t/unit-testing-best-practices-for-prefect-flows-subflows-and-tasks/1070/2
and https://orion-docs.prefect.io/tutorials/testing/
I think this syntax will prevent your pytest error: 
from prefect import flow
from prefect.testing.utilities import prefect_test_harness

@flow
def my_favorite_flow():
    return 42

def test_my_favorite_flow():
  with prefect_test_harness():
      # run the flow against a temporary testing database
      assert my_favorite_flow() == 42
r

Riccardo Tesselli

08/01/2022, 1:39 PM
@Anna Geller that’s not the problem. I get the same error (ValueError: Unable to find block document named test for block type string) even for a Prefect system block (in this case I’ve tried String block). The thing is, for some reason, when running a flow from pytest it fails to set the value for 
PREFECT_API_URL
a

Anna Geller

08/01/2022, 1:48 PM
could you share a minimal reproducible example with your repo showing your flows, blocks and tests that you create and how you do it? hard to say what you are currently testing here
r

Riccardo Tesselli

08/01/2022, 1:49 PM
yeah sure, thank you
:thank-you: 1
I’m not able to replicate the issue in a docker container, so it should be some configuration on my local pc. I’ll let you know if I find something useful
:thank-you: 1
🙌 1
a

Anna Geller

08/01/2022, 6:09 PM
thanks for the update
r

Riccardo Tesselli

08/02/2022, 9:32 AM
@Anna Geller so I found out what happened. The thing is I have a flow which takes as input the name of a configuration block, something like this: 
@flow
def myflow(config_block: str):
    config = MyConfig.load(config_block)
then I want to test the execution of that flow by doing in a test file something like this: 
def test_flow():
    myflow("test-config")
the reason why it fails loading the block it’s because I’ve set this fixture in pytest (as suggested in https://docs.prefect.io/tutorials/testing/#unit-testing-flows) 
@pytest.fixture(autouse=True, scope="session")
def prefect_test_fixture():
    with prefect_test_harness():
        yield
so, when the fixture is in place, it prevents the flow to run on the cloud (which is what I want since I do not want my local tests to be shown in Prefect Cloud), but then I’m not able to load a block from Cloud. Do you have a way/best practice in order to be able to run local tests in the local database but at the same time being able to load blocks from Cloud?
9 Views
#prefect-communityJoin Slack
Powered by