https://prefect.io logo
Channels
Powered by
r

Riccardo Tesselli

08/01/2022, 11:51 AM
hello, I’m struggling with loading a custom block from Prefect 2.0 cloud. I’ve created on cloud a block called 
test
of a custom block type 
Config
, and when trying to load its values within a flow with 
Config.load('test')
I get this error:
Copy code
except prefect.exceptions.ObjectNotFound as e:
>               raise ValueError(
                    f"Unable to find block document named {block_document_name} for block type {block_type_slug}"
                ) from e
E               ValueError: Unable to find block document named test for block type config
the weird thing is that if I load the block outside of a flow it works perfectly, the problem arises only when loading within a flow
ah, maybe I got it, the problem is not the flow, it’s pytest, because if I don’t run pytest it works fine
so, I’ve compared the two runs, the one within 
pytest
and the normal one, they differ in 
prefect.client.get_client
in line 110 
ctx = prefect.context.get_settings_context()
they differ because in the 
pytest
run in this entry for 
ctx
: 
PREFECT_API_URL=None
, while the other has the field set
the weird thing is that 
PREFECT_API_KEY
is set correctly in both runs
I can confirm this, so I’ve put in my flow this line
Copy code
<http://logger.info|logger.info>(f"PREFECT_API_URL: {PREFECT_API_URL.value()}")
when running normally (within
Copy code
if __name__ == '__main__':
) I get the correct URL, but if I run pytest (which then calls the flow) the result is 
PREFECT_API_URL: None
any idea of what’s going on?
a

Anna Geller

08/01/2022, 1:33 PM
you probably didn't register that block
I think this syntax will prevent your pytest error:
Copy code
from prefect import flow
from prefect.testing.utilities import prefect_test_harness

@flow
def my_favorite_flow():
    return 42

def test_my_favorite_flow():
  with prefect_test_harness():
      # run the flow against a temporary testing database
      assert my_favorite_flow() == 42
r

Riccardo Tesselli

08/01/2022, 1:39 PM
@Anna Geller that’s not the problem. I get the same error (ValueError: Unable to find block document named test for block type string) even for a Prefect system block (in this case I’ve tried String block). The thing is, for some reason, when running a flow from pytest it fails to set the value for 
PREFECT_API_URL
a

Anna Geller

08/01/2022, 1:48 PM
could you share a minimal reproducible example with your repo showing your flows, blocks and tests that you create and how you do it? hard to say what you are currently testing here
r

Riccardo Tesselli

08/01/2022, 1:49 PM
yeah sure, thank you
🙏 1
I’m not able to replicate the issue in a docker container, so it should be some configuration on my local pc. I’ll let you know if I find something useful
🙏 1
🙌 1
a

Anna Geller

08/01/2022, 6:09 PM
thanks for the update
r

Riccardo Tesselli

08/02/2022, 9:32 AM
@Anna Geller so I found out what happened. The thing is I have a flow which takes as input the name of a configuration block, something like this:
Copy code
@flow
def myflow(config_block: str):
    config = MyConfig.load(config_block)
then I want to test the execution of that flow by doing in a test file something like this:
Copy code
def test_flow():
    myflow("test-config")
the reason why it fails loading the block it’s because I’ve set this fixture in pytest (as suggested in https://docs.prefect.io/tutorials/testing/#unit-testing-flows)
Copy code
@pytest.fixture(autouse=True, scope="session")
def prefect_test_fixture():
    with prefect_test_harness():
        yield
so, when the fixture is in place, it prevents the flow to run on the cloud (which is what I want since I do not want my local tests to be shown in Prefect Cloud), but then I’m not able to load a block from Cloud. Do you have a way/best practice in order to be able to run local tests in the local database but at the same time being able to load blocks from Cloud?
19 Views
Powered by