https://prefect.io logo
Title
c

Chris Gunderson

09/07/2022, 7:01 PM
Hi Team - I'm getting an error when attempting to use this library in another class from the task. I'm not sure why it is failing
from prefect.blocks.system import *
sraDatabaseSecretName = String.load("sra-database") #This is the name of the secret Traceback (most recent call last): File "/opt/pysetup/.venv/lib/python3.8/site-packages/prefect/client.py", line 1268, in read_block_document_by_name response = await self._client.get( File "/opt/pysetup/.venv/lib/python3.8/site-packages/httpx/_client.py", line 1751, in get return await self.request( File "/opt/pysetup/.venv/lib/python3.8/site-packages/httpx/_client.py", line 1527, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "/opt/pysetup/.venv/lib/python3.8/site-packages/prefect/client.py", line 279, in send response.raise_for_status() File "/opt/pysetup/.venv/lib/python3.8/site-packages/prefect/client.py", line 225, in raise_for_status raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.cause prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url 'http://ephemeral-orion/api/block_types/slug/string/block_documents/name/sra-database?include_secrets=true' Response: {'detail': 'Block document not found'} For more information check: https://httpstatuses.com/404
n

Nate

09/07/2022, 7:08 PM
from a terminal in same environment as your code runs from, do you see the
sra-database
block in the output of
prefect block ls
? I suspect your prefect profile is not pointing at the correct cloud workspace
c

Chris Gunderson

09/07/2022, 7:10 PM
n

Nate

09/07/2022, 7:17 PM
hmm could you share a bit more of your code where the error is being thrown?
c

Chris Gunderson

09/07/2022, 7:17 PM
I stepped over the import in PyCharm... (python is a new language for me). The method in the class I am attempting to use is in another folder. Remote file /opt/project/python/src/main/prefect/reports/allocations/ is mapped to the local path C:\Users\chris.gunderson\source\repos\python\src\main\prefect\reports\allocations\ and can't be found. You can continue debugging, but without the source.
import pandas as pd
import numpy as np
from datetime import datetime
from datetime import date
import logging, os
from sraPy.connections.mysql import sraController
from prefect.blocks.system import *
import boto3
import json

LOG_LEVEL = 'INFO'
LOG_FILE_NAME = os.path.basename(__file__)

class FidelityAllocations:
    @staticmethod
    def run_alloc():
        logging.getLogger(LOG_FILE_NAME).setLevel(LOG_LEVEL)
        <http://logging.info|logging.info>('Connecting to SRSE and SRA instances...')
        <http://logging.info|logging.info>(f'{os.getcwd()}')


        sraDatabaseSecretName = String.load("sra-database")
        srseDatabaseSecretName = String.load("srse-database")
from datetime import datetime
from prefect import task, Flow
#from pydantic import BaseModel
from main.prefect.utility.email import SRAMailer
from main.reports.allocations.fidelity_allocations import FidelityAllocations
from prefect.blocks.system import Secret
from prefect.blocks.notifications import SlackWebhook
from prefect.orion.schemas.states import Completed, Failed

# --------------------------------------------------------------------------------------------------
#class Custodian(BaseModel):
#    name: str


@task(name = 'Fidelity Allocations',
      retries = 2,
      retry_delay_seconds = 60)
def allocations_process():
    logger = get_run_logger()
    excel_file = FidelityAllocations.run_alloc()
@Nate After putting the scripts in the same folder I'm still having this issue. Is the code not meant to be separated into other classes?
n

Nate

09/08/2022, 2:18 AM
apologies but I'm not sure off the top why that would cause an issue, I'd have to investigate when I'm back at my computer. As a sanity check, could you try loading the same blocks directly in your flow?
c

Chris Gunderson

09/08/2022, 5:57 PM
I'm still having an issue with the docker container, but it works locally using Poetry
prefect = "^2.2.0"
That is in the poetry file, whereas, Docker is using 2.3.2.
My docker container is using 2.3
@Nate
Not sure if it is a version issue?
@Nate This was a configuration issue in PyCharm. The interpreter path was not pointing to the virtual environment, so that was the issue.
👍 1