Ian Andres Etnyre Mercader
01/10/2022, 7:39 PMIan Andres Etnyre Mercader
01/10/2022, 7:42 PMdef tcp_logger():
file_logger = logging.getLogger()
if not any(
[
isinstance(handler, TCPLogstashHandler)
for handler in file_logger.handlers
]
):
host = os.environ.get('LOGSTASH_IP')
port = 5959
handle = TCPLogstashHandler(host, port)
file_logger.setLevel(<http://logging.INFO|logging.INFO>)
file_logger.addHandler(handle)
return file_logger
Task code:
from src.pubmed.logger import tcp_logger as get_logger
@task
def prepare_destination():
get_logger().info("starting prepare_destination")
files_path = prefect.context.get("pubmed_jsonl_local_path")
os.makedirs(files_path, exist_ok=True)
files_path = prefect.context.get("pubmed_xmls_local_path")
os.makedirs(files_path, exist_ok=True)
get_logger().info("ending prepare_destination")
return True
When I test my flow on a LocalExcecutor on my laptop and it logs successfully with no error
But when I deploy my flow on my docker environment in the production server I get this error:
Failed to load and execute Flow's environment: FlowStorageError("An error occurred while unpickling the flow:\n TypeError('expected str, bytes or os.PathLike object, not NoneType')")
Kevin Kho
Ian Andres Etnyre Mercader
01/10/2022, 7:50 PMflow.storage = prefect.storage.Local()
Kevin Kho
Ian Andres Etnyre Mercader
01/10/2022, 7:52 PMflow.run_config = LocalRun(labels=['scicarta'])
Kevin Kho
Ian Andres Etnyre Mercader
01/10/2022, 7:58 PMIan Andres Etnyre Mercader
01/10/2022, 8:03 PMfrom src.pubmed.logger import tcp_logger as get_logger
and the lines where i use the get_logger
Ian Andres Etnyre Mercader
01/10/2022, 8:03 PMKevin Kho
Daniel Kornhauser
01/10/2022, 8:07 PMIan Andres Etnyre Mercader
01/10/2022, 8:08 PM