Liliya Angelova
11/26/2025, 3:16 PMUnboundLocalError: cannot access local variable 'batch' where it is not associated with a value when i run it through the deployment which uses docker and kubernertes worker:
import json
from datetime import datetime
from dateutil import parser
from freight_request_processor import get_distance_mapping, process_single
from freight_request_repository import save_requests_pg
from more_itertools import chunked
from prefect import flow, get_run_logger, task, unmapped
from prefect.blocks.system import Secret
from prefect.cache_policies import NO_CACHE
from prefect.task_runners import ThreadPoolTaskRunner
from timocom import Timocom
from utils import (
build_pg_utils_from_block,
get_config,
parse_location_criteria,
publish_refresh_message,
)
@flow(task_runner=ThreadPoolTaskRunner())
def timocom_search_flow(
search_parameters: dict,
pg_secret_block: str,
timocom_secret_block: str,
rabbitmq_secret_block: str,
ors_server_variable: str,
price_calculation_server_variable: str,
predictive_scoring_server_variable: str,
here_maps_api_key_variable: str,
):
logger = get_run_logger()
<http://logger.info|logger.info>(f"Starting Timocom search with parameters: {search_parameters} at {datetime.now()}")
try:
sanitized_search_parameters = sanitize_and_validate_search_parameters(search_parameters)
pg_utils = build_pg_utils_from_block(pg_secret_block)
# Perform Timocom search
# Timocom credentials are stored as "tenant_id": {"user": "username", "pass": "password"}
creds = Secret.load(timocom_secret_block).get()
tenant_ref = sanitized_search_parameters["tenant_ref"]
<http://logger.info|logger.info>(f"Using Timocom credentials for tenant: {tenant_ref}")
tenant_creds = creds.get(tenant_ref)
if not tenant_creds:
raise ValueError(f"No Timocom credentials found for tenant {tenant_ref}")
user = tenant_creds["user"]
password = tenant_creds["pass"]
# Initialize Timocom instance
timocom_instance = Timocom(logger, pg_utils, user, password)
# Create search tasks
search_tasks = timocom_instance.create_search_tasks(sanitized_search_parameters)
search_futures = timocom_instance.search.map(search_tasks)
<http://logger.info|logger.info>("Timocom search process completed.")
config = get_config(
ors_server_variable,
price_calculation_server_variable,
predictive_scoring_server_variable,
here_maps_api_key_variable,
)
for future in search_futures:
for batch in chunked(future.result(), 50):
<http://logger.info|logger.info>(f"Processing batch of {len(batch)} results...")
distance_mapping = get_distance_mapping(batch, config)
proccessed_results = process_single.map(unmapped(config), batch, distance_mapping)
<http://logger.info|logger.info>("Post-processing of freight requests completed.")
save_requests_pg(proccessed_results, logger, pg_secret_block)
<http://logger.info|logger.info>("Requests data persistence completed.")
<http://logger.info|logger.info>(f"Publish message: {datetime.now()}")
publish_refresh_message(
rabbitmq_secret_block,
search_parameters["tab_id"],
search_parameters["job_id"],
"REFRESH",
)
except Exception as e:
logger.error(f"Error during Timocom search initiation: {e}")
publish_refresh_message(
rabbitmq_secret_block, search_parameters["tab_id"], search_parameters["job_id"], "FAILED"
)
raiseMarvin
11/26/2025, 3:16 PMMarvin
11/26/2025, 3:22 PMUnboundLocalError: cannot access local variable 'batch' where it is not associated with a value is a Python scoping error (not Prefect-specific). It usually happens when:
- a loop like for batch in ...: never runs (e.g., empty iterable), and some code path still tries to use batch later, or
- an exception occurs before the first iteration and an error handler/log statement references batch, or
- a closure/comprehension captures batch in an unexpected way.
A few concrete things I’d tighten up in your flow that will both avoid the error and make mapping behave correctly in Prefect 3.x:
1) Materialize your batches before looping and handle the empty case
- If future.result() returns an empty list, your for batch in ... loop won’t run and any accidental reference to batch will blow up. Materializing the chunks first makes this explicit and safer.
2) Wrap non-mapped arguments with unmapped(...)
- In process_single.map(unmapped(config), batch, distance_mapping), Prefect may treat distance_mapping (a dict) as an iterable and try to map it too. If you intend to use the same distance_mapping for all items in the current batch, wrap it with unmapped(distance_mapping).
3) Resolve mapped results before passing to a plain Python function
- If save_requests_pg is not a Prefect task, pass it concrete results with .result() instead of a PrefectFutureList.
Here’s a revised flow excerpt applying those points:
for future in search_futures:
results = future.result() # resolve each search task
batches = list(chunked(results, 50))
if not batches:
<http://logger.info|logger.info>("No results returned for this search task; skipping.")
continue
for i, batch in enumerate(batches, start=1):
<http://logger.info|logger.info>(f"Processing batch {i}/{len(batches)} with {len(batch)} results...")
distance_mapping = get_distance_mapping(batch, config)
# Ensure only `batch` is mapped; config and mapping are static per item
processed_futures = process_single.map(
unmapped(config),
batch,
unmapped(distance_mapping),
)
# If save_requests_pg is a normal function, resolve mapped results first
processed_results = processed_futures.result()
save_requests_pg(processed_results, logger, pg_secret_block)
<http://logger.info|logger.info>("Requests data persistence completed.")
<http://logger.info|logger.info>(f"Publish message: {datetime.now()}")
publish_refresh_message(
rabbitmq_secret_block,
search_parameters["tab_id"],
search_parameters["job_id"],
"REFRESH",
)
Why this likely fixes your issue
- If future.result() returns no items in your K8s/Docker environment (but did locally), your original for batch in ... loop would never bind batch. Any reference to batch elsewhere (even unintentionally via a log/closure/comprehension/etc.) triggers the UnboundLocalError. The pattern above avoids that by checking if not batches: continue.
- unmapped(distance_mapping) ensures Prefect doesn’t try to map over the dict’s keys, which can lead to odd behavior and mismatched mapping lengths.
A couple of quick checks that will help us confirm root cause:
- Could you share the full traceback (including the line number where UnboundLocalError is raised)?
- What Python version are you running locally vs. in your Docker image?
- Which Prefect version are you on (assuming 3.x)? And is save_requests_pg a plain function or a Prefect task?
If you prefer, we can also encapsulate per-batch work into a dedicated task (e.g., @task def process_batch(...)) and map batches directly. That gives you cleaner concurrency and easier debugging in the UI.