Hey all, I'm running into an issue with prefect cl...
# ask-communityc
Chris Jordan
03/18/2021, 4:13 PMHey all, I'm running into an issue with prefect cloud submitting jobs but not doing anything after that.
The issue first presented with a new flow I was writing. I restarted my agent and I upgraded from 14.6 to 14.12. This only had the effect of spreading the issue to the rest of my flows.
I was wondering if you could offer me some guidance in debugging this issue.
Chris Jordan
03/18/2021, 4:31 PMThe place I'm running into an issue seems to be after a container is submitted - here's a flow from yesterday that succeeded vs where I'm hitting an issue now
Chris Jordan
03/18/2021, 4:31 PMI don't think the cloud flow runner is something I have control over. I just generated a new token and restarted my agent with that, it doesn't seem to have helped
m
Mariia Kerimova
03/18/2021, 4:35 PMHello Chris! Looks like the issue is not with agent because your flow is submitted. Can you provide more information how you run your flow or redacted/simplified version of your flow?
c
Chris Jordan
03/18/2021, 4:36 PMI can do that, for sure
Chris Jordan
03/18/2021, 4:38 PMso here's the flow I was working on that started giving me issues
Copy code
from prefect import Flow
from prefect.engine.results import LocalResult
from ..app.data_object import DataObject
from prefect.schedules import CronSchedule
from .reference_handlers import cloud_only_slack_handler
from .reference_tasks import check_api, transform, import_to_database
# daily_schedule = CronSchedule("12 4 * * *")
with Flow(
"draft_prospect_link_flow",
# schedule=daily_schedule,
result=LocalResult(),
state_handlers=[cloud_only_slack_handler]
) as flow:
data_object = DataObject('draft_prospect_link', 'prospects')
check_api_result = check_api(data_object=data_object) Copy code
@task(result=LocalResult(), tags=['api'])
def check_api(data_object, save_to_s3=False, result=LocalResult()):
handler = ApiHandler(
data_object,
context=prefect.context,
save_to_s3=save_to_s3)
handler.get_file()
return handler.data_objectChris Jordan
03/18/2021, 4:39 PMcurrently I'm running flows on prefect cloud. I've got an agent started with this
Copy code
prefect agent docker start \
--log-level DEBUG \
--volume "${WORKDIR}/.config.toml:/opt/prefect/config.toml" \
-t "${token}"Chris Jordan
03/18/2021, 4:41 PMI'm kicking this flow off with the "quick run" button on cloud.prefect.io
Chris Jordan
03/18/2021, 5:04 PMso I think I found the issue. I was adding a block ot the
context.secretsChris Jordan
03/18/2021, 5:04 PMeither way, this change in my config file got me to working flows again
Chris Jordan
03/18/2021, 5:04 PM Copy code
auth_token = abcdefghijklmnop
->
auth_token = 'abcdefghijklmnop'm
Mariia Kerimova
03/18/2021, 5:06 PMI'm glad you found and solved the issue 😁
c
Chris Jordan
03/18/2021, 5:08 PMme too!
Chris Jordan
03/18/2021, 5:10 PMmy current theory is that my old agent was started before some change happened and was grandfathered in. but when I wanted to add some secrets for a new API, I had to restart the agent. In between agent starts, the quoting necessary on the
auth_tokenChris Jordan
03/18/2021, 5:10 PMdoes that make sense? I don't follow the cloud changelog too closely
Chris Jordan
03/18/2021, 5:12 PMalso it's entirely possible that the config file I update is from some olde tyme, and I overwrote one that I had updated with the quotes in production
Chris Jordan
03/18/2021, 5:13 PMsince it's not under version control cause secrets
m
Mariia Kerimova
03/18/2021, 5:19 PMI had similar issue, and as I remember toml files require to wrap strings in double or single quotes
c
Chris Jordan
03/18/2021, 5:24 PMmy main question for myself is, why did past me think it was a good idea to have that string not be quoted?
Chris Jordan
03/18/2021, 5:25 PMdon't really have a good answer to that, but I'm glad it's resolved
👍 1