Lukáš Polák
10/25/2021, 7:32 AMGiovanni Giacco
10/25/2021, 10:15 AMRichard Hughes
10/25/2021, 2:07 PMPhilip MacMenamin
10/25/2021, 2:50 PMraise signals.FAIL('Didn't work')
I do not want the flow to proceed. However, other downstream tasks occur despite the exception being raised. Is this expected behavior?jcozar
10/25/2021, 4:07 PMKevin
10/25/2021, 4:20 PMIhor Bondartcov
10/25/2021, 5:26 PMTony Yun
10/25/2021, 6:56 PMalways_run/all_finished
trigger to this task, but I can’t simply just add it to the end of parameters. Because this is not a @task
way of definition, don’t know how to:
deletePod = kpod.DeleteNamespacedPod(kubernetes_api_key_secret=None)
Steve s
10/25/2021, 7:56 PMcreate_flow_run.map(...)
to run a single flow multiple times with different parameters. I have that working, but unfortunately I need the created flows to run in a specific order. is there a way to force that behavior?KhTan
10/25/2021, 11:37 PMdavzucky
10/26/2021, 12:40 AMorion agent start
and I set the following env variable to connect to the orion server
PREFECT_ORION_API_HOST: orion_server
PREFECT_ORION_API_PORT: 4200
the agent still try to create a database in this case using sqllite on the agent?davzucky
10/26/2021, 12:44 AMTara
10/26/2021, 4:30 AMGabi Pi
10/26/2021, 6:53 AMYou could override the automatic version promotion to unarchive and enable old versions (for example, for A/B testing)How can I do that?
Jai Deo
10/26/2021, 8:02 AMThomas Furmston
10/26/2021, 8:46 AMEric Feldman
10/26/2021, 8:56 AMget_flow_run_info
and until the state is finished?Will List
10/26/2021, 10:13 AMMilly gupta
10/26/2021, 11:53 AMToprak Nihat Deniz Öztürk
10/26/2021, 12:15 PMEric Feldman
10/26/2021, 3:04 PMdef fetch_result(data):
# data ?
raise
class GetData(Task):
def run(self):
return {'data': 7}
recipe_flow = StartFlowRun(flow_name="batch", project_name="proj", wait=True)
with Flow(name='schedule') as sched:
recipe_flow.set_upstream(GetData(), key='parameters')
FunctionTask(fetch_result)(data=recipe_flow)
but the only thing I get as data
is prefect.engine.signals.SUCCESS
object
calling data.state
isn’t really helpfull, and data.state.result
I get the prefect.engine.signals.SUCCESS
all over again
is there any way to get the real tasks/flowrunid/data of the the inner flow?Josh
10/26/2021, 3:51 PMdocker run -env FOO=BAR <my_docker_image>
I would like to be able to execute a flow on this image with a Docker Agent by calling the flow with the parameters
{
"FOO": "BAR"
}
Is this possible with Prefect? Or is there a Prefect idiom for such a concept?
Use cases for changing environment variables on container create/flow execution would be
1. defining which customer code path we want to trigger (database, configs, special methods and permissions)
2. Setting how we want to persist data (which database type to use, how to cache)haf
10/26/2021, 5:39 PM@task(result=...)
because I then have to specify the result type to be either local or GCS. How do you handle this?Dominic Pham
10/26/2021, 6:57 PM@task
def query_that_will_return_a_list(): -> list
@task
def scrapy_api_call_chunks(title_list):
loop_payload = prefect.context.get("task_loop_count", 0)
title_list_grouper = list(grouper(title_list, 10))
if loop_payload <= len(title_list_grouper):
# Each loop will be an iteration of 10 titles. # of loops * 10 will result in the total number of titles looped over so far
raise LOOP(message = 'Running the next 10 items in job titles list')
scraper_class = Scraper()
scraper_class.instantiate_web_scraper(title_list_grouper[loop_payload - 2])
I feel like I don't fully understand how to utilize LOOP in the context of passing information to another function inside the task.Samuel Hinton
10/26/2021, 10:43 PMXinyi Guo
10/27/2021, 9:43 AMArun Giridharan
10/27/2021, 1:13 PMserhan
10/27/2021, 1:22 PMEric Feldman
10/27/2021, 4:02 PMnew_state.result
keeps being an empty dict when running on prefect agent
if i’m running the flow using flow.run
I do see the results of the stateJason Boorn
10/27/2021, 4:08 PM