Matt Liszewski
06/29/2022, 2:37 PMMadison Schott
06/29/2022, 4:01 PMdata_pipeline_prod
that we want to somehow label that it is deprecated? And then change the one with test in its name to the new data_pipeline_prod
? Or any suggested ways to do this?Josh
06/29/2022, 4:37 PMBrandon T. Kowalski
06/29/2022, 6:11 PMwith Flow("HathiTrust Deposit (MARCXML)") as flow:
deposit = Parameter("deposit", default=False)
This is the existing parameter. I want to add a second so naturally I tried adding another underneath the first like this:
with Flow("HathiTrust Deposit (MARCXML)") as flow:
deposit = Parameter("deposit", default=False)
local_test = Parameter("local_test", default=False)
This results in an error:
UserWarning: Tasks were created but not added to the flow: {<Parameter: local_test>}. This can occur when `Task` classes, including `Parameters`, are instantiated inside a `with flow
How does one add a parameter to an existing flow?Yupei Chen
06/29/2022, 6:11 PMBilly McMonagle
06/29/2022, 6:14 PMharis khan
06/29/2022, 6:31 PMredsquare
06/29/2022, 6:41 PMShaoyi Zhang
06/29/2022, 6:54 PMYury Cheremushkin
06/29/2022, 7:45 PMconfig.toml
and access them via Secrets API or prefect.context.secrets.get()
. But I can't find anything similar in 2.0Jessica Smith
06/29/2022, 8:42 PMWei Mei
06/29/2022, 9:21 PM@task(nout=1)
def determine_run(clock):
if clock == "full_run":
<http://logger.info|logger.info>("full_run clock")
run_type = "full"
<http://logger.info|logger.info>(f"{run_type}")
return run_type
Amol Shirke
06/29/2022, 11:51 PMAustin Anderson
06/30/2022, 12:14 AMfrom prefect import flow, task
from prefect_dask.task_runners import DaskTaskRunner
@task
def say_hello(name):
print(f"hello {name}")
@flow(task_runner=DaskTaskRunner(address='tcp://[redacted]:8786'))
def greetings():
say_hello('test')
if __name__ == "__main__":
greetings()
Any ideas?Amogh Kulkarni
06/30/2022, 4:11 AMwonsun
06/30/2022, 6:13 AMSander
06/30/2022, 8:10 AMSurya
06/30/2022, 10:00 AMredsquare
06/30/2022, 10:25 AMharis khan
06/30/2022, 10:28 AMGeorge Shishorin
06/30/2022, 10:52 AMmodule not found
. Could you please help with it?
---> threadAnna Geller
Jehan Abduljabbar
06/30/2022, 2:21 PMJoshua Greenhalgh
06/30/2022, 2:51 PMValueError: the greenlet library is required to use this function. No module named 'greenlet'
Any ideas?JK
06/30/2022, 3:01 PMIsara Ovin
06/30/2022, 3:04 PMfor col in date_columns:
if col in df_columns:
df[col] = df[col].str.replace('/', '-')
with mp.Pool(mp.cpu_count()) as pool:
df[col] = pool.map(get_parsed_date, df[col])
Jessica Smith
06/30/2022, 5:05 PMPagerDutyNotificationAction
(24dfaasdfe-93dd-4b11-8dddc-4xxxxxx3d23
) has failed with 'HTTPError: 202 Accepted 'PagerDuty did not provide error details.' while contacting service API. Check your action configuration. See the Pagerduty docs for more details on error handling.' during execution triggered by FlowSLAFailedEvent
event from hook b625xx8-d3x-4xxe-bxx6-1xxxxxx67f2
.Muhammad Usman Ghani
06/30/2022, 5:12 PMSushma Adari
06/30/2022, 7:37 PMTypeError: Object of type DataFrame is not JSON serializable
Madison Schott
06/30/2022, 8:30 PMFlowRenameTask
File "/Users/madisonschott/dbt_snowflake/data_pipeline_test_new_account.py", line 5, in <module>
from prefect.tasks.prefect.flow_rename import FlowRenameTask
ModuleNotFoundError: No module named 'prefect.tasks.prefect.flow_rename'