itay livni
12/24/2020, 4:19 PMapply_map
the only way to map over ifelse
and case
statements?wiretrack
12/24/2020, 8:18 PMwiretrack
12/24/2020, 8:18 PMcustomer_flow = Flow('customer_flow')
customer_flow.register(project_name="All Flows")
customer_uptodate = check_status(source='db.sqlite3', target='db_target.sqlite3', table='customers')
test_target_db = test_target_db()
test_source_db = test_source_db()
customer_flow.add_task(test_source_db)
customer_flow.add_task(test_target_db)
customer_flow.add_task(customer_uptodate)
I get the error âValueError: Could not infer an active Flow context.â (edited)
but I couldnât understand what exactly iâm doing wrong using the imperative API. registered the flow, registerd the tasks, add_task to flow. i didnât understand what could I do differently, since the error appears right at my first task (customer_uptodate = check_status(source=âdb.sqlite3â, target=âdb_target.sqlite3â, table=âcustomersâ)wiretrack
12/24/2020, 8:19 PMJoël Luijmes
12/25/2020, 11:02 AMjeff n
12/26/2020, 4:43 AMJay Shah
12/27/2020, 9:48 AMself._access_token_expires_at = pendulum.now()
RuntimeError Traceback (most recent call last)
<ipython-input-5-fa84dadfda8e> in <module>
----> 1 flow.register("prefect-workflow")
/opt/conda/lib/python3.8/site-packages/prefect/core/flow.py in register(self, project_name, build, labels, set_schedule_active, version_group_id, no_url, idempotency_key, **kwargs)
1658 self.result = self.storage.result
1659
-> 1660 client = prefect.Client()
1661
1662 registered_flow = client.register(
/opt/conda/lib/python3.8/site-packages/prefect/client/client.py in __init__(self, api_server, api_token)
92 self._access_token = None
93 self._refresh_token = None
---> 94 self._access_token_expires_at = pendulum.now()
95 self._active_tenant_id = None
96 self._attached_headers = {} # type: Dict[str, str]
/opt/conda/lib/python3.8/site-packages/pendulum/__init__.py in now(tz)
209
210 if tz is None or tz == "local":
--> 211 dt = _datetime.datetime.now(local_timezone())
212 elif tz is UTC or tz == "UTC":
213 dt = _datetime.datetime.now(UTC)
/opt/conda/lib/python3.8/site-packages/pendulum/tz/__init__.py in local_timezone()
58 Return the local timezone.
59 """
---> 60 return get_local_timezone()
/opt/conda/lib/python3.8/site-packages/pendulum/tz/local_timezone.py in get_local_timezone()
33
34 if _local_timezone is None:
---> 35 tz = _get_system_timezone()
36
37 _local_timezone = tz
/opt/conda/lib/python3.8/site-packages/pendulum/tz/local_timezone.py in _get_system_timezone()
61 return _get_darwin_timezone()
62
---> 63 return _get_unix_timezone()
64
65
/opt/conda/lib/python3.8/site-packages/pendulum/tz/local_timezone.py in _get_unix_timezone(_root)
240 return TimezoneFile(tzpath)
241
--> 242 raise RuntimeError("Unable to find any timezone configuration")
243
244
RuntimeError: Unable to find any timezone configuration
I get a similar error when I run:
import pendulum
pendulum.now()Slackbot
12/27/2020, 10:36 AMAjith Kumara Beragala Acharige Lal
12/28/2020, 11:58 AMgraphql
, not sure what causing this , any ideaMarwan Sarieddine
12/28/2020, 2:31 PMAjith Kumara Beragala Acharige Lal
12/28/2020, 5:17 PMMatthew Blau
12/28/2020, 7:41 PMmatta
12/28/2020, 11:21 PM{'_schema': 'Invalid data type: None'}
matta
12/28/2020, 11:21 PMmatta
12/28/2020, 11:21 PMmatta
12/28/2020, 11:22 PMmatta
12/28/2020, 11:22 PMMatthew Blau
12/29/2020, 1:23 PMJohn Grubb
12/29/2020, 4:54 PMThis means that Prefect Cloud will ensure that no more than 10 tasks with the "database" tag will be running at any given time.this also applies to mapped tasks, yes? So say I want to backfill some data off of a slow API endpoint, I don't want to bury the application with tons of concurrent requests but I also don't want to switch over to the LocalExecutor for the job. Could I set a concurrency limit of
2
for this API and will the mapped tasks behave as intended?Ryan Abernathey
12/29/2020, 4:57 PMPeter Erickson
12/29/2020, 6:37 PMprefecthq/prefect:0.14.1
image doesn't contain the dask-kubernetes
python module. Does anyone know if there's a different image to use or is it best to build my own image?Henrik VÀisÀnen
12/29/2020, 6:56 PMJeff Brainerd
12/29/2020, 9:11 PMDilip Thiagarajan
12/29/2020, 10:02 PMCloudFlowRunner
and CloudTaskRunner
are used when running a flow using a LocalAgent
? I was expecting FlowRunner
and TaskRunner
to be used (the backend is also âserverâ)Amanda Wee
12/29/2020, 10:08 PMjeff n
12/30/2020, 12:41 AMEquipe AI HOC
12/30/2020, 9:56 AMSébastien
12/30/2020, 10:06 AMprefect.utilities.exceptions.ClientError: 400 Client Error: Bad Request for url: <https://api.prefect.io/graphql>
The following error messages were provided by the GraphQL server:
INTERNAL_SERVER_ERROR: Variable "$input" got invalid value null at
"input.states[0].task_run_id"; Expected non-nullable type UUID! not to be null.
The GraphQL query was:
mutation($input: set_task_run_states_input!) {
set_task_run_states(input: $input) {
states {
id
message
status
}
}
}
Runs (for different flows) after the failed run work again. This failed flow will run again at 4pm (in ~5h). Is this a known issue? Did I do something wrong?Matthew Blau
12/30/2020, 2:24 PMpip3 install prefect
and when I run
prefect backend server
I get a
prefect: command not found
my python version is 3.6.8. What am I doing wrong?kevin
12/30/2020, 5:37 PMdefaults_from_attrs
decorator on PostgresFetch.run()
allow me to build a bunch of queries out like this:
with Flow('foo') as flow:
pw = PrefectSecret('pass')
pg_template = PostgresFetch(host='host',user='user',db_name='name', port=1234)
pg_query1 = pg_template(password=pw, query='select * from foo;')
pg_query2 = pg_template(password=pw, query='select * from bar;')
kevin
12/30/2020, 5:37 PMdefaults_from_attrs
decorator on PostgresFetch.run()
allow me to build a bunch of queries out like this:
with Flow('foo') as flow:
pw = PrefectSecret('pass')
pg_template = PostgresFetch(host='host',user='user',db_name='name', port=1234)
pg_query1 = pg_template(password=pw, query='select * from foo;')
pg_query2 = pg_template(password=pw, query='select * from bar;')
josh
12/30/2020, 6:08 PMkevin
12/30/2020, 6:09 PMrun()
method on the task rather than at instantiationjosh
12/30/2020, 6:11 PMPrefectSecret
task is a preferred way of passing sensitive information to the task and for the value to be read it needs to be passed in at runtimekevin
12/30/2020, 6:12 PMPrefectSecret(os.environ.get('db_pw'))
?josh
12/30/2020, 6:13 PMdb_pw
is the name of the secret then you should be good to go. Take a look at this doc on secrets for some more information about how they are used: https://docs.prefect.io/core/concepts/secrets.htmlkevin
12/30/2020, 6:14 PM