Arun Dass
08/21/2021, 4:32 AMAnh Nguyen
08/21/2021, 7:14 AMChintan Shah
08/21/2021, 4:19 PM--run-task-kwargs
Wilson Bilkovich
08/21/2021, 11:01 PMhost = foo
in one video, and [server] endpoint = baz
in anotherWilson Bilkovich
08/21/2021, 11:32 PMBlake List
08/22/2021, 3:26 AM'0000-00-00 00:00:00'
and t2 will be time.now()
. The next time schedule runs, t1 needs to be the previous value of t2, and t2 will once again be time.now()
. I can see that I may need to use a task to compute the parameters within the script of the parent flow (similar to here) and use the StartFlowRun
task (as seen here), but I'm not quite sure how to put it together.
Any help would be appreciated, thank you!!Chintan Shah
08/22/2021, 11:32 AMAn error occurred (ClientException) when calling the RegisterTaskDefinition operation: Container.image should not be null or empty
.
this is run time error while trying to execute flow . any pointersBen Muller
08/22/2021, 10:27 PMS3(
bucket="es-prefect-flows-staging",
stored_as_script=True,
)
Now after registering my flow I am getting an error after the flow is downloaded: in 🧵Gaylord Cherencey
08/23/2021, 3:51 AMEddie Atkinson
08/23/2021, 5:06 AMmap
to process the data for each site individually, but does so for the entire range at once. The complication arises from the fact that the date range is specified as a parameter, and the site_ids are the result of a task that runs.Chhaya Vankhede
08/23/2021, 6:47 AMItalo Barros
08/23/2021, 11:48 AMFailed to load and execute Flow's environment: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'Submodules\'",)\nThis may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.',)
The error was solved after shutting down the agent and starting again at the same folder of his first execution. I think this is related to the agent metadata, is this behavior normal?Wilson Bilkovich
08/23/2021, 12:29 PMRunConfig
object? Do I need one if I’m already specifying an Executor?Jelle Vegter
08/23/2021, 1:05 PMBastian Röhrig
08/23/2021, 2:14 PMprefect auth list-tenants
I get my tenants, so if feel like I should be authenticated. I'm kind of stumped, maybe someone here has an idea?
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 738, in _request
json_resp = response.json()
File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/prefect", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/cli/execute.py", line 53, in flow_run
result = client.graphql(query)
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 550, in graphql
retry_on_api_error=retry_on_api_error,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 454, in post
retry_on_api_error=retry_on_api_error,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 744, in _request
) from exc
prefect.exceptions.AuthorizationError: Malformed response received from Cloud - please ensure that you are authenticated. See `prefect auth login --help`.
YD
08/23/2021, 4:46 PMteam
option in flow.register
.
is it when you create the project in the cloud UI?
what happens if one team create a project with the same name as a different team ?Alex Furrier
08/23/2021, 6:34 PMfrom prefect import task
from prefect.engine.result.local_result import LocalResult
from my_prefect_tasks import imported_task
@task(result=LocalResult())
def assigned_result_task(x):
return x
imported_task.result_handler = LocalResult()
Zach Schumacher
08/23/2021, 9:07 PMVincent
08/23/2021, 10:00 PMrequests.exceptions.RetryError: HTTPSConnectionPool(host='<http://api.prefect.io|api.prefect.io>', port=443): Max retries exceeded with url: /graphql (Caused by ResponseError('too many 429 error responses'))
Ours jobs are quite parallel, and wonder if there is anything that must be resolved, either user or server side that can enable this parallel execution.Alex Furrier
08/23/2021, 10:37 PMprefect.client.client.FlowRunInfoResult
that's returned by client.get_flow_run_info()
?Wilson Bilkovich
08/23/2021, 10:46 PMUnexpected error occured in FlowRunner: ModuleNotFoundError("No module named 'prefect'")
when I try to run a flow after making a small tweak to the pod spec. Anyone seen that before?Maria
08/24/2021, 1:14 AMState message: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family.
Any advices?Jacob Blanco
08/24/2021, 6:40 AMRobert Hales
08/24/2021, 10:36 AMDaskExecutor
Luiz Bezerra de Oliveira Lima Neto
08/24/2021, 12:09 PMCharles Leung
08/24/2021, 3:11 PMWilson Bilkovich
08/24/2021, 4:00 PMERROR - prefect.FlowRunner | Unexpected error: TypeError('code() takes at most 15 arguments (16 given)')
Is this potentially because I’m running Python 3.9?Kyle Pierce
08/24/2021, 5:03 PMError downloading Flow from S3: No module named 'boto3'
Used the prefecthq/server:core-0.15.3
image. Is there something for extras or something we are missing?Leon Kozlowski
08/24/2021, 7:12 PMwiretrack
08/24/2021, 8:50 PMtenant
field in the database, but could find a way to make the UI tenant specific, neither to register the flow for a specific tenant. Any ideas?wiretrack
08/24/2021, 8:50 PMtenant
field in the database, but could find a way to make the UI tenant specific, neither to register the flow for a specific tenant. Any ideas?nicholas
08/24/2021, 9:11 PMtenant
field is a convention only; the UI normally relies on implicit database sharding to filter views instead of adding an explicit tenant_id
filter to every requestwiretrack
08/24/2021, 9:14 PMregister()
method in the Flow
class instantiates a client
object, which accepts tenant_id
as a parameter. So, excluding the authentication (security) aspect of it, it shouldn’t be a dramatic change to add tenant_id
as a parameter for the register
method, and allow for multitenancy. Maybe the UI would need an extra layer to allow to change tenants, but could also have just a tenant_id
identifier. Far from production ready in the sense of security, but could be a great alternative for separation of concerns.server
repo to allow multi tenancy, I would be happy to contribute as well, both for the server and the UInicholas
08/25/2021, 12:18 AM