Georgi Yanev
07/05/2022, 7:38 AMRobin Weiß
07/05/2022, 8:05 AMprefect.exceptions.PrefectHTTPStatusError: Client error '422 Unprocessable Entity' for url '<https://api-beta.prefect.io/api/accounts/XXXXXXX/workspaces/XXXXXXXX/work_queues/name/k8s>'
Response: {'exception_message': 'Invalid request received.', 'exception_detail': [{'loc': ['path', 'account_id'], 'msg': 'value is not a valid uuid', 'type': 'type_error.uuid'}, {'loc': ['path', 'workspace_id'], 'msg': 'value is not a valid uuid', 'type': 'type_error.uuid'}, {'loc': ['path', 'account_id'], 'msg': 'value is not a valid uuid', 'type': 'type_error.uuid'}, {'loc': ['path', 'workspace_id'], 'msg': 'value is not a valid uuid', 'type': 'type_error.uuid'}, {'loc': ['path', 'workspace_id'], 'msg': 'value is not a valid uuid', 'type': 'type_error.uuid'}], 'request_body': None}
For more information check: <https://httpstatuses.com/422>
An exception occurred.
That error is just so weird that I have no ideas where to look. Any help would be greatly appreciated :)
The agent is in a crashloop because of this. I have followed https://discourse.prefect.io/t/how-to-deploy-a-prefect-2-0-agent-to-a-local-kubernetes-cluster-and-connect-it-to-cloud-2-0-backend/979 exactly. The only difference is that I updated to prefecthq/prefect:2.0b7-python3.9
because the b6 image gave me version incompatibility errors when using the CLI tool.redsquare
07/05/2022, 8:14 AMShivam Bhatia
07/05/2022, 8:16 AMFlorian Kühnlenz
07/05/2022, 1:05 PMUrban Škudnik
07/05/2022, 1:44 PMprefect
would start it's containers? Is that correct?
If so, what's the best path to have everything as a docker-compose file that developers can spin up on request? Ar there any samples for that use case?Tom Klein
07/05/2022, 2:27 PMnode:12-alpine
(which doesn’t seem to have pip
and possibly not python
either)
i tried to add:
RUN apk update
RUN apk add py-pip
RUN pip install prefect[github,aws,kubernetes,snowflake]
first steps finished fine, but the prefect installation seems to be taking forever (over 25 minutes already) and also has a lot of weird warning messages like:
Collecting snowflake-connector-python>=1.8.2
Downloading snowflake_connector_python-1.8.7-py2.py3-none-any.whl (168 kB)
Downloading snowflake_connector_python-1.8.6-py2.py3-none-any.whl (161 kB)
Downloading snowflake_connector_python-1.8.5-py2.py3-none-any.whl (159 kB)
Downloading snowflake_connector_python-1.8.4-py2.py3-none-any.whl (161 kB)
Downloading snowflake_connector_python-1.8.3-py2.py3-none-any.whl (158 kB)
Downloading snowflake_connector_python-1.8.2-py2.py3-none-any.whl (157 kB)
INFO: pip is looking at multiple versions of six to determine which version is compatible with other requirements. This could take a while.
or :
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this
run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: <https://pip.pypa.io/surveys/backtracking>
any ideas?datamongus
07/05/2022, 3:27 PMAustin Anderson
07/05/2022, 4:49 PMKevin
07/05/2022, 5:28 PMChris Reuter
07/05/2022, 5:54 PMIfeanyi Okwuchi
07/05/2022, 7:14 PM.map()
is trying to run the task more times than necessary and the 4th time, the index is a string. . product_categories
is a list containing 3 elements, flow_config
is a dictionary and product_category_variables
is also a dictionary. When the task runs in prefect cloud there are three successful task runs indexed 0, 1, 2 but then it tries to do another run with the index as a string and it fails saying
Task 'set_dynamic_config_settings['dataset_bucket_path']': Starting task run...
TypeError: list indices must be integers or slices, not str
The task set_dynamic_config_settings
looks like this within the flow definition
with Flow(name="flow-name") as:
product_category_variables = get_run_variables(
is_zero_nyp=is_zero_nyp_param,
bucket_base=flow_config["bucket_base"],
ltv_product_categories=product_categories,
return_type="vars",
)
final_config = set_dynamic_config_settings.map(
cfg=unmapped(flow_config),
product_category_variables=unmapped(product_category_variables),
product_category=product_categories,
upstream_tasks=[unmapped(product_category_variables)],
)
Mansour Zayer
07/05/2022, 7:30 PMwait_for_flow_run
, but I get asked for a flow_run_id
, while I don't have a flow_run_id
, I only have a flow_id
for the parent flow flow_AConstantino Schillebeeckx
07/05/2022, 9:17 PMMars
07/05/2022, 9:33 PMflow.run()
?
I’ve tried adding this code from the boto3 docs to my flow:
import boto3
boto3.set_stream_logger() # sets logging.DEBUG
However, ^ that code hasn’t changed the flow.run()
output.
For background, I’m trying to use localstack as an AWS stand-in for local pipeline development. The S3Download
task is giving me errors and I want to debug the boto3 connection to find out if the S3 service URL, bucket, and key are correct.Jeff Kehler
07/06/2022, 3:54 AMAndreas
07/06/2022, 10:38 AMprefect.client.create_flow_run_from_deployment()
is it possible to set the name for the flow_run? I know that there is such an option when calling create_flow_run(name="My flow run name")
that takes as input a flow_model, what about when creating a run from a deployment though?Surya
07/06/2022, 12:16 PMTom Klein
07/06/2022, 1:35 PMJosh Paulin
07/06/2022, 2:04 PMprefect.context.parameters
when setting a Result
location?Nikolaus Landgraf
07/06/2022, 2:12 PMsqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: json_each
[SQL: INSERT INTO flow_run_notification_queue (flow_run_notification_policy_id, flow_run_state_id) SELECT flow_run_notification_policy.id, CAST(? AS CHAR(36)) AS anon_1
FROM flow_run_notification_policy
WHERE flow_run_notification_policy.is_active IS 1 AND (flow_run_notification_policy.state_names = ? OR EXISTS (SELECT 1
FROM json_each(flow_run_notification_policy.state_names) AS json_each
WHERE json_each.value IN (?))) AND (flow_run_notification_policy.tags = ? OR EXISTS (SELECT 1
FROM json_each(flow_run_notification_policy.tags) AS json_each
WHERE json_each.value IN (SELECT 1 FROM (SELECT 1) WHERE 1!=1)))]
[parameters: ('8d01934d-c9c8-4f23-a3af-ec03ee1586a0', '[]', 'Pending', '[]')]
(Background on this error at: <https://sqlalche.me/e/14/e3q8>)
Muddassir Shaikh
07/06/2022, 2:15 PMgraphql_1 | ValueError: Uniqueness violation.
graphql_1 |
graphql_1 | The above exception was the direct cause of the following exception:
graphql_1 |
graphql_1 | Traceback (most recent call last):
graphql_1 | File "/usr/local/lib/python3.7/site-packages/graphql/execution/execute.py", line 674, in await_completed
graphql_1 | return await completed
graphql_1 | File "/usr/local/lib/python3.7/site-packages/graphql/execution/execute.py", line 659, in await_result
graphql_1 | return_type, field_nodes, info, path, await result
graphql_1 | File "/usr/local/lib/python3.7/site-packages/graphql/execution/execute.py", line 733, in complete_value
graphql_1 | raise result
graphql_1 | File "/usr/local/lib/python3.7/site-packages/graphql/execution/execute.py", line 628, in await_result
graphql_1 | return await result
graphql_1 | File "/prefect-server/src/prefect_server/graphql/extensions.py", line 52, in resolve
graphql_1 | result = await result
graphql_1 | File "/prefect-server/src/prefect_server/graphql/tenants.py", line 14, in resolve_create_tenant
graphql_1 | "id": await api.tenants.create_tenant(name=input["name"], slug=input["slug"])
graphql_1 | File "/prefect-server/src/prefect_server/api/tenants.py", line 29, in create_tenant
graphql_1 | tenant_id = await models.Tenant(name=name, slug=slug).insert()
graphql_1 | File "/prefect-server/src/prefect_server/database/orm.py", line 222, in insert
graphql_1 | insert_mutation_name=self.__root_fields__.get("insert"),
graphql_1 | File "/prefect-server/src/prefect_server/database/hasura.py", line 237, in insert
graphql_1 | result = await self.execute_mutations_in_transaction(mutations=[graphql])
graphql_1 | File "/prefect-server/src/prefect_server/database/hasura.py", line 165, in execute_mutations_in_transaction
graphql_1 | as_box=as_box,
graphql_1 | File "/prefect-server/src/prefect_server/database/hasura.py", line 89, in execute
graphql_1 | raise ValueError("Uniqueness violation.")
graphql_1 | graphql.error.graphql_error.GraphQLError: Uniqueness violation.
graphql_1 |
graphql_1 | GraphQL request:2:3
graphql_1 | 1 | mutation ($input: create_tenant_input!) {
graphql_1 | 2 | create_tenant(input: $input) {
graphql_1 | | ^
graphql_1 | 3 | id
Vadym Dytyniak
07/06/2022, 2:23 PMIbrahim Sherif
07/06/2022, 3:47 PMkevin
07/06/2022, 5:34 PM.csv
file from a google cloud bucket. It seems like this task is the correct way to go? An alternative I can see is just implementing a python callable that uses the python library for google cloud storage. I'd like some feedback if I'm approaching this problem correctly?Amogh Kulkarni
07/06/2022, 6:21 PMJosh Paulin
07/06/2022, 6:50 PMSuccess
state, and the results file has been overwritten. If I’m understanding correctly this should be completing with Cached
. Any suggestions?Jon Ruhnke
07/06/2022, 7:19 PMKyle McChesney
07/06/2022, 8:46 PM-l
(I want a login shell). Seems like currently its passing the command like so
overrides
containerOverrides
command": [
"/bin/sh",
"-c",
"prefect execute flow-run"
]
Also, possible or not, is this command stable / is it safe to customize this?Ryan Sattler
07/07/2022, 4:37 AM