Alex Cannon
09/09/2022, 7:19 PMcreateFlowRunInput
type in the GraphQL API doesn't include labels
, is there a way to trigger a flow run with a specific label using the GraphQL API?Solmaz Bagherpour
09/09/2022, 7:36 PMAnkur Sheth
09/09/2022, 7:42 PMEsdras Lopes Nani
09/09/2022, 7:51 PMprefect agent start -q 'queue_name' &
(observe the '&' indicator to run it in background), but once I close my remote terminal the agent is killed. If I keep my terminal open the agent don't die.
Is it a current bug, am I doing something wrong or is it the normal behavior?
Thanks!Michael Dyer
09/09/2022, 8:02 PMprefect orion start --host 0.0.0.0
I have a reverse proxy (traefik) pointing to the container and I can access the UI at <https://my-host:4200>
. The reverse proxy is handling ssl offload.
The UI is displayed, but unable to connect to the API and I'm receiving the following message:
Can't connect to Orion API at <http://127.0.0.1:4200/api>. Check that it's accessible from your machine.
Q. Is there a way to configure the Orion UI so that it uses a relative URL /api
? (i.e. https://my-host:4200/api)?Otacílio Ribeiro do Amaral Neto
09/09/2022, 8:08 PMBen Ayers-Glassey
09/09/2022, 9:03 PMMiles Lauridsen
09/10/2022, 12:26 AMCole Murray
09/10/2022, 1:43 AMyusuf yel
09/10/2022, 3:12 PMScott Walsh
09/11/2022, 1:00 AMCA Lee
09/11/2022, 8:27 AMis_schedule_active
property in deployment.yaml?
Error:
'deployments/func-deployment.yaml' did not conform to deployment spec: ValidationError(model='Deployment', errors=[{'loc': ('is_schedule_active',), 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}])
This is using S3 block storage. If using LocalFileStorage, I believe this error does not occur.
Prefect version: 2.3.2
Removing the field: is_schedule_active: false
, the deployment can be applied successfully, but the user would need to login to the UI to manually disable the schedule. This is especially painful if triggered on each CI/CD run.Ammar Chalifah
09/11/2022, 6:23 PMPREFECT_API_URL=http//0.0.0.0:4200/api
Chris L.
09/11/2022, 6:51 PM[ "python", "-m", "prefect.engine" ]
for KubernetesJob
18:49:52.809 | INFO | prefect.agent - Submitting flow run 'faa2ebac-ee94-4384-9a4f-16ad28d6fa7e'
....
File "/usr/local/lib/python3.9/site-packages/prefect/blocks/core.py", line 171, in __init__
super().__init__(*args, **kwargs)
File "pydantic/main.py", line 342, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for KubernetesJob
command
none is not an allowed value (type=type_error.none.not_allowed)
James Constable
09/12/2022, 2:53 AMAhmed Ezzat
09/12/2022, 4:07 AMNaila Chennit
09/12/2022, 8:41 AMEli Treuherz
09/12/2022, 9:01 AMraise SKIP("not applicable")
in order to skip a task? If I just return Cancelled(message="not applicable")
instead of returning my task’s actual data, will that have the same effect?Dennis Hinnenkamp
09/12/2022, 12:10 PMsnowflake_connector_block
for a dbt shell task, but I get the following error when I use the created `dbt_cli_profile`:
14:07:11.533 | ERROR | Task run 'trigger_dbt_cli_command-321ca940-0' - Encountered exception during execution:
Traceback (most recent call last):
File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect/engine.py", line 1185, in orchestrate_task_run
result = await task.fn(*args, **kwargs)
File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/commands.py", line 136, in trigger_dbt_cli_command
profile = dbt_cli_profile.get_profile()
File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/credentials.py", line 107, in get_profile
"outputs": {self.target: self.target_configs.get_configs()},
File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/configs/snowflake.py", line 78, in get_configs
configs_json = super().get_configs()
File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/configs/base.py", line 53, in get_configs
return self._populate_configs_json({}, self.dict())
File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/configs/base.py", line 34, in _populate_configs_json
configs_json = self._populate_configs_json(configs_json, value)
File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/configs/base.py", line 37, in _populate_configs_json
raise ValueError(
ValueError: The keyword, _is_anonymous, has already been provided in TargetConfigs; remove duplicated keywords to continue
My current code looks like this:
snowflake_connector_block = await SnowflakeConnector.load("wld-snowflake-connection")
connector = snowflake_connector_block
target_configs = SnowflakeTargetConfigs(
connector=connector
)
dbt_cli_profile = DbtCliProfile(
name="profile_name",
target="dev",
target_configs=target_configs
)
debug = await trigger_dbt_cli_command.submit(
"dbt debug",
profiles_dir='profile',
project_dir='wld-dbt-warehouse/dbt-project',
overwrite_profiles=True,
dbt_cli_profile=dbt_cli_profile,
return_state=True
)
Vadym Dytyniak
09/12/2022, 12:22 PMcustomizations=[
{
"op": "add",
"path": "/spec/imagePullSecrets",
"value": [{'name': 'dockerhub'}],
},
],
What is wrong with passed value? I still get an access denied error.Niels Prins
09/12/2022, 1:14 PMfilters
when creating a WorkQueue?
<EDIT> Yes this is correct, see threadPrasanth Kothuri
09/12/2022, 2:03 PMPrasanth Kothuri
09/12/2022, 2:03 PMschedule = Schedule(clocks=[CronClock("0 07,12,14 * * 1-6"),CronClock("0 10 * * 0")])
Prasanth Kothuri
09/12/2022, 2:05 PMKelvin Garcia
09/12/2022, 2:51 PMJosh Cowles
09/12/2022, 3:19 PMSam Garvis
09/12/2022, 5:36 PMValueError: Path /root/.prefect/storage/72ed22dddb494741872cffe99e581ee9 does not exist.
It's an incredibly vague error that is hard to debug. However, the flows all complete successfully, they just show as crashed. But if we were to transition to 2.0, we need reliable error monitoring, and we can't do that with 90% of flows crashing for no reason.Nathaniel Russell
09/12/2022, 6:42 PMKelvin Garcia
09/12/2022, 7:20 PM{
"jobId": reportId,
"s3Location": s3Location,
"validationSettings": validation_settings,
}
but when the flow fails and I need to restart a task that uses that dict as a dependency I get this error
reportId = settings["jobId"]
TypeError: 'NoneType' object is not subscriptable
why the second time I run that task the input is None
, shouldn't it cache the input for restarts?Leon Kozlowski
09/12/2022, 7:55 PMprefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/work_queues/>'
Not sure if I have my api url wrong or some other configuration incorrectLeon Kozlowski
09/12/2022, 7:55 PMprefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/work_queues/>'
Not sure if I have my api url wrong or some other configuration incorrect19:51:19.313 | ERROR | prefect.agent - Failed to create work queue 'dev'.
Serina
09/12/2022, 7:59 PM'<https://api.prefect.cloud/api/accounts/abc-123-456/workspaces/xyz-987-654>'
Leon Kozlowski
09/12/2022, 7:59 PMspec:
containers:
- name: agent
command: [ "prefect", "agent", "start", "-q", "dev" ]
agent Starting v2.3.0 agent connected to <https://api.prefect.cloud>...
agent
agent ___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
agent | _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
agent | _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
agent |_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
agent
agent
agent Agent started! Looking for work from queue(s): dev...
agent 19:51:19.313 | ERROR | prefect.agent - Failed to create work queue 'dev'.
agent Traceback (most recent call last):
agent File "/usr/local/lib/python3.10/site-packages/prefect/agent.py", line 88, in get_work_queues
agent work_queue = await self.client.create_work_queue(name=name)
agent File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 835, in create_work_queue
agent response = await <http://self._client.post|self._client.post>("/work_queues/", json=data)
agent File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1842, in post
agent return await self.request(
agent File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1527, in request
agent return await self.send(request, auth=auth, follow_redirects=follow_redirects)
agent File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 279, in send
agent response.raise_for_status()
agent File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 225, in raise_for_status
agent raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
agent prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/work_queues/>'
agent For more information check: <https://httpstatuses.com/404>
Michael Adkins
09/12/2022, 8:04 PMPREFECT_API_URL
set to in the agent’s container? Serina is correct that it appars to be incorrectly specified.prefect cloud login --key …
then prefect config view
Leon Kozlowski
09/12/2022, 8:05 PM<https://api.prefect.cloud>
Matching the open source chart: https://github.com/PrefectHQ/prefect-helm/blob/f0e838aa20402fa8c23cb4fca8e6c0576d288b1d/charts/prefect-agent/values.yaml#L25Michael Adkins
09/12/2022, 8:06 PMLeon Kozlowski
09/12/2022, 8:06 PMMichael Adkins
09/12/2022, 8:07 PMLeon Kozlowski
09/12/2022, 8:08 PMChristopher Boyd
09/12/2022, 8:10 PMLeon Kozlowski
09/12/2022, 8:10 PMChristopher Boyd
09/12/2022, 8:10 PMLeon Kozlowski
09/12/2022, 8:50 PMChristopher Boyd
09/12/2022, 8:52 PMLeon Kozlowski
09/12/2022, 9:00 PM