prefect-community #prefect-community

prefect-community

Alex Cannon

09/09/2022, 7:19 PM

Hello! I noticed that the

createFlowRunInput

type in the GraphQL API doesn't include

labels

, is there a way to trigger a flow run with a specific label using the GraphQL API?

✅ 1

Solmaz Bagherpour

09/09/2022, 7:36 PM

Hi prefect community, I am looking for a way to set up a slack alert for flows that their runtimes pass a certain expected time while the flow is still running, would appreciate your suggestions or best ways to achieve this, We are on prefect 1 and we use python for our flow runs and we already have slack alerts for failed runs but that is using flow run state changes which is not useful for this case.

✅ 1

Ankur Sheth

09/09/2022, 7:42 PM

Hi All, Has anyone been able to install prefect on centos7 box, I am trying to install prefect on centos7 box, but its failing saying that it needs sqlite3, I have installed latest version available for sqlite on the same centos7 box, but it doesn’t like it, if anyone have experienced such an issue before, please comments your workarounds or solutions

✅ 1

Esdras Lopes Nani

09/09/2022, 7:51 PM

Hey everyone. I'm having some issues with a local agent in linux AWS-EC2. I've started it using

prefect agent start -q 'queue_name' &

(observe the '&' indicator to run it in background), but once I close my remote terminal the agent is killed. If I keep my terminal open the agent don't die. Is it a current bug, am I doing something wrong or is it the normal behavior? Thanks!

✅ 1

Michael Dyer

09/09/2022, 8:02 PM

I am running prefect 2.3 inside a docker container using the following command:

prefect orion start --host 0.0.0.0

I have a reverse proxy (traefik) pointing to the container and I can access the UI at

<https://my-host:4200>

. The reverse proxy is handling ssl offload. The UI is displayed, but unable to connect to the API and I'm receiving the following message:

Can't connect to Orion API at <http://127.0.0.1:4200/api>. Check that it's accessible from your machine.

Q. Is there a way to configure the Orion UI so that it uses a relative URL

/api

? (i.e. https://my-host:4200/api)?

Otacílio Ribeiro do Amaral Neto

09/09/2022, 8:08 PM

Good afternoon. We are having problems with Prefect appointments since 05/09/2022 that were in the queue and did not execute. I checked the processes, stopped the service and then reactivated it, but the failure persisted. As we are heading into the weekend and with impacts on some collection services, how can we resolve this glitch?

Ben Ayers-Glassey

09/09/2022, 9:03 PM

Does Prefect discard the data for old flow runs?.. maybe ones older than 14 days?.. In particular, I had a failed run from Aug 31: https://cloud.prefect.io/zesty-ai/flow-run/385b3c9b-8828-4b46-bda7-5ea32642b266 ...and today I was going to get around to poking at the bug causing it to fail, but instead I see a mostly-blank page with lots of "Uncaught TypeError: Cannot read properties of null (reading 'state')" in the JS console (see attached screenshot). In case it matters, this is for old Prefect, not 2.0.

✅ 1

Miles Lauridsen

09/10/2022, 12:26 AM

Hi all. Having some issues trying to run a basic Prefect flow in a docker instance. Works fine on my local machine (outside docker) but then when run in a container outputs this error. Any suggestions for usage?

✅ 1

Cole Murray

09/10/2022, 1:43 AM

Hi all, Running into a bit of an odd issue around pickling of a result with a recent change to an existing flow. The return type from the task is a List[<dataclass>]. Within the object, the properties are both strings. I’ve tried re-deploying the flow, and currently in progress of upgrading from 2.2.0 to 2.3.2. Anyone experience issues around cloud pickle on returning an object from a task? Stacktrace linked in comments

✅ 2

yusuf yel

09/10/2022, 3:12 PM

Hi everyone, When I try build deployment I get this error:

❓ 1

✅ 1

Scott Walsh

09/11/2022, 1:00 AM

Hello. I'm using Prefect 2.3.2. I would like to choose the name for each of my flow runs (not use the adjective-noun default) so that they are easier to search for in the Orion UI. I am processing images, and would like to name each flow run after its corresponding image. I am not currently using Agents or Deployments or Queues, just importing prefect into python scripts to help organize pipeline code and get each retries and convenient logging. I there a way that I can choose the flow run name when my flow is called? I have tried OrienClient.create_flow_run, but this seems to depend on using agents/deployments. Is there (or could there be) a simpler way I can do this directly when a new flow run starts?

✅ 1

CA Lee

09/11/2022, 8:27 AM

Hello, has anyone encounted an error while setting

is_schedule_active

property in deployment.yaml? Error:

'deployments/func-deployment.yaml' did not conform to deployment spec: ValidationError(model='Deployment', errors=[{'loc': ('is_schedule_active',), 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}])

This is using S3 block storage. If using LocalFileStorage, I believe this error does not occur. Prefect version: 2.3.2 Removing the field:

is_schedule_active: false

, the deployment can be applied successfully, but the user would need to login to the UI to manually disable the schedule. This is especially painful if triggered on each CI/CD run.

✅ 2

Ammar Chalifah

09/11/2022, 6:23 PM

Hi people! I'm trying to run Prefect locally inside a docker container and access the server UI through localhost. To do so, I set the Prefect Orion API Host to 0.0.0.0 (because the default host is localhost, and I can't access localhost through port forwarding from my local machine). However, after building the deployment and applying it, it seems the change is not reflected in the Prefect server UI. I know there is something wrong but I can't spot the mistake. Is there anyone that can help? Thank you 😄 UPDATE: The error in Orion UI no longer exists after I changed the API URL with this env var

PREFECT_API_URL=http//0.0.0.0:4200/api

👍 1

✅ 1

Chris L.

09/11/2022, 6:51 PM

Hello Prefect community! Not sure if this is supposed to be a feature or a bug...but all our k8s deployed flows started failing the last two days with this error on the K8s Agent (v2.3.2): FIXED: Specified`command` arg explicitly as

[ "python", "-m", "prefect.engine" ]

for

KubernetesJob

18:49:52.809 | INFO    | prefect.agent - Submitting flow run 'faa2ebac-ee94-4384-9a4f-16ad28d6fa7e'
....  
  File "/usr/local/lib/python3.9/site-packages/prefect/blocks/core.py", line 171, in __init__
    super().__init__(*args, **kwargs)
  File "pydantic/main.py", line 342, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for KubernetesJob
command
  none is not an allowed value (type=type_error.none.not_allowed)

✅ 1

James Constable

09/12/2022, 2:53 AM

Hi All. I am experimenting with Prefect 2.0 deployments. I am using azure storage as my deployment storage. The simplest example project I can make takes ~25mins to build the deployment. This is way too slow for any kind of CI/CD situation. Is this normal? Is it my connection? Am I doing something wrong?

Ahmed Ezzat

09/12/2022, 4:07 AM

I'm unable to access prefect 2.0 cloud using Github login or reset my own password

Naila Chennit

09/12/2022, 8:41 AM

Hello Prefect community, I would like to know why can't we set a task timeout anymore with prefect 2 please ?

✅ 1

Eli Treuherz

09/12/2022, 9:01 AM

Is there a Prefect 2 equivalent to doing

raise SKIP("not applicable")

in order to skip a task? If I just

return Cancelled(message="not applicable")

instead of returning my task’s actual data, will that have the same effect?

✅ 1

Dennis Hinnenkamp

09/12/2022, 12:10 PM

Hi all, I am trying to read the configuration from a

snowflake_connector_block

for a dbt shell task, but I get the following error when I use the created `dbt_cli_profile`:

14:07:11.533 | ERROR   | Task run 'trigger_dbt_cli_command-321ca940-0' - Encountered exception during execution:
Traceback (most recent call last):
  File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect/engine.py", line 1185, in orchestrate_task_run
    result = await task.fn(*args, **kwargs)
  File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/commands.py", line 136, in trigger_dbt_cli_command
    profile = dbt_cli_profile.get_profile()
  File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/credentials.py", line 107, in get_profile
    "outputs": {self.target: self.target_configs.get_configs()},
  File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/configs/snowflake.py", line 78, in get_configs
    configs_json = super().get_configs()
  File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/configs/base.py", line 53, in get_configs
    return self._populate_configs_json({}, self.dict())
  File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/configs/base.py", line 34, in _populate_configs_json
    configs_json = self._populate_configs_json(configs_json, value)
  File "/Users/d.hinnenkamp/Desktop/test-prefect/.venv/lib/python3.8/site-packages/prefect_dbt/cli/configs/base.py", line 37, in _populate_configs_json
    raise ValueError(
ValueError: The keyword, _is_anonymous, has already been provided in TargetConfigs; remove duplicated keywords to continue

My current code looks like this:

snowflake_connector_block = await SnowflakeConnector.load("wld-snowflake-connection")

connector = snowflake_connector_block

target_configs = SnowflakeTargetConfigs(

connector=connector

dbt_cli_profile = DbtCliProfile(

name="profile_name",

target="dev",

target_configs=target_configs

debug = await trigger_dbt_cli_command.submit(

"dbt debug",

profiles_dir='profile',

project_dir='wld-dbt-warehouse/dbt-project',

overwrite_profiles=True,

dbt_cli_profile=dbt_cli_profile,

return_state=True

✅ 1

Vadym Dytyniak

09/12/2022, 12:22 PM

Hello. I need to pass imagePullSecrets in KubernetesJob. I use customizations arg:

customizations=[
    {
        "op": "add",
        "path": "/spec/imagePullSecrets",
        "value": [{'name': 'dockerhub'}],
    },
],

What is wrong with passed value? I still get an access denied error.

✅ 1

Niels Prins

09/12/2022, 1:14 PM

is it corrct that the Orion gui does not show

filters

when creating a WorkQueue? <EDIT> Yes this is correct, see thread

✅ 1

Prasanth Kothuri

09/12/2022, 2:03 PM

Hi all, is the following schedule allowed? it allows me to register the flow, so assuming thats ok

✅ 1

Prasanth Kothuri

09/12/2022, 2:03 PM

schedule = Schedule(clocks=[CronClock("0 07,12,14 * * 1-6"),CronClock("0 10 * * 0")])

Prasanth Kothuri

09/12/2022, 2:05 PM

Kelvin Garcia

09/12/2022, 2:51 PM

Hi all, I have some flows that fail but when I try to restart it fails because of a different reason, it seems like it doesn't remember the output of a successful task, what should I do to persist data on restart?

👀 1

✅ 1

Josh Cowles

09/12/2022, 3:19 PM

Is there a publicly available roadmap somewhere? Or is it internal to the prefect team only? From this community post it says that vertex in prefect 2 is on the roadmap, but no further info on when it might be coming

✅ 1

Sam Garvis

09/12/2022, 5:36 PM

https://github.com/PrefectHQ/prefect/issues/6779 We are on Prefect 2.3.2 using GCS storage and k8s helm agents. I just commented on somebody else's issue on GitHub, but just to reiterate, this error is the only thing that is preventing our team from truly upgrading to Prefect 2.0. It has been going on since we started transitioning and it won't go away.

ValueError: Path /root/.prefect/storage/72ed22dddb494741872cffe99e581ee9 does not exist.

It's an incredibly vague error that is hard to debug. However, the flows all complete successfully, they just show as crashed. But if we were to transition to 2.0, we need reliable error monitoring, and we can't do that with 90% of flows crashing for no reason.

✅ 1

Nathaniel Russell

09/12/2022, 6:42 PM

how does one run a deployment in python? I have found only how to make a deployment or run a flow separate from a deployment.

✅ 1

Kelvin Garcia

09/12/2022, 7:20 PM

Hi all, I see in the docs (https://docs-v1.prefect.io/core/concepts/persistence.html#input-caching) that prefect does automatic input caching, so in this case I have a task that returns a dict

{
        "jobId": reportId,
        "s3Location": s3Location,
        "validationSettings": validation_settings,
    }

but when the flow fails and I need to restart a task that uses that dict as a dependency I get this error

reportId = settings["jobId"]
TypeError: 'NoneType' object is not subscriptable

why the second time I run that task the input is

None

, shouldn't it cache the input for restarts?

✅ 1

Leon Kozlowski

09/12/2022, 7:55 PM

Hi all, I’m try to deploy an orion agent - I’m getting the following error:

prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/work_queues/>'

Not sure if I have my api url wrong or some other configuration incorrect

✅ 1

Title

Leon Kozlowski

09/12/2022, 7:55 PM

Hi all, I’m try to deploy an orion agent - I’m getting the following error:

prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/work_queues/>'

Not sure if I have my api url wrong or some other configuration incorrect

✅ 1

It first failed when trying to create a work queue:

19:51:19.313 | ERROR   | prefect.agent - Failed to create work queue 'dev'.

Serina

09/12/2022, 7:59 PM

I think it might need to be in this format:

'<https://api.prefect.cloud/api/accounts/abc-123-456/workspaces/xyz-987-654>'

Leon Kozlowski

09/12/2022, 7:59 PM

I’m not making this api call

spec:
      containers:
      - name: agent
        command: [ "prefect", "agent", "start", "-q", "dev" ]

I’m using a helm chart to deploy the agent

(btw im on verison 2.3.0)

Full traceback:

agent Starting v2.3.0 agent connected to <https://api.prefect.cloud>...
agent
agent   ___ ___ ___ ___ ___ ___ _____     _   ___ ___ _  _ _____
agent  | _ \ _ \ __| __| __/ __|_   _|   /_\ / __| __| \| |_   _|
agent  |  _/   / _|| _|| _| (__  | |    / _ \ (_ | _|| .` | | |
agent  |_| |_|_\___|_| |___\___| |_|   /_/ \_\___|___|_|\_| |_|
agent
agent
agent Agent started! Looking for work from queue(s): dev...
agent 19:51:19.313 | ERROR   | prefect.agent - Failed to create work queue 'dev'.
agent Traceback (most recent call last):
agent   File "/usr/local/lib/python3.10/site-packages/prefect/agent.py", line 88, in get_work_queues
agent     work_queue = await self.client.create_work_queue(name=name)
agent   File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 835, in create_work_queue
agent     response = await <http://self._client.post|self._client.post>("/work_queues/", json=data)
agent   File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1842, in post
agent     return await self.request(
agent   File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1527, in request
agent     return await self.send(request, auth=auth, follow_redirects=follow_redirects)
agent   File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 279, in send
agent     response.raise_for_status()
agent   File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 225, in raise_for_status
agent     raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
agent prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/work_queues/>'
agent For more information check: <https://httpstatuses.com/404>

Michael Adkins

09/12/2022, 8:04 PM

What is your

PREFECT_API_URL

set to in the agent’s container? Serina is correct that it appars to be incorrectly specified.

You can see the value you should use by running

prefect cloud login --key …

then

prefect config view

Leon Kozlowski

09/12/2022, 8:05 PM

Its

<https://api.prefect.cloud>

Matching the open source chart: https://github.com/PrefectHQ/prefect-helm/blob/f0e838aa20402fa8c23cb4fca8e6c0576d288b1d/charts/prefect-agent/values.yaml#L25

Michael Adkins

09/12/2022, 8:06 PM

Did you also set https://github.com/PrefectHQ/prefect-helm/blob/f0e838aa20402fa8c23cb4fca8e6c0576d288b1d/charts/prefect-agent/values.yaml#L26-L27 ?

Leon Kozlowski

09/12/2022, 8:06 PM

Let me try that

Michael Adkins

09/12/2022, 8:07 PM

@Serina can you look into adding documentation for this? Some inline comments there would probably be helpful in addition to a prose guide.

👍 1

Leon Kozlowski

09/12/2022, 8:08 PM

These values aren’t considered secrets are they?

Christopher Boyd

09/12/2022, 8:10 PM

your workspace / account are sensitive, but not exactly secrets

your API key is secret

Leon Kozlowski

09/12/2022, 8:10 PM

Right

Christopher Boyd

09/12/2022, 8:10 PM

generally, by default if you’re deploying with a helm chart, values.yaml should be checked for default overriding behavior, in this case, everyones API url will be different, and should be set at runtime, they will never match per person

👍 1

Leon Kozlowski

09/12/2022, 8:50 PM

I’ve made the change in my configmap, but it looks like since the deployment was unchanged the process is still restarting

Christopher Boyd

09/12/2022, 8:52 PM

Hi Leon, I would recommend tearing down the helm install, and installing fresh, supplying it with values.yaml, or even overriding it with a custom <something.yaml> and passing it in directly

I don’t think I’ve ever personally had to update the configmap

These are configured using the prefect agent manifest (not helm), but the end result is the same with the deployment

https://discourse.prefect.io/t/deploying-prefect-agents-on-kubernetes/1298

Leon Kozlowski

09/12/2022, 9:00 PM

I just restarted the deployment, it looks good now

thanks all - appreciate the help

🎉 1

View count: 3