https://prefect.io
Join Slack
I'm struggling to implement <the GCP_CREDENTIALS default secret feature> in a local Prefect flow. - ...
t

Taylor Harless

over 3 years ago
I'm struggling to implement the GCP_CREDENTIALS default secret feature in a local Prefect flow. • I've set an environmental variable PREFECT__CONTEXT__SECRETS_GCP_CREDENTIALS pointing to a full local path for the GCP credentials .json downloaded from the GCP project to which I'd like to upload some files. • I've run the following:
from prefect import Flow
from prefect.tasks.gcp.storage import GCSUpload

with Flow("google-cloud-test") as flow:

    GCSUpload(bucket="test-upload", create_bucket=True)(
        data="test-file.csv", credentials="GCP_CREDENTIALS"
    )

flow.run()
and received an error:
AttributeError: 'str' object has no attribute 'keys'
. I've incorporated the feedback from a similar error discussion recently, but can't figure out what the issue is. Any help is much appreciated.
t
k
a
  • 3
  • 10
  • 69
Hi there, I have this task with a trigger: ``` @task(trigger=any_failed) def alert_email(email, m...
p

Patrick Alves

over 3 years ago
Hi there, I have this task with a trigger:
@task(trigger=any_failed)
def alert_email(email, message):
    sendmail(email, '[Prefect] ADCheck - [FAILED]', message)

with Flow("AD Check") as flow:

    # Parameters
    admin_email = Parameter("admin_email", default="<mailto:patrick@xyz.br|patrick@xyz.br>")

    # Tasks
    message = "some message"

    try:
        ad_conn = connect_active_directory(ad_server, ad_user, ad_pass)

    except Exception as e:
        message = f'Erro ao executar workflow: {e}'

    # If all tasks are successful
    notify_email(admin_email, message)

    # If any task fails
    alert_email(admin_email, message)
When the task fails, I would like to get the Exception error and save it o message variable, but I am getting
Trigger was "all_successful" but some of the upstream tasks failed
@Anna Geller, any tips? I've tried to use the state_handlers, but I need to be able to change parameters of the alert function for each workflow (email, subject, etc.) So triggers seems better. But I could not get the error.
p
a
  • 2
  • 8
  • 69
Hello everyone, what could be the problem? ```Failed to load and execute Flow's environment: Storage...
s

Sergey

almost 4 years ago
Hello everyone, what could be the problem?
Failed to load and execute Flow's environment: StorageError('An error occurred while unpickling the flow:\n AttributeError("Can\'t get attribute \'_unpickle_timestamp\' on <module \'pandas._libs.tslibs.timestamps\' from \'/usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/timestamps.cpython-37m-x86_64-linux-gnu.so\'>")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - prefect: (flow built with \'0.15.3\', currently running with \'0.14.19\')\n - python: (flow built with \'3.7.3\', currently running with \'3.7.4\')')
Is it that I have a version older than on the server?
s
a
  • 2
  • 17
  • 69
<@ULVA73B9P> How do I set logging level to 'debug' within a prefect flow?
m

merlin

6 months ago
@Marvin How do I set logging level to 'debug' within a prefect flow?
m
m
  • 2
  • 3
  • 68
<@ULVA73B9P> Does prefect 3 OSS has authentication?
a

Abhishek Singh

12 months ago
@Marvin Does prefect 3 OSS has authentication?
a
m
  • 2
  • 1
  • 68
Hello ! I'm working on implementing ControlFlow using Docker Compose and have encountered an issue....
a

Ahronn

about 1 year ago
Hello ! I'm working on implementing ControlFlow using Docker Compose and have encountered an issue. I've set up a basic configuration, but I'm running into a connection timeout error when trying to start the Prefect API server. Error message: RuntimeError: Timed out while attempting to connect to ephemeral Prefect API server This error occurs in the following file: /usr/local/lib/python3.12/site-packages/prefect/client/orchestration.py, line 218, in get_client My current docker-compose.yml:
services:
  cf:
    build: .
    environment:
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
      - CONTROLFLOW_DEFAULT_LLM=${CONTROLFLOW_DEFAULT_LLM}
    develop:
     watch:
       - action: sync
         path: ./test.py
         target: /app/test.py
       - action: rebuild
         path: ./test.py
    stdin_open: true
    tty: true
My requirements.txt only includes controlflow and langchain_anthropic. The Python code I'm trying to run:
import controlflow as cf
task = cf.Task(
    "Write a short poem about nature",
    result_type=str,
    instructions="Feel free to choose any poetic form that fits the theme."
)
task.run()
Could you please advise on what might be causing this timeout and how to resolve it? I think I'm totally missing some points... (I'm not familiar at all with Prefect and wanted to test out Controlflow, it seems really incredible !) Thank you for your help!
a
n
d
  • 3
  • 16
  • 68
<@ULVA73B9P> When I cancel a flows that creates subflows using run_deployment, The flows in Late, Sc...
m

Mitch

over 1 year ago
@Marvin When I cancel a flows that creates subflows using run_deployment, The flows in Late, Scheduled, Pending and Running statuses continue to still run. How do I get all of my subflows to cancel. It is very expensive to have 100's of GPU nodes continually running jobs that should be cancelled or having to cancel them all manually
m
m
n
  • 3
  • 7
  • 68
<@ULVA73B9P> why prefect worker is saying None in this case: "Downloading flow code from storage at ...
r

riccardo bernardi

over 1 year ago
@Marvin why prefect worker is saying None in this case: "Downloading flow code from storage at None"
r
m
  • 2
  • 3
  • 68
I upgraded from `2.8.5` to `latest` but now the prefect server won't start and I don't see any helpf...
r

Richard Alexander

over 1 year ago
I upgraded from
2.8.5
to
latest
but now the prefect server won't start and I don't see any helpful errors, so I'm not sure where to start troubleshooting. We have a Postgres database, and some of the errors look related to database upgrades. Any tips on how to get our server back up are much appreciated! The error stack is too long to post, so I'll add it in chunks to this thread...
✅ 1
r
j
  • 2
  • 11
  • 68
Hi all, I'm running into an issue when trying to run Prefect via Kubernetes. I have my job template...
t

Tom

over 1 year ago
Hi all, I'm running into an issue when trying to run Prefect via Kubernetes. I have my job template setup so that it mounts a NFS drive (this works correctly). But when I change the securityContext to the UID of the user that will access the NFS mount, then I get this error with the job:
UserWarning: Failed to create the Prefect home directory at /.prefect
I've tried changing the environment variable to
{"PREFECT_HOME": "/tmp"}
, which does stop this issue, but then my job has issues with cloning the repo. Does anyone know how I can correctly setup a job so that it has a NFS mount and the correct userID working?
t
  • 1
  • 1
  • 68
Previous949596Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by