https://prefect.io
Join Slack
I'm curious if there is a way to use the `prefect deploy` CLI command non-interactively? I've define...
l

Luke Orland

about 2 years ago
I'm curious if there is a way to use the
prefect deploy
CLI command non-interactively? I've defined deployments in
./prefect.yaml
and would like the deployments to be created without answering these questions interactively, e.g. in a continuous deployment setting: •
Would you like to build a custom Docker image for this deployment? [y/n] (n):
•
Would you like to save configuration for this deployment for faster deployments in the future? [y/n]:
l
n
  • 2
  • 2
  • 346
Hi! Thanks for Prefect! I was wondering when we could expect map support in Prefect 2? No worries if...
t

Tim-Oliver

over 4 years ago
Hi! Thanks for Prefect! I was wondering when we could expect map support in Prefect 2? No worries if this can't be answered. Just curious!
✅ 1
t
k
+3
  • 5
  • 67
  • 344
<@ULVA73B9P> How do I run a Prefect server behind a NGINX reverse proxy with SSL termination.
r

Richard Sundqvist

over 1 year ago
@Marvin How do I run a Prefect server behind a NGINX reverse proxy with SSL termination.
r
m
  • 2
  • 1
  • 343
Hello, I am having and issue that relates to serializing my results and setting checkpoint=False. I...
m

Matt Denno

over 5 years ago
Hello, I am having and issue that relates to serializing my results and setting checkpoint=False. I have a task that returns a Google ProtoBuff object, say like:
@task(checkpoint=False)
def fetch() -> ProtoBufObject:
    return fetch_proto_buff_object()
In the past I have set checkpoint=False to avoid serialization errors with the ProtoBuff objects, but now, even with checkpoint=False I am getting the following when i try to register the flow.
TypeError: can't pickle google.protobuf.pyext._message.MessageDescriptor objects
To resolve this I tried to create a serializer for the ProtoBuff like so:
class PBSerializer(Serializer):
    def serialize(self, value: Any) -> bytes:
        # transform a Python object into bytes
        return value.SerializeToString()

    def deserialize(self, value: bytes) -> Any:
        # recover a Python object from bytes
        ts = amanzi_pb2.TimeSeries
        return ts.ParseFromString(value)
and called like:
@task(checkpoint=False, result=LocalResult(serializer=PBSerializer()))
def fetch() -> ProtoBufObject:
    return fetch_proto_buff_object()
But I get the same error still. It doesn't seem like the result serializer is being used, but it is also not causing any errors. I have tried to debug and step through the code but can't figure out what is happening. Any help, guidance, ideas for how to resolve would be welcomed. I am on v 0.13.19
m
z
+2
  • 4
  • 12
  • 341
Hi all, when I create a connection to a db using prefect.task.sql_server and I create a global temp ...
d

Dominic Pham

over 4 years ago
Hi all, when I create a connection to a db using prefect.task.sql_server and I create a global temp table, will the temp table persist even when the flow is done? Or will I have to explicitly close the connection as a final task in the flow?
discourse 1
d
k
a
  • 3
  • 7
  • 340
<@ULVA73B9P> teach me how can i modify prefect ui as per my needs
r

Ritesh

about 1 year ago
@Marvin teach me how can i modify prefect ui as per my needs
r
m
  • 2
  • 11
  • 336
Hi! quick question, what's the recommended approach to define the KubernetesJob resources (cpu and R...
m

Miguel Moncada

over 2 years ago
Hi! quick question, what's the recommended approach to define the KubernetesJob resources (cpu and RAM requests and limits) in a flow deployment to a Kubernetes based Work Pool?
def deploy_hello_flow():
    hello_flow.deploy(
        name="hello_flow_k8s_deployment",
        work_pool_name="default-worker-pool", # k8s work pool
        image="europe-docker.pkg.dev/cambium-earth/prefect-runners-prod/default:latest",
        tags=["prod"],
        schedule=CronSchedule(cron="0 12 1 * *", timezone="UTC"),
    )
🙌 0
✅ 1
m
j
  • 2
  • 20
  • 335
<@ULVA73B9P>, show me please example of base job template for Docker work pool type, where custom en...
m

Michal

about 2 years ago
@Marvin, show me please example of base job template for Docker work pool type, where custom envvars, image and volumes are customized
m
m
  • 2
  • 12
  • 334
<@ULVA73B9P> What is the best way to add kerberos authentication to an image being used by a prefect...
b

Brennan Tolman

almost 3 years ago
@Marvin What is the best way to add kerberos authentication to an image being used by a prefect worker to run a job?
b
m
  • 2
  • 8
  • 333
In Prefect we can define tasks as functions with `@task` decorator, or as classes by subclassing `Ta...
a

Andrew Hannigan

almost 5 years ago
In Prefect we can define tasks as functions with
@task
decorator, or as classes by subclassing
Task
. In general when programming in python and presented with these options, I typically reach for the subclassing option. I find I can think about problems at a higher level with OOP, easier to build up complex components from simpler ones, cleaner abstraction, etc. However I do find it’s a bit clunky with at times when building flows. Is it recommended to avoid the subclassing approach and use a functional approach instead when possible?
a
k
  • 2
  • 5
  • 332
Previous161718Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by