• haf

    haf

    11 months ago
    Hi, what's a the recommended process of transitioning off Prefect cloud into something self-hosted that can run flows? E.g. is it recommended to set up the server software yourself? It's very important that we can run our flows.
    haf
    1 replies
    Copy to Clipboard
  • Gabi Pi

    Gabi Pi

    11 months ago
    Hey everyone, I am trying to run a simple flow with a
    RunNamespacedJob
    task over Kubernetes. I am using the following code:
    storage = S3(bucket="gavriel-test", stored_as_script=True)
    kubernetes_run_conf = KubernetesRun(
        env={
            "AWS_ACCESS_KEY_ID": AWS_ACCESS_KEY_ID,
            "AWS_SECRET_ACCESS_KEY": AWS_SECRET_ACCESS_KEY
        },
        labels=["prefect-poc-k8s-agent"]
    )
    body = {
        'apiVersion': 'batch/v1',
        'kind': 'Job',
        'metadata': {'name': 'echo'},
        'spec':
            {
                'template':
                    {
                        'spec': {
                            'containers': [
                                {
                                    'name': 'echo',
                                    'image': 'alpine:3.7',
                                    'command': ['sh -c  "echo Hello!!!"; sleep 10']
                                }
                            ]
                        }
                    },
                'backoffLimit': 4
            }
    }
    with Flow("kubernetes-CreateNamespacedJob", run_config=kubernetes_run_conf, storage=storage) as flow:
        job = RunNamespacedJob(body=body, namespace="prefect", delete_job_after_completion=False)
    But I keep getting the following error:
    Error during execution of task: MaxRetryError("HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /apis/batch/v1/namespaces/prefect/jobs (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8c8adeb650>: Failed to establish a new connection: [Errno 111] Connection refused'))")
    Do you have any idea how to resolve this?
    Gabi Pi
    Anna Geller
    +1
    7 replies
    Copy to Clipboard
  • Royzac

    Royzac

    11 months ago
    This issue, graphviz.backend.ExecutableNotFound: failed to execute 'dot', make sure the Graphviz executables are on your systems' PATH is occuring through my venv. It would seem to me that the executable would come prepackaged with prefect['viz'] or prefect['dev']. I'm using debian/bash. Anyone else come across this issue?
    Royzac
    a
    +1
    4 replies
    Copy to Clipboard
  • Tony Yun

    Tony Yun

    11 months ago
    hi, how can I start two tasks simultaneously? Now I see it’s running at a single thread, even two tasks have no dependencies, they cannot run at the same time. Is this even impossible?
    Tony Yun
    Kevin Kho
    +1
    4 replies
    Copy to Clipboard
  • Lukáš Polák

    Lukáš Polák

    11 months ago
    Hi everybody, did anybody using local agent notice any interesting spikes in memory usage? We use local agent and Dask executor. Our load is not that intensive at the moment - we run ~30 Flows during that spike. Regularly, we run into situation when the local agent's memory consumption spikes from ~ 150MB to 1.5-2.1GB. Has anybody had similar experience? At the moment, we're diving into debugging and profiling the issue so I'll post our findings here, if they turn out to be relevant.
    Lukáš Polák
    Anna Geller
    +2
    14 replies
    Copy to Clipboard
  • Giovanni Giacco

    Giovanni Giacco

    11 months ago
    Maybe it’s a stupid question but… How can I define the task name with the “_create_flow_run_” function?
    Giovanni Giacco
    Anna Geller
    2 replies
    Copy to Clipboard
  • r

    Richard Hughes

    10 months ago
    Good Morning - I am experiencing an outage on my end w/ self hosted agents something is not allowing my agents to pickup flows - is anyone able to help me - are ports 8080 and 4200 the firewall rules I should check - not sure where to begin
    r
    Kevin Kho
    +1
    52 replies
    Copy to Clipboard
  • p

    Philip MacMenamin

    10 months ago
    Hi, at the top of my flow I'm doing sanity checking type stuff, if something isn't correct I
    raise signals.FAIL('Didn't work')
    I do not want the flow to proceed. However, other downstream tasks occur despite the exception being raised. Is this expected behavior?
    p
    Kevin Kho
    34 replies
    Copy to Clipboard
  • j

    jcozar

    10 months ago
    Hi everybody! I have a flow that is scheduled to be run once every day. However, I would like that the executions during the weekend are moved to the next weekday (monday), which can be done with the adjustment function adjustments.next_weekday. However, I need the “real” date value (saturday or sunday) to run each flow correctly. Is there a way to use the original scheduled time before the adjustemnt? I don’t see any context variable for that. Thank you all in advance!
    j
    Kevin Kho
    10 replies
    Copy to Clipboard