Hello I m thinking about changing my company s orchestrator Prefect Community #prefect-getting-started

Hello, I'm thinking about changing my company's or...

Matheus

08/01/2024, 9:14 PM

Hello, I'm thinking about changing my company's orchestrator to Prefect and I'm having difficulties. This is the scenario: • Self hosted on-premises kubernetes cluster. • I need flows with a frequency of at least 3 seconds: 1. Read a kafka topic 2. Call a api 3. Send back to kafka • Self hosted gitea repository When I try to run the code: >

from prefect import flow

> > # Source for the code to deploy (here, a GitHub repo)
>

SOURCE_REPO="<http://localhost:57095/aignosi/demos.git>"

> >

if __name__ == "__main__":

flow.from_source(

source=SOURCE_REPO,

entrypoint="my_gh_workflow.py:repo_info", # _Specific flow to run_

).deploy(

name="my-first-deployment",

work_pool_name="worker-test", # _Work pool target_

cron="0 1 * * *", # _Cron schedule (1am every day)_

the worker broke with this error: >

prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect.deployments.steps.git_clone

When I try to run the code with SOURCE_REPO="http://gitea-http.gitea.svc.cluster.local:3000/aignosi/demos.git" i got this in my local: >

RuntimeError: Failed to clone repository '<http://gitea-http.gitea.svc.cluster.local:3000/aignosi/demos.git>' with exit code 128.

Question 1: Is prefect the right tool for me? Question2: How do i fiz this?

12 Views

Open in Slack

Previous Next