• Talmaj Marinč

    Talmaj Marinč

    7 months ago
    Hi, I’m running some flows on prefect cloud and tasks are running sequentially instead of in parallel. The reason I decided to use prefect is to easily parallelize task execution. I am using
    task.map
    . What am I doing wrong?
    Talmaj Marinč
    Kevin Kho
    7 replies
    Copy to Clipboard
  • Dev Dasgupta

    Dev Dasgupta

    7 months ago
    Hi All, I'm new to prefect and after making a few refactoring to my existing flow.py. Initially, the folder structure looked the following and it was getting registered successfully (
    prefect register --project my_project -p bar
    )
    foo_root_folder
    ├── README.md
    ├── bar
    │   └── flow.py
    ├── pyproject.toml
    ├── requirements.txt
    ├── setup.cfg
    Since the file
    flow.py
    was cluttered with all utils functions required for the flow execution I refactored it into a separate python package and made calls to those functions in
    flow.py
    from bar.utils import get_some_useful_function
    The resultant folder looked like the following:
    foo_root_folder
    ├── README.md
    ├── bar
    │   ├── __init__.py
    │   ├── utils.py
    │   └── flow.py
    ├── pyproject.toml
    ├── requirements.txt
    ├── setup.cfg
    When I try to register now I am getting error as
    ModuleNotFoundError: No module named 'bar'
    Can anyone please let me know if anyone faced similar issues before and how they resolved it? Thanks
    Dev Dasgupta
    Kevin Kho
    2 replies
    Copy to Clipboard
  • Andrew Lawlor

    Andrew Lawlor

    7 months ago
    im trying out prefect for our etl pipelines. we use GCP as our cloud provider. Id like to use prefect cloud and run agents in GCP. is the best option for doing this the vertex agent? If I do use that, what is the process for getting started? do i just create the agents at the beginning via cli and then leave them running, or do i have to continuously update them? is there a way to set up ci/cd for my flow code?
    Andrew Lawlor
    Kevin Kho
    +1
    108 replies
    Copy to Clipboard
  • Keith Veleba

    Keith Veleba

    7 months ago
    Does anyone know how to run an AWS batch job and wait for completion?
    Keith Veleba
    Kevin Kho
    13 replies
    Copy to Clipboard
  • David Yang

    David Yang

    7 months ago
    Hi All, anyone can get flows from Azure DevOps service successfully using git storage? I got this error "Failed to load and execute Flow's environment: HTTPUnauthorized('No valid credentials provided')". It seems that git storage doesn't support private access token to access azure devops? This is my code: flow.storage = Git(     repo="xxxxxxx/prefect/_git/prefect",                            

    name of repo

        flow_path ="./flows/azuredevops.py",                    

    location of flow file in repo

        branch_name ="main",     repo_host="xxxxx@dev.azure.com",                    

    repo host name

        git_token_secret_name ="azuresecret" )
    David Yang
    Kevin Kho
    +1
    14 replies
    Copy to Clipboard
  • Keith Veleba

    Keith Veleba

    7 months ago
    when you put AWS_CREDENTIALS in via the UI as a Secret, do you use the text or the json options, and do you put it in as a JSON snippet?
    Keith Veleba
    Anna Geller
    +1
    22 replies
    Copy to Clipboard
  • Chris Hemphill

    Chris Hemphill

    7 months ago
    Hey everyone! Chris Hemphill from Actium Health
    Chris Hemphill
    Kevin Kho
    +2
    4 replies
    Copy to Clipboard
  • Dave

    Dave

    7 months ago
    Hi team, our team has recently deployed prefect on gcp kubernete. and it keeps saying
    Couldn't connect to *Prefect Server* at *<http://34.82.119.23:4200/graphql>
    ,* can anyone help?
    Dave
    Kevin Kho
    +3
    106 replies
    Copy to Clipboard
  • Leon Kozlowski

    Leon Kozlowski

    7 months ago
    What will happen if I deploy a new version of a flow while a previous version is in a running state?
    Leon Kozlowski
    Anna Geller
    6 replies
    Copy to Clipboard
  • Daniel Kornhauser

    Daniel Kornhauser

    7 months ago
    Can anybody point to a relatively large project that uses prefect in github to checkout their directory tree structure ? Wondering how they group and name the python files of their flows and tasks when they have a lot of them.
    Daniel Kornhauser
    Kevin Kho
    5 replies
    Copy to Clipboard