how can two projects share code? I have some

utils

I want to be common to all projects, but i want to develop and deploy

projA

and

projB

separately. I'm using the s3 recipe

Why is this discourse post private: https://discourse.prefect.io/t/how-to-deploy-a-prefect-2-0-agent-to-azure-vm-as-your-execution-layer/1097 ?

Hi All. I have an issue with failed flows not deleting their pods in kubernetes. If I run a flow, and then start another run of the same flow, the second (or any subsequent) run will fail with kubernetes throwing a 409, but the pod it was running on doesn’t get deleted and remains in running state - any suggestions how to get around this?

Hey community! I am seeing weird connection issues on Prefect. We are running relatively large flows (~2000 tasks), all of which are

CreateNameSpacedJob

tasks, spawning jobs on our kubernetes cluster. We are hosting our own Prefect server using the docker-compose command directly on a moderately large VM (Standard E8ds v5 VM type from Azure). It seems as if the flow-pod, the pod responsible for orchestrating the tasks, is losing connection to the backend, specifically the

apollo

service as can be seen in the first screenshot. All of a sudden, all

CreateNameSpacedJob

would fail at the same time when the

CloudTaskRunner

went to update the state of the task. I did a bit of digging with

netstat

and it seems that there are quite a bit of TCP connections being created in the

apollo

container, however, I am not entirely sure if that is "business as usual" or a bit on the heavy side for this kind of setup. Have anyone else experience these kinds of hiccups or are using a similar setup that might have ideas? I dont know whether the second screenshot is of relevance but it has started to pop up quite a lot and I cant seem to figure out whats causing it.

trying to trigger a flow after another flow ends following instructions in https://docs.prefect.io/core/idioms/flow-to-flow.html when registering the flows A, B, C and D, in the example, I assume we register them without a schedule, is this correct ? I also do not see the in the python example code the

flow.register(project_name='example')

we do need to run

flow.register

, right ?

Is it possible in Prefect to invoke R script with-in a task. I have a code in R script and that can only run in the R-studio, I don't want to convert it into python. How can I invoke that R code with prefect. Is there an interface between R and Prefect

Does anyone know how to specify the prefect extras I need in the Docker storage? I am getting the following build error;

System Version check: OK
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/prefect/tasks/gcp/__init__.py", line 14, in <module>
    from prefect.tasks.gcp.bigquery import (
  File "/usr/local/lib/python3.7/site-packages/prefect/tasks/gcp/bigquery.py", line 4, in <module>
    from google.cloud import bigquery
ImportError: cannot import name 'bigquery' from 'google.cloud' (unknown location)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/prefect/healthcheck.py", line 151, in <module>
    flows = cloudpickle_deserialization_check(flow_file_paths)
  File "/opt/prefect/healthcheck.py", line 44, in cloudpickle_deserialization_check
    flows.append(cloudpickle.loads(flow_bytes))
  File "/usr/local/lib/python3.7/site-packages/prefect/tasks/gcp/__init__.py", line 24, in <module>
    ) from err
ImportError: Using `prefect.tasks.gcp` requires Prefect to be installed with the "gcp" extra.

I tried;

storage = Docker(
    registry_url="<http://gcr.io/blah/|gcr.io/blah/>",
    image_name=name,
    image_tag="latest",
    build_kwargs={"buildargs":{"EXTRAS":"kubernetes,gcp"}}
)

With no luck 😞

"Can prefect execute a flow in a single thread? See comments for more details and context, I’m trying to get Prefect to run Playwright, but it is single-threaded only." Hi Can Playwright be combined with Prefect?

[tool.poetry.dependencies]
python = "^3.10"
prefect = "^2.6.1"
playwright = "^1.27.1"

import prefect
from playwright.sync_api import Page, sync_playwright

@prefect.task
def go_to_a_web(page: Page):
    page.goto("<https://formulae.brew.sh/cask/slack>")


@prefect.flow
def main_flow():
    with sync_playwright() as pw:
        browser = pw.chromium.launch(headless=False)
        page = browser.new_page()
        go_to_a_web(page)


main_flow()

I got

greenlet.error: cannot switch to a different thread

any help would be much appreciated

@Marvin Process 221986 exited with status code: 1 flow crashing

@Marvin how can i limit the number of concurrent mapped tasks running