<@ULVA73B9P> I have a flow in my repository which ...
# ask-marvin
s
@Marvin I have a flow in my repository which calls flows from other repository which is in different organization. Is there a way where I can clone the other repository files real time and run it on the fly instead of moving the code to my repo?
m
thought for 66.8 seconds
Yes, there's definitely a way to accomplish what you're looking for! In Prefect 3.x, you can use
flow.from_source()
to dynamically pull and run flows from other repositories. Here's how you can do it: 1. For a public repository:
Copy code
python
from prefect import flow

# Reference the flow from the other repository
remote_flow = flow.from_source(
    source="<https://github.com/other-org/other-repo.git>",
    entrypoint="path/to/file.py:flow_name"
)

@flow
def my_flow():
    # Call the remote flow
    result = remote_flow()
    return result
2. For a private repository, you'll need to set up authentication:
Copy code
python
from prefect import flow
from prefect.filesystems import GitRepository
from prefect.blocks.system import Secret

# Set up Git repository with credentials
git_repo = GitRepository(
    url="<https://github.com/other-org/other-repo.git>",
    credentials={"access_token": Secret.load("github-access-token")}
)

remote_flow = flow.from_source(
    source=git_repo,
    entrypoint="path/to/file.py:flow_name"
)

@flow
def my_flow():
    result = remote_flow()
    return result
A few important notes: 1. The
entrypoint
parameter should point to the specific file and flow name in the format
"path/to/file.py:flow_name"
2. For private repositories, you'll need to create a GitHub access token and store it as a Prefect Secret 3. The repository will be cloned at runtime, so you don't need to manually copy any code 4. Make sure any dependencies required by the remote flow are installed in your environment You can then deploy this flow like any other:
Copy code
python
my_flow.deploy(
    name="cross-repo-flow",
    work_pool_name="your-work-pool"
)
This approach gives you the flexibility to keep your code separate while still being able to execute flows from different repositories dynamically. The repository will be cloned fresh each time the flow runs, ensuring you always have the latest code. Would you like me to provide more specific details about any part of this setup, such as creating secrets or handling dependencies?
s
@Marvin Now the other repo has files which are present in another repo but in same org. How to pull both repos, so that it doesn't have dependency affected.
m
🔧 Using:
review_common_3x_gotchas
📊 Tools used: •
display_callable_signature
(17x) •
explore_module_offerings
(34x) •
research_prefect_topic
(17x) •
review_common_3x_gotchas
(14x) •
search_prefect_3x_docs
(17x)
🔧 Using:
search_prefect_3x_docs
📊 Tools used: •
display_callable_signature
(102x) •
explore_module_offerings
(68x) •
get_latest_prefect_release_notes
(17x) •
research_prefect_topic
(17x) •
review_common_3x_gotchas
(17x) •
search_prefect_3x_docs
(83x)