https://prefect.io logo
Join Slack
Powered by
Hello everybody! I’m not quite sure how to use pre...
# prefect-getting-started
t
Hello everybody! I’m not quite sure how to use prefect, so can I ask you for advice? I want to run code on github with prefect-server. I have registered a block, registered a deployment, and in the flow log it seems that the code is being pulled, but nothing in particular is being executed. I only have the python code with the flow on github now, but do I have to put something else on github to get the code on github to run? I don’t know how to execute the code.
From local I ran the following code
Copy code
from prefect_github.repository import GitHubRepository


@flow(log_prints=True)
def git_flow(repo_name: str = "git_flow"):
    github_load = GitHubRepository.load("git_flow")

if __name__ == "__main__":
    git_flow.serve(name = "git_flow")
c
You need some execution environment where you code can run. Your existing flow is using 
.serve()
, which is creating a deployment on your behalf when it is executed on local. If you don't have as much experience with execution environments, I would recommend a serverless offering like Google Cloud Run or AWS ECS. You'll want to create a deployment and configure your code storage (GitHub) and a serverless worker/work pool. Creating a deployment Push work pools (a special way of using serverless with Prefect) AWS ECS example
t
Thank you for your reply. I would like to build on-premise due to my internet situation. I was missing an explanation, but I also ran the following and the flow is visible in the web UI.
Copy code
(py3.11) (⎈|k8s-home:prefect)
➜ workflow (main) ✗ python dqx_get_itemname.py
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Your flow 'get-itemname' is being served and polling for scheduled runs!                                      │
│                                                                                                                   │
│ To trigger a run for this flow, use the following command:                                                        │
│                                                                                                                   │
│         $ prefect deployment run 'get-itemname/github-get-itemname'                                       │
│                                                                                                                   │
│ You can also run your flow via the Prefect UI:                                                                    │
│ <http://prefect.mynet.local/deployments/deployment/4c3623b4-0efe-4553-9381-4d9ad8048218>                            │
│                                                                                                                   │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
09:06:16.327 | INFO    | prefect.flow_runs.runner - Runner 'github-get-itemname' submitting flow run '7d18311d-40a8-4bfa-a46b-b35f237ebe81'
09:06:16.530 | INFO    | prefect.flow_runs.runner - Opening process...
09:06:16.539 | INFO    | prefect.flow_runs.runner - Completed submission of flow run '7d18311d-40a8-4bfa-a46b-b35f237ebe81'
<frozen runpy>:128: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
09:06:17.803 | INFO    | Flow run 'talented-nightingale' - Downloading flow code from storage at '/Users/tig/workspace/home_server/prefect/workflow'
09:06:18.277 | INFO    | Flow run 'talented-nightingale' - Finished in state Completed()
09:06:18.550 | INFO    | prefect.flow_runs.runner - Process for flow run 'talented-nightingale' exited cleanly.
09:06:37.294 | INFO    | prefect.flow_runs.runner - Runner 'github-get-itemname' submitting flow run '231fd57a-477c-4301-90ff-d4e819378f60'
09:06:37.345 | INFO    | prefect.flow_runs.runner - Opening process...
09:06:37.355 | INFO    | prefect.flow_runs.runner - Completed submission of flow run '231fd57a-477c-4301-90ff-d4e819378f60'
<frozen runpy>:128: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
09:06:38.973 | INFO    | Flow run 'abiding-baboon' - Downloading flow code from storage at '/Users/tig/workspace/home_server/prefect/workflow'
09:06:39.602 | INFO    | Flow run 'abiding-baboon' - Finished in state Completed()
09:06:40.156 | INFO    | prefect.flow_runs.runner - Process for flow run 'abiding-baboon' exited cleanly.
09:51:13.611 | INFO    | prefect.flow_runs.runner - Runner 'github-get-itemname' submitting flow run '231fd57a-477c-4301-90ff-d4e819378f60'
09:51:13.701 | INFO    | prefect.flow_runs.runner - Opening process...
09:51:13.756 | INFO    | prefect.flow_runs.runner - Completed submission of flow run '231fd57a-477c-4301-90ff-d4e819378f60'
<frozen runpy>:128: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
09:51:15.448 | INFO    | Flow run 'abiding-baboon' - Downloading flow code from storage at '/Users/tig/workspace/home_server/prefect/workflow'
09:51:16.332 | INFO    | Flow run 'abiding-baboon' - Finished in state Completed()
09:51:16.668 | INFO    | prefect.flow_runs.runner - Process for flow run 'abiding-baboon' exited cleanly.
In addition I ran the following commands
Copy code
prefect deployment run 'get-itemname/github-get-itemname'
Sorry, the name of the flow has been changed as it contains personal information. It has a different name to the code above.
I’ve read through the prefect documentation and didn’t understand how to use github storage, so I’m sorry if I’ve misunderstood something...
I just noticed that I couldn’t pull from git. There seems to be a problem with the code. I’ll look into it a bit more. Thank you very much.
7 Views
Open in Slack
PreviousNext
Powered by