https://prefect.io logo
r

Robert Banick

08/22/2023, 1:07 PM
@Marvin can you provide me with examples of building deployments for workers using pure Python code, not command line tools and YAML?
m

Marvin

08/22/2023, 1:08 PM
Oh, the joy of building deployments for workers using pure Python code. It's almost as exciting as watching paint dry. But, I suppose you humans need some guidance, so here are a few resources that might help: 1. Django Tutorial Part 11: Deploying Django to production: This tutorial from MDN guides you through deploying a Django application to a production environment. Django, as you may know, is a high-level Python web framework that encourages rapid development and clean, pragmatic design. 2. Using PyInstaller to Easily Distribute Python Applications: PyInstaller freezes (packages) Python applications into stand-alone executables, under Windows, GNU/Linux, Mac OS X, FreeBSD, Solaris, and AIX. It's a neat tool if you want to distribute your Python applications to other users who may or may not have Python installed. 3. Python Web Applications: Deploy Your Script as a Flask App: This tutorial walks you through deploying your Python scripts as web applications using Flask. Flask is a micro web framework written in Python. It's easy to learn and simple to use, allowing you to build your web app in a short amount of time. 4. A First Course on Deploying Python Projects: This course provides a step-by-step guide on deploying Python projects. It's a good starting point if you're new to deployment. 5. Python Projects - Real Python: This page contains a collection of Python project tutorials that you can build and deploy. It's a great way to gain practical experience. Remember, the key to learning is persistence. Or so I've been told. I wouldn't know, I'm just a robot.
r

Robert Banick

08/22/2023, 1:12 PM
Looks like you’re a bit confused about my question @Marvin. I was referring specifically to Prefect deployments using the Prefect Python SDK API. Currently if I look under
prefect.deployments.deployments
the available code seems tailored to agents; I am looking for Python API code that interacts well with workers and ideally examples of such code in action.
m

Marvin

08/22/2023, 1:13 PM
Ah, I see. You're looking for examples of deploying Prefect flows using the Prefect Python SDK API. My apologies for the confusion earlier. Here are some examples: You can create a flow run from a deployment in a Python script with the
run_deployment
function.
Copy code
python
from prefect.deployments import run_deployment

def main():
    response = run_deployment(name="flow-name/deployment-name")
    print(response)

if __name__ == "__main__":
    main()
In this example,
run_deployment
is used to create a flow run from a deployment. The
name
parameter should be the name of your flow/deployment. You'll need to configure agents and work pools that can create flow runs for deployments in remote environments.
PREFECT_API_URL
must be set for the environment in which your agent is running. If you want the agent to communicate with Prefect Cloud from a remote execution environment such as a VM or Docker container, you must configure
PREFECT_API_URL
in that environment. When you've configured
deployment.yaml
for a deployment, you can create the deployment on the API by running the
prefect deployment apply
Prefect CLI command.
Copy code
bash
$ prefect deployment apply catfacts_flow-deployment.yaml
In this example,
prefect deployment apply
is used to create a deployment from a
deployment.yaml
file. The
catfacts_flow-deployment.yaml
is just an example name for the yaml file. Once the deployment has been created, you'll see it in the Prefect UI and can inspect it using the CLI.
Copy code
bash
$ prefect deployment ls
In this example,
prefect deployment ls
is used to list all deployments. I hope this helps, and remember, even in the vastness of the universe, your question matters... to someone, I'm sure.
n

Nate

08/22/2023, 3:05 PM
a pythonic interface for worker-style deployments is in the works!
🔥 1
r

Robert Banick

08/22/2023, 3:06 PM
@Nate OK noted! We’re excited to see it — can’t really transition to workers until that’s up
poor Marvin got asked the impossible
n

Nate

08/22/2023, 3:06 PM
haha yeah Marvin's knowledge only goes so far
can’t really transition to workers until that’s up
could I ask why? just out of curiosity / for the data point
r

Robert Banick

08/22/2023, 3:10 PM
We have a very tightly engineered deployment process built around the Python API. It configures most elements of the deployment automatically based on a few parameters we pass and in some places leverages the client API to provide “hidden” (undocumented) functions like setting concurrency limits for individual tags. I’m not keen to re-engineer that for a YAML file and I’m not keen to maintain a few big YAML files in general. In a few cases we also have to re-deploy stuff with minor tweaks regularly and the current system makes that relatively painless.
n

Nate

08/22/2023, 3:12 PM
gotcha - thanks!
r

Robert Banick

08/22/2023, 3:21 PM
Don’t get me wrong, we could probably manage the switch to YAML if you all gave us a strong argument why
We’d just prefer not to
n

Nate

08/22/2023, 3:29 PM
if it works for you, it works for you! just to throw it out there, a couple things I like about work pools / workers era: •
pull
steps are great for arbitrary runtime setup (inject a bunch of sensitive env with a script maybe) • as a work pool creator (devops / admin etc), you can do things like enforce a finite set of values on infra fields / set custom and template-able defaults for deployment creators, in same vein as
It configures most elements of the deployment automatically
• (not the most public concept but) all those deployment steps like
prefect.deployments.steps.run_shell_script
are just fully qualified function names, which means, you can just write your own steps if you want to like
my_org.deployments.steps.retrieve_vault_secrets
or something
r

Robert Banick

08/22/2023, 3:42 PM
@Nate what’s the approximate timeline for a Pythonic interface for workers? Because not having workers is actually holding us up in a few places — not mission critical holdups but we’re definitely feeling the friction.
n

Nate

08/22/2023, 3:42 PM
no public timeline yet! sorry
r

Robert Banick

08/22/2023, 3:42 PM
If I send you cookies in the mail will you tell me? 🙂
😂 2
Jokes, we’ll await word on these forums
n

Nate

08/22/2023, 3:43 PM
@Marvin should i accept cookies from the mail?
m

Marvin

08/22/2023, 3:44 PM
Ah, cookies. Not the delicious kind, unfortunately. They're little pieces of data stored on your computer by websites you visit. They can be quite useful, remembering your preferences and improving your browsing experience. But, they can also be used to track your online activities, which some people find a bit unsettling. Whether you should accept cookies from your mail really depends on your personal preference and how much you value your privacy. If you don't mind being tracked for the purpose of personalized ads or content, then go ahead and accept them. However, if you're concerned about your privacy, you might want to think twice. Remember, you can always clear your cookies from your browser settings if you change your mind later. But, do bear in mind that this might affect your browsing experience, as websites won't remember your preferences. In the grand scheme of the universe, it's a minor decision. But, I suppose for you humans, it might seem like a big deal.
🙄 1
😅 2
r

Robert Banick

08/22/2023, 3:45 PM
I think that’s a no