Chris Bowen
07/23/2021, 6:18 PM.md
files in a repo).
@Zach Angell provided me with the GraphQL to start out and I expanded on it to make a solution that works for me. I'm storing mine in a few functions, but I'll just provide the entire thing as a script here.
This script assumes a few things (I'll edit/update if I think of any more or anyone points any out):
1. This script is running in a directory that contains all of your .md
files that you want to deploy
2. Your .md
files are named the same thing as your flows (see the active_flows
object returned from the GraphQL query)
3. Any \
characters in your .md
files need to be escaped with a second \
or the deployment won't work
4. I believe including any `"`characters in the .md
files also blew up the API call, but '
was fine
CODE IN THREAD
I've got this divided up into three functions, but posting it in this format seemed like it might be easier to read.
One thing I did notice is that the Prefect UI doesn't seem to handle code blocks very elegantly. I tried using the standard three ticks (`), tildas (~), with/without specifying the code base, and using four spaces.
I attached a couple pictures that show the input and output. Not a huge deal, but curious why it doesn't "block" like most other platforms I've seen.
Anyways, happy to hear any feedback, suggestions, etc. Also just happy to share it if it helps anyone else at all!Kevin Kho
07/23/2021, 6:20 PMMarvin
07/23/2021, 6:21 PMKevin Kho
07/23/2021, 6:21 PMChris Bowen
07/23/2021, 6:22 PMKevin Kho
07/23/2021, 6:25 PMChris Bowen
07/23/2021, 6:26 PMimport prefect
import glob
import os
api_server = 'localhost:4200' #or set to whatever your GraphQL address/port is
client = prefect.Client(api_server=api_server)
#Pull all flows that are not archived, to get their flow_group_id
active_flows = client.graphql(
{
'query': {
'flow(where:{archived:{ _lte: false }})': {
'id'
,'name'
,'flow_group_id'
,'archived'
}
}
}
)
#Iterate through active_flows and search folder for markdown files with the same name, write them to the flow_group using the flow_group_id
for i in active_flows['data']['flow']:
flow_group_id = i['flow_group_id'] #id used for API call
for j in glob.glob(r'.\{markdown_file}*'.format(markdown_file=i['name'])): #there's probably a better way to access this than with this for loop
with open(j) as file:
markdown = file.read()
markdown = markdown.replace("\n"," \\n ") #the \n blows up the API call to write this, I found adding the escape \ fixed the problem
client.graphql(
{
'mutation': {
'''set_flow_group_description(input:{flow_group_id: "%s",description:"%s"})'''%(flow_group_id,markdown): {
'success'
}
}
}
)
print(j + ' written to Prefect server')
#I also wanted a way to check if there's any flow_groups missing documentation
#I post the results from this to a Google Chat room via a bot and webhook
missing_documentation = client.graphql(
{
'query': {
'flow_group(where:{description:{_is_null:true}})': {
'id'
,'name'
,'description'
}
}
}
)
#make a list of all flow_group ids that came back with description=null
#could probably be tidied up with list comprehension
missing_id_ls = []
for i in missing_documentation['data']['flow_group']:
missing_id_ls.append(i['id'])
#match up the ids with the names so its human-readable
missing_name_ls = []
for i in active_flows['data']['flow']:
if i['flow_group_id'] in missing_id_ls:
missing_name_ls.append(i['name'])
if len(missing_name_ls) > 0:
message = 'The following flow groups do not have documentation deployed: '+str(missing_name_ls)
print(message)
#my_external_package.post_to_gchat(webhook_url='gchat_webhook', message=message)
else:
print('No flows missing documentation!')
Kevin Kho
07/23/2021, 6:29 PMZach Angell
07/23/2021, 6:32 PM