https://prefect.io logo
Join Slack
Powered by
I'm working on creating a custom worker for runnin...
# prefect-integrations
o
I'm working on creating a custom worker for running services in Snowpark Container Services. I've bumped into a problem - Whenever I adjust the worker configuration to include references to blocks, the Base Job Configuration section of the work pool gets corrupted after I edit the configuration and add a block reference. Additional info in the next message..
Specifically, I've been trying this with different types of credential blocks. First SnowflakeCredentials, then AzureContainerInstanceCredentials just to make sure it's not SnowflakeCredentials specifically that are broken. As far as I can tell, my code for implementing the block configuration is identical to that of the other, Prefect-developed workers. See the attached pictures for what it looks like when: 1. The pool was created by the worker. 2. I've selected the block reference and I'm about to save the change. 3. Post-save corruption of the Base Job Configuration section. This is not just an error in the GUI display, the configuration appears to not be set correctly when running deployments on the worker.
1 - newly created work pool.png,2 - selecting credentials.png,3 - after saving.png
n
hi @Oscar Björhn - its hard to know what's going on without the implementation of the your 
BaseJobConfiguration
subclass is that code public somewhere?
o
Nope, it's all a mess at the moment while I work things out, it's my first time building a worker.. 🙂 But I could paste it in here!
n
sure! feel free to make a gist or yeah just a snippet here with the base job configuration subclass and the BaseWorker subclass
o
Sure, gimme a sec..
n
also if it’s useful here are 2 other custom worker implementations • uv-worker • volcano worker
o
Cool, I will definitely check those out too! Although I feel like the credential block reference part shouldn't be that difficult
https://gist.github.com/bjorhn/e4ff62ad0b05c2554955774d5b479b28 (gist deleted, posted a minmal example instead, further down)
Keep in mind this is a messy proof of concept, so some variables are named azure etc.. I've been wildly copy-pasting stuff from various workers while I try to get this working 😅 Everything works fine by the way, other than this! So if I can resolve this, I can clean this up, add a few more features and release it..
I basically only have snowflake_credentials: SnowflakeCredentials in two places, the basejobconfiguration and the BaseVariables.. Just like how the worker for ACI does it. It doesn't seem that complicated, which is why I'm surprised it causes the config corruption.
👀 1
If it helps, I'd be happy to upload the file source file or the whole project, instead. Or I could put together some minimal example that replicates the issue.. Maybe a bare minumum worker with a credentials reference would do it.
n
that's what I am doing now 🙂
Copy code
Maybe a bare minumum worker with a credentials reference would do it.
feel free to join me in making a minimal example though!
o
Can do!
n
I think this reproduces it (and then 
prefect worker start --type demo --pool asdf
) when i edit my work pool in the UI, and then return to edit it again, everything beneath Base Job Template disappears - that's what you were seeing? so far im not sure if this is a UI specific thing or not going to try changing up the credentials
o
That's exactly what I was seeing, yeah!
When I ran a flow after that, it complained about settings being missing, so I think it's not just a UI thing. I'm not 100% sure though.
I get the same issue with my minimal example: https://gist.github.com/bjorhn/2037a580f57b78813a7caf4419e60cfe
n
can you show what you're seeing?
When I ran a flow after that, it complained about settings being missing, so I think it's not just a UI thing.
if I do 
prefect work-pool inspect <my-pool>
I am so far seeing what I expect
o
Hmm, alright.. It's possible my code was in a bad state when I first bumped into this issue, it could be been caused by something else. Let me try with my example..
Yeah, my bad. The configuration does exist.
UI issue makes it difficult to work with though.. What do you think, am I doing something wrong or could it be a bug in Prefect/Prefect Cloud?
n
all good! there's clearly something amiss though - not seeing anything weird in the network tab going to try this on an oss server
👍 1
o
I gotta go afk for 30 mins, back soon. Let me know if there's anything I can do to help out!
👍 1
I'm curious, did the issue happen on the oss server too? Maybe you haven't had any time to look into, I realize it's Sunday. But if you did.. 🙂
n
ah sorry, yeah it did happen I think I have a hunch as to why, but will have to look a bit more later
o
Cool, thank you. Then I won't try to fix it on my own. Would it be of any help if I created an issue?
n
yes it would indeed be helpful! thank you
👍 1
o
Probably won't happen tonight (getting late here) but it'll be up tomorrow at the latest.
24 Views
Open in Slack
PreviousNext
Powered by