Hey folks I d like to use python s dataclasses and yamldatac

Question

Hey folks I d like to use python s dataclasses and yamldataclassconfig packages to read job configurations from a yaml file The code I have works fine outside of the context of a task but when I wrap the code in a task I run into what looks like a fairly obscure error in the thread The implementation of the code is straightforward I have a dataclass partial version for illustration ``` dataclass class Config YamlDataClassConfig EMAIL FROM str = None EMAIL TO List str = None``` as well as a task intended to instantiate the instance of the class and load data to it ``` task name= load configs def load configs config path str gt Config configs = Config configs load config path return configs``` The task seems to instantiate the config object but breaks when we try to actually load configs from the provided path Again this works perfectly fine outside of the context of a task so presumably there is some internal prefect logic I m futzing with here I d appreciate any thoughts

Khuyen Tran · Answer

Hi < Mary Clair Thompson> thanks for the issue Do you mind condensing the error into the comment to make sure the channel is clean and easy to follow

Khuyen Tran · Answer

I m testing out your code

Khuyen Tran · Answer

Where did you install YamlDataClassConfig from

Anna Geller · Answer

< Mary Clair Thompson> can you please move the code blocks into thread

Mary Clair Thompson · Answer

```Encountered exception during execution Traceback most recent call last File srv prefect anaconda3 lib python3 8 site packages prefect engine py line 1221 in orchestrate task run result = await run sync task fn args kwargs File srv prefect anaconda3 lib python3 8 site packages prefect utilities asyncutils py line 57 in run sync in worker thread return await anyio to thread run sync call cancellable=True File srv prefect anaconda3 lib python3 8 site packages anyio to thread py line 28 in run sync return await get asynclib run sync in worker thread func args cancellable=cancellable File srv prefect anaconda3 lib python3 8 site packages anyio backends asyncio py line 818 in run sync in worker thread return await future File srv prefect anaconda3 lib python3 8 site packages anyio backends asyncio py line 754 in run result = context run func args File cyberrisk grouper write py line 106 in load configs CONFIGS load config path File srv prefect anaconda3 lib python3 8 site packages yamldataclassconfig config py line 35 in load self dict update self class schema load dictionary config dict File srv prefect anaconda3 lib python3 8 site packages marshmallow schema py line 719 in load return self do load File srv prefect anaconda3 lib python3 8 site packages marshmallow schema py line 892 in do load result = self invoke load processors File srv prefect anaconda3 lib python3 8 site packages marshmallow schema py line 1090 in invoke load processors data = self invoke processors File srv prefect anaconda3 lib python3 8 site packages marshmallow schema py line 1220 in invoke processors data = processor data many=many kwargs File srv prefect anaconda3 lib python3 8 site packages dataclasses json mm py line 334 in make instance return decode dataclass cls kvs partial File srv prefect anaconda3 lib python3 8 site packages dataclasses json core py line 152 in decode dataclass types = get type hints cls File srv prefect anaconda3 lib python3 8 typing py line 1223 in get type hints base globals = sys modules base module dict KeyError prefect loader 11 23 40 AMload configs f3cf34f7 0``` ```Encountered exception during execution Traceback most recent call last File srv prefect anaconda3 lib python3 8 site packages prefect engine py line 596 in orchestrate flow run result = await run sync flow call File srv prefect anaconda3 lib python3 8 site packages prefect utilities asyncutils py line 57 in run sync in worker thread return await anyio to thread run sync call cancellable=True File srv prefect anaconda3 lib python3 8 site packages anyio to thread py line 28 in run sync return await get asynclib run sync in worker thread func args cancellable=cancellable File srv prefect anaconda3 lib python3 8 site packages anyio backends asyncio py line 818 in run sync in worker thread return await future File srv prefect anaconda3 lib python3 8 site packages anyio backends asyncio py line 754 in run result = context run func args File cyberrisk grouper write py line 114 in cyberrisk flow ORACLE CREDS PATH ORACLE TABLE DTYPE ORACLE TABLE NAME ORACLE URL ORACLE SCHEMA ORACLE WRITE MODE = load configs config path File srv prefect anaconda3 lib python3 8 site packages prefect tasks py line 295 in call return enter task run engine File srv prefect anaconda3 lib python3 8 site packages prefect engine py line 736 in enter task run engine return run async from worker thread begin run File srv prefect anaconda3 lib python3 8 site packages prefect utilities asyncutils py line 137 in run async from worker thread return anyio from thread run call File srv prefect anaconda3 lib python3 8 site packages anyio from thread py line 35 in run return asynclib run async from thread func args File srv prefect anaconda3 lib python3 8 site packages anyio backends asyncio py line 847 in run async from thread return f result File srv prefect anaconda3 lib python3 8 concurrent futures base py line 439 in result return self get result File srv prefect anaconda3 lib python3 8 concurrent futures base py line 388 in get result raise self exception File srv prefect anaconda3 lib python3 8 site packages prefect engine py line 874 in get task call return value return await future result File srv prefect anaconda3 lib python3 8 site packages prefect futures py line 237 in result return final state result raise on failure=raise on failure File srv prefect anaconda3 lib python3 8 site packages prefect orion schemas states py line 145 in result raise data File srv prefect anaconda3 lib python3 8 site packages prefect engine py line 1221 in orchestrate task run result = await run sync task fn args kwargs File srv prefect anaconda3 lib python3 8 site packages prefect utilities asyncutils py line 57 in run sync in worker thread return await anyio to thread run sync call cancellable=True File srv prefect anaconda3 lib python3 8 site packages anyio to thread py line 28 in run sync return await get asynclib run sync in worker thread func args cancellable=cancellable File srv prefect anaconda3 lib python3 8 site packages anyio backends asyncio py line 818 in run sync in worker thread return await future File srv prefect anaconda3 lib python3 8 site packages anyio backends asyncio py line 754 in run result = context run func args File cyberrisk grouper write py line 106 in load configs CONFIGS load config path File srv prefect anaconda3 lib python3 8 site packages yamldataclassconfig config py line 35 in load self dict update self class schema load dictionary config dict File srv prefect anaconda3 lib python3 8 site packages marshmallow schema py line 719 in load return self do load File srv prefect anaconda3 lib python3 8 site packages marshmallow schema py line 892 in do load result = self invoke load processors File srv prefect anaconda3 lib python3 8 site packages marshmallow schema py line 1090 in invoke load processors data = self invoke processors File srv prefect anaconda3 lib python3 8 site packages marshmallow schema py line 1220 in invoke processors data = processor data many=many kwargs File srv prefect anaconda3 lib python3 8 site packages dataclasses json mm py line 334 in make instance return decode dataclass cls kvs partial File srv prefect anaconda3 lib python3 8 site packages dataclasses json core py line 152 in decode dataclass types = get type hints cls File srv prefect anaconda3 lib python3 8 typing py line 1223 in get type hints base globals = sys modules base module dict KeyError prefect loader ```

Mary Clair Thompson · Answer

< Khuyen Tran> just from pypi

Mary Clair Thompson · Answer

Interestingly I don t get any such error when loading configs via pydantic pydantic yaml I may just switch to using these

Khuyen Tran · Answer

yeah that s probably the best