Hey everyone I m getting some weird behavior between running

Question

Hey everyone I m getting some weird behavior between running a prefect flow locally and running on `DockerContainer` infra on a prefect agent So I m trying to pass an instance of an object as an argument to my first task this object happens to be a dataclass but more complex Locally it works fine but on the prefect agent the engine tries to resolve my object and create a new instance of it before passing it to the task I m on prefect 2 9 0 and python3 11 details about the error in the thread arrow down

Ouail Bendidi · Answer

``` File usr local lib python3 11 concurrent futures base py line 401 in get result raise self exception File usr local lib python3 11 site packages prefect internal concurrency calls py line 218 in run async result = await coro ^^^^^^^^^^ File usr local lib python3 11 site packages prefect engine py line 1132 in get task call return value return await future result ^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect futures py line 240 in result return await final state result raise on failure=raise on failure fetch=True ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect states py line 91 in get state result raise await get state exception state File usr local lib python3 11 site packages prefect task runners py line 207 in submit result = await call ^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect engine py line 1369 in begin task run state = await orchestrate task run ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect engine py line 1459 in orchestrate task run resolved parameters = await resolve inputs parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect engine py line 1852 in resolve inputs return visit collection ^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect utilities collections py line 331 in visit collection items = visit nested k visit nested v for k v in expr items ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect utilities collections py line 331 in lt listcomp gt items = visit nested k visit nested v for k v in expr items ^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect utilities collections py line 273 in visit nested return visit collection ^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect utilities collections py line 337 in visit collection result = typ items if return data else None ^^^^^^^^^^^^ File usr local lib python3 11 site packages kili client py line 104 in init raise AuthenticationFailed api key api endpoint ```

Ouail Bendidi · Answer

To reproduce ```from kili client import Kili from prefect import task flow task def my task kili Kili return kili users flow async def my flow kili = Kili api key= xxxx result = my task kili=kili ```

Zanie · Answer

Is that object a dataclass or pydantic model

Ouail Bendidi · Answer

Yes it s an instance of a dataclass

Zanie · Answer

`from prefect utilities annotations import quote my task kili=quote kili `

Zanie · Answer

If you quote it we will not look for Prefect futures in it

Zanie · Answer

Otherwise yeah we traverse dataclasses to resolve futures by default

Ouail Bendidi · Answer

any idea why it s not the same behavior when running the flow locally

Zanie · Answer

No that part is really weird

Zanie · Answer

Presumably some sort of detail of the error itself We re almost certainly traversing the object in the same way in both cases

Zanie · Answer

Different Prefect versions

Ouail Bendidi · Answer

nope exactly the same I use poetry to pin versions

Ouail Bendidi · Answer

using `quote` makes the `task map` fail ``` File usr local lib python3 11 concurrent futures base py line 449 in result return self get result ^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 concurrent futures base py line 401 in get result raise self exception File usr local lib python3 11 site packages prefect internal concurrency calls py line 218 in run async result = await coro ^^^^^^^^^^ File usr local lib python3 11 site packages prefect engine py line 1030 in begin task map raise MappingLengthMismatch prefect exceptions MappingLengthMismatch Received iterable parameters with different lengths Parameters for map must all be the same length Got lengths kili 1 project 10 ``` I should probably use `unmapped` Example code ```from kili client import Kili from prefect import task flow from prefect utilities annotations import quote task def my task kili Kili gt list dict return kili users def my other task kili Kili user dict print user return user id flow async def my flow kili = Kili api key= xxxx users = my task kili=quote kili my other task map kili=quote kili user=users ``` this time it fails both locally and on the agent^^

Zanie · Answer

Oh yeah I guess you need `unmapped` now

Zanie · Answer

That s a side effect of `quote` inheriting from tuple for serialization with Dask

Zanie · Answer

We should make it `unmapped` by default

Zanie · Answer

Ouail Bendidi · Answer

Cool thanks for the fix I ll be implementing a wrapper for now cause it can be a hassle to keep track of all instances of `kili` in all pipelines and add `quote` and `unmapped` everywhere because for some reason kili decided to make their client a dataclass sweat ```class KiliWrapper def init self kili Kili gt None self kili = kili def getattr self name str gt tp Any return getattr self kili name ```