How to use any global object variable parameter in task defi

Question

How to use any global object variable parameter in task definitions For example my flow fails when I declare the client in the global scope in the below code ` usr bin python3` from prefect import task Flow import boto3 import json client = boto3 client sagemaker region name= us east 1 task def list training jobs response = client list training jobs print response with Flow SageMakerFlow as flow training jobs = list training jobs print training jobs flow register project name= tutorial flow run

fblaze f · Answer

Error ` lt Task list training jobs gt ` Traceback most recent call last File first project py line 21 in lt module gt flow register project name= tutorial File usr local lib python3 7 site packages prefect core flow py line 1669 in register idempotency key=idempotency key File usr local lib python3 7 site packages prefect client client py line 782 in register serialized flow = flow serialize build=build type Any File usr local lib python3 7 site packages prefect core flow py line 1440 in serialize self storage add flow self File usr local lib python3 7 site packages prefect storage local py line 143 in add flow flow location = flow save flow location File usr local lib python3 7 site packages prefect core flow py line 1533 in save cloudpickle dump self f File usr local lib python3 7 site packages cloudpickle cloudpickle fast py line 88 in dump CloudPickler file protocol=protocol dump obj File usr local lib python3 7 site packages cloudpickle cloudpickle fast py line 563 in dump return Pickler dump self obj File usr lib64 python3 7 pickle py line 437 in dump self save obj File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 914 in save set save item File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr local lib python3 7 site packages cloudpickle cloudpickle fast py line 745 in save function self dynamic function reduce obj obj=obj File usr local lib python3 7 site packages cloudpickle cloudpickle fast py line 687 in save reduce pickle5 save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 774 in save tuple save element File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 890 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 890 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 634 in save reduce save args File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 774 in save tuple save element File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 819 in save list self batch appends obj File usr lib64 python3 7 pickle py line 846 in batch appends save tmp 0 File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 638 in save reduce save args File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 774 in save tuple save element File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 638 in save reduce save args File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 774 in save tuple save element File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 549 in save self save reduce obj=obj rv File usr lib64 python3 7 pickle py line 662 in save reduce save state File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 504 in save f self obj Call unbound method with explicit self File usr lib64 python3 7 pickle py line 859 in save dict self batch setitems obj items File usr lib64 python3 7 pickle py line 885 in batch setitems save v File usr lib64 python3 7 pickle py line 524 in save rv = reduce self proto TypeError can t pickle SSLContext objects

Spencer · Answer

It s trying to pickle the boto client which doesn t work Instantiate the boto client inside the task