Hello all I would like to ask if you could help me with a pr

Question

Hello all I would like to ask if you could help me with a problem I m having in creating a deployment with an SFTP server as remote storage I m using Prefect server version 2 6 4 on Windows 10 Specifically I locally created an example project with the following structure ```C USERS TESTBENUTZER PREFECT TEST PROJECT idea some files prefect flows prefect flows py prefect tasks prefect tasks py``` In addition I created a remote storage block via the UI named `sftp server` and the base path `<sftp ACTUAL SFTP URL prefect flows api flow>` After that I used the following command to create a deployment `PS C Users Testbenutzer prefect test project gt prefect deployment build C Users Testbenutzer prefect test project prefect flows prefect flows py api flow n ftp test q ftp test sb remote file system sftp server ` However only the folder `prefect flows` with the file `prefect flows py` is uploaded to the `api flow` directory of the SFTP server not the folder `prefect tasks` Also Prefect gives the following error ```File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages fsspec callbacks py line 65 in relative update self value += inc TypeError unsupported operand type s for += int and NoneType An exception occurred ``` I would write the full traceback into the thread as well as the code of the two Python files and a screenshot of the storage block definition without the actual SFTP URL and login credentials If anyone had any idea what I could be doing wrong here that would be a great help Many thanks in advance

Jens Freund · Answer

Traceback ```PS C Users Testbenutzer prefect test project gt prefect deployment build C Users Testbenutzer prefect test project prefect flows prefect flows py api flow n ftp test q ftp test sb remote file system sftp server Found flow api flow Default prefectignore file written to C Users Testbenutzer prefect test project prefectignore Traceback most recent call last File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages prefect cli utilities py line 41 in wrapper return fn args kwargs File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages prefect utilities asyncutils py line 201 in coroutine wrapper return run async in new loop async fn args kwargs File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages prefect utilities asyncutils py line 152 in run async in new loop return anyio run partial fn args kwargs File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages anyio core eventloop py line 70 in run return asynclib run func args backend options File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages anyio backends asyncio py line 292 in run return native run wrapper debug=debug File C Users Testbenutzer anaconda3 envs prefect test project v2 lib asyncio runners py line 44 in run return loop run until complete main File C Users Testbenutzer anaconda3 envs prefect test project v2 lib asyncio base events py line 646 in run until complete return future result File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages anyio backends asyncio py line 287 in wrapper return await func args File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages prefect cli deployment py line 853 in build File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages prefect deployments py line 720 in build from flow await deployment upload to storage File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages prefect deployments py line 572 in upload to storage file count = await self storage put directory File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages prefect filesystems py line 359 in put directory self filesystem put file f fpath overwrite=True File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages fsspec spec py line 819 in put file callback relative update segment len File C Users Testbenutzer anaconda3 envs prefect test project v2 lib site packages fsspec callbacks py line 65 in relative update self value += inc TypeError unsupported operand type s for += int and NoneType An exception occurred ``` Python code ``` prefect flows py from prefect import flow from prefect tasks prefect tasks import call api flow def api flow url fact json = call api url print fact json prefect tasks py import requests from prefect import task task def call api url response = requests get url print response status code return response json ``` Storage block definition

Q · Answer

TLDR `pip install fsspec gt =2022 10 0` As you can see in the traceback whenever a file gets written to the remote the progress is tracked via a callback that adds the value returned by `AbstractFileSystem open write` to a counter It is assumed that the value represents the number of bytes written to the buffer However in the case of `fsspec implementations sftp SFTPFileSystem` the buffer `paramiko file BufferedFile` returns `None` when `write` is which leads to an exception when `fsspec spec AbstractBufferedFile` attempts to add `None` to an int counter The offending segment in `fsspec==2022 10 0` released 2022 10 19 bumping `fsspec` s version seems to solve the problem

Jens Freund · Answer

Thank you so much < Q> that solved the problem And also thank you very much for the explanation