haris khan

08/04/2022, 10:44 AM
when i build prefect deployment with gcs storage it tries to copy all my files including the virtaul enviroment to the gcs bucket and then fails (venv) haris@haris-Inspiron-3593:~/Desktop/snowflake$ prefect deployment build workflow/ -n genvenueflow_deployment -t n1-standard-4 -i kubernetes-job --storage-block gcs/prefect2-snowflake /home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/prefect/ UserWarning: A flow named 'gen_venue' and defined at 'workflow/' conflicts with another flow. Consider specifying a unique
parameter in the flow definition:
@flow(name='my_unique_name', ...)
warnings.warn( Found flow 'gen_venue' Manifest created at '/home/haris/Desktop/snowflake/gen_venue_flow-manifest.json'. Traceback (most recent call last): File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/prefect/cli/", line 41, in wrapper return fn(*args, **kwargs) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/prefect/utilities/", line 193, in wrapper return run_async_in_new_loop(async_fn, *args, **kwargs) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/prefect/utilities/", line 140, in run_async_in_new_loop return, *args, **kwargs)) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/anyio/_core/", line 56, in run return, *args, **backend_options) # type: ignore File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/anyio/_backends/", line 233, in run return native_run(wrapper(), debug=debug) File "/usr/lib/python3.8/asyncio/", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.8/asyncio/", line 616, in run_until_complete return future.result() File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/anyio/_backends/", line 228, in wrapper return await func(*args) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/prefect/cli/", line 502, in build file_count = await storage.put_directory() File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/prefect/", line 428, in put_directory return await self.filesystem.put_directory( File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/prefect/", line 250, in put_directory self.filesystem.put_file(f, fpath) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/fsspec/", line 86, in wrapper return sync(self.loop, func, *args, **kwargs) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/fsspec/", line 66, in sync raise return_result File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/fsspec/", line 26, in _runner result[0] = await coro File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/gcsfs/", line 1045, in _put_file await simple_upload( File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/gcsfs/", line 1596, in simple_upload j = await fs._call( File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/gcsfs/", line 392, in _call status, headers, info, contents = await self._request( File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/", line 221, in fun return await caller(func, *(extras + args), **kw) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/gcsfs/", line 115, in retry_request return await func(*args, **kwargs) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/gcsfs/", line 384, in _request validate_response(status, contents, path, args) File "/home/haris/Desktop/snowflake/venv/lib/python3.8/site-packages/gcsfs/", line 100, in validate_response raise ValueError("Bad Request: %s\n%s" % (path, msg)) ValueError: Bad Request: Payload size invalid. Expected size: 2. Actual size: 3. An exception occurred.
is there a way to exclude the virtual enviroment ?
solved it by moving my virtual enviroment folder outside the directory but there is another issue everytime i run the deployment build command with storage it overwrites all files again and not just the updated one
ps : ive got multiple deployments

Khuyen Tran

08/04/2022, 6:08 PM
Yes, you can do that with
. It is similar to
For example, if you want to exclude a virtual environment named
, add the following to your `.prefectignore`: