<@ULVA73B9P> I am running a worker on kubernetes and calling a dbt cloud job. It reliably gives this...
j

James Constable

over 1 year ago
@Marvin I am running a worker on kubernetes and calling a dbt cloud job. It reliably gives this error after one hour.
Error occurred while streaming logs - Job will continue to run but logs will no longer be streamed to stdout.
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_proto.py", line 92, in connection_lost
    uncompleted = self._parser.feed_eof()
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_parser.pyx", line 508, in aiohttp._http_parser.HttpParser.feed_eof
aiohttp.http_exceptions.TransferEncodingError: 400, message:
  Not enough data for satisfy transfer length header.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/prefect_kubernetes/worker.py", line 1041, in _stream_job_logs
    async for line in logs.content:
  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 52, in __anext__
    rv = await self.read_func()
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 352, in readline
    return await self.readuntil()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 386, in readuntil
    await self._wait("readuntil")
  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 347, in _wait
    await waiter
aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed: <TransferEncodingError: 400, message='Not enough data for satisfy transfer length header.'>
Hello , I am trying to run a prefect in docker. I looked in this community. But it seems that there ...
o

omiakif

almost 2 years ago
Hello , I am trying to run a prefect in docker. I looked in this community. But it seems that there is no proper docker compose file for prefect. I was trying to use this for the prefect docker compose. I am very new to docker. So, I tried to write a docker compose to up and prefect. I am also learning prefect very recently. I tried to write the docker compose below following this example. Is there any proper docker compose.yml file that I can use to create nodes for prefect and view the prefect dashboard? The dashboard is not showing. I am looking for docker-compose.yml for prefect v3. Thank you in advance for any assistance. https://github.com/PrefectHQ/prefect-background-task-examples/blob/main/chaos-duck/docker-compose.yaml But I seem to be encountering this issue in the docker compose logs.
prefect  | During handling of the above exception, another exception occurred:
prefect  | 
prefect  | Traceback (most recent call last):
prefect  |   File "/usr/local/bin/prefect", line 8, in <module>
prefect  |     sys.exit(cli())
prefect  |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1130, in __call__
prefect  |     return self.main(*args, **kwargs)
prefect  |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1055, in main
prefect  |     rv = self.invoke(ctx)
prefect  |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
prefect  |     return _process_result(sub_ctx.command.invoke(sub_ctx))
prefect  |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
prefect  |     return _process_result(sub_ctx.command.invoke(sub_ctx))
prefect  |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
prefect  |     return ctx.invoke(self.callback, **ctx.params)
prefect  |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 760, in invoke
prefect  |     return __callback(*args, **kwargs)
prefect  |   File "/usr/local/lib/python3.7/site-packages/prefect/cli/server.py", line 674, in start
prefect  |     ["docker-compose", "down"], cwd=compose_dir_path, env=env
prefect  |   File "/usr/local/lib/python3.7/subprocess.py", line 411, in check_output
prefect  |     **kwargs).stdout
prefect  |   File "/usr/local/lib/python3.7/subprocess.py", line 488, in run
prefect  |     with Popen(*popenargs, **kwargs) as process:
prefect  |   File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
prefect  |     restore_signals, start_new_session)
prefect  |   File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
prefect  |     raise child_exception_type(errno_num, err_msg, err_filename)
prefect  | FileNotFoundError: [Errno 2] No such file or directory: 'docker-compose': 'docker-compose'
My docker-compose.yml
services:

  postgres:
    image: postgres:16
    restart: always
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: password
      POSTGRES_DB: prefect
    volumes:
      - postgres-data:/var/lib/postgresql/data
    networks:
      - prefect-net

  prefect:
    image: prefecthq/prefect:latest
    container_name: prefect
    restart: always
    environment:
      PREFECT_HOME: /data
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:password@postgres/prefect>
    command: prefect server start
    ports:
      - "4200:4200"
    volumes:
      - prefect-data:/data
    depends_on:
      - postgres
    networks:
      - prefect-net

volumes:
  postgres-data:
  prefect-data:

networks:
  prefect-net: