https://prefect.io
Join Slack
Hi, I'm just getting started. When I run prefect I get an error, but I'm not sure if it's a bug or j...
# prefect-getting-started
p
Hi, I'm just getting started. When I run prefect I get an error, but I'm not sure if it's a bug or just my mistake
Copy code
# prefect
Traceback (most recent call last):
  File "/usr/local/bin/prefect", line 5, in <module>
    from prefect.cli import app
  File "/usr/local/lib/python3.12/site-packages/prefect/cli/__init__.py", line 22, in <module>
    import prefect.cli.kubernetes
  File "/usr/local/lib/python3.12/site-packages/prefect/cli/kubernetes.py", line 36, in <module>
    get_prefect_image_name(),
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/dockerutils.py", line 56, in get_prefect_image_name
    else "sha-" + prefect.__version_info__["full-revisionid"][:7]
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
TypeError: 'NoneType' object is not subscriptable
n
hi @Patrick Wspanialy! can you share the output of 
pip list | grep prefect
?
p
pip list | grep prefect prefect 2.18.1 prefect-client 2.18.1
this is the dockerfile I'm running in.
Copy code
FROM python:3.12

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

RUN curl -sSL <https://install.python-poetry.org> | python3 - && \
    cd /usr/local/bin && \
    ln -s ~/.local/bin/poetry && \
    poetry config virtualenvs.create false

COPY app/backend/app /app

ENV PYTHONPATH=/app
WORKDIR /app

RUN pip install -r requirements.txt
It works using the official docker
Copy code
prefect:
    image: prefecthq/prefect:2.18-python3.12
    command: prefect server start
albeit with some error messages
Copy code
| _ \ _ \ __| __| __/ __|_   _| 
prefect-1     | |  _/   / _|| _|| _| (__  | |  
prefect-1     | |_| |_|_\___|_| |___\___| |_|  
prefect-1     | 
prefect-1     | Configure Prefect to communicate with the server with:
prefect-1     | 
prefect-1     |     prefect config set PREFECT_API_URL=<http://127.0.0.1:4200/api>
prefect-1     | 
prefect-1     | View the API reference documentation at <http://127.0.0.1:4200/docs>
prefect-1     | 
prefect-1     | Check out the dashboard at <http://127.0.0.1:4200>
prefect-1     | 
prefect-1     | 
prefect-1     | 
prefect-1     | /usr/local/lib/python3.12/contextlib.py:144: SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_desc
prefect-1     |   next(self.gen)
prefect-1     | /usr/local/lib/python3.12/contextlib.py:144: SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_asc
prefect-1     |   next(self.gen)
n
> prefect-1 | /usr/local/lib/python3.12/contextlib.py144 SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_desc > prefect-1 | next(self.gen) > prefect-1 | /usr/local/lib/python3.12/contextlib.py144 SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_asc > prefect-1 | next(self.gen) these are warnings specifically related to prefect with a sqlite db, not errors i am able to run fine in a fresh container with 2.18.1
what is your objective with your resulting image?
p
just playing around , following the tutorial. tried running 
prefect cloud login
n
hmm im not sure why the full revision id would be 
None
🤔
Copy code
else "sha-" + prefect.__version_info__["full-revisionid"][:7]
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
TypeError: 'NoneType' object is not subscriptable
p
oh, got it working. I upgraded from 2.17 but had typer and uvicorn versions specified that weren't compatible. sorry about that and thanks for troubleshooting
catjam 1
n
no worries!
fastest way to dockerize a deployment imo is like this (set 2 env vars on container to run against cloud instead of ephemeral API)
Copy code
FROM python:3.12-slim

WORKDIR /app

# uv is a faster pip (<https://github.com/astral-sh/uv>) - highly recommend!
RUN pip install -U uv
RUN uv venv

# replace `prefect` with `-r requirements.txt` if you have one
RUN uv pip install prefect

# copy the flow definition into the image
COPY serve.py /app

ENTRYPOINT ["/app/.venv/bin/python", "/app/serve.py"]
Copy code
from pydantic import BaseModel

from prefect import flow


class SomeInput(BaseModel):
    a: int
    b: int


@flow
def test(i: SomeInput) -> int:
    return i.a + i.b


if __name__ == "__main__":
    test.serve(__file__)
Copy code
» docker build . -t test

...

» docker run -e PREFECT_API_KEY=$PREFECT_API_KEY -e PREFECT_API_URL=$PREFECT_API_URL test
Your flow 'test' is being served and polling for scheduled runs!

To trigger a run for this flow, use the following command:

        $ prefect deployment run 'test/serve'

You can also run your flow via the Prefect UI: <https://app.prefect.cloud/account/xxx/workspace/xxx/deployments/deployment/xxx>
i
Anyone got a docker compose for prefecthq/prefect:2.18-python3.12
n
hi ivan, what do you mean by
a docker compose
?
i
a docker-compose is a much more prefered way to launch a docker container, since you can define everything (multiple containers as a stack) in a single YAML file. It is much more prefered than a CLI run command because it is more readable, maintainable but also backupable
n
sorry, yeah i am familiar with docker compose, but a specific docker compose would need to opinionated for a use case generally speaking right? so what do you want it for is my question? just running a server?
i
In majority of selfhostable projects I see, it is usual to find in the documentation quite a bit of examples of those files
I want to self host prefect, and use my existing DB.
n
this might be a good reference we're demoing background tasks here (like celery but prefect) but youll find that we're using the compose to run a server here as well
here's a different example that uses postgres instead of sqlite
i
this is what I get
no usefull feedback
n
sorry its unclear what you're trying when you get that output
i
this is the docker container CLI output I get when starting the container.
n
are you running your container with 
prefect server start
like in the examples I shared?
i
yes
n
that seems correct if you're trying to start the server
i
however, this is below:
Copy code
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 538, in _prepare_and_execute
    self._rows = await prepared_stmt.fetch(*parameters)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch
    data = await self.__bind_execute(args, 0, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 241, in __bind_execute
    data, status, _ = await self.__do_execute(
                      ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 230, in __do_execute
    return await executor(protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 207, in bind_execute
asyncpg.exceptions.InsufficientPrivilegeError: permission denied for schema public

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1971, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 919, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 572, in execute
    self._adapt_connection.await_(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 131, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 550, in _prepare_and_execute
    self._handle_exception(error)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 501, in _handle_exception
    self._adapt_connection._handle_exception(error)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 789, in _handle_exception
    raise translated_error from error
sqlalchemy.dialects.postgresql.asyncpg.AsyncAdapt_asyncpg_dbapi.ProgrammingError: <class 'asyncpg.exceptions.InsufficientPrivilegeError'>: permission denied for schema public

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/prefect/_vendor/starlette/routing.py", line 736, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/server/api/server.py", line 635, in lifespan
    await run_migrations()
  File "/usr/local/lib/python3.12/site-packages/prefect/server/api/server.py", line 529, in run_migrations
    await db.create_db()
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/interface.py", line 56, in create_db
    await self.run_migrations_upgrade()
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/interface.py", line 64, in run_migrations_upgrade
    await run_sync_in_worker_thread(alembic_upgrade)
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 136, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/alembic_commands.py", line 24, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/alembic_commands.py", line 53, in alembic_upgrade
    alembic.command.upgrade(alembic_config(), revision, sql=dry_run)
  File "/usr/local/lib/python3.12/site-packages/alembic/command.py", line 403, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.12/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/migrations/env.py", line 175, in <module>
    apply_migrations()
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 288, in coroutine_wrapper
    return run_async_from_worker_thread(async_fn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 222, in run_async_from_worker_thread
    return anyio.from_thread.run(call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/from_thread.py", line 47, in run
    return asynclib.run_async_from_thread(func, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 906, in run_async_from_thread
    return f.result()
           ^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/migrations/env.py", line 169, in apply_migrations
    await connection.run_sync(do_run_migrations)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/ext/asyncio/engine.py", line 886, in run_sync
    return await greenlet_spawn(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/migrations/env.py", line 137, in do_run_migrations
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.12/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.12/site-packages/alembic/runtime/migration.py", line 610, in run_migrations
    self._ensure_version_table()
  File "/usr/local/lib/python3.12/site-packages/alembic/runtime/migration.py", line 548, in _ensure_version_table
    self._version.create(self.connection, checkfirst=True)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/schema.py", line 1288, in create
    bind._run_ddl_visitor(ddl.SchemaGenerator, self, checkfirst=checkfirst)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2461, in _run_ddl_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/visitors.py", line 664, in traverse_single
    return meth(obj, **kw)
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 956, in visit_table
    )._invoke_with(self.connection)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 314, in _invoke_with
    return bind.execute(self)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1422, in execute
    return meth(
           ^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 180, in _execute_on_connection
    return connection._execute_ddl(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1533, in _execute_ddl
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1850, in _execute_context
    return self._exec_single_context(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1990, in _exec_single_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2357, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1971, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 919, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 572, in execute
    self._adapt_connection.await_(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 131, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 550, in _prepare_and_execute
    self._handle_exception(error)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 501, in _handle_exception
    self._adapt_connection._handle_exception(error)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 789, in _handle_exception
    raise translated_error from error
sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class 'asyncpg.exceptions.InsufficientPrivilegeError'>: permission denied for schema public
[SQL: 
CREATE TABLE alembic_version (
        version_num VARCHAR(32) NOT NULL, 
        CONSTRAINT alembic_version_pkc PRIMARY KEY (version_num)
)

]
(Background on this error at: <https://sqlalche.me/e/20/f405>)

Application startup failed. Exiting.
Server stopped!
 *  Terminal will be reused by tasks, press any key to close it.
n
this seems like its about your postgres instance?
File "asyncpg/protocol/protocol.pyx", line 207, in bind_execute
asyncpg.exceptions.InsufficientPrivilegeError: permission denied for schema public
did you setup a 
postgres
container in your compose and then set the connection string like in the linked example?
i
finally, it works. I forgot to give access to the public schema. moron. Thanks a lot
👍 1
but now I cant map any directory.
Copy code
server:
    image: prefecthq/prefect:2.18-python3.12
    restart: always
    volumes:
      - /docker/appdata/prefect/volumes/prefect:/root/.prefect
      - /docker/appdata/prefect/volumes/tmp:/tmp:rw
      - /docker/appdata/prefect/volumes/flows:/opt/prefect/flows
    entrypoint: ["/bin/bash", "-c"]
    command: ["prefect server start --host 0.0.0.0"]
    ports:
      - "4200:4200"
I create files and folders but they are empty or dont exist
inside of the container
54 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by