https://prefect.io logo
Join Slack
Powered by
Hi, I'm just getting started. When I run prefect I...
# prefect-getting-started
p
Hi, I'm just getting started. When I run prefect I get an error, but I'm not sure if it's a bug or just my mistake
Copy code
# prefect
Traceback (most recent call last):
  File "/usr/local/bin/prefect", line 5, in <module>
    from prefect.cli import app
  File "/usr/local/lib/python3.12/site-packages/prefect/cli/__init__.py", line 22, in <module>
    import prefect.cli.kubernetes
  File "/usr/local/lib/python3.12/site-packages/prefect/cli/kubernetes.py", line 36, in <module>
    get_prefect_image_name(),
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/dockerutils.py", line 56, in get_prefect_image_name
    else "sha-" + prefect.__version_info__["full-revisionid"][:7]
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
TypeError: 'NoneType' object is not subscriptable
n
hi @Patrick Wspanialy! can you share the output of 
pip list | grep prefect
?
p
pip list | grep prefect prefect 2.18.1 prefect-client 2.18.1
this is the dockerfile I'm running in.
Copy code
FROM python:3.12

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

RUN curl -sSL <https://install.python-poetry.org> | python3 - && \
    cd /usr/local/bin && \
    ln -s ~/.local/bin/poetry && \
    poetry config virtualenvs.create false

COPY app/backend/app /app

ENV PYTHONPATH=/app
WORKDIR /app

RUN pip install -r requirements.txt
It works using the official docker
Copy code
prefect:
    image: prefecthq/prefect:2.18-python3.12
    command: prefect server start
albeit with some error messages
Copy code
| _ \ _ \ __| __| __/ __|_   _| 
prefect-1     | |  _/   / _|| _|| _| (__  | |  
prefect-1     | |_| |_|_\___|_| |___\___| |_|  
prefect-1     | 
prefect-1     | Configure Prefect to communicate with the server with:
prefect-1     | 
prefect-1     |     prefect config set PREFECT_API_URL=<http://127.0.0.1:4200/api>
prefect-1     | 
prefect-1     | View the API reference documentation at <http://127.0.0.1:4200/docs>
prefect-1     | 
prefect-1     | Check out the dashboard at <http://127.0.0.1:4200>
prefect-1     | 
prefect-1     | 
prefect-1     | 
prefect-1     | /usr/local/lib/python3.12/contextlib.py:144: SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_desc
prefect-1     |   next(self.gen)
prefect-1     | /usr/local/lib/python3.12/contextlib.py:144: SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_asc
prefect-1     |   next(self.gen)
n
> prefect-1 | /usr/local/lib/python3.12/contextlib.py144 SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_desc > prefect-1 | next(self.gen) > prefect-1 | /usr/local/lib/python3.12/contextlib.py144 SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_asc > prefect-1 | next(self.gen) these are warnings specifically related to prefect with a sqlite db, not errors i am able to run fine in a fresh container with 2.18.1
what is your objective with your resulting image?
p
just playing around , following the tutorial. tried running 
prefect cloud login
n
hmm im not sure why the full revision id would be 
None
🤔
Copy code
else "sha-" + prefect.__version_info__["full-revisionid"][:7]
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
TypeError: 'NoneType' object is not subscriptable
p
oh, got it working. I upgraded from 2.17 but had typer and uvicorn versions specified that weren't compatible. sorry about that and thanks for troubleshooting
catjam 1
n
no worries!
fastest way to dockerize a deployment imo is like this (set 2 env vars on container to run against cloud instead of ephemeral API)
Copy code
FROM python:3.12-slim

WORKDIR /app

# uv is a faster pip (<https://github.com/astral-sh/uv>) - highly recommend!
RUN pip install -U uv
RUN uv venv

# replace `prefect` with `-r requirements.txt` if you have one
RUN uv pip install prefect

# copy the flow definition into the image
COPY serve.py /app

ENTRYPOINT ["/app/.venv/bin/python", "/app/serve.py"]
Copy code
from pydantic import BaseModel

from prefect import flow


class SomeInput(BaseModel):
    a: int
    b: int


@flow
def test(i: SomeInput) -> int:
    return i.a + i.b


if __name__ == "__main__":
    test.serve(__file__)
Copy code
» docker build . -t test

...

» docker run -e PREFECT_API_KEY=$PREFECT_API_KEY -e PREFECT_API_URL=$PREFECT_API_URL test
Your flow 'test' is being served and polling for scheduled runs!

To trigger a run for this flow, use the following command:

        $ prefect deployment run 'test/serve'

You can also run your flow via the Prefect UI: <https://app.prefect.cloud/account/xxx/workspace/xxx/deployments/deployment/xxx>
i
Anyone got a docker compose for prefecthq/prefect:2.18-python3.12
n
hi ivan, what do you mean by
a docker compose
?
i
a docker-compose is a much more prefered way to launch a docker container, since you can define everything (multiple containers as a stack) in a single YAML file. It is much more prefered than a CLI run command because it is more readable, maintainable but also backupable
n
sorry, yeah i am familiar with docker compose, but a specific docker compose would need to opinionated for a use case generally speaking right? so what do you want it for is my question? just running a server?
i
In majority of selfhostable projects I see, it is usual to find in the documentation quite a bit of examples of those files
I want to self host prefect, and use my existing DB.
n
this might be a good reference we're demoing background tasks here (like celery but prefect) but youll find that we're using the compose to run a server here as well
here's a different example that uses postgres instead of sqlite
i
this is what I get
no usefull feedback
n
sorry its unclear what you're trying when you get that output
i
this is the docker container CLI output I get when starting the container.
n
are you running your container with 
prefect server start
like in the examples I shared?
i
yes
n
that seems correct if you're trying to start the server
i
however, this is below:
Copy code
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 538, in _prepare_and_execute
    self._rows = await prepared_stmt.fetch(*parameters)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch
    data = await self.__bind_execute(args, 0, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 241, in __bind_execute
    data, status, _ = await self.__do_execute(
                      ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 230, in __do_execute
    return await executor(protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 207, in bind_execute
asyncpg.exceptions.InsufficientPrivilegeError: permission denied for schema public

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1971, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 919, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 572, in execute
    self._adapt_connection.await_(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 131, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 550, in _prepare_and_execute
    self._handle_exception(error)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 501, in _handle_exception
    self._adapt_connection._handle_exception(error)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 789, in _handle_exception
    raise translated_error from error
sqlalchemy.dialects.postgresql.asyncpg.AsyncAdapt_asyncpg_dbapi.ProgrammingError: <class 'asyncpg.exceptions.InsufficientPrivilegeError'>: permission denied for schema public

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/prefect/_vendor/starlette/routing.py", line 736, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/server/api/server.py", line 635, in lifespan
    await run_migrations()
  File "/usr/local/lib/python3.12/site-packages/prefect/server/api/server.py", line 529, in run_migrations
    await db.create_db()
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/interface.py", line 56, in create_db
    await self.run_migrations_upgrade()
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/interface.py", line 64, in run_migrations_upgrade
    await run_sync_in_worker_thread(alembic_upgrade)
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 136, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/alembic_commands.py", line 24, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/alembic_commands.py", line 53, in alembic_upgrade
    alembic.command.upgrade(alembic_config(), revision, sql=dry_run)
  File "/usr/local/lib/python3.12/site-packages/alembic/command.py", line 403, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.12/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/migrations/env.py", line 175, in <module>
    apply_migrations()
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 288, in coroutine_wrapper
    return run_async_from_worker_thread(async_fn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 222, in run_async_from_worker_thread
    return anyio.from_thread.run(call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/from_thread.py", line 47, in run
    return asynclib.run_async_from_thread(func, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 906, in run_async_from_thread
    return f.result()
           ^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/migrations/env.py", line 169, in apply_migrations
    await connection.run_sync(do_run_migrations)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/ext/asyncio/engine.py", line 886, in run_sync
    return await greenlet_spawn(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/server/database/migrations/env.py", line 137, in do_run_migrations
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.12/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.12/site-packages/alembic/runtime/migration.py", line 610, in run_migrations
    self._ensure_version_table()
  File "/usr/local/lib/python3.12/site-packages/alembic/runtime/migration.py", line 548, in _ensure_version_table
    self._version.create(self.connection, checkfirst=True)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/schema.py", line 1288, in create
    bind._run_ddl_visitor(ddl.SchemaGenerator, self, checkfirst=checkfirst)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2461, in _run_ddl_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/visitors.py", line 664, in traverse_single
    return meth(obj, **kw)
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 956, in visit_table
    )._invoke_with(self.connection)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 314, in _invoke_with
    return bind.execute(self)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1422, in execute
    return meth(
           ^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 180, in _execute_on_connection
    return connection._execute_ddl(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1533, in _execute_ddl
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1850, in _execute_context
    return self._exec_single_context(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1990, in _exec_single_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2357, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1971, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 919, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 572, in execute
    self._adapt_connection.await_(
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 131, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 550, in _prepare_and_execute
    self._handle_exception(error)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 501, in _handle_exception
    self._adapt_connection._handle_exception(error)
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 789, in _handle_exception
    raise translated_error from error
sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class 'asyncpg.exceptions.InsufficientPrivilegeError'>: permission denied for schema public
[SQL: 
CREATE TABLE alembic_version (
        version_num VARCHAR(32) NOT NULL, 
        CONSTRAINT alembic_version_pkc PRIMARY KEY (version_num)
)

]
(Background on this error at: <https://sqlalche.me/e/20/f405>)

Application startup failed. Exiting.
Server stopped!
 *  Terminal will be reused by tasks, press any key to close it.
n
this seems like its about your postgres instance?
File "asyncpg/protocol/protocol.pyx", line 207, in bind_execute
asyncpg.exceptions.InsufficientPrivilegeError: permission denied for schema public
did you setup a 
postgres
container in your compose and then set the connection string like in the linked example?
i
finally, it works. I forgot to give access to the public schema. moron. Thanks a lot
👍 1
but now I cant map any directory.
Copy code
server:
    image: prefecthq/prefect:2.18-python3.12
    restart: always
    volumes:
      - /docker/appdata/prefect/volumes/prefect:/root/.prefect
      - /docker/appdata/prefect/volumes/tmp:/tmp:rw
      - /docker/appdata/prefect/volumes/flows:/opt/prefect/flows
    entrypoint: ["/bin/bash", "-c"]
    command: ["prefect server start --host 0.0.0.0"]
    ports:
      - "4200:4200"
I create files and folders but they are empty or dont exist
inside of the container
55 Views
Open in Slack
PreviousNext
Powered by