• James Keegan

    James Keegan

    4 months ago
    Hello! I'm trying to clone a bitbucket repository into a Prefect Docker image (using the
    ShellTask
    task) but keep running into this error. Any suggestions? The SSH key is set up correctly, but there doesn't appear to be a user associated with the
    ShellTask
    commands. This wasn't an issue previously, as older flow images can pull git projects just fine. Only came up recently after I rebuilt the image again (Prefect version & python version are still the same).
    Cloning into './dbt_project'...
    No user exists for uid 1000190000
    fatal: Could not read from remote repository.
    
    Please make sure you have the correct access rights
    and the repository exists.
    James Keegan
    Kevin Kho
    6 replies
    Copy to Clipboard
  • James Keegan

    James Keegan

    4 months ago
    cloudpickle
    v2.1.0 was just released an hour ago and it has broken the healthcheck when we try to register a flow. We're using the
    prefecthq/prefect:0.15.6-python3.7
    as a base image. We now get this error when building the docker image.
    Step 21/21 : RUN python /opt/prefect/healthcheck.py '["/opt/prefect/flows/prefect-dbt-run-modelling.prefect"]' '(3, 7)'
     ---> Running in 1d14c333ced9
    Beginning health checks...
    System Version check: OK
    Traceback (most recent call last):
      File "/opt/prefect/healthcheck.py", line 152, in <module>
        flows = cloudpickle_deserialization_check(flow_file_paths)
      File "/opt/prefect/healthcheck.py", line 44, in cloudpickle_deserialization_check
        flows.append(cloudpickle.loads(flow_bytes))
    AttributeError: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from '/usr/local/lib/python3.7/site-packages/cloudpickle/cloudpickle.py'>
    James Keegan
    Kevin Kho
    +3
    13 replies
    Copy to Clipboard
  • Florian Guily

    Florian Guily

    4 months ago
    Hello, more a python issue but encountered when trying to execute this tuto. I have a project structures that looks like the attached image. This structure is the same as @Anna Geller github repo for the tuto. In her flow codes, she imports functions from a
    config.py
    file in
    flow_utilities
    folder like this
    from flow_utilities.config import funcA funcB
    . When i do so, i get an error. So i tried to reproduced it with the dummy code you have on the image. I get the same error which is
    ModuleNotFoundError: No module named 'flow_utilities'
    . This really seems stupid to me as it is just importing some func from local file but somehow i can't make it work with this file architecture. What am i doing wrong ?
    Florian Guily
    Anna Geller
    4 replies
    Copy to Clipboard
  • Florian Guily

    Florian Guily

    4 months ago
    New issue: When using
    Secret.get()
    it apparently only search for local secrets. The doc says i have to change
    prefect.config.use_local_secrets
    to false but i can't understand how... I suppose it is in the config.toml file but as the flow has to run on an eks cluster, how can i specify this for the cluster ?
    Florian Guily
    Kevin Kho
    2 replies
    Copy to Clipboard
  • Mike Vanbuskirk

    Mike Vanbuskirk

    4 months ago
    does anyone have a link to an example Dockerfile for a custom image that a flow might run on?
    Mike Vanbuskirk
    Anna Geller
    +1
    8 replies
    Copy to Clipboard
  • Malthe Karbo

    Malthe Karbo

    4 months ago
    When attaching a log handler (e.g., slack) to prefect logger, then whenever running a flow with dask, it will log multiple times with the handler - at least this is what we see with our implementation - Is there a good example for attaching a custom log handler to prefect orion logging?
    Malthe Karbo
    Kevin Kho
    2 replies
    Copy to Clipboard
  • Alvaro Durán Tovar

    Alvaro Durán Tovar

    4 months ago
    if I want to create flows dynamically like this (registration happens after)
    def build(...):
        with Flow(...) as Flow:
            ...
        return flow
    is it possible to use docker storage? thinking on possible issues trying to find the path of the flow inside the docker file, there won't any "flow" variable on the module level
    Alvaro Durán Tovar
    Kevin Kho
    +1
    12 replies
    Copy to Clipboard
  • p

    Philip MacMenamin

    4 months ago
    Hello - can I get some help with debugging a failing workflow?
    p
    Kevin Kho
    22 replies
    Copy to Clipboard
  • Nash Taylor

    Nash Taylor

    4 months ago
    I feel like I'm going to gain a reputation as the guy who only ever asks about type safety but... I've got pyright integrated into my environment now, which is working far better with Prefect than Mypy did. It just requires one small hack, and it makes me wonder why this isn't part of the library: I had to add
    py.typed
    on my own. I see that there is a
    py.typed
    on the master branch; was there a conscious decision made not to include this marker file in 2.0? If so, why? Without the marker file, I get:
    from prefect.flows import flow
    
    reveal_type(flow)  # Unknown
    With the marker file, I get:
    from prefect.flows import flow
    
    reveal_type(flow)  # Overload[(__fn: (**P@flow) -> R@flow, /) -> Flow[P@flow, R@flow], (*, name: str = None, version: str = None, task_runner: BaseTaskRunner = ConcurrentTaskRunner, description: str = None, timeout_seconds: int | float = None, validate_parameters: bool = True) -> (((**P@flow) -> R@flow) -> Flow[P@flow, R@flow]), (__fn: Unknown | None = None, *, name: str = None, version: str = None, task_runner: BaseTaskRunner = ConcurrentTaskRunner, description: str = None, timeout_seconds: int | float = None, validate_parameters: bool = True) -> (Flow[P, R] | (((**P) -> R) -> Flow[P, R]))]
    which is much better. For example, now when I decorate a function with
    @task
    and use it in an
    @flow
    , the return type of the task is known to have a
    .result()
    method. When it was just an Unknown, pyright would complain that I'm accessing a method that doesn't exist. For someone who enforces fully passing mypy (now pyright) checks on every pull request, this is kind of a necessity.
    Nash Taylor
    Michael Adkins
    +1
    8 replies
    Copy to Clipboard
  • GGK Kellie

    GGK Kellie

    4 months ago
    Hello, when I run prefect 2.0b5 on windows machine, there is an error message "ValidationError: 1 validation error for Settings PREFECT_ORION_API_PORT value is not a valid integer (type=type_error.integer)". I try to reset it with "prefect config set PREFECT_ORION_API_PORT=4200". It showed the same error message. Any idea to fix it? Thanks.
    GGK Kellie
    Kevin Kho
    4 replies
    Copy to Clipboard