Hi all, I am experimenting with nested tasks and t...
# prefect-ui
d
Hi all, I am experimenting with nested tasks and the UI appears to just show the parent task as an isolated rectangle with no dependency or encapsulation of its child tasks. Is this expected behavior or am I doing something wrong? I was hoping for it to enclose the child tasks the way that sub-flows do.
1
d
Hey Daniel, We just did a quick test internally and we're seeing nested tasks encapsulated by their parent as we expect. Can you share a brief MRE (minimum reproducible example) along with your Prefect version & any other pertinent environment info?
d
Certainly. Flow and deployment code below. Server is hosted on K8s and is
2.19.8
. Result of running the deployment in the UI is attached as image. Python file for flow tasks:
Copy code
from prefect import flow, get_run_logger, deploy
from prefect.deployments import DeploymentImage
from prefect import task


@task(log_prints=True)
def child_task():
    print("I am a child task")


@task(log_prints=True)
def parent_task():
    print("I am a parent task")
    return child_task()


@flow()
def test_flow_nested_task_mre():
    return parent_task()


if __name__ == "__main__":
    deployment_container_image = DeploymentImage(
        name="{registry remove for confidentiality}/prefect-test-flow-nested-task-mre",
        tag="latest",
        dockerfile="Dockerfile",
        platform="linux/amd64",
    )

    test_flow_nested_task_mre.deploy(
        name=f"test-flow-nested-task-mre-primary",
        work_pool_name="default",
        image=deployment_container_image,
    )
Dockerfile:
Copy code
FROM prefecthq/prefect:2.20.10-python3.11
COPY requirements.txt /opt/prefect/prefect/requirements.txt
# Hotfix for oscrypto openssl bug (<https://community.snowflake.com/s/question/0D5Do000010NAVzKAO/hi-snowflake-teami-am-facing-issues-while-connecting-to-snowflake-using-snowflake-connector-with-oscryptoerrorslibrarynotfounderror-error-detecting-the-version-of-libcrypto>)
RUN python -m pip install oscrypto@git+https://github.com/wbond/oscrypto.git@d5f3437ed24257895ae1edd9e503cfb352e635a8
RUN python -m pip install -v -r /opt/prefect/prefect/requirements.txt
COPY . /opt/prefect/prefect/
WORKDIR /opt/prefect/prefect/
Requirements.txt
Copy code
prefect>=2.17.1,<3.0.0
prefect-dbt>=0.4.1,<1.0.0
prefect-snowflake>=0.27.3,<1.0.0
dbt-core>=1.8.6,<2.0.0
dbt-snowflake>=1.8.3,<2.0.0
pygit2>=1.14.1,<2.0.0
azure-identity>=1.16.0,<2.0.0
azure-keyvault-secrets>=4.8.0,<5.0.0
msal>=1.28.0,<2.0.0
snowflake-connector-python>=3.8.1,<4.0.0
html-to-json>=2.0.0,<3.0.0
cryptography>=42.0.5,<43.0.0
black>=24.4.0,<25.0.0
Result in the UI:
I upgraded server and worker to
2.20.10
and still got the same result
d
This is a great MRE! Thanks for being so thorough 💯
Will take a look at this and get back to you
Meantime, here's the example we whipped up if you'd like to compare
{removed, it wasn't that helpful}
So far the consensus is that we wouldn't expect 2.x to properly render nesting for tasks in the way your example outlines
Similar code does nest in 3.x:
Copy code
rom prefect import flow, task


@task(log_prints=True)
def child_task():
    print("I am a child task")
    return 1


@task(log_prints=True)
def parent_task():
    print("I am a parent task")
    result = child_task()
    print(result)
    return result


@flow()
def test_flow_nested_task_mre():
    result = parent_task()
    return result


if __name__ == "__main__":
    test_flow_nested_task_mre()
i
.00
d
@Dylan After upgrading to prefect 3 nested tasks in my MRE are rendering correctly. Unfortunately they are still rendering as separate unassociated entities in my production code. I am having a difficult time re-producing this in an MRE. Any chance we could connect on this?
d
Hey Daniel, definitely! You can connect with an engineer on our team here: https://calendly.com/prefect-experts/prefect-product-advocates?month=2024-11
d
I figured it out. The UI stops nesting child tasks if the parent task has more than one argument. For example, it stops working with the below modifications to the previous mre. If you remove
test_arg_2
the UI renders it as a nested task again.
Copy code
from prefect import flow, task, deploy
from prefect.docker.docker_image import DockerImage


@task(log_prints=True)
def child_task():
    print("I am a child task")
    return 1


@task(log_prints=True)
def parent_task(test_arg, test_arg_2):
    print("I am a parent task")
    result = child_task()
    print(result)
    return result


@flow()
def test_flow_nested_task_mre():
    result = parent_task(
        test_arg="hello",
        test_arg_2="world",
    )
    return result


if __name__ == "__main__":
    custom_docker_image = DockerImage(
        name="[REDACTED]/test-nested-tasks-mre",
        tag="latest",
        dockerfile="Dockerfile",
    )

    flow_deployment = test_flow_nested_task_mre.to_deployment(
        name=f"test-nested-tasks-mre",
    )

    deploy(
        flow_deployment,
        work_pool_name="kubernetes",
        image=custom_docker_image,
    )
You don't even want to know how much trial and error that took to find 🤣
d
That's certainly a bug! Would you mind opening an issue with the UI tag and we'll investigate 👍
d
Done! Though I don't think I have permissions to add labels so here it is: https://github.com/PrefectHQ/prefect/issues/15933
d
Thank you!
🙌 1
j
@Dylan I have subtasks that are created with task.submit that are not showing up either -- they have arguments, so probably the same bug, but thought I'd add another data point