n

    Nicholas Chammas

    1 year ago
    I have a
    Parameter
    being fed into a
    DatabricksRunNow
    Task
    as a
    notebook_param
    . I can see the edge between the parameter and the task in the UI fine, but for whatever reason the Databricks notebook is not receiving the parameter value when I run the flow. What should I try to troubleshoot the reason why the notebook task is not receiving parameter values?
    Kevin Kho

    Kevin Kho

    1 year ago
    Hi @Nicholas Chammas, could you show me the code of the
    DatabricksRunNow
    task and how you pass it in? Just remove any sensitive stuff.
    n

    Nicholas Chammas

    1 year ago
    The flow code is roughly this:
    def databricks_job(job_id, **kwargs):
        if "name" in kwargs:
            name_prefix = f"{kwargs['name']} - "
        else:
            name_prefix = ""
        kwargs.setdefault("task_run_name", f"{name_prefix}databricks_job[{job_id}]")
        kwargs.update(job_id=job_id)
        return DatabricksRunNow(**kwargs)
    
    
    GenerateCounts = databricks_job(
        name="Generate Counts",
        job_id=GENERATE_COUNTS_JOB_ID,
    )
    
    
    with le_flow(name="...", state_handlers=[HANDLER]) as flow:
        version = Parameter("version", required=True)
    
        generate_counts = GenerateCounts(
            databricks_conn_secret=DATABRICKS_CONN,
            notebook_params={
                "attemptId": version,
            },
        )
    Kevin Kho

    Kevin Kho

    1 year ago
    It looks fine to me and the source code looks fine. Does it work if you don’t use the param? I assume
    notebook_params
    does not get passed?
    n

    Nicholas Chammas

    1 year ago
    Hmm, it looks like if I replace
    version
    inside
    "attemptId": version,
    with a plain string, the notebook still doesn’t get any parameters. I guess the problem is not the parameter then! That’s an obvious test I should have tried. 😅 Guess I need to understand what’s going wrong with
    notebook_params
    to begin with.
    Kevin Kho

    Kevin Kho

    1 year ago
    Does your job go through though? Meaning the connection secret got used?
    n

    Nicholas Chammas

    1 year ago
    Yes, the job is triggered but always with no/empty notebook parameter values.
    Kevin Kho

    Kevin Kho

    1 year ago
    Stupid question, but just making sure you’re re-registering right? This seems weird notebook_params is not respected. What version are you on?
    Does it work if you just use the task and call the
    Task.run()
    without using a Flow?
    n

    Nicholas Chammas

    1 year ago
    I’m on Prefect 0.14.22. The reregistration is working. If I provide the
    notebook_params
    as static kwargs (i.e. alongside things like
    name
    and
    job_id
    ) it works, but when I provide them as runtime/dynamic
    kwargs
    that’s when they are being ignored. I’ll try
    Task.run()
    and report back.
    Kevin Kho

    Kevin Kho

    1 year ago
    Very weird because the code seems to just grab the init arguments and use them at runtime if there are done provided
    n

    Nicholas Chammas

    1 year ago
    OK, I think I found the issue. Our ECS Prefect agent was running Prefect 0.14.12. Updating that to 0.14.22 fixed the problem. I got suspicious when running the flow locally (per your suggestion) worked, but running it on our deployed ECS agent didn’t.
    Kevin Kho

    Kevin Kho

    1 year ago
    Ohh wow. Good to know!