Thread
#prefect-community
    s

    Sébastien

    1 year ago
    New server, this error isn't telling much:
    #> prefect auth login -t "<my token>"
    Error attempting to communicate with Prefect Cloud
    I can confirm that
    ~/.prefect/config.toml
    isn't being created. Trying to use
    python3 start.py
    to register any flow gives:
    prefect.utilities.exceptions.ClientError: Malformed response received from Cloud - please ensure that you have an API token properly configured.
    which unfortunately doesn't add any info. Pinging
    <http://api.prefect.io|api.prefect.io>
    works:
    PING <http://api.prefect.io|api.prefect.io> (34.96.72.220) 56(84) bytes of data.
    64 bytes from <http://220.72.96.34.bc.googleusercontent.com|220.72.96.34.bc.googleusercontent.com> (34.96.72.220): icmp_seq=1 ttl=116 time=3.71 ms
    Any ideas?
    j

    josh

    1 year ago
    Ah we can definitely expose that error better! Could you try something:
    from prefect import Client
    
    client = Client(api_token=<your_token>)
    
    output = client.graphql(
                query={"query": {"user": {"default_membership": "tenant_id"}}}
            )
    
    client.login_to_tenant(            tenant_id=output.data.user[0].default_membership.tenant_id
            )
    And see what error is raised
    s

    Sébastien

    1 year ago
    @josh Figured it out — I needed a personal access token to use
    prefect auth login -t X
    while I was trying to use my RUNNER token from my other server's
    config.toml
    And registering a flow requires the personal access token apparently.
    @josh I've got a mapped task that works properly, and just registered 2 additional agents (they show up in the Agents tab) + all 3 share a label which I used to tag the Flow. But the mapped task seems to be running on a single worker. What did I miss?
    j

    josh

    1 year ago
    @Sébastien multiple agents has no baring on the concurrency for the flow. All the agents do is create the job that runs the flow. Take a look at setting an executor on your flow to achieve parallelism https://docs.prefect.io/orchestration/flow_config/executors.html
    s

    Sébastien

    1 year ago
    Oh. I thought registering multiple agents for a flow would allow them to pool on a flow. (Without using Dask — my mistake)
    Thanks for the info
    @josh Sorry to bug you again, but
    executor=DaskExecutor(adapt_kwargs={"maximum": 10})
    didn't help, still running the map on a single worker - taking exactly as much time. Is there an easy way to add logs to check the executor?
    The goal was to get the DaskExecutor running with a temporary cluster.
    Tried
    executor=DaskExecutor(cluster_kwargs={"n_workers": 10})
    as well thinking it would be simpler. It runs, but same issue as before.
    debug = true
    
    [logging]
    level = "DEBUG"
    and
    executor=DaskExecutor(..., debug=True)
    no debug logs to be found
    Shut down all workers except one + deleted from Cloud, moved to LocalDaskExecutor, and tried using
    "processes": False, "threads_per_worker": 8
    — exact same result.
    (Yes, published the new version each time, the version counter is going up)
    Matched settings to OS' CPU & thread count as reported by
    lscpu
    just to make sure — no change.