https://prefect.io logo
Channels
Powered by
s

Sébastien

12/09/2020, 9:36 PM
New server, this error isn't telling much:
Copy code
#> prefect auth login -t "<my token>"
Error attempting to communicate with Prefect Cloud
I can confirm that 
~/.prefect/config.toml
isn't being created. Trying to use 
python3 start.py
to register any flow gives:
Copy code
prefect.utilities.exceptions.ClientError: Malformed response received from Cloud - please ensure that you have an API token properly configured.
which unfortunately doesn't add any info. Pinging 
<http://api.prefect.io|api.prefect.io>
works:
Copy code
PING <http://api.prefect.io|api.prefect.io> (34.96.72.220) 56(84) bytes of data.
64 bytes from <http://220.72.96.34.bc.googleusercontent.com|220.72.96.34.bc.googleusercontent.com> (34.96.72.220): icmp_seq=1 ttl=116 time=3.71 ms
Any ideas?
j

josh

12/09/2020, 9:46 PM
Ah we can definitely expose that error better! Could you try something:
Copy code
from prefect import Client

client = Client(api_token=<your_token>)

output = client.graphql(
            query={"query": {"user": {"default_membership": "tenant_id"}}}
        )

client.login_to_tenant(            tenant_id=output.data.user[0].default_membership.tenant_id
        )
And see what error is raised
s

Sébastien

12/09/2020, 9:48 PM
@josh Figured it out — I needed a personal access token to use 
prefect auth login -t X
while I was trying to use my RUNNER token from my other server's 
config.toml
👍 1
And registering a flow requires the personal access token apparently.
@josh I've got a mapped task that works properly, and just registered 2 additional agents (they show up in the Agents tab) + all 3 share a label which I used to tag the Flow. But the mapped task seems to be running on a single worker. What did I miss?
j

josh

12/09/2020, 10:21 PM
@Sébastien multiple agents has no baring on the concurrency for the flow. All the agents do is create the job that runs the flow. Take a look at setting an executor on your flow to achieve parallelism https://docs.prefect.io/orchestration/flow_config/executors.html
s

Sébastien

12/09/2020, 10:22 PM
Oh. I thought registering multiple agents for a flow would allow them to pool on a flow. (Without using Dask — my mistake)
Thanks for the info
@josh Sorry to bug you again, but 
executor=DaskExecutor(adapt_kwargs={"maximum": 10})
didn't help, still running the map on a single worker - taking exactly as much time. Is there an easy way to add logs to check the executor?
The goal was to get the DaskExecutor running with a temporary cluster.
Tried 
executor=DaskExecutor(cluster_kwargs={"n_workers": 10})
as well thinking it would be simpler. It runs, but same issue as before.
Copy code
debug = true

[logging]
level = "DEBUG"
and
Copy code
executor=DaskExecutor(..., debug=True)
no debug logs to be found
Shut down all workers except one + deleted from Cloud, moved to LocalDaskExecutor, and tried using 
"processes": False, "threads_per_worker": 8
— exact same result.
(Yes, published the new version each time, the version counter is going up)
Matched settings to OS' CPU & thread count as reported by 
lscpu
just to make sure — no change.
Powered by