Ajith Kumara Beragala Acharige Lal

    Ajith Kumara Beragala Acharige Lal

    1 year ago
    Hi Prefect experts , am trying to deploy a workflow on prefect-cloud setup on our Kubernetes cluster, when I run
    prefect auth create-token -n my-runner-token -s RUNNER
    i get this error
    Traceback (most recent call last):
    File "/home/ajith/.local/lib/python3.8/site-packages/prefect/client/client.py", line 451, in _request
    json_resp = response.json()
    File "/usr/lib/python3/dist-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
    File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 518, in loads
    return _default_decoder.decode(s)
    File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
    File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
    simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
    The above exception was the direct cause of the following exception:
    Traceback (most recent call last):
    File "/home/ajith/.local/bin/prefect", line 8, in <module>
    sys.exit(cli())
    File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
    File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
    File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
    File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
    File "/home/ajith/.local/lib/python3.8/site-packages/prefect/cli/auth.py", line 214, in create_token
    output = client.graphql(
    File "/home/ajith/.local/lib/python3.8/site-packages/prefect/client/client.py", line 298, in graphql
    result = <http://self.post|self.post>(
    File "/home/ajith/.local/lib/python3.8/site-packages/prefect/client/client.py", line 213, in post
    response = self._request(
    File "/home/ajith/.local/lib/python3.8/site-packages/prefect/client/client.py", line 454, in _request
    raise ClientError(
    prefect.utilities.exceptions.ClientError: Malformed response received from Cloud - please ensure that you have an API token properly configured.
    setup is like this
    agents registered
    any idea, why i cannot create a token on this cloud ?
    Chris White

    Chris White

    1 year ago
    Hi Ajith, this looks like you are mixing backends; Prefect Server does not have any concept of users or tokens
    Ajith Kumara Beragala Acharige Lal

    Ajith Kumara Beragala Acharige Lal

    1 year ago
    Hi @Chris White - thanks, if I understand your answer correct, - We cannot setup "cloud" mode on docker, only the "server" mode ?
    @Chris White - and I tried to run a simple ShellTask after switching back to
    server
    then i get this error
    requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: /graphql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4fcbcc5130>: Failed to establish a new connection: [Errno 111] Connection refused'))
    Chris White

    Chris White

    1 year ago
    No, that’s not what I mean; Prefect consists of two components: • Prefect Core: the workflow engine / builder that you write your workflows with and test them / run them locally • A Prefect Backend / API: this is an API that your workflows can be configured to talk to; this API is where persistence of state and other interesting orchestration features happens. You have two mutually exclusive options for the API: you can self host the open source version of the API (“Prefect Server”), and in doing so you need to make sure both your agents and the workflows they submit have network access to this API that you have stood up. Alternatively, you can use Prefect Cloud, which is the API that we manage. Your issues are resulting from either trying to use Cloud settings with a Server API or vice versa, or because your agents / flows do not have network access to the Server API that you have created.
    Ajith Kumara Beragala Acharige Lal

    Ajith Kumara Beragala Acharige Lal

    1 year ago
    Thank you very much @Chris White, sorry to bugging you - if agent is on docker - how can I register a flow ? ( i have prior experience when agent is running on a machine instead docker) , any example/or documentation?
    Chris White

    Chris White

    1 year ago
    No worries at all! I totally understand your confusion - there are a lot of moving pieces, and we’re definitely trying to streamline our documentation to make these things easier. The key thing is that you need to make sure the docker containers have network access to the API - give me a minute and I can try to find some docs / an example for you
    Ajith Kumara Beragala Acharige Lal

    Ajith Kumara Beragala Acharige Lal

    1 year ago
    Thanks a lot @Chris White - if I'm not mistaken agent and server has established the communication - http://161.35.244.65:8080/default?agents=
    I guess , i don't know where to execute and register the script
    hi @Chris White, It'd be really great if you can give some reference today, as the my Jira task related to this for this almost met the deadline 🙏🙏🙏🙏
    Chris White

    Chris White

    1 year ago
    Hi @Ajith Kumara Beragala Acharige Lal — it looks like you are running the kubernetes agent, which should be much simpler because Kubernetes should be handling the networking heavy lifting; here is a helm chart template for the k8s agent against a server deployment that you might want to follow: https://github.com/PrefectHQ/server/blob/master/helm/prefect-server/templates/agent/deployment.yaml For your flow, I recommend using
    Docker
    storage in this particular instance with a registry that your k8s cluster has access to
    Ajith Kumara Beragala Acharige Lal

    Ajith Kumara Beragala Acharige Lal

    1 year ago
    Thanks a lot @Chris White, i'll get back to you with results ... Good night!
    Chris White

    Chris White

    1 year ago
    For sure — good luck!!