q

    Questionnaire

    2 years ago
    Hello folks, I'm trying to register my flow but my API status in local UI remains Attempting to connect... I'm sharing
    config.toml
    and UI screenshot. Please help. and getting an error.
    config.toml
    =>
    [server]
    host = "<http://localhost>"
    port = "4200"
    host_port = "4200"
    endpoint = "${server.host}:${server.port}"
    
    	[server.database]
    	host = "localhost"
    	port = "5432"
    	host_port = "5432"
    	name = "dev"
    	username = "root"
    	password = "password"
    	connection_url = "postgresql://${server.database.username}:${server.database.password}@${server.database.port}/${server.database.name}"
    
    	[server.graphql]
    	host = "0.0.0.0"
    	port = "4201"
    	host_port = "4201"
    	debug = true
    	path = "/graphql/"
    
    	[server.hasura]
    	host = "localhost"
        port = "3000"
        host_port = "3000"
    	admin_secret = ""
    	claims_namespace = "hasura-claims"
    	graphql_url = "http://${server.hasura.host}:${server.hasura.port}/v1alpha1/graphql"
    	ws_url = "ws://${server.hasura.host}:${server.hasura.port}/v1alpha1/graphql"
    	execute_retry_seconds = 10
    
    	[server.ui]
    	host = "<http://localhost>"
    	port = "8080"
    	host_port = "8080"
    	endpoint = "${server.ui.host}:${server.ui.port}"
    error
    =>
    Result check: OK
    Traceback (most recent call last):
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
        chunked=chunked,
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 426, in _make_request
        six.raise_from(e, None)
      File "<string>", line 3, in raise_from
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 421, in _make_request
        httplib_response = conn.getresponse()
      File "/usr/lib/python3.6/http/client.py", line 1356, in getresponse
        response.begin()
      File "/usr/lib/python3.6/http/client.py", line 307, in begin
        version, status, reason = self._read_status()
      File "/usr/lib/python3.6/http/client.py", line 268, in _read_status
        line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
      File "/usr/lib/python3.6/socket.py", line 586, in readinto
        return self._sock.recv_into(b)
    ConnectionResetError: [Errno 104] Connection reset by peer
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/tanmay/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
        timeout=timeout
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 725, in urlopen
        method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 403, in increment
        raise six.reraise(type(error), error, _stacktrace)
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
        raise value.with_traceback(tb)
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
        chunked=chunked,
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 426, in _make_request
        six.raise_from(e, None)
      File "<string>", line 3, in raise_from
      File "/home/tanmay/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 421, in _make_request
        httplib_response = conn.getresponse()
      File "/usr/lib/python3.6/http/client.py", line 1356, in getresponse
        response.begin()
      File "/usr/lib/python3.6/http/client.py", line 307, in begin
        version, status, reason = self._read_status()
      File "/usr/lib/python3.6/http/client.py", line 268, in _read_status
        line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
      File "/usr/lib/python3.6/socket.py", line 586, in readinto
        return self._sock.recv_into(b)
    urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "test.py", line 10, in <module>
        flow.register()
      File "/home/tanmay/.local/lib/python3.6/site-packages/prefect/core/flow.py", line 1429, in register
        no_url=no_url,
      File "/home/tanmay/.local/lib/python3.6/site-packages/prefect/client/client.py", line 640, in register
        version_group_id=version_group_id,
      File "/home/tanmay/.local/lib/python3.6/site-packages/prefect/client/client.py", line 218, in graphql
        token=token,
      File "/home/tanmay/.local/lib/python3.6/site-packages/prefect/client/client.py", line 178, in post
        token=token,
      File "/home/tanmay/.local/lib/python3.6/site-packages/prefect/client/client.py", line 304, in _request
        response = <http://session.post|session.post>(url, headers=headers, json=params, timeout=30)
      File "/home/tanmay/.local/lib/python3.6/site-packages/requests/sessions.py", line 578, in post
        return self.request('POST', url, data=data, json=json, **kwargs)
      File "/home/tanmay/.local/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
        resp = self.send(prep, **send_kwargs)
      File "/home/tanmay/.local/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
        r = adapter.send(request, **kwargs)
      File "/home/tanmay/.local/lib/python3.6/site-packages/requests/adapters.py", line 498, in send
        raise ConnectionError(err, request=request)
    requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
    @Jim Crist-Harif can you help me in this?
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    Hello @Questionnaire! My guess would be that the graphql service is not up, how are you starting the rest of the services for server? Docker? If so what is the output of
    docker ps
    , or one step further, of the
    docker logs
    command against the graphql container?
    q

    Questionnaire

    2 years ago
    hey @Laura Lorenz (she/her) , I'm doing
    prefect server start
    yeah which lifts docker file only, So,
    docker ps
    outputs:
    CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                                     NAMES
    eaa352a76c5a        prefecthq/ui:0.11.2            "/intercept.sh"          2 hours ago         Up 2 hours          80/tcp, 0.0.0.0:8080->8080/tcp            tmp_ui_1
    63ea0676e7da        prefecthq/server:0.11.2        "python src/prefect_…"   2 hours ago         Up 2 hours                                                    tmp_scheduler_1
    1fd04acd8f1c        prefecthq/apollo:0.11.2        "npm run serve"          2 hours ago         Up 2 hours          0.0.0.0:4200->4200/tcp                    tmp_apollo_1
    4fb46e7d56ec        prefecthq/server:0.11.2        "bash -c 'prefect-se…"   2 hours ago         Up About a minute   0.0.0.0:4201->4201/tcp                    tmp_graphql_1
    0d82aa8247ee        hasura/graphql-engine:v1.1.0   "graphql-engine serve"   2 hours ago         Up About a minute   0.0.0.0:3000->3000/tcp                    tmp_hasura_1
    c432bfe4de63        postgres:11                    "docker-entrypoint.s…"   2 hours ago         Up 2 hours          0.0.0.0:5432->5432/tcp
    @Laura Lorenz (she/her)
    docker logs
    outputs:
    {"type":"pg-client","timestamp":"2020-05-21T12:39:27.199+0000","level":"warn","detail":{"message":"postgres connection failed, retrying(0)."}}
    {"type":"pg-client","timestamp":"2020-05-21T12:41:38.206+0000","level":"warn","detail":{"message":"postgres connection failed, retrying(1)."}}
    {"internal":"could not connect to server: Connection timed out\n\tIs the server running on host \"5432\" (0.0.21.56) and accepting\n\tTCP/IP connections on port 5432?\n","path":"$","error":"connection error","code":"postgres-error"}
    {"type":"startup","timestamp":"2020-05-21T12:41:44.529+0000","level":"info","detail":{"kind":"server_configuration","info":{"live_query_options":{"batch_size":100,"refetch_delay":1},"transaction_isolation":"ISOLATION LEVEL READ COMMITTED","plan_cache_options":{"plan_cache_size":100},"enabled_log_types":["http-log","websocket-log","startup","webhook-log"],"server_host":"HostAny","enable_allowlist":false,"log_level":"info","auth_hook_mode":null,"use_prepared_statements":true,"unauth_role":null,"stringify_numeric_types":false,"enabled_apis":["metadata","graphql","config","pgdump"],"enable_telemetry":true,"enable_console":true,"auth_hook":null,"jwt_secret":null,"cors_config":{"allowed_origins":"*","disabled":false,"ws_read_cookie":null},"console_assets_dir":null,"admin_secret_set":false,"port":3000}}}
    {"type":"startup","timestamp":"2020-05-21T12:41:44.529+0000","level":"info","detail":{"kind":"postgres_connection","info":{"retries":1,"database_url":"<postgresql://root>:...@5432/data_model_dev"}}}
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    Cool thanks @Questionnaire! From the graphql logs it looks like graphql can’t start because it can’t connect to postgres. Either the postgres container is sick, or possibly the configuration is off. Is there any weird output from your postgres container’s logs that indicate it wasn’t able to start up all the way? Can you connect to your postgres container with the credentials from your config.toml (with another client like with pgadmin GUI or psql?)
    q

    Questionnaire

    2 years ago
    Let me check one
    @Laura Lorenz (she/her) I confirmed the connection is settuped using client and ran a postgres task also using
    flow.run()
    method.
    Hey @Laura Lorenz (she/her), Let me tell you what I did:
    pip install prefect
    Put 
    config.toml
     in 
    ~/.prefect
    Then 
    prefect backend server
     after that 
    prefect server start
    . And my flow is:
    import prefect
    from prefect import task, Flow
    
    @task
    def hello_task():
        logger = prefect.context.get("logger")
        <http://logger.info|logger.info>("Hello, Cloud!")
    
    flow = Flow("hello-flow", tasks=[hello_task])
    flow.register()
    and my
    .toml
    is already here in thread for ur reference.
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    @Questionnaire Awesome thank you! With all that I can try and reproduce with your situation locally for me as a next step. I’,m not 100% sure this is the issue but I feel like something is up between how
    prefect server start
    configured your postgres server against your configuration: I’m suspicious because config.toml specifies that the databsae should be called
    dev
    but the graphql output looks like its trying to access a db called
    data_model_dev
    Ok I can reproduce your issue! Going to mess around with it for a bit
    @Questionnaire Ok I think it’s a typo in your config.toml! server.database.connection_url is missing the host interpolation right now it is:
    connection_url = "postgresql://${server.database.username}:${server.database.password}@${server.database.port}/${server.database.name}"
    but it should be:
    connection_url = "postgresql://${server.database.username}:${server.database.password}@${server.database.host}:${server.database.port}/${server.database.name}"
    If I change that then I go from having your same issue, to it all working! I do think you might have an env var that is overwriting your database name in your config.toml in your local terminal, based on your logs. If you run
    env | grep -i data_model_dev
    in the terminal you were starting prefect server from, are there any matches?
    q

    Questionnaire

    2 years ago
    env | grep -i data_model_dev
    it returns me nothing on terminal
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    Ok maybe I’m paranoid then 😅, if you don’t mind updating your config.toml with the new connection_url and giving it a try, we can just see if it’s ok now and if not, send me some the new docker logs please 🙂
    q

    Questionnaire

    2 years ago
    AHH, no no I updated it also .... @Laura Lorenz (she/her)
    Is it necessary to name dbname=
    dev
    ???
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    Technically the only necessary thing is to have config.toml to have the name the same as a database that exists in your postgres container. FWIW when the postgres container first starts up from an image it will create a database off of the name that is in config.toml at that time so if you are changing config.toml between container starts it might be confusing it.
    q

    Questionnaire

    2 years ago
    Yeah, so I've db in my postgres with name
    data_model_dev
    that's why I put the name as same.... I was just asking to confirm that is this not the case where graphql only search for
    dev
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    Gotcha, no it’s not hardcoded, it will pull it off of your config.toml’s server.database.name, so even if you made a database called
    super_awesome_database
    independently in your postgres instance, if you change config.toml and restart graphql it will connect to
    super_awesome_database
    instead
    q

    Questionnaire

    2 years ago
    yeah @Laura Lorenz (she/her), So now my docker logs is saying this:
    ":{"x-hasura-role":"admin"},"request_id":"fb1e6596-af31-4e13-9987-b715d7cac239","response_size":40,"request_read_time":1.2532e-5},"http_info":{"status":200,"http_version":"HTTP/1.1","url":"/v1alpha1/graphql","ip":"192.168.48.5","method":"POST","content_encoding":"gzip"}}}
    {"type":"http-log","timestamp":"2020-05-21T15:42:49.798+0000","level":"info","detail":{"operation":{"query_execution_time":5.558238e-3,"user_vars":{"x-hasura-role":"admin"},"request_id":"cc3eb829-b154-4570-961c-b9bbfe332ad3","response_size":44,"request_read_time":9.29e-6},"http_info":{"status":200,"http_version":"HTTP/1.1","url":"/v1alpha1/graphql","ip":"192.168.48.5","method":"POST","content_encoding":"gzip"}}}
    and api is connected also
    yaay
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    Yay awesome!!!
    q

    Questionnaire

    2 years ago
    @Laura Lorenz (she/her) you're STAR🌟
    @Laura Lorenz (she/her) Can you just tell me one thing like I've to pass parameters in my flow... How will I pass them after registering?
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    @Questionnaire Depends how you are going to start the flow runs! You can pass them when you call flow.run() if you are starting it from a python script (see https://docs.prefect.io/core/concepts/parameters.html#parameters), when you start the flow from the UI (the “Run” button NOT the “Quick Run” button - see https://docs.prefect.io/orchestration/ui/flow.html#run), or you can add default parameters to your clocks if you are going to schedule the flow (see https://docs.prefect.io/core/concepts/schedules.html#complex-schedules).
    q

    Questionnaire

    2 years ago
    Ok, @Laura Lorenz (she/her)
    flow.register()
    CronClock('0 0 * * *', parameter_defaults={
      "key:: "value"
    }).events()
    Is this a right way to add it???
    Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    You need to attach the schedule to the flow before registering it. It looks like there is some syntax errors in your parameter defaults dictionary, and you don’t need to call the events method too. Here’s an example with an IntervalClock with no parameters, but hopefully it helps with the gist: https://github.com/PrefectHQ/prefect/blob/master/examples/github_release_cycle.py#L40-L45
    q

    Questionnaire

    2 years ago
    Thanks @Laura Lorenz (she/her) 🙂