https://prefect.io logo
Powered by
Title
s

Steve Aby

03/08/2021, 5:44 PM
Hello Everyone....Newbie to Slack and newbie to prefect as well. I am having an issue, probably simple, but can't solve it. I have a python prefect script that runs fine when I run from pyCharm or the CLI. I now want to get that script to run on the backend server and use the UI but having a problem. I have the backend server and UI installed and I can access that without any issues. I have created my first project there. I have modified my python script replacing flow.run(executor=DaskExecutor()) with flow.register(project_name="FirstFlow") in an attempt to register my flow from python and then was going to execute from the UI When I try to run that script, I keep getting the error: "requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://localhost:4200/graphql". I can access graphql from that port when I try through Chrome but python is giving me the error. 1) I also attempted the basic tutorial and had the same issue. Am I missing another step to just run the script using the backend server and UI?
d

Dylan

03/08/2021, 5:49 PM
Hi @Steve Aby! Can you share the script you’re running to register the flow?
s

Steve Aby

03/08/2021, 6:28 PM
Thanks Dylan. Here is one example. It runs fine if executed as is on python 3.7.9 from CLI or from PyCharm Professional 2020.3.3. If I comment out the flow.run run and uncomment the flow.register line, I receive the "requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://localhost:4200/graphql" error. I get that error regardless of whether I have the UI running or not. I assume I am missing some very basic config option
teststuff.py
d

Dylan

03/08/2021, 6:32 PM
Do you have an instance of Prefect Server running on that same machine? Using 
prefect server start
or the like?
s

Steve Aby

03/08/2021, 6:32 PM
I should also add that I am running this on Windows 10 and do have Docker desktop installed. I can start the UI without any issues
I have started the server using that command
actually. I use prefect backend server and then prefect server start before I start the UI....Let me do that and try running the script again just to be sure
yes, if I execute those two commands and then run the script, it works fine for flow.run and I get the 4200 error for flow.register
I also noticed on the console log that it is using port 4201 which I assume is in addition to 4200
d

Dylan

03/08/2021, 6:44 PM
flow.run()
doesn’t communicate with the orchestration layer, so that makes sense
k

Kyle Moon-Wright

03/08/2021, 7:19 PM
Hey @Steve Aby - what version of Prefect are you running? Also, are you setting 
prefect backend server
where you are running the python script?
s

Steve Aby

03/08/2021, 7:31 PM
hi Kyle.  I am executing prefect backend server in a shell window and not in the python code.  I am on prefect 0.14.10
k

Kyle Moon-Wright

03/08/2021, 7:41 PM
Very strange, I’m unable to reproduce this error with the information provided. This detail in particular is of interest, but I’m unsure of what to make of it:
I get that error regardless of whether I have the UI running or not.
As far as I can tell, you aren’t missing any steps so something is afoot! I will keep digging.
Can you post the full error traceback you’re seeing from registering the script?
s

Steve Aby

03/08/2021, 7:50 PM
Sure thing...thanks for your help......:\Users\abyst\AppData\Local\Continuum\anaconda3\python.exe C:/Users/abyst/Documents/Source/Python/SonicLedgerPrefect/teststuff.py
Traceback (most recent call last):
  File "C:/Users/abyst/Documents/Source/Python/SonicLedgerPrefect/teststuff.py", line 29, in <module>
    main()
  File "C:/Users/abyst/Documents/Source/Python/SonicLedgerPrefect/teststuff.py", line 24, in main
    flow.register(project_name="FirstFlow")
  File "C:\Users\abyst\AppData\Local\Continuum\anaconda3\lib\site-packages\prefect\core\flow.py", line 1675, in register
    idempotency_key=idempotency_key,
  File "C:\Users\abyst\AppData\Local\Continuum\anaconda3\lib\site-packages\prefect\client\client.py", line 788, in register
    project = self.graphql(query_project).data.project  # type: ignore
  File "C:\Users\abyst\AppData\Local\Continuum\anaconda3\lib\site-packages\prefect\client\client.py", line 304, in graphql
    retry_on_api_error=retry_on_api_error,
  File "C:\Users\abyst\AppData\Local\Continuum\anaconda3\lib\site-packages\prefect\client\client.py", line 220, in post
    retry_on_api_error=retry_on_api_error,
  File "C:\Users\abyst\AppData\Local\Continuum\anaconda3\lib\site-packages\prefect\client\client.py", line 460, in _request
    session=session, method=method, url=url, params=params, headers=headers
  File "C:\Users\abyst\AppData\Local\Continuum\anaconda3\lib\site-packages\prefect\client\client.py", line 375, in _send_request
    response.raise_for_status()
  File "C:\Users\abyst\AppData\Local\Continuum\anaconda3\lib\site-packages\requests\models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <http://localhost:4200/graphql>
Process finished with exit code 1
k

Kyle Moon-Wright

03/08/2021, 8:05 PM
And you said this will register no problem via the CLI with something like 
prefect register flow -f teststuff.py -p FirstFlow
?
The only other thing I can think of is updating the 
requests
(I’m on 2.25.1) to double check that module from the traceback, as this must be coming from the Python client?
s

Steve Aby

03/08/2021, 8:16 PM
When I try to register from the CLI, I get an error that is a ValueError("No flow found in file."
if I take the flow context and the register out of the python routine and just leave it naked in the module, I do get the same 404 error as before with the 4200/graphql
as far as requests, I am on 2.25.1 as well
z

Zanie

03/08/2021, 9:19 PM
Hi @Steve Aby. When you open the graphql url in chrome do you see an interactive API page? Can you share the output of 
docker ps
?
s

Steve Aby

03/08/2021, 9:38 PM
attached are three screenshots that I hope give you what you asked for. One is my chrome screenshot of graphql and two relate to Docker Desktop.
here is the output of docker ps
docker_ps.txt
z

Zanie

03/08/2021, 9:52 PM
Thanks! That all looks right to me. Very confusing.
Perhaps https://github.com/psf/requests/issues/879 ?
Perhaps run 
set no_proxy=localhost
in a shell then try registering your flow?
s

Steve Aby

03/08/2021, 9:57 PM
I will give that a try
that didn't help but I will read the article. I also have another PC with python 3.8 that has no proxy settings that I can try this on with a clean install of prefect
thanks for you help
z

Zanie

03/08/2021, 10:06 PM
Another one of our engineers will take a look when they're free as well
s

Steve Aby

03/08/2021, 10:11 PM
Thanks! One other piece of data. When I start the UI, it won't let me add a new project. I enter the name in the dialog box and then it says it look like your project wasnt added, please try again. When I switch to the cloud, it validates my identity and will let me add a new project
n

nicholas

03/08/2021, 11:13 PM
Hi @Steve Aby - thanks for working through this with us; could you share what you see on the connection menu in the top right of the Server UI? The menu will look something like this:
In addition - could you confirm that you don't have any prefect-specific environment variables set (they would begin with 
PREFECT__SERVER__*
) and send over the contents of your config file (located at 
~/.prefect/config.toml
)? Please redact any sensitive info beforehand since this is a public channel 🙂
s

Steve Aby

03/08/2021, 11:19 PM
HI as far as the connection menu, I can go back and forth between cloud and server. When I go to Cloud, it connects me through Google successfully and lets me add projects. If I go to Server, there is no recognition of my identity and it says Connected to Prefect Server at http://localhost:4200/graphql
n

nicholas

03/08/2021, 11:20 PM
Ah! Perfect. So that part is set correctly. Have you run 
prefect server create-tenant --name <<tenant_name>> --slug <<tenant_slug>>
?
s

Steve Aby

03/08/2021, 11:20 PM
config,toml
no I have not.....I will try that now......and getting the config file
👍 1
ok I see two toml files. One is in the site-packages and the other is in a src file that came when I downloaded the tutorial from github
config.toml from tutorials.toml
config.toml from site packages.toml
config.toml from site packages.toml
also, when I run from the CLI 
prefect server create-tenant --name abyst --slug myslug
I get an error from requests 404 Client error: Not Found for url: http://localhost:4200/graphql
Also, I have not set any environment variables
z

Zanie

03/09/2021, 12:49 AM
When you get a chance can you try running 
python -c "import requests; requests.get('<http://localhost:4200/graphql').raise_for_status()>"
and 
python -c "import requests; requests.get('<http://localhost:8080/').raise_for_status()>"
and (if you have curl) 
curl '<http://localhost:4200/graphql>' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: <http://localhost:4200>' --data-binary '{"query":"# Write your query or mutation here\n"}' --compressed
s

Steve Aby

03/09/2021, 1:14 AM
the first python cmd failed with the same 404 error saying could not find localhost:4200 ; the second python command ran with no error
z

Zanie

03/09/2021, 1:19 AM
Okay that narrows down that this is not an issue with 
requests
since it can contact the UI fine. Sounds like the apollo service has silently failed or something. We'll keep trying to figure out what's going on here...
s

Steve Aby

03/09/2021, 1:22 AM
as far as the curl command, it did not like that giving me this error: Invoke-WebRequest : Cannot bind parameter 'Headers'. Cannot convert the "Accept-Encoding: gzip, deflate, br" value of type "System.String" to type "System.Collections.IDictionary".
At line:1 char:41
+ ... calhost:4200/graphql' -H 'Accept-Encoding: gzip, deflate, br' -H 'Con ...
+                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Invoke-WebRequest], ParameterBindingException
    + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
I didnt change the "query":"write your......let me know if I was supposed to put something elsae there
I also just ran curl 'http//localhost:4200/graphql' and that failed with a 404 Not Found error
but if I run http://localhost:4200/graphql from Chrome, it does open the screen
Thanks to everyone for looking at this today. I have some good news but not really an explanation so if you want some more information about my setup tomorrow, I will be glad to help. Since I wasn't paying close attention tonight, let me tell you what I think happened, I had three windows open on my laptop: 1) powershell which was running the server; 2) Chrome which had the UI; 3) pyCharm which had my scripts. I also had the TV on so this is why I am a little fuzzy. Everything was basically how it had been all day. I shut down pycharm first and then noticed on the shell prompt I was seeing the prefect logo. This was not something I had seen before. I then went to Chrome and was able to interact more on that than I had been able to before. I was able to create a new project and access Account. I was also able to start my test script. I have to do more to understand if it worked but this was total success. My only guess is that something in pyCharm was conflicting with one of the services. I will try to recreate in the morning but wanted to share the "good" news. Thanks again for all of your help. This is an annoying issue.
z

Zanie

03/09/2021, 2:47 PM
Very weird, thanks for reporting back Steve!
We'll try to reproduce something like this on Windows ourselves but yeah it's hard to say what would have held it up like that
s

Steve Aby

03/09/2021, 3:23 PM
I did a little more research but I don't know how valuable it is. Using a program called tcpview, I do see pycharm, along with Docker listening to port 4200. In trying to understand why pycharm may be doing that, it seems related to Angular. I do have npm and node installed on my pc as well as the angular project framework which is one of my default project templates. I don't know as of now how to turn that off. For now, I am going to run my scripts through the CLI or UI if I can and use pycharm only to edit the code. Thanks again for all of your help!
3 Views
#prefect-uiJoin Slack
Powered by