Dear Prefect Community.
I am new to prefect and I'...
# prefect-servert
Todd Lindstrom
04/22/2021, 9:25 PMDear Prefect Community.
I am new to prefect and I'm evaluating it vs Airflow and the why-not-airflow paper on the website was very convincing but my first experience was not positive - thus reaching out here because I dont want to give up on a technology that might just make my life easier later by giving up too early. I will paste the startup error in this thread - but the problem is on prefect server start - I am getting connection errors, and then later when I pull up the UI :8080 I get to the getting started screen but the graphql connect fails with more docker error logs scrolling. I can move the UI question to the other room if this is the wrong place.
Todd Lindstrom
04/22/2021, 9:27 PMhere is docker images:
toddlindstrom@Todds-Mac-mini rn % docker psTodd Lindstrom
04/22/2021, 9:30 PMfollowed by my startup error:
towel_1 reports error can not connect to 172.22.0.3
That is not my public IP on my mac - I am guessing that is a docker IP
Todd Lindstrom
04/22/2021, 9:30 PM Copy code
towel_1 | {"severity": "ERROR", "name": "prefect-server.HasuraClient", "message": "Encountered internal API exception: [Errno 111] Connect call failed ('172.22.0.3', 3000)", "exc_info": "Traceback (most recent call last):\n File \"/prefect-server/src/prefect_server/utilities/exceptions.py\", line 87, in reraise_as_api_error\n yield\n File \"/prefect-server/src/prefect_server/utilities/graphql.py\", line 64, in execute\n timeout=30,\n File \"/usr/local/lib/python3.7/site-packages/httpx/_client.py\", line 1385, in post\n timeout=timeout,\n File \"/usr/local/lib/python3.7/site-p.......Todd Lindstrom
04/22/2021, 9:34 PMlater I get a graphql_1 error connection refused.
z
Zanie
04/22/2021, 9:34 PM
Hey @Todd Lindstrom -- how much memory do you have allocated to Docker Desktop?
t
Todd Lindstrom
04/22/2021, 9:35 PM4 CPUs 2.0 GB swap 1GB
Todd Lindstrom
04/22/2021, 9:35 PMjust default
Todd Lindstrom
04/22/2021, 9:36 PMi have a brand new apple chip based mac mini
Todd Lindstrom
04/22/2021, 9:48 PMi changed mem to 4GB with no effect - so I deleted all containers and images and I'm pulling them fresh again as a test.
z
Zanie
04/22/2021, 10:38 PM
Did that resolve it? I had to bump the default memory -- 1GB is definitely not enough for the full Server stack.
Zanie
04/22/2021, 10:38 PM
Can you provide the full output of
prefect server startprefect diagnosticst
Todd Lindstrom
04/22/2021, 11:44 PMtoddlindstrom@Todds-Mac-mini /tmp % prefect diagnostics
{
"config_overrides": {},
"env_vars": [],
"system_information": {
"platform": "macOS-11.2.3-arm64-arm-64bit",
"prefect_backend": "server",
"prefect_version": "0.14.16",
"python_version": "3.9.4"
}
}
Todd Lindstrom
04/23/2021, 12:22 AMmessage has been deleted
Todd Lindstrom
04/23/2021, 12:37 PMtowel_1 process is trying to connect to 172.19.0.3 address? Is that something that docker arranges internally? that is the first error i'm getting towel_1 cant connect to the 172 address port 3000
z
Zach Angell
04/23/2021, 12:57 PM
Hi @Todd Lindstrom I ran into similar errors when I was setting up server (hasura exiting with code 137). Bumping up docker memory to 8 GB solved the issue for me. Could you try that?
t
Todd Lindstrom
04/23/2021, 12:57 PMsure
Todd Lindstrom
04/23/2021, 12:58 PMone other thing i saw that python 3.9 was experimental but my install of prefect is installing into my /opt/homebrew/bin folder and it is requiring python 3.9
Todd Lindstrom
04/23/2021, 12:58 PMis that a possible problem
z
Zach Angell
04/23/2021, 12:58 PM
yeah I would recommend downgrading python and using your favorite python environment manager (conda, pipenv, pyenv, etc)
t
Todd Lindstrom
04/23/2021, 12:59 PMi'm very new to mac - i istalled python with brew - is that a mistake ?
z
Zanie
04/23/2021, 1:02 PM
It kind of looks like there are 3 graphql services running
Zanie
04/23/2021, 1:02 PM
In that log, there are three separate colors and each one is attempting to run database migrations
Zanie
04/23/2021, 1:03 PM
I'd recommend not forcefully stopping on exit, not all of the containers are torn down then.
Zanie
04/23/2021, 1:03 PM
Can you kill all your docker containers with
docker kill $(docker ps -q)prefect server startupvote 1
z
Zach Angell
04/23/2021, 1:05 PM
no brew is great for global python on your machine, for specific python versions or projects I'd recommend a python package manager
I use
conda Copy code
$ conda create -n "py38prefect" python=3.8
$ conda activate py38prefect
$ (py38prefect) # pip install stuff in the shell nowt
Todd Lindstrom
04/23/2021, 1:11 PMok i am battling getting python 3.8 installed and reinstall prefect - I will let you know how it goes
👍 1
Todd Lindstrom
04/23/2021, 2:45 PMinstalled conda - installed python 3.8 and then prefect server start worked ok and I've created my first flow. Mission 1 accomplished - now to see if it will work for us.
😄 1
z
Zanie
04/23/2021, 2:46 PM
Yay! Let us know if you need any more help