Shailendra Thakur
01/31/2021, 11:43 PMPeyton Runyan
02/01/2021, 1:10 PMPREFECT__CONTEXT__SECRETS__GITHUB_ACCESS_TOKEN= <my_token>
The flow is
storage = GitHub(
repo="my_repo",
path="simple_flow.py",
secrets=["GITHUB_ACCESS_TOKEN"]
)
with Flow("github_flow", storage=storage) as f:
If I remove storage and run it, it runs fine. With storage, it hangs indefinitely on scheduled
.
Any ideas?Anton Rasmussen
02/01/2021, 1:45 PMTraceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 451, in _request
json_resp = response.json()
File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/prefect", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/cli/execute.py", line 49, in flow_run
result = client.graphql(query)
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 298, in graphql
result = <http://self.post|self.post>(
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 213, in post
response = self._request(
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 454, in _request
raise ClientError(
prefect.utilities.exceptions.ClientError: Malformed response received from Cloud - please ensure that you have an API token properly configured.
Peter Roelants
02/02/2021, 10:25 AMPREFECT_SERVER__LOGGING__LEVEL: WARN
and PREFECT__LOGGING__LEVEL: WARN
environment variables to the docker-compose file, however I still see INFO
log messages.Scott Jones
02/02/2021, 6:46 PMRui Rego
02/02/2021, 8:10 PMfrom prefect.agent.local import LocalAgent
After that, all scheduled executions of the flow are failing with:LocalAgent().start()
Failed to load and execute Flow's environment: StorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'src\'")\nThis may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')What am I doing wrong? I am using Windows 10, python 3.9 and a virtual environment (venv).
BK Lau
02/02/2021, 10:30 PMDavid Kuda
02/03/2021, 4:42 PMSean Talia
02/03/2021, 5:49 PMprefect server start
just fine the other day, and in the last 10-15 minutes when I've tried running it, i'm getting sporadic errors for the various docker compose services that need to be pulled. one time i might get ERROR: for apollo unauthorized: authentication required
, another time i get ERROR: for postgres unauthorized: authentication required
, a 3rd time i get the same for graphql
, or some combination of the abovejosh
02/03/2021, 6:25 PMNikul
02/03/2021, 6:38 PMJosh Greenhalgh
02/03/2021, 8:02 PMkevin
02/03/2021, 10:47 PMTimo
02/04/2021, 8:21 AMflow.register( idempotency_key=flow.serialized_hash(),...)
. If I register my flows with this method, then, if I change the code of a single task, it creates no new version. Only if I make a change at the flow itself. Is it possible to make a new version if the code of the task changes (but not every time, like without idempotency_key)? ThanksBob Primusanto
02/04/2021, 9:49 AMUnexpected error: AttributeError("'StartFlowRun' object has no attribute 'run_config'")
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 867, in get_task_run_state
logger=self.logger,
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 298, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 453, in method
kwargs.setdefault(attr, getattr(self, attr))
AttributeError: 'StartFlowRun' object has no attribute 'run_config'
Michael Hadorn
02/04/2021, 10:57 AM> prefect get flow-runs
NAME FLOW NAME STATE AGE START TIME ID
spiffy-scorpion Load Scheduled 13 minutes ago 17a13e07-d217-4a10-a821-5947657a1a48I use the local agent. In the UI can see the agent and also if I stop the UI, the agent complains. Also the prefect server start env is used for the whole setup. Would be very amazing to get some help. Thanks
Rui Rego
02/04/2021, 2:57 PMUnexpected error: TypeError("cannot pickle 'weakref' object")
Traceback (most recent call last):
File "d:\302_investing\venv\lib\site-packages\prefect\engine\runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "d:\302_investing\venv\lib\site-packages\prefect\engine\task_runner.py", line 891, in get_task_run_state
result = self.result.write(value, **formatting_kwargs)
File "d:\302_investing\venv\lib\site-packages\prefect\engine\results\local_result.py", line 116, in write
value = self.serializer.serialize(new.value)
File "d:\302_investing\venv\lib\site-packages\prefect\engine\serializers.py", line 70, in serialize
return cloudpickle.dumps(value)
File "d:\302_investing\venv\lib\site-packages\cloudpickle\cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "d:\302_investing\venv\lib\site-packages\cloudpickle\cloudpickle_fast.py", line 563, in dump
return Pickler.dump(self, obj)
TypeError: cannot pickle 'weakref' object
Why the different behavior between Core and Server?Samuel Hinton
02/04/2021, 3:35 PMprefect local agent start --label any
in my terminal, it works beautifully. (Edit: Actually now when I run it its stuck on “Waiting for flow runs”). Inside a docker container (using local or docker agents) - connects fine by then always says “No flow runs found”. This is the same whether I run a docker or local agent in the docker-compose, with and without attaching labels to anything, with both local storage and S3, and with local, universal and docker run configs explicitly set.Joseph
02/04/2021, 6:27 PMflow.run()
) it successfully completes and using diagnostic tools for my cluster I can see the jobs complete. I can also register the flow with the server (flow.register
). But when I quick run the flow, from the UI’s perspective it looks as if it never completes (it is stuck in the “running” state). Using my cluster diagnostics I can see the actual jobs get scheduled and complete. So I’d guess whatever sort of events are meant to be sent to the server in order to indicate state changes in the jobs are not being sent? How can I diagnose what’s happening?Neeraj Vyas
02/05/2021, 8:50 AMJoao Erik Melo
02/05/2021, 8:27 PMwith Flow("conditional-branches") as flow:
cond1 = check_condition1()
with case(cond1, True):
val1 = task1()
cond2 = check_condition2()
with case(cond2, True):
val2 = task2()
cond3 = check_condition3()
with case(cond3, True):
val3 = task3()
flow.set_dependencies(
task=val1,
downstream_tasks=[val2])
flow.set_dependencies(
task=val2,
downstream_tasks=[val3])
Sagun Garg
02/08/2021, 4:29 AM$ prefect backend server && prefect server create-tenant --name default --slug default
This is the error I get on my CLI
raise ClientError(result["errors"])
prefect.utilities.exceptions.ClientError: [{'message': '[Errno -2] Name or service not known', 'locations': [{'line': 2, 'column': 5}], 'path': ['create_tenant'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': '[Errno -2] Name or service not known'}}}]
I am getting this error in my prefect-graphql container
2021-02-07 01:57:13[Errno -2] Name or service not known7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:13GraphQL request:2:37e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:131 | mutation ($input: create_tenant_input!) {7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:132 | create_tenant(input: $input) {7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:13| ^7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:133 | id7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:13Traceback (most recent call last):7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:13File "/usr/local/lib/python3.7/site-packages/graphql/execution/execute.py", line 628, in await_result7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:13return await result7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:13File "/prefect-server/src/prefect_server/graphql/extensions.py", line 52, in resolve7e86527e64b44aa992ab290af4938b13
2021-02-07 01:57:13result = await result7e86527e64b44aa992ab290af4938b13
I have setup prefect-server on AWS Fargate ECS using the docker-compose.yml as a guidleine to create my own task definition where I am doing a multi-container setup with ENV defined in Key, Values of each container.
I think there is some issue in ENV Hasura Admin Secret in prefect-graphql, prefect-apollo as I see some errors in headers while the tenant is being created. So I have removed the Hasura Admin password from my prefect-hasura container. I am still getting the above errors.
ANY HELP IN THIS REGARD WILL BE APPRECIATED...I THINK I AM 1 STEP AWAY FROM SEEING PREFECT-SERVER RUN AGAIN 🙂 I HAD IT RUNNING BEFORE BUT HIT THIS ISSUE.Mathijs Schoorl
02/08/2021, 1:00 PMChris Goddard
02/08/2021, 2:12 PMDana Merrick
02/08/2021, 4:39 PMOops! It looks like something went wrong when trying to connect; make sure Prefect Server is running at the URL above and try again.
Edward Chen
02/09/2021, 9:15 AMName or service not known
error message. I confirmed that GraphQL is not running based on the error message on the top right of the prefect UI.
Stack trace:
apollo_1 | Checking GraphQL service at <http://graphql:4201/health> ...
apollo_1 | Checking GraphQL service at <http://graphql:4201/health> ...
graphql_1 |
graphql_1 | Running Alembic migrations...
graphql_1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
graphql_1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
graphql_1 |
graphql_1 | Could not upgrade the database!
graphql_1 | Error: HTTPConnectionPool(host='hasura', port=3000): Max retries exceeded with url: /v1/query (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x40133ab7d0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
apollo_1 | Checking GraphQL service at <http://graphql:4201/health> ...
Based on the stack trace, it confirms that there is an issue with graphql, I’m just not sure how to dive deeper. I have tried Dagster and initially came across some issues with compatibility with the M1 chip so I created an EC2 instance and tried the prefect first flow tutorial in that one as well. I hit an issue with docker compose not being found so rather than splitting into two separate paths, I’m going to try to focus on getting the first flow tutorial working on my local instance (macos)Joël Luijmes
02/09/2021, 11:02 AMBen Epstein
02/09/2021, 10:34 PMrequirements.txt
or conda.yaml
with a job? Thanks!Michael Hadorn
02/10/2021, 7:49 PMKeyError: 'Task slug XX not found in the current Flow; this is usually caused by changing the Flow without reregistering it with the Prefect API.'
Best
MichaelSi Wong
02/11/2021, 8:00 AM