Nitin Madhavan
09/05/2021, 4:50 AMHari Krishna Sunkari
09/05/2021, 5:24 PM/Users/hari/Library/Python/3.8/bin/prefect server start
im getting the below error
Docker version 20.10.8, build 3967b7d
Docker Compose version v2.0.0-rc.2
⠋ Container _postgres_1 Creating 0.0s
Error response from daemon: Invalid container name (_postgres_1), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed
Hari Krishna Sunkari
09/05/2021, 5:25 PM/Users/hari/Library/Python/3.8/bin/prefect server config
Hari Krishna Sunkari
09/05/2021, 5:26 PMIf high availability is important for your instance it is encouraged that you inquire about a community maintained deployment method
Morgan Yu
09/05/2021, 7:37 PMArun Dass
09/06/2021, 6:25 AMprefect server config > docker-compose.yaml
prefect server has errors but when i run
prefect server start
its fine. Do these two commands create the same docker compose file?Alfie
09/06/2021, 1:47 PMprefect.utilities.exceptions.ClientError: Flow run ID not found: "3b51f006-cbbe-459c-b74f-6a832f22db96"
Michael Hadorn
09/06/2021, 1:57 PMC:\Users\hadornm\Miniconda3\envs\prefect-3.9\python.exe S:/cdwh-prefect/cdwh_flow_register.py
[2021-09-06 15:56:30+0200] INFO - prefect | Do DEV Register with image: xxx/xxx:1.x.x
Traceback (most recent call last):
File "S:\cdwh-prefect\cdwh_flow_register.py", line 91, in <module>
flow.register(project_name='cdwh ' + env_name, idempotency_key=flow_hash)
File "C:\Users\hadornm\Miniconda3\envs\prefect-3.9\lib\site-packages\prefect\core\flow.py", line 1727, in register
registered_flow = client.register(
File "C:\Users\hadornm\Miniconda3\envs\prefect-3.9\lib\site-packages\prefect\client\client.py", line 1122, in register
project = self.graphql(query_project).data.project # type: ignore
File "C:\Users\hadornm\Miniconda3\envs\prefect-3.9\lib\site-packages\prefect\client\client.py", line 548, in graphql
result = <http://self.post|self.post>(
File "C:\Users\hadornm\Miniconda3\envs\prefect-3.9\lib\site-packages\prefect\client\client.py", line 451, in post
response = self._request(
File "C:\Users\hadornm\Miniconda3\envs\prefect-3.9\lib\site-packages\prefect\client\client.py", line 737, in _request
response = self._send_request(
File "C:\Users\hadornm\Miniconda3\envs\prefect-3.9\lib\site-packages\prefect\client\client.py", line 647, in _send_request
response.raise_for_status()
File "C:\Users\hadornm\Miniconda3\envs\prefect-3.9\lib\site-packages\requests\models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 405 Client Error: Not Allowed for url: <https://xxx.xx:443/>
Is this a known error?Mikkel Antonsen
09/07/2021, 1:18 PMTomás Emilio Silva Ebensperger
09/07/2021, 7:07 PMSatyam Tandon
09/07/2021, 7:24 PM_run
creates a new key by default inside the context object so it can be available to every task that is added to the flow
class MyFlow(prefect.Flow):
def _run(
self,
parameters: Dict[str, Any],
runner_cls: type,
run_on_schedule: bool,
**kwargs: Any
) -> "prefect.engine.state.State":
context = dict(my_variable="my_variable")
return super()._run(
parameters,
runner_cls,
run_on_schedule=run_on_schedule,
context=context,
**kwargs
)
This works as intended when I call .run
and the task is able to access that key but it doesn’t work if I register the flow with a local ui and agent and try running it from the UI. Why is that and how would I make it work if I need to register a flow and run it from the UI?
@task(name="log-my-variable")
def log_run_id():
prefect.context.get("logger").info(f"{prefect.context.my_variable}")
with MyFlow("experimental-flow") as flow:
result = log_run_id()
if __name__ == "__main__":
# flow.run()
flow.register(project_name="test-flows")
sean williams
09/07/2021, 11:11 PMprefect backend server
on an M1 mac? It seemed to have issues with hasura and graphql, so I killed the process after about 30 mins. ThanksBouke Krom
09/08/2021, 9:04 AMMathias Herberts
09/08/2021, 9:36 AMGaylord Cherencey
09/08/2021, 10:58 PMinject
labels in a flow based on a project it is in. A bit of context, I want to run a flow on a Kubernetes cluster namespace based on the project it is part of (e.i each flow in a project A
should run on namespace A
) and I would like to use the agent affinity but don't enforce the user to specify those labels.Gani Simsek
09/09/2021, 4:01 PMAiden Price
09/10/2021, 2:16 AMMarko Jamedzija
09/10/2021, 7:10 AMThomas Fredriksen
09/10/2021, 8:02 AMFlow
class so that all tasks will report state to an external service. From what I undersand, subclassing the Flow
class may not be a good idea as the prefect server will not use the subclass when managing the flows.
I like that prefect has support for state change hooks (state_handlers
), and I was wondering if I can use such a hook to modify the tasks of all tasks of a flow in order to add tasks state handlers that will report the task state change.
Something like this:
def task_state_handler(task, old_state, new_state):
report_state(task, old_state, new_state)
def flow_state_handler(flow, old_state, new_state):
was_pending = old_state.is_queued() or old_state.is_scheduled() or old_state.is_pending() or old_state.is_submitted()
if was_pending and new_state.is_running():
for task in flow.tasks:
task.state_handlers.append(task_state_handler)
Marcel Gutsche
09/10/2021, 9:32 AMdocker.StartContainer
task. Thanks for your help!Mikkel Antonsen
09/10/2021, 12:28 PMLawrence Finn
09/10/2021, 2:29 PMWill Milner
09/10/2021, 3:59 PMFailed to load and execute Flow's environment: ModuleNotFoundError("No module named '/root/'")
Sam Werbalowsky
09/10/2021, 5:17 PMraise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='<my-deployment-name>-apollo.<namespace>', port=4200): Max retries exceeded with url: /graphql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb5d1bd6510>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Tenant pod logs ending with…:
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd0c274e990>: Failed to establish a new connection: [Errno 111] Connection refused'))
Not sure which settings need to be updated in the values.yaml file, but it’s gotta be something…Kevin Mullins
09/10/2021, 9:47 PMprefect server start
not working due to trying to create an invalid container name _postgres_1
?
I can’t figure out what changed, it worked last time I started the server a couple of days ago and now I can’t get it to work.Gani Simsek
09/13/2021, 10:19 AMLuca Schneider
09/13/2021, 5:26 PMSantiago Gonzalez
09/13/2021, 6:52 PMFilip Lindvall
09/14/2021, 7:53 AMMikkel Antonsen
09/14/2021, 9:09 AMMikkel Antonsen
09/14/2021, 9:09 AMKevin Kho
09/14/2021, 2:01 PMMikkel Antonsen
09/15/2021, 11:05 AMKevin Kho
09/15/2021, 1:54 PM