Matthias
07/29/2022, 8:03 AMMatthias
07/29/2022, 8:05 AMflurven
07/29/2022, 9:20 AMChristian
07/29/2022, 9:29 AMRajvir Jhawar
07/29/2022, 10:14 AMLeonardo Melo
07/29/2022, 11:57 AMDaniel Manson
07/29/2022, 12:29 PMGit
storage clones the full repo, so I'm trying to use that rather than the GitHub
storage as that looks to just clone the one flow file (having looked at the code). However, when using Git
storage I get the following error as the only log in my flow:
{'type': ['Unsupported value: Git']}
Tom Klein
07/29/2022, 12:55 PMJoshua Greenhalgh
07/29/2022, 1:04 PMcreate_flow_run
?
Want to do something like;James Phoenix
07/29/2022, 1:24 PMEli Treuherz
07/29/2022, 2:00 PMTRIGGERFAIL
passed to them as their first argument. If this sounds completely bizarre I can see if I can narrow down a minimal exampleR
07/29/2022, 2:13 PMSebastian
07/29/2022, 2:27 PMorchester
in which I organize all my orchestration code.
In Prefect 1.x the deployment pattern was akin to this:
from orchester.flows import hello_flow
hello_flow.storage = Module(__name__ + ".flows")
hello_flow.register()
In Prefect 2.x-beta I was able to achieve something similar like this:
from orchester.flows import hello_flow
Deployment(
name="entw",
flow=hello_flow,
packager=OrionPackager(serializer=ImportSerializer()),
)
# After that I created the deployment on the local Orion server with the CLI.
Now with Prefect 2.0.0 I am unsure what the deployment pattern would look like. My guess would be to run prefect deployment build .\orchester\flows\hello_flow.py:hello_flow --name entw
from the cli and to then manually edit the hello_flow-manifest.json
. I am not sure whether this is the the best approach or whether it needs to be done differently.
{
"flow_name": "hello_flow",
"import_path": ".\\orchester\\flows\\hello_flow.py:hello_flow", # Do I just need to change this line to a module path?
"parameter_openapi_schema": {
"title": "Parameters",
"type": "object",
"properties": {
"name": {
"title": "name",
"type": "string"
}
},
"required": [
"name"
],
"definitions": null
}
}
TLDR; Does anyone have a good suggestion how to deploy flows that live in their own Python module with Prefect 2.0.0?Zach Schumacher
07/29/2022, 2:54 PMMuhammad Mikaal Saleem Anwar
07/29/2022, 3:27 PMHafsa Junaid
07/29/2022, 3:43 PMJeffrey Lai
07/29/2022, 4:27 PMAndrew Richards
07/29/2022, 5:28 PMAccess to XMLHttpRequest at '<https://api.prefect.cloud/api/accounts/><MY ACCOUNT ID>/workspaces/<MY WORKSPACE ID>/block_documents/<MY BLOCK ID>' from origin '<https://app.prefect.cloud>' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
It seems like CORS may be unhappy with the UI trying to do a cross-domain request from app.prefect.cloud to api.prefect.cloud?Oleg Sheyner
07/29/2022, 5:29 PMJake
07/29/2022, 6:46 PMprefect.exceptions.ClientError: Malformed response received from API.
Here is the full stack trace:
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/cli/agent.py", line 305, in start
start_agent(KubernetesAgent, image_pull_secrets=image_pull_secrets, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/cli/agent.py", line 128, in start_agent
agent.start()
File "/usr/local/lib/python3.7/site-packages/prefect/agent/agent.py", line 186, in start
self._setup_api_connection()
File "/usr/local/lib/python3.7/site-packages/prefect/agent/agent.py", line 870, in _setup_api_connection
self.client.attach_headers({"X-PREFECT-AGENT-ID": self._register_agent()})
File "/usr/local/lib/python3.7/site-packages/prefect/agent/agent.py", line 831, in _register_agent
agent_config_id=self.agent_config_id,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 1788, in register_agent
tenant_id=self.tenant_id,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 264, in tenant_id
self._tenant_id = self._get_default_server_tenant()
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 176, in _get_default_server_tenant
response = self.graphql({"query": {"tenant": {"id"}}})
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 449, in graphql
retry_on_api_error=retry_on_api_error,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 405, in post
retry_on_api_error=retry_on_api_error,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 646, in _request
raise ClientError("Malformed response received from API.") from exc
prefect.exceptions.ClientError: Malformed response received from API.
Here is the .yml:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: prefect-agent
namespace: prefect
name: prefect-agent
spec:
replicas: 1
selector:
matchLabels:
app: prefect-agent
template:
metadata:
labels:
app: prefect-agent
spec:
containers:
- args:
- prefect agent kubernetes start --job-template "<s3://xxx>"
command:
- /bin/bash
- -c
env:
- name: PREFECT__CLOUD__AGENT__AUTH_TOKEN
value: ''
- name: PREFECT__CLOUD__API
value: <https://api.prefect.io>
- name: NAMESPACE
value: prefect
- name: IMAGE_PULL_SECRETS
- name: PREFECT__CLOUD__AGENT__LABELS
value: '[''xxx'', ''xxxxx'']'
- name: JOB_MEM_REQUEST
value: ''
- name: JOB_MEM_LIMIT
value: ''
- name: JOB_CPU_REQUEST
value: ''
- name: JOB_CPU_LIMIT
value: ''
- name: IMAGE_PULL_POLICY
value: Always
- name: SERVICE_ACCOUNT_NAME
value: ''
- name: PREFECT__BACKEND
value: cloud
- name: PREFECT__CLOUD__AGENT__AGENT_ADDRESS
value: http://:8080
- name: PREFECT__CLOUD__API_KEY
value: xxx
- name: PREFECT__CLOUD__TENANT_ID
value: xxx
- name: PREFECT__LOGGING__EXTRA_LOGGERS
value: "['xxxxx', 'xxxxxxxx']"
image: prefecthq/prefect:1.2.2-python3.7
imagePullPolicy: Always
livenessProbe:
failureThreshold: 2
httpGet:
path: /api/health
port: 8080
initialDelaySeconds: 40
periodSeconds: 40
name: agent
Ruggiero Dargenio
07/29/2022, 6:50 PMBen Muller
07/29/2022, 8:13 PMAll tasks must be called from within a flow. Tasks may not call other tasks directly.
Does this mean we can't call task.run() like in 1.0?
That was quite useful...Nova Westlake
07/29/2022, 8:51 PMNova Westlake
07/29/2022, 9:12 PMKevin Grismore
07/29/2022, 9:52 PMEOFError EOF when reading a line
Creating a profile for this Prefect Cloud login. Please specify a profile name: An exception occurred.
Error: Process completed with exit code 1.
I don't think it's possible for me to respond to a prompt like that via Actions. Is there a way around this?Jeffrey Lai
07/29/2022, 9:53 PMdatamongus
07/29/2022, 10:41 PMCA Lee
07/30/2022, 4:46 AMprefect deployment build
, the deployment.yaml
file is overwritten ?Surawut Jirasaktavee
07/30/2022, 7:21 PMSlackbot
07/30/2022, 8:03 PM