Aqib Fayyaz
11/16/2021, 6:48 PMhttps://www.youtube.com/watch?v=EwsMecjSYEU&t=332s▾
Tom Shaffner
11/17/2021, 4:53 PMprefect server start --expose --ui-port=80 --server-port=1720
), I can at least get to the UI then, but the UI gives me a failure message about not connecting to graphql and a link to this slack (second picture below). I've tried setting the graphql port flag 5060 or 443 or even also to 1720 or using the no port mapping flag (https://docs.prefect.io/api/latest/cli/server.html), all to no avail. I'm new to this so I'm probably doing something stupid but I can't figure out what; any thoughts?
In case it's related, I'm also unable to start a user agent. When I try prefect agent local start
I get requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=1720): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d60a34e20>: Failed to establish a new connection: [Errno 111] Connection refused'))
. I'm unclear on whether the server is automatically starting agents of its own or if I'd have to do this myself in addition to the server.
I could potentially have the option to open some additional ports, but it takes several days, a long approval process that might get rejected, and I'm not even clear on which I'd need to open or why. After all, these are mostly internal ports to the same machine, right?
I know that's a lot but I'm a bit at a loss here; any help?Aqib Fayyaz
11/18/2021, 10:58 AM[server]
host = "http://<EXTERNAL-IP>"
as mentioned in docs to point the cli to server. i have created the tenat using command
prefect backend server && prefect server create-tenant --name default --slug default
now when i open the server ui i shows the getting-started page and if i click on dashboardAqib Fayyaz
11/18/2021, 3:16 PMTom Klein
11/19/2021, 2:16 PMTom Klein
11/19/2021, 2:21 PMAnders Lindstrom
11/19/2021, 7:05 PMModule
storage in a Docker container. I've been trying to get it working for half a day now and I'm stumped. I'm sure it's a very simple problem but I'm unable to find any working examples as a basis.
So I have a flow that is installed (via pip install
) in a Docker image. The flow is called flows.docker_2
. It looks like this:
import prefect
from prefect import task, Flow
from prefect.run_configs import DockerRun
from prefect.storage import Module
@task
def say_hello():
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("Hello, docker 2!")
with Flow("docker_2_flow") as flow:
flow.storage = Module("flows.docker_2")
flow.run_config = DockerRun(
image="prefect_docker_2_image",
host_config={
"auto_remove": False
}
)
say_hello()
if __name__ == "__main__":
flow.register(project_name="docker_2_project", labels=["docker_flows"])
I am able to register the flow by executing it inside the container:
docker run --network prefect-server --rm --name prefect_docker_2 -it prefect_docker_2:latest python -m flows.docker_2
So I know that the module is installed and can be run inside the container.
However, when I run the flow, the module is not found. The docker agent reports:
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'flows'
Is there something else I have to do (other than `pip install`ing the module) to make the flow code visible?Ovo Ojameruaye
11/20/2021, 4:44 AMRyan Sattler
11/22/2021, 5:42 AMpostgresql: persistence: enabled:
from false
to true
in the chart and helm install
, the chart appears to install successfully as normal and the various server pods are created, but the Postgres pod never appears and there are no error messages, nothing in kubcetl get deployments
or kubectl describe replicasets
etc. With the value false
it works fine (except for the part where it loses all data when the pod dies). Does anyone know how to debug this?Sylvain Hazard
11/22/2021, 11:01 AMcreate_flow_run
paradigm to create multiple flow runs with differing parameters using map
?
Been trying it out and can't seem to make it work. Code below.Sylvain Hazard
11/22/2021, 2:13 PMwith Flow(name="Parent Flow") as flow:
docs = Parameters("docs", default=[1, 2, 3])
small_task = SmallTask().map(docs)
big_task = create_flow_run.map(flow_name=unmapped("Child flow"), project_name="whatever", parameters=small_task)
wait_big_task = wait_for_flow_run.map(big_task, raise_final_state=unmapped(True))
another_small_task = AnotherSmallTask().map(wait_big_task)
but I'm not sure how or where to define the result of a flow in order to use it downstream.Vaibhav Ariyur
11/22/2021, 3:00 PMvolkerjaenisch
11/22/2021, 11:14 PMvolkerjaenisch
11/22/2021, 11:16 PMvolkerjaenisch
11/22/2021, 11:17 PMDiego Alonso Roque Montoya
11/23/2021, 2:28 AMStéphan Taljaard
11/23/2021, 8:36 AMhome_dir = "/mnt/disks/prefect_data/.prefect"
to .prefect/config.toml
, but results are still saved on my main diskBruno Kuasney
11/23/2021, 10:26 AMmutation {
set_schedule_inactive(input: {flow_id: "67d364da-375d-4d0f-b151-d46435351c50"}) {
success
}
}
I get:
"message": "field \"set_schedule_active\" not found in type: 'mutation_root'"
I’m basically trying to use the query from the documentation hereSylvain Hazard
11/23/2021, 10:45 AMAlexis Lucido
11/23/2021, 3:58 PMScarlett King
11/23/2021, 5:45 PMLana Dann
11/23/2021, 7:45 PM"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateSecurityGroup",
"ec2:CreateTags",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs",
"ec2:DeleteSecurityGroup"
Joël Luijmes
11/24/2021, 11:09 AMif data_size > 1000000: # 1 mb max
raise ValueError("Artifact data payload exceedes 1Mb limit.")
Alexis Lucido
11/24/2021, 12:52 PMMariusz Olszewski
11/24/2021, 2:38 PMSylvain Hazard
11/24/2021, 3:16 PMget_task_run_result
to access data from a subflow.
This example should reproduce the issue :
from prefect import Flow, task, Parameter, unmapped
from prefect.tasks.prefect import create_flow_run, get_task_run_result
@task(slug="plus-one")
def plus_one(x: int):
return x+1
with Flow("Child") as child_flow:
x = Parameter("x", default=10)
po = plus_one(x)
@task
def get_params():
return [{"x": i} for i in range(5)]
with Flow("Parent") as parent_flow:
params = get_params()
flow_run_ids = create_flow_run.map(flow_name=unmapped("Child"), parameters=params)
results = get_task_run_result.map(flow_run_ids, unmapped("plus-one"))
When running the parent flow, I get issues that look like :
Error during execution of task: ValueError("No task runs found while querying for task runs where {'task': {'slug': {'_eq': 'plus-one'}}, 'flow_run_id': {'_eq': '3316e187-adda-461d-814f-29a6ffc059a4'}, 'map_index': {'_eq': -1}}")
Is there something I'm missing with how slugs work ?Aleksandr Glushko
11/24/2021, 4:34 PMwith Flow('flow_name') as flow:
a = prefect.Parameter('a', 'required'=True)
<http://logger.info|logger.info>(a)
how can i get the parameter value to be used? So i want a
to be 1, but I get: a = <Parameter: a>
Im running the flow in the following way:
prefect_client.graphql(query= """
mutation{
create_flow_run(input: { flow_id: %s , parameters: %s}) {
id
}
}
""" % (train_flow_id, json.dumps(flow_params).replace('"', '\"'))
)
where flow_params is a dictionarySaurabh Indoria
11/25/2021, 3:23 AMRyan Sattler
11/25/2021, 3:40 AMError: secret "prefect-server-postgresql" not found
. I thought this secret would be created automatically? Am I supposed to create it manually and if so how?Qwame
11/25/2021, 11:19 AMQwame
11/25/2021, 11:19 AMAnna Geller
11/25/2021, 11:25 AM