Severin Ryberg [sevberg]
02/12/2021, 1:24 AMLaura Vaida
02/12/2021, 8:48 AMINFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
does somebody know, what might be a reason for that?Nikola Milushev
02/12/2021, 5:28 PMschedule = Schedule(clocks=[IntervalClock(timedelta(days=1))])
with Flow(name=name, schedule=schedule) as flow:
start_date = Parameter(name="start_date",
required=False,
default=(datetime.now() - timedelta(days=2)).strftime("%Y-%m-%d"))
end_date = Parameter(name="end_date",
required=False,
default=(datetime.now() - timedelta(days=1)).strftime("%Y-%m-%d"))
Robin
02/12/2021, 7:19 PM{'type': ['Missing data for required field.']}
as single log directly in the beginning of a flow that directly fails.
What could be possible reasons and did you encounter the same? 😮
PS: the flow runs locally but throws this error on Prefect Cloud 🤔Danny Vilela
02/12/2021, 7:20 PM@task
decorator is lacking a positional argument. For example:
@task
def add_one(x: int) -> int:
return x + 1
@task()
def add_two(x: int) -> int:
return x + 2
The @task
decorator over add_one
is how I’ve seen it in the docs, but it seems like the decorator for add_two
is how PyCharm likes it. Am I missing something?Alex Welch
02/12/2021, 7:45 PM{'type': ['Unsupported value: UniversalRun']}.
It works locally. Here is my code:
import prefect
from prefect import task, Flow
@task
def say_hello():
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("Hello, Cloud!")
with Flow("hello-flow") as flow:
say_hello()
flow.register(project_name="tutorial")
Felipe Montefuscolo
02/12/2021, 8:52 PMMatthew Blau
02/12/2021, 9:09 PMitay livni
02/14/2021, 11:22 PMStartFlowRun
is run from within another flow.
Unexpected error: ClientError('400 Client Error: Bad Request for url: https://api.prefect.io/graphql\nThis is likely caused by a poorly formatted GraphQL query or mutation but the response could not be parsed for more details')
Traceback (most recent call last):
File "/home/ilivni/miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/client/client.py", line 361, in _send_request
response.raise_for_status()
File "/home/ilivni/miniconda3/envs/py38lmap/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.prefect.io/graphql
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/engine/task_runner.py", line 865, in get_task_run_state
value = prefect.utilities.executors.run_task_with_timeout(
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/utilities/executors.py", line 299, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/utilities/tasks.py", line 454, in method
return run_method(self, *args, **kwargs)
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/tasks/prefect/flow_run.py", line 177, in run
flow_run_id = client.create_flow_run(
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/client/client.py", line 1094, in create_flow_run
res = self.graphql(create_mutation, variables=dict(input=inputs))
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/client/client.py", line 298, in graphql
result = <http://self.post|self.post>(
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/client/client.py", line 213, in post
response = self._request(
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/client/client.py", line 445, in _request
response = self._send_request(
File "miniconda3/envs/py38lmap/lib/python3.8/site-packages/prefect/client/client.py", line 374, in _send_request
raise ClientError(f"{exc}\n{graphql_msg}") from exc
prefect.utilities.exceptions.ClientError: 400 Client Error: Bad Request for url: <https://api.prefect.io/graphql>
This is likely caused by a poorly formatted GraphQL query or mutation but the response could not be parsed for more details
Thanks in advanceRob Fowler
02/15/2021, 4:58 AMsark
02/15/2021, 8:59 AM@prefect.task
def log(item):
<http://prefect.context.logger.info|prefect.context.logger.info>(item)
this works when you pass in the string output of an upstream task as the entirety of the input to the logging task
log(output)
but what if i want to do something like
log("the output of the previous task was: " + output)
here we have an issue because +
is not a task and so we will actually be logging something like
the output of the previous task was: Task<blah blah>
Ruben
02/15/2021, 10:32 AMVitaly Shulgin
02/15/2021, 12:17 PMFailed to load and execute Flow's environment: StorageError("An error occurred while unpickling the flow:\n DisplayNameError('0.0')")
Matthew Blau
02/15/2021, 3:39 PMDavid Clements
02/16/2021, 12:20 AMMatthew Alhonte
02/16/2021, 12:31 AMJacob Blanco
02/16/2021, 6:03 AMrafaqat ali
02/16/2021, 7:43 AMJonas Sølvsteen
02/16/2021, 9:13 AMkromash
02/16/2021, 9:49 AMPrashant Kumar
02/16/2021, 1:07 PMAlex Welch
02/16/2021, 2:17 PMValueError: Failed to infer default networkConfiguration, please explicitly configure using --run-task-kwargs
error. I have tried all teh different combinations I can think of and been through the docs (both prefect and ECS) but cant find a solution that works. Has anyone else been able to solve this?
export networkConfiguration="{'awsvpcConfiguration': {'assignPublicIp': 'ENABLED', 'subnets': ['<EC2 instance subnet>'], 'securityGroups': ['<EC2 instance security group>']}}"
prefect agent ecs start --token $RUNNER_TOKEN_FARGATE_AGENT \
--task-role-arn=arn:aws:iam::<AWS ACCOUNT ID>:role/ECSTaskS3Role \
--log-level INFO --label prod --label s3-flow-storage \
--name prefect-prod
Vincent
02/16/2021, 2:36 PMSamuel Hinton
02/16/2021, 3:36 PMPREFECT__SERVER__UI__PORT
and PREFECT__SERVER__UI__HOST_PORT
to try and follow the config.toml, but the UI service is still launching and saying “👾👾👾 UI running at localhost:8080 👾👾👾” and I dont know what else to change in the UI service 😞 As a follow up, what is the difference between the UI port and the UI host port?
EDIT: I see echo UI running at localhost:8080
is hardcoded in ./start_server.sh in the UI service, is this meant to be the case?David Clements
02/16/2021, 5:00 PMJohn Grubb
02/16/2021, 5:33 PMmap
on case
?
with case.map(should_do_the_thing, True):
does not agree...Samuel Hinton
02/16/2021, 6:48 PMPeyton Runyan
02/16/2021, 8:21 PMsql_task = SqlServerFetch(
db_name=prefect.config.sql_server.database,
user=prefect.config.sql_server.user,
host=prefect.config.sql_server.server,
query=get_manual_override_rows,
fetch='many',
fetch_count=3,
result=result_formatter,
name="Database Query"
# commit: bool = False,
)
with Flow("config_flow") as f:
mode = Parameter("Mode", default="dev")
password_var = set_creds(mode)
password_task = EnvVarSecret()
password = password_task(password_var)
s = sql_task(password=password)
v = view_sql(sql_task)
Suchindra
02/16/2021, 9:16 PMLocalDaskExecutor
My flow looks something like this:
list_of_files = Parameter()
list_of_frames = extract_fn.map(list_of_files) # This is indeed getting executed in parallel
How can I pass both the result of the previous mapped fn and also the input parameters to my transform fn? I understand that I could probably refactor the transform_fn to avoid using the list_of_files
but wanted to know if there is any other way of handling this.
transform_fn.map(list_of_frames, list_of_files)
As you can tell I am very new to prefect. Thanks a lot!Peyton Runyan
02/16/2021, 10:58 PM