William Grim10/30/2021, 7:26 PM
? I've got a very long-running flow, and I don't see any output. Is prefect buffering? Do I need to use
prefect agent local start -l mylabel --show-flow-logs 2>&1 | tee /my/file/here.log
when running the process?
Mike Lev10/30/2021, 7:44 PM
mr Memedi10/30/2021, 10:13 PM
Fina Silva-Santisteban10/31/2021, 2:37 PM
Eric Feldman10/31/2021, 2:57 PM
to a task, but it failed when querying GraphQL since
isn’t json serializable
I can pass
~/Library/Caches/pypoetry/virtualenvs/recipe-t7nMXg6Y-py3.8/lib/python3.8/site-packages/prefect/client/client.py in graphql(self, query, raise_on_error, headers, variables, token, retry_on_api_error) 550 server=self.api_server, 551 headers=headers, --> 552 params=dict(query=parse_graphql(query), variables=json.dumps(variables)), 553 token=token, 554 retry_on_api_error=retry_on_api_error,
and deserialize it, but I rather not
Todd de Quincey10/31/2021, 5:39 PM
Harish10/31/2021, 10:16 PM
Daniel Siles11/01/2021, 9:15 AM
Execution time: 31.85s Is there a way to reduce the overhead of a task?
@task() def transform_url(url): try: if '?' in url: return url[:url.find('?')] except Exception: pass return url urls = #list with 1k urls with Flow("flow") as flow: transformed_urls = transform_url.map(urls) import time start = time.time() flow.run() print(time.time() - start)
Marko Herkaliuk11/01/2021, 9:27 AM
Eric Feldman11/01/2021, 12:23 PM
Adam Brusselback11/01/2021, 2:18 PM
Jean-Michel Provencher11/01/2021, 3:00 PM
Joe11/01/2021, 3:40 PM
. I tried to create a project in the UI, but I get an "uh-oh" error - looking at the GraphQL response I can see the error is "Variable "$tenantId" of non-null type "UUID!" must not be null.". I think a tenant is a team, and if I try to update the team name from the profile page I see a similar GraphQL error: "Field update_tenant_name_input.tenant_id of required type UUID! was not provided". If I try to create a project using the CLI (
prefect server config
), I get the error: "You have not set an API key for authentication" - I can't see a way to setup an API key that doesn't seem to related to Prefect Cloud? If I try to create a tenant with the CLI (
prefect create project "Test Project"
) I get: "To create a tenant with Prefect Cloud, please signup at ...". Have I somehow missed creating the 'default' team - is there a way I can fix that?
prefect server create-tenant --name="Test"
Payam K11/01/2021, 3:42 PM
Joe Still11/01/2021, 5:31 PM
Vamsi Reddy11/01/2021, 6:01 PM
@task(log_stdout=True, max_retries=3, retry_delay=datetime.timedelta(minutes=5),state_handler=[post_to_slack]) def some_task(): return something
Andrey Kozhevnikov11/01/2021, 6:20 PM
Guozheng Kuang11/01/2021, 6:23 PM
Vamsi Reddy11/01/2021, 6:37 PM
Phil11/01/2021, 10:01 PM
If I got it right, Task D only gets a reference to the previous task object instead of the reduced list of DataFrames. The same logic with the task decorator, on the other hand, works without any problems. There is no difference in both logics, which is why I am quite irritated by the different results. Please, can someone help me out? It's probably just a small thing, but I just can't find it 😕
TypeError: Task is not iterable. If your task returns multiple results, pass `nout` to the task decorator/constructor, or provide a `Tuple` return-type annotation to your task.
Donny Flynn11/01/2021, 10:25 PM
The task failed with this error message: `Error during execution of task: ValueError("Could not infer an active Flow context while creating edge to <Task: LessThan>. This often means you called a task outside a
with Flow(FLOW_NAME, storage=STORAGE, run_config=RUN_CONFIG,) as flow: Task_with_secret_pulling()
block. If you're trying to run this task outside of a Flow context, you need to call `LessThan(...).run(...)`")`. Then I brought all secrets to the flow level which created them as separate tasks, but I am curious: is it the Prefect intention to have each "get" of a Secret be a separate task?
Todd de Quincey11/02/2021, 6:36 AM
Charles Hunt11/02/2021, 9:04 AM
Marcos San Miguel11/02/2021, 9:41 AM
Klemen Strojan11/02/2021, 11:14 AM
Ondřej Melichar11/02/2021, 11:32 AM
Billy McMonagle11/02/2021, 2:28 PM
haf11/02/2021, 2:32 PM