justin
06/18/2020, 9:19 PM#prefect_test.py
import prefect
from prefect import task, Flow
@task
def hello_task():
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("Hello, Cloud!")
flow = Flow("hello-flow", tasks=[hello_task])
flow.register()
flow.run_agent()
I've got my prefect server running. I started with the following command: prefect server start --hasura-port=3050
.
So I then see that when I run the prefect_test.py file above it submits the flow and I have a url where I want to go run the flow. Something like this: <http://localhost:8080/flow/6e437b8f-9fc2-41c1-9862-311d9a423442>
I follow this link to use the UI to run the flow and I have an empty screen.
Anything obvious I am missing?Darragh
06/18/2020, 10:27 PMOleksandr Lobunets
06/18/2020, 10:38 PMMark McDonald
06/19/2020, 1:23 AMcreate_cloud_hook
mutation?
I'm getting the following error:
{
"graphQLErrors": [
{
"path": [
"create_cloud_hook"
],
"message": "Expected type JSON, found {to: \"<mailto:test@test.com|test@test.com>\"}; 'ObjectValueNode' object has no attribute 'value'",
"extensions": {
"code": "INTERNAL_SERVER_ERROR",
"exception": {
"message": "Expected type JSON, found {to: \"<mailto:test@test.com|test@test.com>\"}; 'ObjectValueNode' object has no attribute 'value'",
"locations": [
{
"line": 2,
"column": 146
}
],
"path": null
}
}
}
],
"networkError": null,
"message": "GraphQL error: Expected type JSON, found {to: \"<mailto:test@test.com|test@test.com>\"}; 'ObjectValueNode' object has no attribute 'value'"
}
I'm pulling the mutation straight out of the docs here
This was previously working for me.Emmanuel Klinger
06/19/2020, 8:34 AMitay livni
06/19/2020, 12:40 PMjustin
06/19/2020, 4:18 PMjustin
06/19/2020, 4:18 PMCreateContainer(image_name='vgg',
detach=True,
ports={'5000/tcp': 5050},
name=container_name,
)
justin
06/19/2020, 4:19 PMjustin
06/19/2020, 4:19 PMERROR - prefect.TaskRunner | Unexpected error: TypeError("__init__() got an unexpected keyword argument 'ports'",)
justin
06/19/2020, 4:19 PMJeremiah
james.lamb
06/19/2020, 4:35 PMIs there any way to write a custom Storage class that extends one of the Prefect built-in Storage classes?Background: I'm trying to extend prefect.environments.storage.S3 , to add some custom logic. I don't need to change anything in the interface, but I want to change the behavior of one or more methods in a my-use-case-specific way. Code samples and more context in thread.
Jeremiah
Darragh
06/19/2020, 6:21 PM0.12.0
release:
• Has something changed with the FlowRunTask? We were using without any issues up until we upgraded to 0.12.0 , but now our flow fails looking for a project_name. I can see in the code this is specific to backed being cloud, but we set ‘prefect backend server’ before starting the server and haven’t had issues with it before..
• I had asked about a week ago about DFE for mapped tasks on Fargate, and was told it would be in 0.12.0, but doesn’t appear to be working for us - mapped tasks are still running sequentially…Jeremiah
Kai Weber
06/21/2020, 1:18 PMDaniel Kim
06/21/2020, 2:37 PMCody Webb
06/22/2020, 1:37 AMJacob Blanco
06/22/2020, 11:06 AMNo value for argument 'fn' in function callpylint(no-value-for-parameter)
It happens on lines:
@task(name="Something")
As an error in VScode when working with my flow, any ideas on how to get it to stop showing me that?Rafal
06/22/2020, 12:05 PMDarragh
06/22/2020, 3:48 PMFrank Lind
06/22/2020, 6:57 PMFrank Lind
06/22/2020, 6:57 PMFrank Lind
06/22/2020, 6:58 PMJackson Maxfield Brown
06/22/2020, 7:30 PMDarragh
06/22/2020, 9:01 PMdocker_server_url
but I’ve no idea what that might be for this case.
The error I get is:
DockerException("Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))",)
Which traces back to line 302 in containers.py
- there are some threads out there that suggest /`var/run` is the right answer, but no luck with itMark McDonald
06/22/2020, 9:52 PMFailed to set task state with error: ClientError([{'path': ['set_task_run_states'], 'message': 'State update failed for task run ID a8c171e6-49fc-4370-90ee-ae9319f5848a: provided a running state but associated flow run 5ab28d3b-0dad-4327-8063-f019e5cbbdd5 is not in a running state.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}])
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/cloud/task_runner.py", line 119, in call_runner_target_handlers
cache_for=self.task.cache_for,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 1085, in set_task_run_state
version=version,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 225, in graphql
raise ClientError(result["errors"])
prefect.utilities.exceptions.ClientError: [{'path': ['set_task_run_states'], 'message': 'State update failed for task run ID a8c171e6-49fc-4370-90ee-ae9319f5848a: provided a running state but associated flow run 5ab28d3b-0dad-4327-8063-f019e5cbbdd5 is not in a running state.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]
it's preceded by an error related to the LocalResultHandler like this (this flow is still on version 0.10.2 of core):
Task 'DQ Test - postgres extractions and test creation[40]': unexpected error while running task: FileNotFoundError(2, 'No such file or directory')
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 271, in run
state=state, upstream_states=upstream_states
File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 590, in get_task_inputs
handler
File "/usr/local/lib/python3.7/site-packages/prefect/engine/result/base.py", line 294, in to_result
value = self.result_handler.read(self.value)
File "/usr/local/lib/python3.7/site-packages/prefect/engine/result_handlers/local_result_handler.py", line 62, in read
with open(fpath, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/src/prefect-result-2020-06-22t04-16-48-214966-00-00'
Any guidance as to what's going on here? Sometimes the flow runs just fine without these errors.Peter
06/23/2020, 12:01 AMflow.run()
. I've read through the documentation on logging which recommends you use prefect's utility function get_logger()
to hook into a flow's logs, which works fine for creating a new application but would be a ton of work for us to change our existing applications that use python's standard logging practice - essentially defining logger = logging.getLogger(__name___)_
at the top of the module. Unfortunately, when we call flow.run()
with tasks that invoke our existing application that uses standard logging it looks like prefect disables those logs. Is there any way to tell prefect to log from existing applications that use python's standard logging.getLogger
instead of prefect's get_logger
utility?Peter
06/23/2020, 12:01 AM