• s

    SK

    1 year ago
    [xxxxxserverxxxxxx]$ python prefect_sample.py Traceback (most recent call last): File "prefect_sample.py", line 5, in <module> import prefect ImportError: No module named prefect
    s
    Michael Adkins
    +1
    16 replies
    Copy to Clipboard
  • b

    BK Lau

    1 year ago
    Q: Can Prefect Agent be set up to run Hashicorp
    Nomad
    as executor instead of
    Dask
    ?? I want to use Prefect on the Cloud to drive a
    Nomad
    cluster installed on-premise
    b
    j
    2 replies
    Copy to Clipboard
  • m

    Matic Lubej

    1 year ago
    Hi again! I'm trying to run a process over a Fargate cluster using
    dask_cloudprovider
    API. running tutorials and sample code from
    dask
    this works great, but for prefect I have created a dedicated docker image which I provide to the cluster initializer. The cluster gets created, but as soon as the flow starts, after 10 s I get the following time-out error:
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/prefect/engine/runner.py", line 48, in inner
        new_state = method(self, state, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/prefect/engine/flow_runner.py", line 418, in get_flow_run_state
        with self.check_for_cancellation(), executor.start():
      File "/usr/local/lib/python3.8/contextlib.py", line 113, in __enter__
        return next(self.gen)
      File "/usr/local/lib/python3.8/site-packages/prefect/executors/dask.py", line 203, in start
        with Client(self.address, **self.client_kwargs) as client:
      File "/usr/local/lib/python3.8/site-packages/distributed/client.py", line 748, in __init__
        self.start(timeout=timeout)
      File "/usr/local/lib/python3.8/site-packages/distributed/client.py", line 953, in start
        sync(self.loop, self._start, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/distributed/utils.py", line 340, in sync
        raise exc.with_traceback(tb)
      File "/usr/local/lib/python3.8/site-packages/distributed/utils.py", line 324, in f
        result[0] = yield future
      File "/usr/local/lib/python3.8/site-packages/tornado/gen.py", line 762, in run
        value = future.result()
      File "/usr/local/lib/python3.8/site-packages/distributed/client.py", line 1043, in _start
        await self._ensure_connected(timeout=timeout)
      File "/usr/local/lib/python3.8/site-packages/distributed/client.py", line 1100, in _ensure_connected
        comm = await connect(
      File "/usr/local/lib/python3.8/site-packages/distributed/comm/core.py", line 308, in connect
        raise IOError(
    OSError: Timed out trying to connect to <tcp://172.31.40.184:8786> after 10 s
    [2021-01-19 20:14:19+0000] ERROR - prefect.Execute process | Unexpected error occured in FlowRunner: OSError('Timed out trying to connect to <tcp://172.31.40.184:8786> after 10 s')
    Traceback (most recent call last):
      File "s3_process_l2a_2019.py", line 114, in <module>
        assert status.is_successful()
    AssertionError
    Any ideas what is going on? Is the dask scheduler having issues connecting to the workers? Or might it be something else? Thanks!
    m
    j
    2 replies
    Copy to Clipboard
  • Billy McMonagle

    Billy McMonagle

    1 year ago
    I have a flow with docker storage, running with the k8s agent. When I run my flow and look at the logs, it seems to be executing an old version of my flow. Details in thread...
    Billy McMonagle
    j
    +2
    72 replies
    Copy to Clipboard
  • Chris Jordan

    Chris Jordan

    1 year ago
    Hey, I've got a quick question about persisting temporary files between tasks. It looks like if I pass a
    tmpfile
    object between tasks, it'll work as expected, as in
    @task(name="save records to a tmp file")
    def save_some_data(result=PrefectResult()):
        logger = prefect.context.get("logger")
        the_file = tempfile.NamedTemporaryFile()
        for i in range(5000000):
            the_file.write(b"lorem ipsum\n")
        the_file.seek(0)
        <http://logger.info|logger.info>("wrote the file")
        return the_file
    
    @task(name="reread that data")
    def read_some_data(the_file, result=PrefectResult()):
        logger = prefect.context.get("logger")
        output = the_file.read()
        <http://logger.info|logger.info>(f"read the file")
        <http://logger.info|logger.info>(f"length of file is {len(output)}")
    
    
    with Flow("save_retrieve_file_flow") as flow:
        f = save_some_data()
        g = read_some_data(f)
    but passing the name of the file won't work - the file will not be found
    @task(name="save records to a tmp file")
    def save_some_data(result=PrefectResult()):
        logger = prefect.context.get("logger")
        the_file = tempfile.NamedTemporaryFile()
        for i in range(500000):
            the_file.write(b"lorem ipsum\n")
        the_file.seek(0)
        <http://logger.info|logger.info>(f"wrote the file to {the_file.name}")
        return the_file.name
    
    @task(name="reread that data")
    def read_some_data(the_file_name, result=PrefectResult()):
        logger = prefect.context.get("logger")
        <http://logger.info|logger.info>(f"trying to open {the_file_name}")
        
        with open(the_file_name, 'r') as the_file:
            output = the_file.read()
            <http://logger.info|logger.info>(f"read the file")
            <http://logger.info|logger.info>(f"length of file is {len(output)}")
    
    
    with Flow("save_retrieve_file_flow2", 
            state_handlers=[cloud_only_slack_handler]
           ) as flow:
        f = save_some_data()
        g = read_some_data(f)
    what's going on here? is the file system being reset between tasks? does prefect clean up temporary files in the flow if they're not in memory directly? something else?
    Chris Jordan
    Michael Adkins
    10 replies
    Copy to Clipboard
  • h

    Hui Zheng

    1 year ago
    Hello Prefect, One our flow run got an Connection Error at the start of the flow run, which we never seen before. Do you know what caused that?
    HTTPSConnectionPool(host='10.124.0.1', port=443): Max retries exceeded with url: /apis/batch/v1/namespaces/default/jobs (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa7a7705668>: Failed to establish a new connection: [Errno 111] Connection refused',))
    h
    2 replies
    Copy to Clipboard
  • r

    Riley Hun

    1 year ago
    Hi Prefect Team, I upgraded to Prefect version
    0.14.4
    and I can no longer run
    flow.environment.executor = executor
    . I get the following error
    AttributeError: 'NoneType' object has no attribute 'executor'
    . Does anyone know how to attach an executor to a flow registered to Prefect Server using Docker Storage in the latest version of Prefect. Had no issues with this is Prefect version
    0.13.18
    . In the meantime, I'll downgrade to
    0.13.8
    for the time being.
    r
    Chris White
    2 replies
    Copy to Clipboard
  • s

    SK

    1 year ago
    I have setup prefect in aws ec2 and able to register the flow and the flow is running too, but the aws prefect server page is not showing any flow and the page looks like the below. Please help
  • s

    SK

    1 year ago
  • Aiden Price

    Aiden Price

    1 year ago
    Hi Prefect people, I'm wondering how I can get the KubernetesRun to use a specific image tag that I want to provide dynamically (or at least whenever I reinstall the server and agent with Helm). The backstory is that I have a custom base image for my Agent, Kubernetes jobs and Dask cluster, and when I launch a KubernetesRun there's a delay before the job starts because I call for the
    latest
    tag and
    Always
    pull so I can be sure I'm getting the latest dependencies. I'd like to minimise the delay by changing to a
    IfNotPresent
    image pull policy, but then I'd need to pin the image tag to a specific version, but then I'd want to make sure that specific tag could be updated as I keep pushing updates. Does anyone have any advice? Thanks.