• Max Kolasinski

    Max Kolasinski

    5 months ago
    Hi, I saw that there were some issues earlier today with Prefect Cloud. We’re currently still seeing a lot of our ECSRun tasks fail to begin execution correctly- are we potentially seeing delayed after effects from the earlier service disruption?
    Max Kolasinski
    Anna Geller
    +1
    5 replies
    Copy to Clipboard
  • j

    Jason

    5 months ago
    Howdy - I've attempting to push a hello world job to ECR using the Docker storage for use with ECS. It looks like it fails on the last part, after passing health checks, where it's attempting to push to the registry. It's triggering this exception:
    <https://github.com/PrefectHQ/prefect/blob/master/src/prefect/storage/docker.py#L610-L613>
    Has anyone run into a similar exception? I'm successfully authed to ECR from my local docker --login as well.
    j
    Kevin Kho
    3 replies
    Copy to Clipboard
  • j

    Jason

    5 months ago
    I have an ECSRun for a hello world that appears to be hanging. I've successfully pushed the image to ECR via the Docker() storage, and the task shows up as scheduled in Prefect Cloud - which can also see the ECS Agent I added following the guide (https://docs.prefect.io/orchestration/agents/ecs.html#running-ecs-agent-in-production). I'm a bit puzzled on how to proceed to debug why the agent isn't popping the task from the queue in the cloud.
    j
    Kevin Kho
    7 replies
    Copy to Clipboard
  • Alexander Belikov

    Alexander Belikov

    5 months ago
    Hi I have a question regarding scheduling prefect flows with parameters, I think I'm missing something simple : how do you pass db creds, if not through parameters (without hard-coding passwords, paths, etc)? I've posted my questions on SO: https://stackoverflow.com/questions/71875658/running-registering-prefect-flows-with-parameters-on-schedule, please let me know what you think. thanks!
    Alexander Belikov
    Anna Geller
    +1
    9 replies
    Copy to Clipboard
  • egk

    egk

    5 months ago
    Is it possible to return a Socket from a task?
    egk
    Kevin Kho
    16 replies
    Copy to Clipboard
  • Aric Huang

    Aric Huang

    5 months ago
    Hi, I'm looking into running Prefect on Kubernetes clusters in different regions, and had a question about handling GCSResults. If we use the
    @task(result=GCSResult(bucket=<bucket>))
    method of configuring a task result, is the bucket path fixed at flow registration time? If so, is there a way it can be dynamically set at flow run time? What I'm hoping to do is have flows that can be registered to run on different clusters (using agent labels), and have their GCSResult bucket path be configured via an env var on the cluster. That way we can re-use the same flow code across different clusters but have different results buckets depending on the cluster.
    Aric Huang
    Anna Geller
    +1
    9 replies
    Copy to Clipboard
  • j

    Jason

    5 months ago
    Do you need to set the image on ECSRun for the task to use a a specific image or will it associate the Docker storage pushed to ECR in the same flow as the base image to use?
    j
    Kevin Kho
    4 replies
    Copy to Clipboard
  • p

    Philip MacMenamin

    5 months ago
    Cryptic errors in ShellTask - Hi, I'm running a flow with a number of shellTasks, and encountering errors like:
    2022-04-14 09:42:06-0600] ERROR - prefect.TaskRunner | Task 'ShellTask[0]': Exception encountered during task execution!
    Traceback (most recent call last):
      File "/blah/python3.9/site-packages/prefect/engine/task_runner.py", line 880, in get_task_run_state
        value = prefect.utilities.executors.run_task_with_timeout(
      File "/blah/python3.9/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
        return task.run(*args, **kwargs)  # type: ignore
      File "/blah/python3.9/site-packages/prefect/utilities/tasks.py", line 456, in method
        return run_method(self, *args, **kwargs)
      File "/blah/python3.9/site-packages/prefect/tasks/shell.py", line 131, in run
        tmp.write(command.encode())
    AttributeError: 'list' object has no attribute 'encode'
    I have a couple of questions: • Is there a way to tag shell_tasks such that you can see some clue as to which one failed? • Can I get a better description of the failure At the moment I have
    shell_task = ShellTask(log_stderr=True, return_all=True, stream_output=True)
    p
    Kevin Kho
    17 replies
    Copy to Clipboard
  • egk

    egk

    5 months ago
    Is it a valid use-case to use Prefect for the sake of having a managed way of running tasks rather than "data pipelines"? I just want the workflow aspect of it.
    egk
    Kevin Kho
    6 replies
    Copy to Clipboard
  • egk

    egk

    5 months ago
    Can I run Prefect fully on-premise without cloud features whatsoever?
    egk
    Kevin Kho
    +1
    6 replies
    Copy to Clipboard