• j

    Jasono

    1 year ago
    Is there a way to quickly start running an existing flow starting with a task in the middle of the dependency graph? (treating like all its dependencies have already been met)
    j
    Dylan
    14 replies
    Copy to Clipboard
  • a

    Andrew Hannigan

    1 year ago
    Suppose I have a class called
    ReadTable
    which extends
    Task.
    This class just SELECTs data from a big SQL table. I want to perform this SELECT in parallel with say n sub-Tasks. So the
    ReadTable(n)
    object should spawn n sub-Tasks when it runs. How would this work within Prefect's OOP paradigm, where
    ReadTable
    inherits
    Task
    ?
    a
    1 replies
    Copy to Clipboard
  • c

    Cab Maddux

    1 year ago
    Hi! Quick question, we had flow concurrency for a specific label set to 5, we scheduled 20 flows and the first 5 ran for 12 hours (and are still running). We updated flow concurrency for that label to 20 but the remaining 15 flows have remained in the scheduled state. We've confirmed that we can run more than 5 flows concurrently by triggering a brand new flow which gets submitted and run immediately, but we can get those original 15 to submit. Any suggestions?
    c
    Dylan
    15 replies
    Copy to Clipboard
  • Dor Menachem

    Dor Menachem

    1 year ago
    Hello Prefect Community, for a reason I don't understand my file can't recognize the prefect 'from prefect import task, Flow'. although it works, how can I fix that?
  • Alexander

    Alexander

    1 year ago
    Does all tables in DB created by prefect have proper FK relations so if i remove let say a flow via graphql, all connected flow runs and logs will be removed too?
    Alexander
    1 replies
    Copy to Clipboard
  • j

    Jasono

    1 year ago
    Hi, some of my tasks import modules from a distant path using
    sys.path.append()
    . This seems to work fine when I run the flow locally with flow.run(), but not when running it from the Prefect Cloud/Server. (The error says the module can’t be found) Does it sound right? If so, why would it be the case?
    j
    s
    6 replies
    Copy to Clipboard
  • o

    Omar Sultan

    1 year ago
    Hi Everyone, I'm using the imperative API, and was wondering if there was any tutorials or samples for the use of Parameters with imperative API
    o
    Dor Menachem
    2 replies
    Copy to Clipboard
  • Amanda Wee

    Amanda Wee

    1 year ago
    Hello! Would it be possible to add to the Prefect CLI and/or the Core client an option to create projects only if they don't already exist? My use case is that I intend to create projects and register flows when an ECS task starts up, and then the ECS task will go on to serve as a Prefect agent. The new
    idempotency_key
    option and
    serialized_hash()
    method for flow registration makes this really nice, but I'll still get an error for the project creation when the ECS task restarts. I could ignore this error, but then that could be problem if somehow the error is not due to the project already existing, but a genuine failure to create a new project.
    Amanda Wee
    1 replies
    Copy to Clipboard
  • JC Garcia

    JC Garcia

    1 year ago
    Hello guys, I am seeing the following error when trying to build a flow in an azure devops pipeline:
    WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f74c98d0710>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/cryptography/
    ERROR: Could not find a version that satisfies the requirement cryptography (from versions: none)
    ERROR: No matching distribution found for cryptography
    
    Removing intermediate container f73d8367b237
    The command '/bin/sh -c pip install requests cryptography snowflake-connector-python wheel' returned a non-zero code: 1
    Traceback (most recent call last):
      File "./flows/frieze-viewing-room-flow/flow.py", line 113, in <module>
        flow.storage.build()
      File "/usr/local/lib/python3.6/dist-packages/prefect/environments/storage/docker.py", line 351, in build
        self._build_image(push=push)
      File "/usr/local/lib/python3.6/dist-packages/prefect/environments/storage/docker.py", line 418, in _build_image
        "Your docker image failed to build!  Your flow might have "
    ValueError: Your docker image failed to build!  Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.
    ##[error]Bash exited with code '1'.
    any ideas on why this might be happening? it looks like it cannot connect to pypi for some odd reason. ideas/thoughts welcome. thanks!
  • deltikron

    deltikron

    1 year ago
    Hi! I'm running prefect server and my flows keep getting stuck on
    submitted
    . I originally started with a kubernetes agent and thought it was a configuration issue between prefect and kubernetes, but I get the same behavior with a docker agent. The agents pull the correct docker image and submit it, but the flows don't enter a running state. I'd be grateful for any ideas or pointers! Cheers
    deltikron
    ale
    +1
    14 replies
    Copy to Clipboard