• m

    Max Lei

    1 year ago
    Hi All, How do I include default loggers to be logged in the prefect UI? Right now I have a function that setup logs, for example
    def setup_logs():
        # Example config
        'root': {
                'level': 'DEBUG',
                'handlers': [
                    'console',
                    'info_file_handler',
                    'fluent'
                ]
         }
         logging.config.dictConfig(logging_config)
    
    class DoSomething():
        def run(self):
            setup_logs()
            <http://logger.info|logger.info>("Here")
    The logging configurations sets up a default logger to stdout, a file handler, and also to a 3rd party log streaming service using fluent. I see the local file output, but not through the prefect UI.
    export PREFECT__LOGGING__EXTRA_LOGGERS="['root']"
    
    [logging]
    extra_loggers = "['root']"
    I have setup a toml file, and tried to set the environmental variable before running the agent, and the flow itself. However I still do not see the logs in the prefect UI. Is there something else I need to do?
    m
    1 replies
    Copy to Clipboard
  • n

    Nakul Gowdra

    1 year ago
    Hi Team, Has anyone encountered issues running flows on prefect cloud, the flow runs perfectly locally ( on windows pc) but fails with "Failed to load and execute Flow's environment: TypeError('an integer is required (got type bytes)')" . I have tried to register via flow.register() as well as prefect cmd. Thanks in advance - BTW it is using a boto3 aws library and python 3.7.2
    n
    1 replies
    Copy to Clipboard
  • Newskooler

    Newskooler

    1 year ago
    Hi Prefect community ! 👋 Could someone (or more) please share any best practices in regards to registering a flow? If I have my code in a repository and then this repository deployed to a server where prefect is running (server and agent) what is an elegant way of registering a flow? currently i need to open some kind of python (ipython or notebook) and register manually each flow (and flow version). Surely there must be a more elegant way?: )
    Newskooler
    Robin
    +2
    18 replies
    Copy to Clipboard
  • Robin

    Robin

    1 year ago
    Is there a possibility to access variables that were defined in tasks in the state handler? For example, in each task of a mapped task, I want to make a notification about the state change. And in that notication I would like to add a system id that is different for each task instance of the mapped task. Does somebody know how to access a variable from the task within the state handler?
    Robin
    Kyle Moon-Wright
    5 replies
    Copy to Clipboard
  • s

    Sven Teresniak

    1 year ago
    Is there any way I can identify a task-run by some runtime information? I need to query task-runs by information that is only available during runtime, based on
    Parameter
    . I cannot use slug or tags because I cannot set them to parameter values (or can I?). I still work on a Lock-like ResourceManager but thats very difficult when it comes to scheduling-/parameter-dependent locking. Creating a Lock on constants (e.g. a constant string) seems rather easy. What I need (and trying to build) is tag concurrency limits for the standalone version. 😃
    s
    Kyle Moon-Wright
    30 replies
    Copy to Clipboard
  • ale

    ale

    1 year ago
    Hi folks, we’re configuring Prefect to run tasks with Fargate Agent. Is there a way to tell Prefect to create task definitions with a predefined naming convention instead of flow_name along with tags flow_id and flow_version?
    ale
    1 replies
    Copy to Clipboard
  • i

    itay livni

    1 year ago
    Hi - Is there an example or documentation for naming mapped tasks? I could not find it 🙃. Thanks
    i
    j
    2 replies
    Copy to Clipboard
  • ale

    ale

    1 year ago
    Hey folks, finally I was able to configure Fargate Agent. Now I want to run a task on Fargate. I’m using the following config:
    FargateTaskEnvironment(
        taskRoleArn=ETL_TASK_ROLE_ARN,  # ARN of the task role
        executionRoleArn=ETL_EXECUTION_ROLE_ARN  #  ARN of the execution role
    )
    But I get the following error back from the Agent:
    An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Fargate requires task definition to have execution role ARN to support ECR images
    Any suggestions?
    ale
    s
    +1
    37 replies
    Copy to Clipboard
  • h

    Hui Zheng

    1 year ago
    Hello, I have a question about flow container docker registry. Could the docker container be registered with multiple image-tags? https://docs.prefect.io/api/latest/environments/storage.html#docker
    h
    Kyle Moon-Wright
    2 replies
    Copy to Clipboard
  • k

    Kyle Pierce

    1 year ago
    If I want to make a task reusable across flows do i need to make it into a package or is there someway to access modules in other files? I looked on the documentation there isnt anything the explains how to reuse tasks.
    k
    Kyle Moon-Wright
    2 replies
    Copy to Clipboard