s

    Sandeep Aggarwal

    2 years ago
    Is there any architecture diagram for prefect core. It can be really helpful when self hosting core server and tell how various components in system interact with each other. Currently I am facing issues where individual components stops working/responding due to various reasons (Majority of them being triggered by system resource constraints). The architecture diagram will help understand if it is possible to selectively scale individual components.
    j

    josh

    2 years ago
    Hi @Sandeep Aggarwal I am not immediately aware of a diagram at the ready. We definitely should add one to the docs, if you want to open a docs ticket on the repo that would be great! There are a few resources that have been linked in this channel before which may give you some context:

    https://youtu.be/FETN0iivZps

    https://docs.google.com/presentation/d/1TfOsYmsjgbwXRkiItb2ZeTW_oYxXWAWKMtEnEFOyPiA/edit#slide=id.g7453fd8b20_0_129
    a

    An Hoang

    2 years ago
    The Pycon 2020 slide really helped! Thanks @josh 🙂
    s

    Sandeep Aggarwal

    2 years ago
    Hey @josh thanks for quick response and sharing the resources. I will take a look at them sometime tomorrow. Can you also help with the list of components that support more than one running instances. From what i have observed, agent and scheduler can't have multiple instances running (at least targeting same workflows), but what about graphql, apollo & hasura? (Excuse me if this sounds too lame as my knowledge with Graphql ecosystem is quite limited as of now.)
    j

    josh

    2 years ago
    For the agent you can have multiple running but it isn’t really beneficial unless each agent has a different set of labels pulling different flows. Similar for the scheduler you probably wouldn’t need more than one instance. As for the others I’m pretty sure multiple instances would help but I’m going to @Zachary Hughes for his opinion 🙂
    Zachary Hughes

    Zachary Hughes

    2 years ago
    Definitely not a "lame" question! I'll echo what Josh said: having more than one agent and scheduler are unlikely to be super beneficial. There's absolutely no harm in horizontally scaling any of your Apollo, GQL, or Hasura pods though. You might need to make some tweaks depending on how you're deploying, but multiple copies should be good! 👍
    s

    Sandeep Aggarwal

    2 years ago
    Thats really helpful. Thanks for your inputs @josh @Zachary Hughes
    j

    Javier Garcia

    2 years ago
    @josh - I couldn't access the slide, it seems like the link is now private. I requested access but not sure who will see that
    j

    Javier Garcia

    2 years ago
    same issue
    j

    josh

    2 years ago
    Here’s a PDF copy 🙂
    j

    Javier Garcia

    2 years ago
    thanks!