Hey guys, curious if someone can help me understand task mapping / flattening for an iterated for loop. This loop calculates tracking error for each account in several strategies, and as such takes quite some time. I know task mapping would be of use here, but can't quite seem to get it right from the docs.

strategies = ['strat1', 'strat2', 'strat3']

for strat in strategies:
    strat = [strat]
    
    # this gets the list of applicable accounts
    lstAccnts = GetRiskModelAccnts(strategy = strat)

    # this gets the tracking error risk on an account level (each account has between 1 and 3 strategies)
    dfAccntsTEc, dfAccntsTE, dfAccntsMoments = RiskWrapper(
                                                           lstAccnt = lstAccnts,
                                                           benchmark = 'SPY',
                                                           lstAllowStrats = strat,
                                                           AggregationLevel = ['accnt']
                                               )

    # this gets the tracking error risk on an account-strategy level 
    dfAccntsTEc, dfAccntsTE, dfAccntsMoments = RiskWrapper(
                                                           lstAccnt = lstAccnts,
                                                           benchmark = 'SPY',
                                                           lstAllowStrats = strat,
                                                           AggregationLevel = ['accnt', 'strategy']
                                               )

You'll notice the aggregation level is also something that is iterated, and there is a third aggregation level ['accnt', 'strategy', 'strategyLevel'] that isn't shown above. I know mapping could be of use here as well but I'm not sure the correct syntax to implement it.

Hey @Joseph Loss, I think I know what you’re saying. In general there is no two-stage mapping. The approach would be to map on strategies, get the

lstAccnts

so you have a list of lists,

flatten

that so you end up with 1 list.

map

on that. If you need some sort of traceability with the list of lists, you can have intermediate tasks that go reorganize the list.

Do I map strats in both "GetRiskModelAccnts" and in RiskWrapper? The above was the excerpt of the problem, this is the code that I've written so far:

with Flow("load-tblTrackingErrorTicker ParallelExecution") as flow:

    strats = Parameter("strats", default=["MII","HEP","OYE","OYENT"])
    lstAccnts = GetRiskModelAccnts(mapped(strats))

    dfAccntsTEc, dfAccntsTE, dfAccntsMom=RiskWrapper(lstAccnt = mapped(lstAccnts),
                                                       strBenchTicker = 'SPY',
                                                       lstShowPlots = [False, False],
                                                       lstAllowStrats = mapped(strats),
                                                       lstAggLevels = ['accnt'],
                                                       bDataFields = True)

flow.run_config = LocalRun()

# Use a `LocalDaskExecutor` to run this flow
# This will run tasks in a thread pool, allowing for parallel execution
flow.executor = LocalDaskExecutor()

flow.run()

Wondering why you are using the

mapped

instead of

GetRiskModelAccnts.map()

? I think you so the

.map(strats)

.

lstAccnts

becomes a list of lists. You can flatten with

RiskWrapper(lstAccnt = flatten(lstAccnts))

mapped() - no reason other than I saw it in a doc example lol. The examples were simple arithmetic so I was a bit lost at something like this, but I will give that a shot and use unmapped() for the other variables! lstAllowStrats, the 4th argument, was usually 1 strat in that for loop. Since I am mapping strats to lst accnts, would I pass in the same here? Referring to RiskWrapper()

Can I see that example? Ah I see maybe keep your for loop because you will get parallelization anyway of the downstream mapping

ah okay that makes sense! I'll give it a try this evening and let you know how it goes!