I’m creating stock trading signals based on price and indicator data. I’m wondering what the most efficient way to add indicators to a frame of price data, with the catch that I can fill NaN, null, or similar when the frame length is not long enough to support the indicator.

Here’s what I’m doing now:

    watchlistname = 'Russell_3000'
    universe_symbols = norgatedata.watchlist_symbols(watchlistname)
    print(f'Looking at {len(universe_symbols)} symbols in recent_ipo')
    # print(universe_symbols)

    # signal code
    count  = 0
    signal = []
    for symbol in universe_symbols:
        try:
            count += 1
            if(count%300 == 0):
                print(f'We have looked at {count} symbols.')

            symbol_data = get_data(symbol)


            if count == 1:
                print(f'Looking at data on {symbol_data.iloc[-(test_days_back + 1), symbol_data.columns.get_loc("Date")]}')

            if test_days_back != 0:
                symbol_data = symbol_data[:-test_days_back]

            ATR_length = 10
            liquidity_ma_length = 10
            ROC_Length = 40
            HHV_Length = 10

            ROC = 'ROC_' + str(ROC_Length)

            ATR = 'ATRr_' + str(ATR_length)

            roc_short = symbol_data.ta.roc(length=ROC_Length)
            atr = symbol_data.ta.atr(length=ATR_length)

            symbol_data = pd.concat([symbol_data, roc_short, atr], axis=1)

            symbol_data['HHV'] = symbol_data['Close'].rolling(HHV_Length).max()


            symbol_data['C*V'] = symbol_data['Close'] * symbol_data['Volume']
            symbol_data['Liquidity'] = symbol_data['C*V'].rolling(liquidity_ma_length).min()

            symbol_data.insert(0, 'bar_num', range(0, 0 + len(symbol_data)))
            # print(symbol_data)

            symbol_data = symbol_data.drop(columns=['Turnover', 'Unadjusted Close', 'Dividend', 'C*V'])
            symbol_data = symbol_data.rename(columns={ROC:'ROC', ATR: 'ATR', 'Open':'open', 'High':'high',
                                                      'Low':'low', 'Close':'close', 'Volume':'volume',
                                                      'Liquidity':'liquidity'})

            signal.append(symbol_data.iloc[-1])

            current_trade_date = symbol_data.iloc[-1, symbol_data.columns.get_loc('Date')]
            previous_trade_date = symbol_data.iloc[-2, symbol_data.columns.get_loc('Date')]
        except Exception as e:
            print(symbol, "+", e)

    signal_data = pd.DataFrame(signal)
    signal_data = signal_data.reset_index(drop=True)

This code does:

1. Creates a list of equities to loop through
2. Runs a for-loop for each equity
3. Gets OHLC data for the current symbol
4. Adds indicators to it.
5. Appends the last line of the OHLC frame to a list. this last line is the latest data and represents the current signals.
6. Take this list of ‘last lines’ to create a dataframe of all of the equities and indicator values associated with these equities.

I use a try/except block to handle the event that I pass a dataframe with, say, 15 rows of historical data. This short dataframe would raise an error as the indicators cannot be calculated.

In the current iteration, if one indicator is not able to be added(due to not enough dataframe length or otherwise), we skip the entire symbol. I would like to change this to add all indicators that can be calculated or fill with NaN, null, etc.

My leading idea is to create a function for each indicator, and to provide a check on the dataframe length before calculating the values, or even a try/except block for each indicator. My concern is this doesn’t seem very scalable, the full system will calculate hundreds of indicators on each symbol.

Am I on the right track? Am I missing anything? This seems inefficient.